Linear engineering external environment hidden danger low-altitude intelligent inspection system
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-20
- Publication Date
- 2026-08-11
AI Technical Summary
现有系统缺乏针对数据传输与存储的有效安全机制,数据易被未授权访问或窃取,不仅影响巡检数据的完整性与保密性,还可能引发更大的工程安全漏洞
其一,本申请提供了一种线性工程外部环境隐患低空智能巡检系统,系统通过构建包含巡检管理平台、无人机终端、数据回传系统、云端智能分析平台及隐患隔离与警报模块的协同架构,实现了低空遥感巡检的全流程自动化与智能化。该系统能够基于多源数据自动生成并执行巡检任务,在终端侧利用边缘智能实现泥石流隐患的实时初筛与标记,并通过云端深度分析精准识别地质灾害的演变趋势。其建立的从预警发布、现场核查到模型优化与策略调整的闭环反馈机制,显著提升了巡检效率、预警准确性与系统自适应能力,有效克服了传统方法在复杂环境下检测精度低、响应滞后且缺乏持续优化能力的缺陷;
Smart Images

Figure CN122239176B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent inspection technology, and more specifically, to a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects. Background Technology
[0002] In the field of low-altitude remote sensing intelligent inspection, existing technologies mainly rely on manual inspection or simple automated equipment, which suffers from low detection accuracy, slow response speed, and poor environmental adaptability. Especially under complex meteorological and terrain conditions, such as the rainy season in mountainous corridors or areas prone to geological disasters, traditional methods struggle to achieve efficient and accurate early identification and dynamic monitoring of debris flows, leading to delayed warnings and increased risks to engineering operations and personnel safety. Furthermore, with the widespread application of Industrial Internet technology, the large amount of sensitive data involved in inspection systems, such as high-precision geographic information, real-time image sequences, and geological disaster records, faces serious threats of information leakage during collection, transmission, and processing. Existing systems lack effective security mechanisms for data transmission and storage, making data vulnerable to unauthorized access or theft, which not only affects the integrity and confidentiality of inspection data but may also lead to larger engineering security vulnerabilities. Meanwhile, existing technologies lack closed-loop feedback and adaptive optimization capabilities based on real-time field data, and cannot dynamically adjust monitoring strategies and model parameters according to the evolution of debris flows. They are rigid in responding to sudden and variable geological disasters, further exacerbating the risk of increased losses due to delayed response or decision-making errors, and making the system more vulnerable to new threats or complex interference.
[0003] Therefore, this application provides a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects to solve one of the aforementioned technical problems. Summary of the Invention
[0004] The purpose of this application is to provide a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects, which can solve at least one of the aforementioned technical problems. The specific solution is as follows: According to a specific embodiment of this application, in a first aspect, this application provides a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects, comprising: The inspection management platform is used to generate and issue inspection instructions based on the input inspection requirements, combined with geographic information, environmental data and historical potential hazards. The drone terminal is used to execute the inspection command, including acquiring images and using an integrated edge intelligence processing module to perform real-time analysis on the acquired images. Based on the results of the real-time analysis, the flight altitude and the intensity of the downwash airflow are adjusted to identify and mark the location of the debris flow and obtain time-series image data. The data feedback system is used to receive and upload time-series image data marked by the UAV terminal; The cloud-based intelligent analysis platform is used to perform in-depth change detection and risk modeling on uploaded time-series image data in order to determine the type of debris flow and identify its evolution trend. The hazard isolation and alarm module is used to adjust the flight altitude and the direction of the downwash airflow based on the type and evolution trend of the debris flow, so as to keep the debris away from the engineering equipment and issue alarms through multiple channels to protect the engineering equipment.
[0005] Furthermore, the adjustment of flight altitude and downwash intensity based on real-time analysis results includes: S21, when the edge intelligent processing module determines in real-time analysis that the location of the debris flow cannot be located due to fog obscuring the view, it controls the UAV terminal to reduce its flight altitude and simultaneously increases the rotor speed to increase the intensity of the downwash airflow. S22, while increasing the downwash airflow intensity, adjust the upwash airflow intensity of the UAV terminal to balance the fuselage pressure difference caused by the decrease in altitude and the enhancement of the downwash airflow, and maintain stable flight attitude; S23, the enhanced downwash airflow is used to disperse the fog below the current position, and after the airflow stabilizes, the image acquisition device acquires an image of the current area. S24, if the fog below the current position is not dispersed, return to S21 and readjust the flight altitude and downdraft intensity of the drone terminal. S25, the edge intelligent processing module analyzes the image, and if the current downwash airflow's area of action is within the location where the debris flow occurs, the current location is recorded and marked. S26, control the UAV terminal to move horizontally to the adjacent unexplored area, and repeat steps S23 to S24 until the boundary of the debris flow location is completely recorded and marked; S27, Generate time-series image data based on the acquired images.
[0006] Furthermore, controlling the drone terminal to move horizontally to an adjacent unexplored area includes: S261, Set the initial detection direction and control the UAV terminal to move horizontally along the initial detection direction; S262, During the movement, the edge intelligent processing module continuously analyzes the acquired images to determine whether the boundary features between the area where the debris flow occurred and the area where the debris flow did not occur are identified. S263, if the boundary feature is not identified, the drone terminal is controlled to continue moving in the current direction; if the boundary feature is identified, the current position is determined to be the boundary point of the debris flow and recorded, and then the boundary tracking mode is entered. S264, In boundary tracking mode, starting from the current boundary point, control the UAV terminal to move along the tangential direction of the estimated debris flow boundary and continuously acquire images. S265, adjust the flight path based on continuously acquired images, so that the UAV terminal keeps flying along the boundary of the debris flow location, and continuously records and marks the flight trajectory; S266, Repeat steps S264 to S265 until the flight trajectory forms a closed area, which is the location of the identified debris flow.
[0007] Furthermore, the edge intelligence processing module uses a neural network model based on a hybrid CNN-Transformer architecture for image analysis; The hybrid CNN-Transformer architecture includes: A multimodal preprocessing unit is used for image de-raining and fog enhancement processing; Hybrid backbone network units are used to extract local features of the enhanced image through CNN layers and global features through Transformer layers; The multi-task deep interaction unit is used to perform feature interaction between object detection and semantic segmentation tasks based on local and global features. The dynamic loss optimization unit is used to balance the multi-task losses in the image analysis process using an uncertainty weighting method.
[0008] Furthermore, the multimodal preprocessing unit adopts a defogging framework that combines physical-driven and data-driven approaches. The physical drive is based on an improved atmospheric scattering model, which estimates transmittance and atmospheric light value by jointly using dark channel priors and color attenuation priors.
[0009] Furthermore, the multi-task deep interaction unit includes a cross-attention module, which is used to implement bidirectional feature guidance for detection and segmentation tasks. In this process, the confidence of the detection box is used as a spatial attention weight to weight the segmentation feature map, and the semantic probability of the segmentation map is used as a contextual feature input to the detection head.
[0010] Furthermore, the total loss function of the dynamic loss optimization unit consists of uncertainty-weighted loss and contrast loss.
[0011] Furthermore, the hazard isolation and alarm module is configured as follows: S51, Receive the analysis results of debris flow type and evolution trend of debris flow output by the cloud intelligent analysis platform; S52, determine whether the debris flow is a debris flow flowing through the engineering site; S53, If it is a debris flow passing through the engineering site, then based on the location of the debris flow, the preset location of the engineering equipment, and the evolution trend of the debris flow, calculate the evolution direction of the debris flow and the direction vector from the location of the debris flow to the engineering equipment. S54. If the calculated direction of the debris flow evolution is different from the direction vector from the location of the debris flow to the engineering equipment, then an alarm message containing the location, type and evolution trend of the debris flow is generated and released through multiple channels.
[0012] Furthermore, if the calculation determines that the evolution direction of the debris flow is the same as the direction vector from the location of the debris flow to the engineering equipment, then an isolation operation is performed, which includes: S541, control the drone terminal to fly directly above the preset position coordinates of the engineering instrument; S542, control the UAV terminal to lower the flight altitude a second time, and adjust the vector direction of the downwash airflow so that it forms an outward-sloping airflow below the UAV; S543, the intensity of the upwash airflow of the UAV terminal is adjusted for the second time to balance the pressure difference of the fuselage caused by the decrease in altitude and the adjustment of the direction of the downwash airflow, so as to maintain the stability of the flight attitude; S544, using the outward-sloping downward airflow, the gravel located around the engineering equipment is moved away from the engineering equipment. S545, control the UAV terminal to rotate horizontally around the vertical line between the geometric center of the UAV terminal and the ground as an axis; S546, through the coordinated operation of steps S542 to S545, a ring-shaped cleaning area is formed around the engineering equipment, so that the accumulated gravel in the area forms an isolation ring to protect the engineering equipment.
[0013] Furthermore, the types of debris flows include debris flows that flow through the engineering site and debris flows that do not flow through the engineering site. If the debris flow is a debris flow that does not flow through the engineering site, then an alarm message containing the location, type and evolution trend of the debris flow is generated and released through multiple channels.
[0014] Compared with the prior art, the above-described solutions of this application have at least the following beneficial effects: Firstly, this application provides a low-altitude intelligent inspection system for potential hazards in the external environment of linear engineering projects. The system achieves full automation and intelligence of low-altitude remote sensing inspections by constructing a collaborative architecture comprising an inspection management platform, UAV terminals, a data transmission system, a cloud-based intelligent analysis platform, and a hazard isolation and alarm module. This system can automatically generate and execute inspection tasks based on multi-source data. At the terminal side, it utilizes edge intelligence to achieve real-time initial screening and marking of debris flow hazards, and accurately identifies the evolution trend of geological disasters through in-depth cloud analysis. Its closed-loop feedback mechanism, from early warning issuance and on-site verification to model optimization and strategy adjustment, significantly improves inspection efficiency, early warning accuracy, and system adaptability, effectively overcoming the shortcomings of traditional methods such as low detection accuracy, slow response, and lack of continuous optimization capabilities in complex environments. Secondly, in the debris flow detection and situational awareness stage, this invention introduces physical intervention methods such as actively lowering altitude and enhancing downwash airflow to disperse dust or fog, combined with a collaborative control method that synchronously adjusts upwash airflow to maintain attitude stability. This solves the key technical problem of traditional UAV inspections failing to observe in harsh environments such as rain, fog, and dust, significantly improving the early detection rate and positioning accuracy of debris flows. Furthermore, through boundary tracking algorithms based on real-time visual feedback and time-series image data generation technology, the system achieves automatic, accurate delineation and dynamic recording of irregular disaster area boundaries, overcoming the technical shortcomings of traditional point-based alarms that cannot reflect the scope and evolution of the disaster, thereby comprehensively improving the automation level, spatial accuracy, and timeliness of geological disaster situational awareness. Thirdly, in the intelligent decision-making and proactive protection phase, this invention establishes a linked architecture of cloud-based deep analysis and edge-based real-time execution, and employs a vector-based quantitative threat assessment model. This addresses the prominent problems of traditional response strategies being singular and decision-making lagging behind changes in the disaster situation, significantly improving the accuracy of early warning issuance and the scientific nature of decision-making. Particularly noteworthy is the innovative technique of commanding drones to form an outwardly rotating airflow field to construct a debris flow isolation zone. This is the first time the system has achieved dynamic and proactive physical isolation of critical engineering equipment using drones, overcoming the passive situation of traditional monitoring systems that only issue alarms but do not take action. This directly reduces the immediate risk of critical assets being damaged by debris flows, buying valuable time for manual emergency response. Attached Figure Description
[0015] Figure 1 A block diagram of a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering, according to an embodiment of this application, is shown. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0017] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.
[0018] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0019] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.
[0020] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”
[0021] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.
[0022] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.
[0023] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.
[0024] The embodiments provided in this application are embodiments of a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects.
[0025] The following is combined Figure 1 The embodiments of this application will be described in detail.
[0026] Figure 1 A unit block diagram of a low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects, according to an embodiment of this application, is shown. Figure 1 As shown, the low-altitude intelligent inspection system for potential external environmental hazards in linear engineering includes an inspection management platform, a drone terminal, a data transmission system, a cloud-based intelligent analysis platform, and a hazard isolation and alarm module.
[0027] In some embodiments, the system achieves intelligent inspection throughout the entire process from task planning to closed-loop feedback through the collaborative work of the inspection management platform, drone terminal, data feedback system, cloud intelligent analysis platform, and hazard isolation and alarm module. The following detailed description is provided in conjunction with specific embodiments.
[0028] In some embodiments, the workflow of the low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects includes a task planning phase, an autonomous inspection phase, a data feedback phase, an intelligent analysis and early warning phase, and an early warning issuance and closed-loop feedback phase. By integrating edge intelligent processing and multi-temporal depth change detection, the system achieves real-time identification, risk modeling, and adaptive optimization of geological hazards in corridors, forming a complete logical closed loop from data acquisition to model iteration.
[0029] In some embodiments, the task planning phase is performed by the inspection management platform. Based on the inspection requirements input by maintenance personnel, the platform generates and issues inspection instructions by combining geographic information, environmental data, and historical hazards. For example, inspection requirements may include specified sections, inspection types, and priorities; geographic information includes terrain and obstacle data; environmental data includes real-time weather information; and historical hazards include locations where geological disasters have occurred. Path generation calculates the UAV's flight path, altitude, and speed parameters through comprehensive analysis of these factors. Instructions are transmitted to the UAV terminal via a satellite link.
[0030] In some embodiments, the autonomous inspection phase is performed by the UAV terminal. The UAV terminal receives and executes inspection commands, including automatic takeoff, flight along a planned path, and acquisition of corridor images. Data acquisition is achieved by capturing high-resolution images at a set frequency using an onboard visible light camera. An edge intelligent processing module is integrated into the UAV terminal to perform real-time analysis of the acquired images. Based on the results of the real-time analysis, it adjusts the flight altitude and the intensity of the downwash to identify and mark the location of debris flows, obtaining time-series image data. For example, the edge intelligent processing module uses a neural network model based on a hybrid CNN-Transformer architecture, achieving hazard detection through multimodal preprocessing, a hybrid backbone network, deep multi-task interaction, and dynamic loss optimization. Emergency response includes environmental adaptive adjustments, such as activating waterproofing measures in windy or rainy weather, or adjusting the course via an obstacle avoidance controller to avoid collisions.
[0031] In this embodiment, the edge intelligence processing module uses a neural network model based on a hybrid CNN-Transformer architecture for image analysis; The hybrid CNN-Transformer architecture includes a multimodal preprocessing unit, a hybrid backbone network unit, a multi-task deep interaction unit, and a dynamic loss optimization unit. Specifically, the multimodal preprocessing unit performs rain and fog removal enhancement on the corridor images; the hybrid backbone network unit extracts local features from the enhanced corridor images through CNN layers and global features through Transformer layers; the multi-task deep interaction unit performs feature interaction between object detection and semantic segmentation tasks based on local and global features; and the dynamic loss optimization unit balances the multi-task losses using an uncertainty-weighted approach during image analysis.
[0032] As a specific embodiment, the multimodal preprocessing unit of the edge intelligent processing module adopts a hybrid rain and fog removal framework combining physical and data-driven approaches. The physical-driven approach is based on an improved atmospheric scattering model, which jointly estimates transmittance t(x) and atmospheric light value A through dark channel priors and color attenuation priors, as shown in the formula: Where I(x) is the input foggy image and J(x) is the target clear image. (x) represents the raindrop scattering noise term. Data-driven enhancement uses a generative adversarial network to generate rain-fog-clear image pairs, improving the model's generalization ability through adversarial training. Detail enhancement employs multi-scale detail fusion to separate low-frequency and high-frequency components of the image, amplifying the high-frequency components through adaptive gain to highlight potential hazards.
[0033] in, (x) represents the raindrop scattering noise term. High-frequency noise is suppressed by morphological filtering (kernel size 5×5) to restore a clear image. .in, The target clear image, t0 = 0.08, to avoid over-enhancement.
[0034] For ease of understanding, the following explains each parameter involved in the above embodiments.
[0035] I(x): The observed hazy image (input image); Physical meaning: Represents the pixel values (RGB three channels) of the image actually captured by the UAV camera in a rainy and foggy environment. Actual scenario: The inspection images of mountain corridors in the rainy season are often blurred due to rain and fog. I(x) is the original input of the edge AI preliminary screening algorithm and needs to be dehazed and restored to a clear image J(x) before potential hazards (such as cracks and falling rocks) can be effectively detected.
[0036] J(x): The clear haze-free image (target restored image); Physical meaning: Represents the pixel values of the ideal clear image of the same scene without rain and fog interference (i.e., the real scene directly "seen" by the camera). Actual scenario: This is the ultimate goal of the dehazing algorithm - to restore J(x) from I(x) by estimating t(x) and A, thereby enhancing the distinguishability of potential hazard features (such as crack edges and rockfall outlines). The actually restored clear haze-free image with a superscript, and without a superscript is the target restoration effect of the clear haze-free image.
[0037] t(x): Transmission rate (Transmission Map); Physical meaning: Describes the proportion of light that penetrates the atmosphere from scene point x to reach the camera without being scattered / absorbed (value range 0 < t(x) ≤ 1). When t(x) = 1, it means that light completely penetrates the atmosphere (no scattering / absorption), which is common in clear and fog-free scenarios; when t(x) ≈ 0, it means that light is almost completely scattered by the atmosphere (such as in a thick fog area), resulting in extremely low image brightness and poor contrast. Actual scenario: In rain and fog, water droplets / particles scatter light, causing t(x) to decrease. For example, in the area where a landslide occurred in section K35, the concentration of fog droplets may increase due to accumulated debris, resulting in a lower local t(x) and a blurrier image.
[0038] A: Atmospheric light value (Atmospheric Light Intensity); Physical meaning: Represents the scattering contribution of suspended particles (such as raindrops and dust) in the atmosphere to light, which is a global constant (scalar). Actual scenario: The atmospheric light A is determined by environmental illumination (such as cloudy / sunny) and atmospheric turbidity. In the rainy season, due to the high water vapor content in the air, A is usually larger than in sunny days, resulting in the overall image being "grayish-white" (the scattered light of the fog is superimposed on the real scene).
[0039] (x): Rain Scattering Noise; Physical meaning: Represents the noise unique to the rainy season caused by raindrops directly scattering into the camera (not the dominant term of atmospheric scattering). Real-world scenario: Raindrops are small (approximately 0.1-1mm) and densely distributed, directly reflecting / scattering ambient light (such as car headlights or skylight) towards the camera, forming random bright spots or blurred areas, causing I(x) and There is a discrepancy.
[0040] In some embodiments, data-driven enhancement uses CycleGAN to generate rain / fog-clear image pairs, inputting real rain / fog images into the generator G to obtain clear images. Discriminator Distinguishing between real and clear images and Adversarial training is used to improve the model's ability to generalize to rain and fog.
[0041] In some embodiments, detail enhancement employs multi-scale detail fusion (MSDF), which separates the low-frequency components (global structure) and high-frequency components (edge details) of the original image, and then uses adaptive gain (…). =1.5) After amplifying the high-frequency components, the components are fused to highlight the details of the crack (0.5mm wide) and the fallen rocks (sharp edges).
[0042] As a specific implementation, the hybrid backbone network unit of the edge intelligent processing module extracts local features of the enhanced corridor image through CNN layers and extracts global features through Transformer layers. Specifically, the shallow layer uses CNN blocks to extract edge and texture features; the middle layer introduces Transformer blocks to compute self-attention within the window, capturing the contextual relationships of the hazard area; the deep layer designs a cross-modal attention fusion module, which concatenates the CNN local features and Transformer global features and then computes complementary weights through multi-head attention to enhance the multi-dimensional feature expression of the hazard area.
[0043] As a specific embodiment, the multi-task deep interaction unit of the edge intelligent processing module includes a cross-attention module for bidirectional feature guidance between object detection and semantic segmentation tasks. The object detection branch outputs a set of candidate boxes, including box coordinates, category, and confidence score; the semantic segmentation branch outputs a pixel-level probability map, including category labels. In the cross-attention interaction, the confidence score of the detection boxes is used as a spatial attention weight to weight the segmentation feature map, highlighting potentially problematic areas; the semantic probability of the segmentation map is used as a contextual feature input to the detection head, enhancing the detection of associated regions.
[0044] In some embodiments, the edge intelligence processing module employs a neural network model based on a hybrid CNN-Transformer architecture for image analysis. This architecture, through a hierarchical design, fuses local features and global context, combining multi-task deep interaction and dynamic loss optimization to improve the accuracy and real-time performance of identifying potential geological hazards. The following detailed description, in conjunction with specific embodiments, further illustrates this.
[0045] This invention provides an implementation of an edge intelligent processing module integrated into a UAV terminal for real-time analysis of acquired corridor images. The module employs a hybrid CNN-Transformer architecture, utilizing multimodal preprocessing, a hybrid backbone network, multi-task deep interaction, dynamic loss optimization, and adaptive post-processing to extract, interact with, and optimize hazard features, forming a complete processing flow from feature extraction to result output.
[0046] In some embodiments, the hybrid backbone network unit adopts a hierarchical hybrid architecture, using CNNs to extract local features in the shallow layers and Transformers to model global relationships in the deep layers. Specifically, the shallow layers extract edge and texture features through CNN blocks, the middle layers introduce Transformer blocks to compute self-attention within the window to capture the contextual relationships of potential problem areas, and the deep layers design a cross-modal attention fusion module to fuse local and global features. For example, the hierarchical structure includes layers C2, C3-C4, and C5, enhancing feature representation capabilities through a progressive layer-by-layer approach.
[0047] As a specific implementation, the shallow C2 layer uses ConvNeXt blocks instead of the traditional ResNet Bottleneck. Through depthwise separable convolutions, dynamic convolutional kernels, and inversely designed global response normalization, it enhances local feature representation while maintaining computational efficiency. Depthwise separable convolutions decompose standard convolutions into depthwise and pointwise convolutions, reducing the number of parameters; dynamic convolutional kernels adaptively adjust kernel weights based on input features, improving feature adaptability; and global response normalization enhances feature contrast through normalization operations.
[0048] As a specific implementation, the middle layers C3-C4 introduce a Swing Transformer Tiny block with a window size of 7×7, employing a shifted window attention mechanism. After dividing the feature map output by the CNN into blocks, Q, K, and V matrices are generated through linear layer projection. Self-attention weights within the window are calculated to capture the contextual relationship between the hazard area and its surrounding environment, such as the contrast between falling rocks and terrain, or the texture difference between cracks and the background. The shifted window attention expands the receptive field through cyclic shifting, avoiding window boundary effects.
[0049] As a specific implementation, the deep C5 layer is designed with a cross-modal attention fusion module, which integrates the local features F extracted by the CNN. CNN With the global features F extracted by Transformer Trans The concatenation process calculates complementary weights between features using a multi-head attention mechanism. The multi-head attention mechanism uses eight heads, and the query Q=F is used to perform the concatenation. CNN Key K=F Trans Sum of values V=F Trans Calculate cross-modal correlation using the following formula: ; The Attention module queries Q=F. CNN Key K=V=F Trans Calculating cross-modal correlations links local features of CNNs with global features of Transformers, enhancing the multi-dimensional feature representation of potential problem areas. fusion F represents the final feature representation after multimodal feature fusion. CNN F represents the local features extracted by the CNN; Trans This represents the global features extracted by the Transformer, and Attention() represents the multi-head attention mechanism. For cross-modal attention mechanisms, Concat() represents the concatenation operation, and MLP represents a multilayer perceptron. The fused features F fusion Further optimization using the concat stitching operation and the multilayer perceptron (MLP) enhances the multidimensional feature representation of potential hazard areas.
[0050] In some embodiments, the multi-task deep interaction unit includes a cross-attention module to enable mutual guidance of features from the object detection task and the semantic segmentation task. Task feature extraction extracts task-related features from the output of the hybrid backbone network through the object detection branch and the semantic segmentation branch, respectively, and the cross-attention interaction enhances the features of the two tasks through bidirectional attention gating.
[0051] As a specific implementation, the object detection branch is based on the DINOv2 detection head and outputs a candidate bounding box set {bi,ci,si}, where bi is the bounding box coordinates, ci is the class label, and si is the confidence score. Detection feature F det Extracted from CNN features designed specifically for detection tasks in a hybrid backbone network, typically from shallow or mid-level local features, the bounding box coordinates and category are directly output through regression operations.
[0052] As a specific implementation, the semantic segmentation branch is based on the HRNet-W32 network, outputting a pixel-level probability map P, where H and W are the image height and width, and C is the number of categories, for example, crack category 1, landslide category 2, and background category 0. Segmentation features Fseg Extracted from high-resolution CNN features designed specifically for segmentation tasks in a hybrid backbone network, preserving more spatial details, and outputting pixel-level probabilities through convolution and upsampling operations.
[0053] As a specific implementation, the cross-attention interaction includes detection-guided segmentation and segmentation-guided detection. Detection-guided segmentation uses the confidence score si of the detection box as the spatial attention weight, weights the segmentation feature map P, and highlights potential problem areas through element-wise multiplication, as shown in the formula: The segmentation result is Expand(si) expands the confidence score si into a mask of the same size as P. Segmentation-guided detection obtains the feature P from the semantic probabilities of the segmentation map through global average pooling. avg The Transformer layer of the input detection head is enhanced with detection features after mapping through a Multilayer Perceptron (MLP), as shown in the formula. Among them, P avg represents the global average pooling feature of the segmentation map P (representing the "context of the overall hazard distribution"); ML represents the multilayer perceptron (mapping the global context to the detection feature space).
[0054] In some embodiments, the dynamic loss optimization unit employs uncertainty-weighted loss and contrastive loss to balance the optimization directions of multiple tasks and enhance feature discriminative power. Uncertainty-weighted loss assigns a learnable variance parameter to each task, dynamically adjusting the loss weights; contrastive loss constrains the feature space distance by sampling positive and negative samples.
[0055] As a specific implementation, the uncertainty-weighted loss function is: ,in The detection loss represents the loss of the target detection branch (e.g., CIoU Loss + Focal Loss), used to optimize the localization and classification of targets such as falling rocks. The segmentation loss (Dice Loss + Cross Entropy Loss) is used to optimize pixel-level segmentation of cracks, landslide areas, etc. The model automatically learns σ through backpropagation to balance the importance of tasks. The variance parameter represents the learnable variance parameter (automatically updated through model training), and represents the uncertainty of object detection and semantic segmentation tasks (i.e., the difficulty or noise level of task learning). This represents the weighted task loss. The smaller the variance (the more certain the task), the larger the weight (i.e., more emphasis is placed on optimizing the task). This represents the regularization term, used to constrain the range of the variance parameter (to prevent excessively large or small variances from causing training instability).
[0056] As a specific implementation example, the contrastive loss function is: ; in, Characteristics of potential hazards For negative sample features, =0.1 represents the temperature parameter. This loss forces the model to learn discriminative features between "hazard vs. background". This loss forces the model to learn discriminative features between hazards and background by sampling positive and negative samples from the same image.
[0057] In some embodiments, the adaptive post-processing module employs Graph Neural Network Simulation (NMS) and a risk grading strategy to improve the accuracy and reliability of the detection results. Graph Neural Network Simulation treats candidate boxes as graph nodes and learns node importance through a graph convolutional network; risk grading combines hazard type, size, and context to calculate a comprehensive risk value.
[0058] As a specific embodiment, this application introduces Graph Neural Network (GNN) NMS and a risk grading strategy to improve the accuracy of the results: In this approach, GNN NMS treats candidate boxes as graph nodes, with edge weights representing the IoU values between boxes. It learns the importance of nodes through two layers of GCN (Graph Convolutional Network), as shown below: ; Here, i and j are indices of nodes in the graph, where i represents the target node to be updated, and j represents the neighboring nodes of node i. All nodes correspond to candidate boxes detected by the UAV. l represents the index of the current graph convolutional layer, starting from 0. This embodiment uses two layers of graph convolution. , Let the edge weight be (IoU). For the first l The learnable weight matrix of the layer. High-importance nodes are ultimately retained (based on confidence level). >0.8 and not suppressed). Indicates the first l Nodes in the layer graph i The hidden state (feature vector). This represents the hidden state of node j, which is adjacent to node i, at level l. Represents a node i The set of neighboring nodes (i.e., the candidate box) i (Other candidate boxes that overlap). Represents a node j For nodes i Attention weights (representing) j right i (Importance). Indicates the first l The learnable weight matrix of the layer. Indicates the activation function (such as ReLU). As a specific implementation, the risk grading strategy calculates the comprehensive risk value R using the following formula: ; The scoring system comprises TypeScore (rockfall = 0.8, crack = 0.6), SizeScore (higher score for larger width / volume), and ContextScore (1.0 for distance ≤ 5m from the line, 0.5 otherwise). A risk value R ≥ 0.7 is marked as "high risk" and triggers a real-time warning. R represents the comprehensive risk value (range [0,1]), used to assess the severity of the hazard. TypeScore represents the type score (severity of the hazard type). SizeScore represents the size score (size of the hazard). ContextScore represents the context score (locational relationship between the hazard and the line).
[0059] In some embodiments, real-time optimization and hardware adaptation achieve a single-frame processing time of no more than 0.8 seconds through model lightweighting, hardware-aware optimization, and parallel computing. Model lightweighting reduces the number of network layers and parameters, hardware-aware optimization leverages specific hardware features to improve computational efficiency, and parallel computing fully utilizes resources through pipelines and multithreading.
[0060] As a specific implementation, the model lightweighting includes adopting the Swing Transformer Tiny backbone network, reducing the number of layers to 6, and setting the hidden dimension to 64, which reduces the computational cost by 40% compared to the standard Swing-T; the detection head uses the lightweight design of YOLOv8n, and the number of convolutional kernels is reduced from 256 to 128.
[0061] As a specific implementation, hardware-aware optimization utilizes the Jetson AGX Orin's Tensor Core to support FP16 inference, quantizing the model to FP16 precision, halving the computational cost and reducing the precision loss to less than 1%; it dynamically sparsifies the feature maps of non-hazardous areas, setting a threshold of 0.2 to skip invalid computations, which is expected to reduce FLOPs by 30%.
[0062] As a specific implementation, parallel computing executes preprocessing (de-raining and enhancement) and backbone network inference in parallel through CUDA streaming, while detection and segmentation branches are computed asynchronously through multi-threading, making full use of the parallel computing capabilities of the GPU.
[0063] In some embodiments, the data backhaul phase is performed by a data backhaul system. The data backhaul system receives time-series imagery data tagged by the UAV terminal, performs edge preprocessing and edge caching. Edge preprocessing includes filtering redundant images and compressing critical data, for example, by reducing data volume through a set compression rate; edge caching provides local storage services in case of cloud reception failure, ensuring data integrity. Data is uploaded to a cloud-based intelligent analysis platform via a high-speed relay link.
[0064] In some embodiments, the intelligent analysis and early warning phase is executed by a cloud-based intelligent analysis platform. This platform performs depth change detection and risk modeling on the uploaded time-series image data to determine the type of debris flow and identify its evolution trend. Depth change detection is achieved through a multi-temporal change detection model, including time-series image data preprocessing, multi-scale temporal feature extraction, dynamic time alignment, and temporal change modeling.
[0065] In some specific embodiments, the cloud-based intelligent analysis platform includes a time-series image data preprocessing module, a multi-scale time-series feature extraction module, a feature extraction module at different scales, a dynamic time alignment module, and a time-series change modeling module.
[0066] The time-series image data preprocessing module is used to perform spatiotemporal consistency denoising, adaptive illumination normalization, and multi-scale registration on the time-series image data; the multi-scale temporal feature extraction module is used to extract features at different scales from the preprocessed time-series image data; the dynamic time alignment module is used to align the image data at different time steps after preprocessing; and the temporal change modeling module is used to identify abnormal changes based on contrastive learning.
[0067] As a specific implementation, the temporal image data preprocessing module performs spatiotemporal consistent denoising, adaptive illumination normalization, and multi-scale registration. Spatiotemporal consistent denoising employs a spatiotemporal convolutional network (ST-CNN) that combines images from adjacent time steps (e.g., the first 3 frames, the last 3 frames) to suppress noise (such as local brightness anomalies caused by raindrop scattering) while preserving details such as crack edges. The formula is: ; in, This represents the pixel value of the denoised visible light image at coordinates (x, y) in frame t (where x, y are spatial coordinates and t is the time step). This represents the pixel value of the original noisy image in frame t at coordinates (x+i, y+j) (where i and j are the offsets within the spatiotemporal window, ranging from [...]). T,T]). The spatiotemporal weight kernel (learned through training) represents the contribution weight of the pixel at position (i,j) within the window to the denoising result at position (x,y) (e.g., the center pixel has a higher weight, and the edge pixels have a lower weight). T represents the size of the spatiotemporal window (e.g., T=2 means the window covers 2 frames before and after, and 2 pixels to the left and right).
[0068] In some embodiments, adaptive illumination normalization employs a contrast-limited adaptive histogram equalization algorithm to enhance local contrast.
[0069] In some specific implementations, CLAHE (Contrast-Limited Adaptive Histogram Equalization) is used to enhance local contrast (such as the brightness difference between a crack and the surrounding mountains) to address changes in lighting (e.g., from cloudy to sunny). This makes the model more effective at capturing subtle changes. Formula: CLAHE avoids over-enhancement by statistically analyzing pixel histograms in blocks and cropping highlight areas. This represents the pixel value at (x,y) of the normalized image. This represents the pixel values of the original image after denoising. CLAHE (Contrast Limited Adaptive Histogram Equalization) is a local histogram equalization algorithm that avoids over-enhancement by statistically analyzing pixel histograms in blocks and cropping highlight areas.
[0070] In some embodiments, multi-scale registration is based on characteristic optical flow and cross-correlation optimization to find the optimal spatial offset to ensure temporal image alignment.
[0071] In some specific embodiments, visible light images at different time steps may experience spatial offsets due to UAV attitude fluctuations. Optical flow registration based on SIFT features is employed, combined with cross-correlation in the temporal dimension to optimize alignment accuracy. The optimal offsets Δx and Δy are found by maximizing cross-correlation, ensuring spatial alignment of the temporal images. The formula is: ; in, This represents the cross-correlation function, which measures the similarity between two image frames. This represents the pixel value at (x,y) of the t-th frame. Indicates the t-th The pixel value at (x,y) after offset (Δx, Δy) in a frame of image (Δx, Δy are the spatial offsets to be optimized). This represents the summation of all spatial coordinates of the image.
[0072] As a feasible implementation, the multi-scale temporal feature extraction module includes three parallel branches: the high-resolution detail feature extraction branch uses HRNet combined with ConvLSTM network to preserve spatial details and capture temporal dependence; the global context feature extraction branch uses Swin Transformer combined with temporal attention mechanism to extract global features and dynamically weight different time steps; and the temporal dynamic feature extraction branch uses temporal convolutional network to capture dynamic differences in a sliding window in the time dimension.
[0073] As a specific implementation, the three parallel branches include the following branch 1, branch 2 and branch 3, and are specifically configured as follows.
[0074] In this embodiment, optical images need to capture both local details (such as crack edges) and global context (such as the relationship between cracks and surrounding terrain), so a multi-scale feature extraction network is designed.
[0075] Branch 1: High-resolution detail feature extraction (HRNet+ConvLSTM). The backbone network adopts HRNet-W32 (high-resolution network). Through parallel multi-resolution subnetworks (1 / 1, 1 / 2, 1 / 4, 1 / 8 resolution) and repeated feature fusion, high-resolution details (such as the edge of a 0.5mm wide crack) are preserved.
[0076] Temporal modeling: ConvLSTM is used to capture spatial dependencies in the time dimension. ConvLSTM captures spatial dependencies in the time dimension through gating mechanisms (input gate, forget gate, output gate) (e.g., cracks from t). The expansion trend from frame 1 to frame t), and the temporal changes that preserve high-resolution details. The formula is: ; in, σ represents the input gate of frame t, which controls the degree to which new information enters the memory unit (σ is the sigmoid activation function). This represents the ForgetGate of frame t, which controls the degree to which old information is retained in the memory unit. Represents the memory unit (CellState) of frame t, which stores historical information of the time series, and ⊙ represents element-wise multiplication. This represents the output gate (OutputGate) of frame t, which controls the contribution of memory cell information to the hidden state. This represents the HRNet feature map of frame t (H×W×C, where H and W are spatial dimensions and C is the number of feature channels). Indicates the t-th Hidden state of 1 frame (H×W×C). W xi To act on the current input The convolution kernel, W hi To apply to the hidden state of the previous time step The convolution kernel, W xf and W hf W represents the weight matrix of the forget gate, which is applied to the current input and the hidden state at the previous time step, respectively. xo and W ho W represents the weight matrix of the output gate, which is applied to the current input and the hidden state at the previous time step, respectively. xc and W hc Here, b represents the weight matrix of the candidate memory unit, which is applied to the current input and the hidden state at the previous time step, respectively. c For the bias term of candidate memory units, b o b is the bias term for the output gate. i For the bias term of the input gate, b f is the bias term of the forget gate, h represents the hidden state at a certain time, and is the final output feature of ConvLSTM. This represents a two-dimensional convolution operation (extracting spatial features).
[0077] Branch 2: Global contextual feature extraction. The backbone network uses window self-attention to divide the image into blocks (7×7 windows) and extract global contextual features (such as the texture difference between cracks and surrounding mountains). Temporal modeling introduces TemporalAttention to dynamically weight features at different time steps. The formula is: ; in, The attention weight of frame t to frame k is represented by (representing...) right The importance of sim ( ) represents the cosine similarity function ( ( ), which measures the similarity of features between two frames. and These represent the hidden states of frame t and frame k, respectively (from the global context features of the SwinTransformer). T represents the temporal length (total number of frames). This represents the temporal attention feature of frame t (which incorporates information from historical keyframes).
[0078] Branch 3: Temporal Dynamic Feature Extraction (TemporalConvNet). The backbone network uses a temporal convolutional network (TemporalConvNet) that slides a window (window size 5) along the time dimension to capture temporal dynamic differences. The formula is as follows: ; in, This represents the temporal convolutional feature of frame t. This represents the learnable temporal convolution kernel (length K represents the window size). Indicates the t-th Hidden states of k frames (k=1,2,…,K). This represents a weighted summation of the features of historical frames within a time window.
[0079] In some embodiments, visible light images at different time steps may become misaligned due to fluctuations in the drone's sampling frequency (e.g., unstable frame rate caused by signal interference) or cloud cover (e.g., missing frames). This requires alignment using a dynamic time alignment module. Specifically, Dynamic Time Warping (DTW) finds the optimal match between two time series by minimizing the cumulative distance of the alignment path, as shown in the formula: ; Where DTW(A,B) represents the DTW distance between sequences A and B (the smaller the distance, the better the alignment). A and B represent two time series to be aligned (such as pixel value sequences or feature sequences of visible light images). π represents the alignment path ( , indicating that the i-th frame of A is aligned with the j-th frame of B). (i,j)∈π represents a time step pair in the path (i is the time step of A, and j is the time step of B).
[0080] In some embodiments, the temporal variation modeling module enhances the model's ability to distinguish between "minor changes" and "normal changes" through contrastive learning, focusing on "the difference between the current frame and historical normal frames." The positive and negative samples and the contrastive loss are configured as follows.
[0081] Positive sample pairs: normal images (such as consecutive frames without cracks) of the same region at adjacent time steps.
[0082] Negative sample pairs: anomalous images of the same region at different time steps (such as the current frame with cracks and the historical frame without cracks).
[0083] Comparative losses: This represents the contrast loss value (the smaller the value, the stronger the model's ability to distinguish between positive and negative samples). This represents the feature vector at the current time step t (the output of multi-scale feature fusion). Indicates the previous time step t Normal characteristics of 1 (positive sample, compared with) (Same region, no change). Represents the anomalous features (negative samples, compared to) of the first k frames. Within the same region, but with variations, k=1,2,…,K). sim( ) represents the cosine similarity function (measures the similarity of feature vectors). τ represents the temperature parameter (controls the sharpness of the distribution; the smaller τ is, the higher the discrimination between positive and negative samples). exp( ) represents an exponential function (amplifying the differences in similarity).
[0084] In some embodiments, the cloud-based intelligent analysis platform further includes a multi-scale attention fusion module, which uses high-resolution detail features as queries and global context and temporal dynamic features as keys and values to achieve dynamic fusion of multi-scale features.
[0085] Cross-Scale Attention, a multi-scale feature fusion technique, complements features at different scales (high-resolution details, global context, and temporal dynamics) to enhance the representation of subtle changes. Specifically, it employs a cross-scale attention mechanism, using high-resolution detail features as the query and global context and temporal dynamic features as the key and value, dynamically associating multi-scale information. The formula is: ; Where Q represents the query feature, which comes from the high-resolution detail branch (HRNet output). K represents the key feature, which comes from the global context branch (the output of the Swing Transformer, projected to the same dimension as Q). V represents the value feature, which comes from the temporal dynamic branch (the output of the temporal convolution, projected to the same dimension as Q). ); The dimension representing the key feature ( ), used to scale the dot product similarity (to avoid gradient vanishing). Softmax represents normalization of the attention weights (ensuring...). ) In the above embodiments, the significance of this operation is that when a high-resolution feature detects a suspected crack (Q), multi-scale attention will extract the comparison information between the region and the surrounding terrain (such as whether the crack is located on a slope) from the global context (K,V), and extract the temporal change trend of the region (such as whether the crack continues to expand) from the temporal dynamics (K,V), thereby enhancing the discriminative power of the crack.
[0086] As one specific embodiment, the drone terminal also includes an adaptive post-processing module. The adaptive post-processing module employs a graph neural network (NMS) and assesses the severity of potential hazards through a risk grading strategy.
[0087] In some embodiments, the risk classification strategy calculates a comprehensive risk value based on the type, size, and location relationship of the hazard to the line, and triggers a real-time warning when the risk value exceeds a preset threshold.
[0088] As a specific implementation, the temporal change modeling module identifies abnormal changes based on contrastive learning and uses a contrastive loss function to enhance the model's discriminative ability. Anomaly detection and segmentation outputs the changed region through adaptive threshold segmentation and morphological optimization. Based on the output of temporal change modeling (change score map), the final potential hazard change region is output through threshold segmentation and morphological optimization.
[0089] Change score map: The score of each pixel represents the degree of difference between the current frame and the historical normal frames (range [0,1], the larger the value, the more likely it is a change area).
[0090] Adaptive threshold segmentation: The threshold is dynamically adjusted based on historical data (e.g., the threshold is lowered during the rainy season to avoid interference from rain and fog; the threshold is raised during the dry season to reduce false detections), and the formula is: ; in, This represents the adaptive threshold for frame t (used to segment changing regions). This represents the mean of the change scores in the first T frames (reflecting the distribution of change scores in a normal scene). The standard deviation of the change score in the first T frames (reflecting the fluctuation of the change score in a normal scene).
[0091] Morphological optimization: Closing is used to fill small holes in the variable region, and opening is used to remove isolated noise points. The formula is as follows: .
[0092] In some embodiments, post-processing optimization improves accuracy through graph neural networks and multi-frame verification. Graph construction treats pixels in changing regions as nodes, edge weights are calculated based on spatial distance and feature similarity, and GNN inference updates node confidence through graph convolutional networks.
[0093] In some embodiments, the early warning issuance and closed-loop feedback phase is executed by the hazard isolation and alarm module. This module issues early warning information through multiple channels, including SMS, mobile app, and audible and visual alarms, and collects feedback from on-site inspections. On-site inspections verify hazards using portable devices, such as handheld crack gauges to measure crack width. Inspection results are uploaded to the inspection management platform, where the model iteration platform adds cases to the training set to optimize the algorithm model; the task scheduling engine adjusts subsequent inspection plans to achieve adaptive adjustment.
[0094] The low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects provided in this application achieves full automation and intelligence of low-altitude remote sensing inspections by constructing a collaborative architecture that includes an inspection management platform, UAV terminals, a data transmission system, a cloud-based intelligent analysis platform, and a hazard isolation and alarm module. This system can automatically generate and execute inspection tasks based on multi-source data, utilize edge intelligence at the terminal side for real-time initial screening and marking of hazards, and accurately identify the evolution trend of geological disasters through deep cloud analysis. Its closed-loop feedback mechanism, from intelligent early warning and proactive isolation to model optimization and strategy adjustment, significantly improves inspection efficiency, early warning accuracy, and system adaptability, effectively overcoming the shortcomings of traditional methods such as low detection accuracy, slow response, and lack of continuous optimization capabilities in complex environments.
[0095] The low-altitude intelligent inspection system for potential external environmental hazards in linear engineering projects, as described in this invention, achieves efficient detection, accurate early warning, and closed-loop management of geological disasters along corridors through end-to-end integration and algorithm optimization. The system forms a logical closed loop from task planning to feedback adjustment, maintains consistent terminology, and ensures system reliability and real-time performance through the collaborative work of all components. By employing edge intelligent processing and multi-temporal depth change detection, it effectively solves the problems of hazard identification and risk warning in complex environments, improving inspection efficiency and accuracy.
[0096] In some embodiments, when a debris flow occurs in the external environment of a linear engineering project, the accompanying dust or naturally occurring fog may prevent the UAV terminal from accurately collecting data on the target location. In this case, adjusting the flight altitude and the intensity of the downwash airflow based on real-time analysis results includes: S21, when the edge intelligent processing module determines in real-time analysis that the location of the debris flow cannot be located due to fog obscuring the view, it controls the UAV terminal to reduce its flight altitude and simultaneously increases the rotor speed to increase the intensity of the downwash airflow. S22, while increasing the downwash airflow intensity, adjust the upwash airflow intensity of the UAV terminal to balance the fuselage pressure difference caused by the decrease in altitude and the enhancement of the downwash airflow, and maintain stable flight attitude; S23, the enhanced downwash airflow is used to disperse the fog below the current position, and after the airflow stabilizes, the image acquisition device acquires an image of the current area. S24, if the fog below the current position is not dispersed, return to S21 and readjust the flight altitude and downdraft intensity of the drone terminal. S25, the edge intelligent processing module analyzes the image, and if the current downwash airflow's area of action is within the location where the debris flow occurs, the current location is recorded and marked. S26, control the UAV terminal to move horizontally to the adjacent unexplored area, and repeat steps S23 to S24 until the boundary of the debris flow location is completely recorded and marked; S27, Generate time-series image data based on the acquired images.
[0097] In some embodiments, when a drone terminal flies to the airspace above a suspected debris flow location according to an inspection command, its onboard multispectral camera, such as a visible light and infrared thermal imaging composite sensor, begins to acquire raw image streams. An integrated edge intelligence processing module, such as a high-performance embedded AI computing unit, performs real-time analysis of the image stream. The analysis algorithm first detects the presence of debris flow areas in the image through the thermal imaging channel and analyzes the texture, density, and motion characteristics of large areas of dust or fog obscuring the image through the visible light channel. If it is determined that the proportion of the target area in the visible light image effectively obscured by dust or fog exceeds a preset threshold, such as 70%, and the imaging signal is blurred due to dust or fog scattering, it is determined that the location of the debris flow cannot be located due to dust or fog obscuring the image. Subsequently, a first control command is generated: controlling the drone's flight control system to reduce its flight altitude while simultaneously increasing the rotational speed of all rotor motors. Reducing the altitude shortens the distance between the drone and the dust or fog, creating conditions for the effective action of the downwash airflow; increasing the rotor speed directly increases the dynamic pressure and velocity of the downwash airflow, improving its ability to penetrate and disperse smoke.
[0098] While the flight control system executes commands to reduce altitude and enhance downwash, a cooperative control loop is triggered. Because the UAV is at low altitude and its propeller speed is increased, the air pressure below the fuselage increases significantly, while the air pressure above the fuselage changes relatively little. This generates an additional lift force that pushes the fuselage upwards and may induce unstable moments in the pitch or roll directions. To address this, the flight control system simultaneously adjusts the auxiliary rotor assembly, dedicated to attitude control, to generate a controllable upwash, or adjusts the pitch angle of some rotors to create a compensating low-pressure zone in a specific area above the fuselage. This operation aims to dynamically balance the overall fuselage pressure difference caused by altitude reduction and enhanced downwash. Its control objective is to maintain the fuselage attitude angle fluctuations fed back by the inertial measurement unit within a safe threshold, thereby ensuring the UAV's flight attitude stability under strong disturbances and providing a stable platform for high-definition image acquisition.
[0099] Once the drone reaches a new, lower altitude and its rotor speed stabilizes, the enhanced downwash acts like a concentrated jet of air, impacting the dust or fog layer directly below. This airflow disperses the dust or fog, temporarily creating a viewing window within it. The flight control system waits for a brief period of airflow stabilization, and after the platform is confirmed to be stable by the airframe vibration sensor and visual optical flow sensor, it triggers the high-definition visible light camera to capture an image of the area affected by the current airflow. The image acquired at this point should have significantly improved clarity and contrast compared to the previous image, as the dust or fog has been partially cleared.
[0100] The edge intelligence processing module rapidly evaluates the newly acquired imagery in S23. Evaluation criteria include: calculating the image sharpness index of the core area of the downwash airflow, such as the Brenner gradient function value, and the dust or fog concentration characteristic value. If the evaluation results show that the sharpness does not meet the preset standard, or the smoke concentration is still higher than the threshold, it is determined that the smoke below the current location has not been effectively dispersed. At this point, the system logic returns to step S21, but this adjustment will be more targeted: for example, further slightly reducing the altitude within safe limits, and / or increasing the rotor speed with a larger gradient. This iterative cycle continues until a clear image meeting the analysis requirements is acquired, or the preset maximum number of attempts or minimum safe altitude is reached, after which the system transitions to an anomaly handling process.
[0101] Once images meeting the analysis requirements are acquired, the edge intelligence processing module runs its lightweight debris flow identification model, such as a cropped convolutional neural network, to analyze the images. The model not only determines whether a debris flow has occurred but also accurately analyzes the spatial relationship between the current downwash area and the identified debris flow location. If the analysis determines that the main body of the debris flow is located within the core area of the downwash, then the current downwash area is considered to be within the debris flow location. The system then records the high-precision GNSS coordinates, altitude, and nose-pointing at this time and overlays a uniquely identified location bounding box onto the image frame. This location is considered a confirmed debris flow point.
[0102] After marking a point within a debris flow, the UAV moves horizontally along the sequence of discovered boundary points, continuously discovering and marking new boundary points until its flight path forms a closed polygon. This closed polygon is defined as the location where the debris flow occurred.
[0103] Throughout the exploration and boundary delineation process, the drone continuously caches image frames locally, containing timestamps, spatial coordinates, and marker information. When the boundary closes or the mission reaches a specific stage, such as completing a full exploration cycle, the system packages this chronologically ordered image data, flight trajectory logs, and all marked debris flow locations / boundary coordinates into a structured "time-series image data" file. This file is then uploaded to a cloud-based intelligent analysis platform via a data transmission system, such as 4G / 5G or satellite links, for subsequent use in depth change detection and risk modeling.
[0104] This invention addresses the technical problem of passive observation failure of UAVs in dusty or foggy environments by actively lowering flight altitude and enhancing downwash intensity. This transforms the UAV from an observer to an intervener, enabling it to actively disperse localized smoke and create temporary clear observation windows, significantly improving its ability to initially detect and approach debris flow locations under poor visibility conditions. Furthermore, by synchronously adjusting upwash to balance fuselage pressure differences, it solves the technical problem of UAV attitude instability and difficulty in achieving high-quality imaging caused by strong downwash operations, improving data acquisition in complex airflow environments. The system improves image clarity and stability; through the technical means of evaluating and iteratively adjusting the purging effect, it solves the technical problem that a single airflow intervention may not be able to effectively disperse dust or fog or adapt to different dust or fog concentrations, forming a closed-loop control loop based on real-time perception feedback, which significantly improves the adaptability and final success rate of dust or fog dispersal operations and avoids exploration interruptions due to single operation failures; through the technical means of generating and uploading time-series image data, it solves the technical problem that a single image or discrete data cannot reflect the dynamic evolution process of debris flow, improving the accuracy and timeliness of debris flow evolution trend prediction.
[0105] The control of the drone terminal to move horizontally to an adjacent unexplored area includes: S261, Set the initial detection direction and control the UAV terminal to move horizontally along the initial detection direction; S262, During the movement, the edge intelligent processing module continuously analyzes the acquired images to determine whether the boundary features between the area where the debris flow occurred and the area where the debris flow did not occur are identified. S263, if the boundary feature is not identified, the drone terminal is controlled to continue moving in the current direction; if the boundary feature is identified, the current position is determined to be the boundary point of the debris flow and recorded, and then the boundary tracking mode is entered. S264, In boundary tracking mode, starting from the current boundary point, control the UAV terminal to move along the tangential direction of the estimated debris flow boundary and continuously acquire images. S265, adjust the flight path based on continuously acquired images, so that the UAV terminal keeps flying along the boundary of the debris flow location, and continuously records and marks the flight trajectory; S266, Repeat steps S264 to S265 until the flight trajectory forms a closed area, which is the location of the identified debris flow.
[0106] In some embodiments, after the drone terminal has initially identified a point inside the debris flow by blowing away dust or fog, its core task shifts to efficiently and accurately delineating the complete two-dimensional spatial extent of the debris flow. To achieve this goal, the system executes the following intelligent boundary level exploration and tracking process: First, an initial detection direction is determined. This direction can be set based on various strategies: for example, starting from a marked point inside the debris flow, choosing a direction pointing towards the nearest map boundary or perpendicular to the corridor's direction; or randomly selecting an azimuth angle; or setting it in the opposite direction of the initial dust dispersion to infer the possible upwind boundary of the debris flow. After determining the direction, the flight control system controls the UAV terminal to maintain the current operating altitude and move horizontally in a straight line along the set direction. The movement speed is set to a low value, such as 1-2 meters per second, to ensure continuous image acquisition and allow sufficient image processing time.
[0107] During horizontal movement, the visible light gimbal mounted on the drone continuously captures images of the area directly below, creating a video stream. The edge intelligence processing module performs real-time online analysis on each frame or every few frames. A lightweight semantic segmentation model determines whether the boundary features between areas where debris flows have occurred and those where they haven't are identified. This lightweight semantic segmentation model performs pixel-level classification of the input image, outputting a binary segmentation map of the debris flow area and the normal background. Simultaneously, by combining multispectral or topographic data, the contours of surface damage or humidity anomalies in the image are calculated. Boundary features are specifically defined as: in a continuous frame sequence, the leading edge of the segmentation map or surface anomaly contour map exhibits a stable, spatially coherent gradient change edge. For example, a transition from a continuous area with debris flow material texture, flow path, or high humidity / water content characteristics to an area characterized by intact vegetation, stable soil, or normal water content, with this transition zone maintaining continuity in position and shape over several frames.
[0108] If the boundary feature is not identified, it means that the drone is either completely within the debris flow area or completely within an area where no debris flow has occurred. The flight controller will control the drone to continue moving in the current direction until the preset maximum unidirectional exploration distance is reached. If the boundary feature is identified, the algorithm will accurately locate the pixel column with the most dramatic gradient change in the current frame, i.e., the boundary line, and map its center point to the geographic coordinate system. The system immediately determines that this geographic location is a debris flow boundary point, records its precise GNSS coordinates, timestamp, and boundary normal direction, and then switches from radial exploration mode to boundary tracking mode. The core objective changes from finding the boundary to walking along the boundary.
[0109] In boundary tracking mode, the system uses the most recently calibrated boundary point as the path starting point. To predict the extension direction of the boundary, i.e., the local tangent direction, the algorithm uses a sliding window to review several recently recorded boundary points, such as the first 3-5 points. Using the coordinates of these points, a short line segment is fitted using the least squares method or a moving average direction vector is calculated, which serves as the estimated tangent direction of the local boundary orientation. The flight controller then controls the UAV terminal to move horizontally a small fixed distance, such as 1-1.5 meters, along this estimated tangent direction. During the movement, the gimbal maintains ground observation and continuously acquires images. After each small step the drone takes and acquires new imagery, the edge processing module immediately analyzes the new imagery to quickly locate the actual observed boundary line position within the current field of view. The system uses the deviation between the estimated tangent direction and the actual observed boundary line position in the image as the control input. Specifically, if the observed boundary line is to the left of the image's center line, a command to adjust the heading to the right is generated, and vice versa. Through this PID control based on real-time visual feedback or more advanced tracking algorithms, the flight controller dynamically and continuously adjusts the drone's horizontal flight path, ensuring that the drone consistently strives to lock the boundary line near the center of its field of view. In this way, the drone can adaptively maintain flight along the true boundary of the winding debris flow location. During this process, each observation point used for control feedback or each boundary point confirmed after fine-tuning is continuously recorded and marked, forming a high-precision flight trajectory / boundary line composed of a dense point cloud.
[0110] During flight, the algorithm continuously checks the latest recorded sequence of boundary points. It periodically checks whether the current point spatially closes with historical points, especially the initial boundary point. The closure condition is typically set as follows: the drone's flight path ensures the straight-line distance between the latest boundary point and the initial boundary point is less than a closure threshold, such as 3 meters, and the trajectory has completed a circle of greater than a certain angle, such as 270°. Once the closure condition is met, the algorithm considers the complete exploration of the debris flow boundary finished. Connecting all recorded boundary points sequentially forms a polygonal closed region, which is formally defined internally as the location of the debris flow identified in this mission. Its geographical extent, perimeter, area, and other parameters can be calculated in real-time and uploaded along with the concurrent imagery data.
[0111] This invention addresses the inefficiency of blindly or randomly searching for boundaries from a single debris flow occurrence point by employing a technique that moves along an initial direction and identifies boundary features in real time. It provides a rapid and directional initial boundary screening method, significantly shortening the time to locate the first boundary point and improving the initial efficiency of boundary exploration. Furthermore, by moving along a predicted tangent direction and combining visual servoing for closed-loop path adjustment, it solves the problem of irregular debris flow boundaries and their susceptibility to terrain, making automatic and accurate tracking difficult. This enables UAVs to adaptively and smoothly track irregular debris flow boundaries, greatly improving the trajectory accuracy, continuity, and automation level of boundary delineation while reducing reliance on preset paths or high-precision maps. Finally, by performing closed-loop detection and forming polygonal closed regions, it solves the problem that traditional point- or line-based disaster reports cannot intuitively reflect the scope and area of debris flow impact. It can automatically generate complete, closed geographic polygons representing the debris flow coverage area, greatly improving the intuitiveness, accuracy, and decision support value of debris flow situation assessment.
[0112] The hazard isolation and alarm module is configured as follows: S51, Receive the analysis results of debris flow type and evolution trend of debris flow output by the cloud intelligent analysis platform; S52, determine whether the debris flow is a debris flow flowing through the engineering site; S53, If it is a debris flow passing through the engineering site, then based on the location of the debris flow, the preset location of the engineering equipment, and the evolution trend of the debris flow, calculate the evolution direction of the debris flow and the direction vector from the location of the debris flow to the engineering equipment. S54. If the calculated direction of the debris flow evolution is different from the direction vector from the location of the debris flow to the engineering equipment, then an alarm message containing the location, type and evolution trend of the debris flow is generated and released through multiple channels.
[0113] If the calculated direction of debris flow evolution is the same as the direction vector from the location of debris flow occurrence towards the engineering equipment, then an isolation operation is performed, which includes: S541, control the drone terminal to fly directly above the preset position coordinates of the engineering instrument; S542, control the UAV terminal to lower the flight altitude a second time, and adjust the vector direction of the downwash airflow so that it forms an outward-sloping airflow below the UAV; S543, the intensity of the upwash airflow of the UAV terminal is adjusted for the second time to balance the pressure difference of the fuselage caused by the decrease in altitude and the adjustment of the direction of the downwash airflow, so as to maintain the stability of the flight attitude; S544, using the outward-sloping downward airflow, the gravel located around the engineering equipment is moved away from the engineering equipment. S545, control the UAV terminal to rotate horizontally around the vertical line between the geometric center of the UAV terminal and the ground as an axis; S546, through the coordinated operation of steps S542 to S545, a ring-shaped cleaning area is formed around the engineering equipment, so that the accumulated gravel in the area forms an isolation ring to protect the engineering equipment.
[0114] The types of debris flows include debris flows that flow through the engineering site and debris flows that do not flow through the engineering site. If the debris flow is a debris flow that does not flow through the engineering site, an alarm message containing the location, type and evolution trend of the debris flow will be generated and released through multiple channels.
[0115] In some embodiments, after the cloud-based intelligent analysis platform completes in-depth analysis, its structured assessment results, including debris flow type, precise location, and evolution trend, are pushed in real time to the hazard isolation and alarm module deployed on-site or in the regional command center. This module, acting as the intelligent hub for decision-making and execution, transforms the assessment information into specific alarms or physical intervention actions. Its workflow is as follows: The module determines whether the debris flow type is a debris flow flowing through the engineering site. If it is determined to be a debris flow flowing through the engineering site, the module immediately initiates a threat assessment calculation. Its core is based on spatial geometry and trend prediction to quantify the threat level of the debris flow to critical assets, extracting its main evolution direction angle (e.g., due east, 0°). This direction angle is converted into a unit vector. Taking the boundary point or centroid closest to the engineering equipment on the current debris flow boundary as the starting point, and the precise preset position coordinates of the engineering equipment retrieved from the preset digital twin model as the ending point, the vector connecting the two is calculated and normalized to obtain a unit vector. The dot product of the two unit vectors is calculated. A dot product value close to +1 indicates that the evolution direction is highly consistent with the direction pointing towards the equipment, posing a significant threat; a negative dot product value indicates that the dominant evolution direction deviates from the equipment.
[0116] If the calculation determines that the debris flow's evolution direction is far from engineering equipment—for example, if the dot product is negative, or if it is positive but very small and the evolution rate is slow—then the current scenario is determined to be a non-direct threat. The system then generates a structured alarm message, which includes at least: the geographical extent of the debris flow, the confirmed type of engineering site it passes through, the predicted evolution direction, and the estimated impact time. This alarm is simultaneously sent through an integrated multi-channel distribution interface, including but not limited to: pushing to the large-screen command system of the inspection management platform, sending to handheld terminals of on-site workers via a dedicated wireless broadcast band, automatically triggering audible and visual alarm devices installed along the corridor, and sending SMS messages to relevant responsible persons via mobile networks.
[0117] If the threat assessment determines that the debris flow is evolving toward engineering equipment and the dot product exceeds a positive threshold, the module immediately skips the regular alarm and prioritizes active physical isolation operations, instructing the nearest drone terminal to perform the following tasks: The coordinates of the engineering equipment are sent to the UAV flight controller. The UAV terminates or pauses other tasks and flies to the preset position coordinates of the engineering equipment at the fastest and safest speed, and performs initial stabilization at a relatively high hovering altitude, such as 20 meters. To maximize ground effect and airflow influence, the flight control system lowers the UAV's altitude a second time to an extremely low operating altitude, for example, 3-5 meters above the equipment or the ground. Simultaneously, the vector direction of the downwash airflow is adjusted: by independently controlling the rotational speed and pitch of different rotor assemblies, the combined downwash airflow is no longer vertically downward, but rather forms an airflow at a certain angle to the vertical, such as 45°, angled outwards. The purpose of this airflow is to push the debris outwards rather than compress it. Strong oblique airflow at extremely low altitudes generates complex unbalanced torques and ground effect interference on UAVs. The flight control system makes secondary adjustments to the upwash airflow intensity and the power distribution of each motor, performing millisecond-level dynamic compensation. This is typically achieved through a feedforward-feedback composite control algorithm. This algorithm aims to maintain the fuselage level and resist the reaction force of crosswinds, precisely balancing the complex fuselage pressure difference and torque caused by the rapid decrease in altitude and changes in airflow vectors, ensuring the absolute stability of the UAV in maintaining its ultra-low-altitude flight attitude under strong disturbances. After stabilizing and hovering, a strong, outward-sloping downwash acts on the periphery of the engineering equipment. The airflow directly acts on objects such as gravel and loose soil on the ground, generating sufficient horizontal force to cause them to slide or roll in the centrifugal direction, thus moving them away from the engineering equipment and initially clearing the area around the equipment. To achieve 360-degree protection, the flight control terminal of the drone rotates slowly and uniformly horizontally around its geometric center perpendicular to the ground, for example, at 5-10 revolutions per minute. This rotational motion causes the oblique outward airflow to act continuously and evenly in all directions around the device, like a rotating jet compass. By combining low-altitude, outward-sloping airflow with continuous horizontal rotation, a persistent radial airflow field is generated around the engineering equipment. This airflow field continuously blows away newly moved debris or loose material from the area. Ultimately, a ring-shaped isolation zone composed of debris accumulation is formed on the ground around the engineering equipment—a dynamically maintained physical isolation ring. This isolation ring effectively buffers and blocks the direct impact of the debris flow or scattered debris, buying crucial time for subsequent manual disposal.
[0118] This invention employs a technique for quantitative threat assessment by calculating the dot product of the debris flow evolution direction vector and the pointing instrument vector. This addresses the subjectivity and lag issues inherent in relying on manual visual inspection or simple distance-based threat judgments, providing an objective and real-time mathematical threat quantification method that significantly improves the accuracy and foresight in predicting risks faced by engineering equipment. Furthermore, by employing a technique of secondary altitude reduction and adjustment of the downwash airflow vector to create a strong outward-sloping airflow field, it overcomes the problem that traditional UAVs' vertical downward airflow can only suppress rather than clear debris, failing to effectively construct a protective barrier. This invention creates a method capable of directionally blowing debris away. The proactive cleanup mechanism significantly improves the feasibility and effectiveness of using UAV airflow for ground-based physical isolation. By employing a secondary adjustment of the upwash airflow to achieve dynamic equilibrium under extreme low-altitude and complex flow fields, it solves the technical problem of UAVs being prone to attitude instability and even crashes when performing operations in strong, asymmetric airflow conditions. This ensures that defensive actions can be executed at high intensity under safe and stable conditions, greatly improving the flight safety and mission reliability of UAVs in extreme defense operation modes. Furthermore, by using a diagonally outward downwash airflow to directionally blow away debris around engineering equipment, it addresses the issue of loose debris or flowing materials at the forefront of mudslides. The direct threat of impacts on critical equipment can be mitigated by clearing a buffer zone before the main body arrives, effectively reducing the direct risk of damage to critical engineering equipment. By controlling drones to rotate horizontally at a constant speed around a vertical axis, the technical problem of blind spots and inability to achieve full-circumference protection in static or unidirectional airflow defenses is solved, enabling directional airflow protection to uniformly cover a 360-degree area around the equipment, comprehensively improving the protective capability of the physical isolation zone. Furthermore, the configuration of a special alarm directly triggered for debris flows not directly threatening the engineering site solves the problem of ineffective proactive attempts when debris flows do not directly threaten core assets. The key issues of defense and delayed targeted early warning have been addressed, achieving optimal matching between disaster assessment and response strategies. This has maximized the decision-making efficiency of emergency command and reduced the risk of resource waste caused by inappropriate responses. Together, they constitute a closed-loop emergency response system from intelligent risk assessment to precise proactive defense. This upgrades the traditional debris flow monitoring and alarm system into an intelligent protection system with proactive intervention capabilities. Overall, it has significantly improved the survival probability of critical facilities in linear engineering projects under the threat of debris flows, reduced the risk of major asset losses, and bought valuable strategic time for professional manual disposal, creating safer operating conditions.
[0119] Although the operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.
[0120] The methods and systems of this application can be implemented using standard programming techniques, utilizing rule-based logic or other logic to implement various method steps. It should also be noted that the terms "system" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.
[0121] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.
[0122] The foregoing description of implementations of this application has been provided for illustrative and descriptive purposes. The foregoing description is not exhaustive and is not intended to limit this application to the exact forms disclosed. Various modifications and variations may exist in accordance with the foregoing teachings, or may arise from practice of this application. These embodiments were chosen and described to illustrate the principles of this application and its practical application, enabling those skilled in the art to utilize this application in various implementations and modifications to suit the specific purpose of the concept.
[0123] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.
[0124] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.
[0125] It is further understood that although the operations are described in a specific order in the accompanying drawings in the embodiments of this application, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.
[0126] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the field of this application that are not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0127] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
[0128] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A linear engineering external environment hidden danger low-altitude intelligent inspection system, characterized in that, include: The inspection management platform is used to generate and issue inspection instructions based on the input inspection requirements, combined with geographic information, environmental data and historical potential hazards. The drone terminal is used to execute the inspection command, including acquiring images and using an integrated edge intelligence processing module to perform real-time analysis on the acquired images. Based on the results of the real-time analysis, the flight altitude and the intensity of the downwash airflow are adjusted to identify and mark the location of the debris flow and obtain time-series image data. The data feedback system is used to receive and upload time-series image data marked by the UAV terminal; The cloud-based intelligent analysis platform is used to perform in-depth change detection and risk modeling on uploaded time-series image data in order to determine the type of debris flow and identify its evolution trend. The hazard isolation and alarm module is used to adjust the flight altitude and the direction of the downwash airflow based on the type and evolution trend of the debris flow. By combining low-altitude, oblique outward airflow with continuous horizontal rotation, a continuous outward radial airflow field is generated around the engineering equipment, which blows away newly moved debris or loose material in this area. Finally, a ring-shaped isolation zone composed of debris accumulation is formed on the ground around the engineering equipment to keep the debris away from the engineering equipment and issue alarms through multiple channels to protect the engineering equipment.
2. The system of claim 1, wherein, The adjustment of flight altitude and downwash intensity based on real-time analysis results includes: S21, when the edge intelligent processing module determines in real-time analysis that the location of the debris flow cannot be located due to fog obscuring the view, it controls the UAV terminal to reduce its flight altitude and simultaneously increases the rotor speed to increase the intensity of the downwash airflow. S22, while increasing the downwash airflow intensity, adjust the upwash airflow intensity of the UAV terminal to balance the fuselage pressure difference caused by the decrease in altitude and the enhancement of the downwash airflow, and maintain stable flight attitude; S23, the enhanced downwash airflow is used to disperse the fog below the current position, and after the airflow stabilizes, the image acquisition device acquires an image of the current area. S24, if the fog below the current position is not dispersed, return to S21 and readjust the flight altitude and downdraft intensity of the drone terminal. S25, the image is analyzed by the edge intelligent processing module. If the area affected by the current downwash airflow is within the location where the debris flow occurs, the current location is recorded and marked. S26, control the UAV terminal to move horizontally to the adjacent unexplored area, and repeat steps S23 to S24 until the boundary of the debris flow location is completely recorded and marked; S27, Generate time-series image data based on the acquired images.
3. The system according to claim 2, characterized in that, The control of the drone terminal to move horizontally to an adjacent unexplored area includes: S261, Set the initial detection direction and control the UAV terminal to move horizontally along the initial detection direction; S262, During the movement, the edge intelligent processing module continuously analyzes the acquired images to determine whether the boundary features between the area where the debris flow occurred and the area where the debris flow did not occur are identified. S263, if the boundary feature is not identified, the drone terminal is controlled to continue moving in the current direction; if the boundary feature is identified, the current position is determined to be the boundary point of the debris flow and recorded, and then the boundary tracking mode is entered. S264, In boundary tracking mode, starting from the current boundary point, control the UAV terminal to move along the tangential direction of the estimated debris flow boundary and continuously acquire images. S265, adjust the flight path based on continuously acquired images, so that the UAV terminal keeps flying along the boundary of the debris flow location, and continuously records and marks the flight trajectory; S266, Repeat steps S264 to S265 until the flight trajectory forms a closed area, which is the location of the identified debris flow.
4. The system according to claim 1, characterized in that, The edge intelligent processing module uses a neural network model based on a hybrid CNN-Transformer architecture for image analysis; The hybrid CNN-Transformer architecture includes: A multimodal preprocessing unit is used for image de-raining and fog enhancement processing; Hybrid backbone network units are used to extract local features of the enhanced image through CNN layers and global features through Transformer layers; The multi-task deep interaction unit is used to perform feature interaction between object detection and semantic segmentation tasks based on local and global features. The dynamic loss optimization unit is used to balance the multi-task losses in the image analysis process using an uncertainty weighting method.
5. The system according to claim 4, characterized in that, The multimodal preprocessing unit adopts a rain and fog removal framework that combines physical driving and data driving. The physical drive is based on an improved atmospheric scattering model, which estimates transmittance and atmospheric light value by jointly using dark channel priors and color attenuation priors.
6. The system according to claim 4, characterized in that, The multi-task deep interaction unit includes a cross-attention module, which is used to implement bidirectional feature guidance for detection and segmentation tasks. In this process, the confidence of the detection box is used as a spatial attention weight to weight the segmentation feature map, and the semantic probability of the segmentation map is used as a contextual feature input to the detection head.
7. The system according to claim 4, characterized in that, The total loss function of the dynamic loss optimization unit consists of uncertainty-weighted loss and contrast loss.
8. The system according to claim 1, characterized in that, The hazard isolation and alarm module is configured as follows: S51, Receive the analysis results of debris flow type and evolution trend of debris flow output by the cloud intelligent analysis platform; S52, determine whether the debris flow is a debris flow flowing through the engineering site; S53, If it is a debris flow passing through the engineering site, then based on the location of the debris flow, the preset location of the engineering equipment, and the evolution trend of the debris flow, calculate the evolution direction of the debris flow and the direction vector from the location of the debris flow to the engineering equipment. S54. If the calculated direction of the debris flow evolution is different from the direction vector from the location of the debris flow to the engineering equipment, then an alarm message containing the location, type and evolution trend of the debris flow is generated and released through multiple channels.
9. The system according to claim 8, characterized in that, If the calculated direction of debris flow evolution is the same as the direction vector from the location of debris flow occurrence towards the engineering equipment, then an isolation operation is performed, which includes: S541, control the drone terminal to fly directly above the preset position coordinates of the engineering instrument; S542, control the UAV terminal to lower the flight altitude a second time, and adjust the vector direction of the downwash airflow so that it forms an outward-sloping airflow below the UAV; S543, the intensity of the upwash airflow of the UAV terminal is adjusted for the second time to balance the pressure difference of the fuselage caused by the decrease in altitude and the adjustment of the direction of the downwash airflow, so as to maintain the stability of the flight attitude; S544, using the outward-sloping downward airflow, the gravel located around the engineering equipment is moved away from the engineering equipment. S545, control the UAV terminal to rotate horizontally around the vertical line between the geometric center of the UAV terminal and the ground as an axis; S546, through the coordinated operation of steps S542 to S545, a ring-shaped cleaning area is formed around the engineering equipment, so that the accumulated gravel in the area forms an isolation ring to protect the engineering equipment.
10. The system according to claim 1, characterized in that, The debris flow types include debris flows that flow through the engineering site and debris flows that do not flow through the engineering site. If the debris flow type is a debris flow that does not flow through the engineering site, an alarm message containing the location, type and evolution trend of the debris flow is generated and released through multiple channels.
Citation Information
Patent Citations
Tilt rotor-based linear multi-rotor unmanned aerial vehicle (UAV) structure for crop protection and control method thereof
US12084210B1
Unmanned aircraft
WO2024171294A1