A method for autonomous inspection of a UAV flying on the ground

Abstract

The application discloses a kind of unmanned aerial vehicle autonomous inspection ground simulation flight method, it is related to unmanned aerial vehicle autonomous control technical field, the method includes: real-time acquisition topographic data by multi-source sensor;Based on lightweight hybrid architecture model, topographic data is carried out pixel-level semantic segmentation, and output topographic material classification label;Fusion semantic segmentation result and laser point cloud spatial topology data, extract topographic structure characteristics;According to the combination of topographic material classification label and topographic structure characteristics, match preset flight mode, generate dynamic control instruction;Smooth switching is realized across flight mode by double-ended queue constraint;Based on sensor feedback data, construct closed loop control, correct decision deviation, the method realizes accurate perception and adaptive flight under complex terrain, improve the safety, stability and intelligent level of unmanned aerial vehicle autonomous inspection, applicable to complex terrain scene such as power inspection, mountain environment monitoring.

Description

Technical Field

[0001] This invention relates to the field of autonomous control technology for unmanned aerial vehicles (UAVs), specifically a method for autonomous inspection and terrain-following flight of a UAV. Background Technology

[0002] Currently, the method of autonomous drone inspection using terrain-following flight is mainly applied to complex terrain scenarios such as power line inspection, photovoltaic power station inspection, and mountain environment monitoring. In these application scenarios, terrain-following flight technology is one of the key technologies to ensure the quality and safety of drone inspection operations. Its core principle is to dynamically adjust the drone's flight altitude to adapt to changes in terrain undulations, ensuring that the drone maintains a relatively constant altitude difference with the ground or the target inspection object. This ensures both the accuracy and effectiveness of inspection data (such as laser point cloud and image data) collection and avoids the risk of collisions due to sudden terrain changes. With the large-scale application of drone inspection technology in fields such as energy and geological exploration, complex terrain scenarios (such as mountainous vegetation areas, areas around water bodies, and areas with dense power transmission lines) place higher demands on the terrain adaptability, scenario compatibility, and intelligent decision-making of terrain-following flight technology.

[0003] In existing technologies, UAV terrain-following flight schemes mainly rely on terrain elevation data to achieve altitude control. The mainstream technical paths include two categories: one is based on real-time collected elevation data, such as directly acquiring terrain elevation information through lidar, or adjusting the flight altitude by constructing a digital elevation model (DEM) in real time. For example, Chinese invention patent CN116627164B, "A method and system for UAV terrain-following flight control based on terrain height," calculates the terrain elevation difference by constructing a DEM model in real time, and then dynamically adjusts the UAV's flight altitude. The other category is based on a pre-constructed elevation model, such as pre-generating variable-altitude waypoints based on a digital surface model (DSM). For example, Chinese invention patent CN111966129B, "A photovoltaic inspection UAV and its terrain-following flight method," pre-plans the flight trajectory and altitude of the photovoltaic inspection area through a DSM model to achieve globally unified terrain-following flight control. Furthermore, in the scenario of power transmission line inspection, existing technologies mostly plan the simulation path based on the geometric elevation of the line; in the scenario of photovoltaic inspection, they mostly rely on DSM to pre-generate waypoints with fixed altitude intervals. The above solutions can achieve basic terrain simulation functions in scenarios with relatively simple terrain conditions and simple terrain features.

[0004] However, existing terrain-following flight technology has a core flaw: it relies solely on terrain geometric elevation data to adjust flight altitude, without incorporating terrain material semantic information (such as different material categories like water surfaces, sandy land, tree canopies, bare rocks, power lines, and buildings) and terrain structural features (such as slope, vegetation NDVI value, bare rock spectral reflectance, and power line sag gradient) into the flight decision-making logic. As a result, it cannot dynamically switch flight modes based on real-time perceived terrain attributes. This core defect prevents drones from adapting to optimal flight strategies based on terrain characteristics in complex inspection scenarios, leading to critical issues such as inaccurate obstacle avoidance judgments. For example, in water scenarios, relying solely on geometric elevation data makes it difficult to distinguish between the mirror reflection of the water surface and the actual ground, resulting in distortion of the lidar point cloud and the drone potentially misjudging its altitude and flying too close to the water surface. In canopy scenarios, the inability to identify canopy thickness and NDVI characteristics can easily lead to the drone mistaking the top of the canopy for the ground and setting the flight altitude too low, causing collisions with the canopy. In areas with dense power lines, the inability to perceive the sag gradient of the power lines and relying solely on preset safety distances for obstacle avoidance may result in collision risks due to missing the lowest point of the sag. These inaccurate obstacle avoidance judgments directly threaten the safety of drone inspection operations and are the core bottleneck restricting the reliable application of existing terrain-following flight technology in complex terrain scenarios.

[0005] In summary, while existing UAV terrain-following flight technology can meet the basic altitude tracking requirements in simple terrain scenarios, it has significant shortcomings in terrain semantic understanding and dynamic decision-making, particularly lacking the integration and utilization of terrain material semantics and structural features. This leads to insufficient reliability in obstacle avoidance in complex inspection scenarios. Therefore, there is an urgent need in this field for an adaptive terrain-following flight technology that can integrate terrain material semantic recognition and structural feature analysis. This technology would address the problem of inaccurate obstacle avoidance judgments caused by existing technologies relying solely on geometric elevation data, thereby improving the safety, stability, and intelligence of UAV autonomous inspection in complex terrain scenarios and meeting the practical needs of high-precision, high-reliability inspection operations in fields such as power, photovoltaics, and mountain monitoring. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for autonomous terrain-following flight of unmanned aerial vehicles (UAVs). This method involves equipping the UAV with multi-source sensors to collect terrain data in real time, using a lightweight PSPNet-MobileNetV2 hybrid architecture model to perform pixel-level semantic segmentation of the data to output terrain material classification labels, and then fusing the semantic segmentation results with laser point cloud spatial topology data to extract terrain structure features. Based on the combination of these two data, a preset flight mode is matched and dynamic control commands are generated. Simultaneously, smooth cross-modal switching is achieved through dual-end queue constraints, and closed-loop control is formed by combining sensor feedback. This allows the UAV to adaptively adjust its flight strategy for different complex terrains, accurately adapt to terrain attributes, effectively avoid inaccurate obstacle avoidance judgments, and improve the safety, stability, and intelligence level of autonomous terrain-following flight of UAVs in complex scenarios.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV), comprising the following steps:

[0008] S100: Real-time terrain data is collected by the onboard multi-source sensors, including a lightweight lidar, a multispectral camera, and an ultrasonic radar. The ranging accuracy of the lightweight lidar is ±3cm, the multispectral camera covers 5 spectral bands, and the obstacle avoidance accuracy of the ultrasonic radar is ±5cm.

[0009] S200, based on a lightweight PSPNet-MobileNetV2 hybrid architecture model, performs pixel-level semantic segmentation on real-time collected terrain data and outputs terrain material classification labels, which include water surface, sand, tree canopy, bare rock, power transmission line and building;

[0010] S300. By fusing semantic segmentation results with laser point cloud spatial topology data, terrain structure features are extracted. These features include cliffs with slopes greater than 60°, valleys with continuous negative terrain, vegetation canopies with NDVI greater than 0.7, and bare rocks with spectral reflectance greater than 0.4.

[0011] S400: Based on the combination of terrain material classification labels and terrain structure features, a preset flight mode is matched to generate dynamic control commands; the flight mode includes:

[0012] When a water surface is detected and the reflectivity is higher than the threshold, the low-disturbance mode is activated, the drone's altitude is reduced to 3m and the downward-looking laser is turned off;

[0013] When the tree canopy is identified and the NDVI is greater than 0.7, the penetration scanning mode is activated, the height is raised to 2m above the canopy, and multispectral penetration imaging is initiated.

[0014] When bare rock is identified and the slope is greater than 30°, the high-precision reconstruction mode is activated, the gimbal tilt is adjusted to -45° and the laser scanning frequency is increased to 100Hz.

[0015] When a dense area of ​​power transmission lines is identified and there is a sag gradient, the line-following obstacle avoidance mode is activated, and the plane flies along the electromagnetic field strength gradient with a lateral obstacle avoidance distance of 5m.

[0016] The S500 achieves smooth switching across flight modes by constraining the height deviation between adjacent frames to no more than 20% through a double-ended queue.

[0017] The S600 evaluates the effectiveness of control command execution in real time during flight and corrects decision deviations based on sensor feedback data, forming a closed-loop control.

[0018] Furthermore, in step S200, the semantic segmentation model removes 20% of redundant layers through channel pruning and compresses the model size to 0.8MB using INT8 quantization.

[0019] The model introduces the FocalLoss function to address dynamic disturbances, and its expression is: ;

[0020] in, This represents the predicted probability value that the current pixel belongs to a specific terrain material category. This represents the weight adjustment coefficient for the current terrain material category. This indicates a regulating factor that focuses on difficult-to-distinguish samples.

[0021] Furthermore, the extraction of terrain structure features in step S300 includes:

[0022] The point cloud sampling density is forcibly increased by 50% in valley and cliff areas, and a three-dimensional obstacle avoidance path is generated.

[0023] Based on the semantic distribution of the transmission line, the lowest point of the sag is predicted, and a counterclockwise loop path with a radius of 10m is generated to avoid electromagnetic blind spots.

[0024] Furthermore, the dynamic switching logic of the flight mode in step S400 includes:

[0025] When the tree canopy accounts for more than 40%, it automatically switches from ground-level mode to penetration scanning mode, and simultaneously increases the vegetation filtering intensity of the lidar by 2 times;

[0026] When a sudden increase in wind speed exceeding 8m / s is detected, the simulated line inspection distance is shortened to 10km and the standby pressure constant altitude mode is switched.

[0027] Furthermore, the smooth switching algorithm across flight modes in step S500 constrains altitude changes using the following formula: ;

[0028] in, Indicates the rate of change of height between adjacent frames. This indicates the current altitude measurement of the drone. This indicates the measured altitude of the drone at the previous moment.

[0029] Furthermore, asynchronous fusion methods for multi-source sensors include:

[0030] The LiDAR data sampling frequency is 10Hz, and the visual data sampling frequency is 30Hz.

[0031] Timestamp alignment reduces computational overhead by 35%.

[0032] Furthermore, the closed-loop control feedback mechanism in step S600 includes:

[0033] The actual distance to obstacles is monitored in real time by ultrasonic radar. When the deviation from the preset obstacle avoidance threshold exceeds ±10%, the flight mode is rematched.

[0034] In areas with dense power transmission lines, the flight heading angle is dynamically calibrated based on the electromagnetic field strength gradient direction, and the horizontal error is controlled within ±0.5m.

[0035] Furthermore, the joint output of terrain material classification and structural features includes:

[0036] Simultaneously generate 6 types of material labels and 4 types of structural feature labels;

[0037] The model inference time is less than 50ms and it is deployed on the NVIDIA Jetson TX2 embedded platform.

[0038] Furthermore, optimized deployment in resource-constrained scenarios includes:

[0039] LiDAR and visual data are asynchronously fused through timestamp alignment;

[0040] Under the memory constraints of embedded devices, model pruning and quantization are used to reduce computational resource consumption.

[0041] Furthermore, the flight modal decision matrix is ​​constructed based on the following logic:

[0042] When the terrain semantic type is identified as water surface, and its structural features are flat and its reflectivity is higher than a preset threshold, the low-disturbance flight mode is enabled.

[0043] When the terrain semantic type is identified as tree canopy, and its structural features meet the requirements of NDVI value greater than 0.7 and canopy thickness exceeding the set standard, the penetration scan flight mode is enabled.

[0044] When the terrain semantic type is identified as bare rock and its structural feature is a slope greater than 30°, high-precision reconstruction of flight modes is enabled.

[0045] When the terrain semantic type is identified as a dense area of ​​power transmission lines and its structural features have sag gradients, the line-following obstacle-circling flight mode is activated.

[0046] Compared with existing technologies, this method for autonomous inspection and terrain-following flight of unmanned aerial vehicles (UAVs) has the following advantages:

[0047] I. This invention utilizes multi-source sensors mounted on a drone to collect terrain data in real time. Based on a lightweight PSPNet-MobileNetV2 hybrid architecture model, the collected terrain data undergoes pixel-level semantic segmentation, outputting terrain material classification labels including categories such as water surfaces, sandy areas, and tree canopies. The semantic segmentation results are then fused with laser point cloud spatial topology data to extract terrain structural features. Finally, based on the combination of terrain material classification labels and terrain structural features, a preset flight mode is matched and dynamic control commands are generated. This enables precise perception and adaptive flight control of complex terrain, allowing the drone to adapt to optimal flight strategies for different terrain attributes during inspections. It effectively solves the problem of inaccurate obstacle avoidance judgments caused by existing technologies that rely solely on terrain geometric elevation data for flight altitude adjustment without incorporating terrain material semantic information and terrain structural features into the flight decision-making logic. This improves the safety and reliability of drones in autonomous inspections in complex terrain scenarios, meeting the practical needs of high-precision, high-reliability terrain-following flight in fields such as power line inspection and mountain environment monitoring.

[0048] Second, this invention achieves smooth switching across flight modes by constraining the altitude deviation between adjacent frames through a dual-ended queue, avoiding drastic fluctuations in the aircraft's attitude caused by sudden changes in altitude during different flight modes. Simultaneously, it evaluates the effectiveness of control command execution in real time during flight, corrects decision deviations based on sensor feedback data to form closed-loop control, and achieves lightweight deployment by employing channel pruning and quantization processing on the semantic segmentation model. This improves the stability and control accuracy of the UAV's terrain-following flight while ensuring model inference efficiency and control real-time performance. This solves the problems of existing technologies that easily lead to loss of control during cross-scene flight mode switching, as well as control command execution deviations caused by environmental interference or sensor errors, ensuring the continuous stability of the UAV's autonomous inspection operations in long-term, complex environments.

[0049] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from an examination of the following, or may be learned from the practice of the invention. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0051] Figure 1 This is a flowchart illustrating the operation of the present invention.

[0052] Figure 2 This is a flowchart illustrating the steps of the present invention;

[0053] Figure 3 This is a diagram of the architecture of the multi-sensor fusion decision control system of the present invention. Detailed Implementation

[0054] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0055] Example 1: As Figure 2 and Figure 3 As shown, this embodiment addresses the high safety and adaptability requirements of UAVs for terrain-following flight in complex terrain (including water bodies, vegetation canopies, bare rock cliffs, and densely populated areas of power transmission lines) during 220kV power transmission line inspection in mountainous terrain. Specifically, it implements a method for autonomous UAV inspection using terrain-following flight. This method utilizes a UAV equipped with multi-source sensors to achieve real-time terrain data acquisition. A lightweight semantic segmentation model is used to classify terrain materials, and laser point cloud topology data is fused to extract terrain structural features. Dynamic flight modes are matched based on the combination of materials and structures, and a smooth switching algorithm and closed-loop control mechanism ensure flight safety and inspection accuracy. This embodiment details the specific operation process, parameter design basis, and working principle of each technical step, verifying the feasibility and superiority of this method in complex mountainous power transmission line inspection scenarios. It effectively solves the problem of inaccurate obstacle avoidance judgment caused by existing technologies relying solely on terrain elevation data, improving the intelligence and reliability of UAV inspection.

[0056] Scenarios and System Deployment:

[0057] The application scenario of this embodiment is a 220kV transmission line inspection area in a mountainous region under the jurisdiction of a provincial power company. This area spans approximately 20km and encompasses various complex terrains: three mountain water bodies with an area greater than 0.5km² along the line, a densely vegetated canopy area (mainly pine forest) of approximately 5km in length, two sections of bare rock cliffs with slopes exceeding 60°, and a valley crossing section with a significant sag gradient of the transmission line (the maximum sag difference reaches 8m). In this scenario, the transmission line inspection must simultaneously meet three core requirements: "avoiding collisions with terrain / obstacles," "ensuring the accuracy of line detail acquisition," and "adapting to complex weather interference (such as sudden gusts)." Traditional terrain-following flight solutions based on pre-generated DSMs are prone to problems in this scenario, such as "misjudging the water surface leading to low-altitude close approach," "canopy collision risk," and "missing the detection of the lowest point of sag." The method of this invention is needed to achieve dynamic adaptive control.

[0058] The deployment of the unmanned aerial vehicle system in this embodiment:

[0059] 1. UAV Platform Selection: A multi-rotor industrial-grade UAV with a maximum payload of 1.5kg and a flight time of 40min was selected. It has a multi-sensor integrated interface and an open-source flight control system (supporting the access of custom control commands). The payload capacity and flight time of this platform can meet the long-term load requirements of multi-source sensors, while the open-source flight control provides hardware support for the real-time issuance of dynamic control commands.

[0060] 2. Multi-source sensor installation and parameter design:

[0061] Lightweight LiDAR: Mounted at the center of the lower part of the drone's fuselage, it employs the TOF (Time-of-Flight) ranging principle, with a ranging range of 0.5-100m and a designed ranging accuracy of ±3cm. This accuracy is based on the following: During power line inspection, the safe distance between the drone and the line must be controlled within 5m. The ±3cm ranging error ensures that deviations in the laser point cloud's calculation of the line's location and terrain elevation do not affect safety decisions. Simultaneously, the lightweight design (weighing less than 300g) avoids excessive load on the drone's payload, ensuring extended flight time.

[0062] Multispectral camera: Mounted below the drone's gimbal, at a 15° angle to the lidar (to avoid mutual obstruction of view), covering five spectral bands: 450-520nm (blue light), 520-600nm (green light), 630-690nm (red light), 760-900nm (near-infrared), and 900-1700nm (shortwave infrared). The core purpose of this band selection is as follows: the red and near-infrared bands are used to calculate the vegetation NDVI (Normalized Difference Vegetation Index) to determine the density of the vegetation canopy; the shortwave infrared band can penetrate some vegetation leaves to obtain topographic information of the sub-canopy; and the blue and green bands are used to improve the classification accuracy of materials such as water surfaces and bare rocks.

[0063] Ultrasonic radar: Four units are installed, deployed around the drone's fuselage (front, rear, left, and right), with a detection range of 0.1-5m and an obstacle avoidance accuracy designed to be ±5cm. This sensor is designed for "near-range sudden obstacle detection," such as protruding rocks or low shrubs encountered during low-altitude, ground-hugging flight. The ±5cm accuracy ensures that when the distance to an obstacle is within 1m, deviations in the feedback data will not lead to incorrect obstacle avoidance decisions.

[0064] 3. Embedded Computing Platform Deployment: An NVIDIA Jetson TX2 embedded platform is used, installed inside the UAV fuselage. It connects to multiple sensors via a USB 3.0 interface to achieve real-time data reception and processing; simultaneously, it communicates with the flight control system via a UART interface to issue control commands. The platform was chosen because its integrated GPU (Pascal architecture, 256 cores) can meet the real-time inference requirements of lightweight semantic segmentation models, and its power consumption is only 15W, which will not excessively consume the UAV's battery power and meets the energy constraints of long-term inspection.

[0065] Multi-source sensor data acquisition and asynchronous fusion (corresponding to step S100):

[0066] The core objective of this step is to acquire comprehensive, real-time terrain data through multi-source sensors and to solve the problems of data redundancy and processing delay caused by differences in the sampling frequencies of different sensors through asynchronous fusion methods, so as to provide high-quality input data for subsequent semantic segmentation and feature extraction.

[0067] 3.1 Data acquisition process for each sensor;

[0068] 1. Lightweight LiDAR Data Acquisition: The LiDAR emits a laser beam at a sampling frequency of 10Hz, generating approximately 100,000 laser points per frame. Each laser point contains three-dimensional coordinates (x, y, z, based on the UAV coordinate system) and a reflection intensity value. The sampling frequency of 10Hz is chosen because the time scale of changes in mountainous terrain is relatively slow (the UAV's flight speed is approximately 5m / s; 10Hz sampling ensures that terrain elevation data is acquired every 0.1s, covering a flight distance of 50cm, which is sufficient to capture the macroscopic undulations of the terrain). If the sampling frequency is too high (e.g., 20Hz), the amount of point cloud data per frame will double, increasing the computational burden on the embedded platform, and the improvement in terrain detail will be limited. If it is too low (e.g., 5Hz), elevation changes in local steep terrain (e.g., cliff edges) may be missed, leading to collision risks. During the acquisition process, the LiDAR transmits the point cloud data to the embedded platform in real time via a USB 3.0 interface. Each frame of data is accompanied by a timestamp (accurate to the millisecond level) for subsequent data fusion.

[0069] 2. Multispectral Camera Data Acquisition: The multispectral camera captures terrain images at a sampling frequency of 30Hz. Each frame has a resolution of 1280×720 pixels, and each pixel corresponds to the reflectance value of 5 spectral bands (output after internal camera calibration). The design logic of the 30Hz sampling frequency is as follows: visual data needs to capture more detailed terrain textures (such as the delicate structure of power lines and the reflective texture of water surfaces), and a higher frame rate can avoid image blurring caused by drone movement; at the same time, the frequency ratio of 30Hz to the 10Hz of the LiDAR is 3:1, which facilitates subsequent timestamp alignment to achieve "3 frames of visual data matching 1 frame of LiDAR point cloud data", ensuring the integrity of visual details without generating too much redundant data. During acquisition, the multispectral camera synchronously records the timestamp of each frame of image (calibrated based on the same clock source as the LiDAR timestamp) and packages the image data and reflectance data for transmission to the embedded platform.

[0070] 3. Ultrasonic Radar Data Acquisition: All four ultrasonic radars detect the distance to surrounding obstacles in real time at a sampling frequency of 20Hz. When the detection distance is less than a preset threshold (initially set to 3m), a high-frequency warning is triggered (the sampling frequency is increased to 50Hz) to ensure rapid response to nearby obstacles. The basic sampling frequency of 20Hz can meet the near-range monitoring needs of the UAV during normal flight, while the 50Hz warning frequency is for sudden obstacles (such as protruding bushes encountered during low-altitude flight), allowing sufficient reaction time for obstacle avoidance decisions (when the UAV flies at a speed of 5m / s, the obstacle distance can be updated every 0.02s with 50Hz sampling, ensuring that the obstacle avoidance process of "detection-decision-execution" is completed within 0.1s).

[0071] 3.2 Implementation of Asynchronous Fusion of Multi-Source Sensors:

[0072] Because of the differences in sampling frequencies among lidar (10Hz), multispectral cameras (30Hz), and ultrasonic radar (20Hz / 50Hz), using a "synchronous acquisition-synchronous processing" mode would cause high-frequency sensor data to wait for low-frequency sensor data, resulting in data accumulation and processing delays. Therefore, this embodiment adopts a "timestamp alignment + asynchronous fusion" strategy, the specific process of which is as follows:

[0073] 1. Timestamp Calibration: During system startup, the timestamps of the three sensors are uniformly calibrated using the embedded platform's clock module to ensure that the timestamp deviation of all sensors is less than 1ms (to avoid data matching errors caused by time asynchrony). The calibration method is as follows: using the embedded platform's system clock as a reference, a synchronization trigger signal is sent to each sensor, and the difference between the timestamp fed back by each sensor and the system clock is recorded. In subsequent data acquisition, timestamp unification is achieved through difference compensation.

[0074] 2. Data Alignment and Filtering: Using the LiDAR sampling timestamp as the "reference time axis" (because LiDAR data is the core source of terrain elevation and spatial topology, it needs to be used as the fusion reference), the multispectral camera data and ultrasonic radar data are aligned:

[0075] For 30Hz data from a multispectral camera, every 3 frames of images correspond to 1 frame of LiDAR data (the 3 frames with the smallest timestamp difference). The reflectivity data of these 3 frames are fused using a bilinear interpolation algorithm to generate 1 frame of "fused visual data" that matches the LiDAR timestamp, thus avoiding the influence of noise from single-frame visual data.

[0076] For ultrasonic radar data, all radar data with a timestamp difference of less than 50ms from lidar data are collected, and the average value is calculated as the "effective obstacle distance data" at that moment. This ensures the real-time nature of the data and reduces measurement noise through averaging filtering.

[0077] 3. Fusion Efficiency Optimization: By employing the aforementioned timestamp alignment strategy, only "valid data on the baseline timeline" is processed subsequently, avoiding calculations on unaligned redundant data (such as images from multispectral cameras that do not match LiDAR timestamps), ultimately achieving a 35% reduction in computational overhead. The principle behind this optimization is that unaligned data, if processed, leads to redundant calculations (such as the same terrain area being repeatedly analyzed from multiple frames of visual data). Aligned data, however, retains only the visual and obstacle information corresponding to terrain elevation, significantly reducing the computational load on the embedded platform and ensuring the real-time performance of subsequent semantic segmentation and feature extraction.

[0078] Pixel-level semantic segmentation processing of terrain data (corresponding to step S200):

[0079] This section utilizes a lightweight PSPNet-MobileNetV2 hybrid architecture model to perform pixel-level semantic segmentation on asynchronously fused terrain data (LiDAR reflectivity data + multispectral fused visual data), outputting six terrain material classification labels (water surface, sand, tree canopy, bare rock, power lines, and buildings), providing a material category foundation for subsequent terrain structure feature extraction. This section requires a detailed explanation of the model architecture design, optimization process, loss function application, and semantic segmentation implementation process.

[0080] 4.1 Selection and advantages of the PSPNet-MobileNetV2 hybrid architecture;

[0081] The PSPNet-MobileNetV2 hybrid architecture was chosen as the semantic segmentation model because it balances "multi-scale terrain feature capture capability" and "lightweight deployment characteristics." The specific design logic is as follows:

[0082] 1. The Role of PSPNet's Pyramid Pooling Module: In mountainous power transmission line inspection scenarios, the scale differences of terrain materials are significant (e.g., large areas of water, small-scale power transmission lines, and medium-scale tree canopies). Traditional semantic segmentation models (such as U-Net) are prone to problems such as "incomplete segmentation of large-scale targets" and "missed detection of small-scale targets" when processing multi-scale targets. PSPNet's pyramid pooling module uses four different scale pooling operations (e.g., 1×1, 2×2, 3×3, and 6×6 pooling) to capture global terrain features (e.g., the distribution of the entire water body), medium-scale features (e.g., the canopy of a single tree), and small-scale features (e.g., a single power transmission line). Then, through feature fusion, it integrates multi-scale information to ensure the segmentation accuracy of the six material labels, especially improving the recognition rate of small-scale targets such as power transmission lines to over 95% (verified by preliminary sample testing).

[0083] 2. Lightweight support for MobileNetV2:

[0084] The embedded platform (NVIDIA Jetson TX2) on the drone has limited memory (8GB) and computing power (1.3 TFLOPS). The traditional PSPNet, based on ResNet50 as its backbone, has a model size exceeding 100MB and an inference time exceeding 200ms, which cannot meet the real-time processing requirements (when the drone flies at 5m / s, one frame of data needs to be processed every 50ms to avoid lag). MobileNetV2 adopts a "depth-separable convolution + linear bottleneck structure," decomposing standard convolution into depthwise convolution (channel-wise convolution) and pointwise convolution (1×1 convolution), reducing the number of parameters by more than 90% while maintaining feature extraction capabilities. At the same time, the linear bottleneck structure solves the feature degradation problem caused by depthwise separable convolution, ensuring model accuracy. Using MobileNetV2 as the backbone of PSPNet can achieve model lightweighting while retaining multi-scale segmentation capabilities.

[0085] 4.2 Optimization process of semantic segmentation model (channel pruning and INT8 quantization):

[0086] This embodiment performs dual optimization of the PSPNet-MobileNetV2 hybrid architecture using "channel pruning + INT8 quantization". The specific process is as follows:

[0087] 1. Channel pruning: Remove 20% of redundant layers:

[0088] Pruning prerequisite: Through "channel contribution analysis" during model training, the contribution value of each channel in each convolutional layer to the final semantic segmentation accuracy is statistically analyzed (the contribution value is calculated by the mutual information between the output feature map and the label of the channel; the lower the mutual information, the smaller the contribution).

[0089] The pruning process involves: first, freezing the core channels (the top 80% of channels in terms of contribution) of the backbone network (MobileNetV2) and pyramid pooling module to avoid affecting the model's core feature extraction capabilities; then, removing the bottom 20% of channels in the transitional convolutional layers (convolutional layers connecting different modules), where the mutual information between the output feature maps and labels of these channels is less than 0.1 (testing showed that the model accuracy only decreased by 0.5% after removal, which is within an acceptable range); finally, fine-tuning the pruned model by updating the weight parameters of the remaining channels through backpropagation to compensate for the slight accuracy loss caused by pruning.

[0090] Pruning effect: The number of model parameters was reduced from 2.5M to 2.0M, and the computational cost was reduced by 20%, laying the foundation for subsequent quantization and compression.

[0091] 2. INT8 quantization: Compress the model size to 0.8MB.

[0092] Quantization principle: Traditional deep learning models store weight parameters and activation values ​​using 32-bit floating-point numbers (FP32), which consumes a large amount of memory. INT8 quantization maps FP32 data to 8-bit integers (INT8), and achieves data conversion through the linear quantization formula (INT8 value = (FP32 value - mean) / scaling factor). At the same time, "calibration set calibration" ensures that the accuracy loss of the quantized model is controllable.

[0093] Quantization process: First, 1000 frames of terrain data from a mountain power transmission line inspection scene were selected as the calibration set, covering all 6 types of terrain materials; then, the FP32 model was run on the calibration set, and the distribution range of weights and activation values ​​of each layer (maximum, minimum, and mean) was statistically analyzed, and the optimal scaling factor was calculated (to ensure that the INT8 data can cover the effective range of the FP32 data and reduce truncation error); finally, the FP32 model was converted into an INT8 model based on the scaling factor, and the accuracy of the quantized model was verified (the mIOU (mean intersection-union ratio) of semantic segmentation decreased from 89.2% in FP32 to 88.7% in INT8, with an accuracy loss of only 0.5%).

[0094] Quantization results: The model size was compressed from 8MB (FP32) after pruning to 0.8MB (INT8), memory usage was reduced by 90%, and inference time was reduced from 80ms after pruning to 45ms, meeting the real-time processing requirements of embedded platforms (inference time <50ms).

[0095] 4.3 Application and Semantic Segmentation Implementation of FocalLoss Function:

[0096] In the scenario of inspecting power transmission lines in mountainous terrain, there is a significant imbalance in the pixel distribution of terrain materials: water surfaces and power transmission lines account for only 5%-8% of the pixels, while bare rock and ground account for more than 60%. If the traditional cross-entropy loss function is used, the model will tend to predict the majority class (bare rock and ground), resulting in excessively low segmentation accuracy for the minority class (power transmission lines and water surfaces). Therefore, this embodiment introduces the FocalLoss function to solve the class imbalance problem. The specific application process is as follows:

[0097] 1. The design and significance of the parameters of the FocalLoss function:

[0098] The expression for the FocalLoss function is: , This is the predicted probability value of the current pixel belonging to a specific terrain material category. It is calculated by the output layer (Softmax layer) of the semantic segmentation model and ranges from [0,1]. For example, the probability that a pixel is predicted by the model as "power transmission line". The probability of it being predicted as "bare rock" Then the pixel The category is "transmission line".

[0099] The weighting adjustment coefficient for the current terrain material category is set based on the distribution ratio of pixels in each category, with the aim of balancing the loss contribution of the majority and minority classes. In this embodiment... The values ​​are: power transmission lines (0.7), water surfaces (0.6), tree canopies (0.5), bare rocks (0.3), sandy areas (0.4), and buildings (0.5). These values ​​are based on the fact that power transmission lines and water surfaces are minority categories, thus assigned higher values. To increase its loss weight; bare rock is the majority class, and is assigned a lower weight. This reduces the loss weight and avoids drowning out the loss signals of minority classes.

[0100] To focus on the adjustment factor for difficult-to-differentiate samples, a value of 2 was chosen (based on previous testing). At that time, the model had the best focusing effect on difficult samples (such as blurred transmission line pixels and bare rock pixels that are similar to water surface reflections). The mechanism of action is as follows: for easily separable samples ( (Close to 1, like a clear bare rock pixel). When the value is close to 0, the loss value is suppressed; for hard-to-separate samples ( Close to 0.5, such as blurry transmission line pixels). When the value approaches 0.25, the loss value is amplified, and the model will prioritize learning the features of hard-to-distinguish samples.

[0101] 2. Calculation of the FocalLoss function and model training:

[0102] Loss calculation process: Taking a "fuzzy power line pixel" in a frame of terrain data as an example, the true label of this pixel is "power line", and the model predicts it as "power line". , , Substitute into the formula to calculate:

[0103] If the pixel is mispredicted as "bare rock" The loss value is:

[0104] A higher loss value will guide the model to adjust its parameters and reduce such misjudgments.

[0105] Model training process: The stochastic gradient descent (SGD) optimizer was used, with the learning rate initially set to 0.001 and decayed to 0.1 every 10 epochs. The training dataset contained 5,000 frames of labeled data of mountain power transmission line inspection scenes (each frame was labeled with 6 material labels). The training rounds were 100 epochs, until the model's mIOU on the validation set was stably converged (fluctuation less than 0.1%).

[0106] 3. Semantic segmentation result output:

[0107] After the trained INT8 quantization model was deployed on the NVIDIA Jetson TX2 embedded platform, it was used to process the asynchronously fused terrain data (each frame of data includes a lidar reflection intensity map and a multispectral fusion map) in real time. The model first normalizes the input data (scaling the pixel values ​​to [0,1]), then extracts basic features through the MobileNetV2 backbone network, fuses multi-scale features through the PSPNet pyramid pooling module, and finally outputs the material category probability of each pixel through the Softmax layer. The category with the highest probability is selected as the final material label of the pixel, generating a "terrain material semantic segmentation map" with the same resolution as the input data (each pixel corresponds to one of the 6 labels). The semantic segmentation results are stored in the cache of the embedded platform for subsequent terrain structure feature extraction.

[0108] Topographic structure feature extraction (corresponding to step S300):

[0109] The core of this step is to integrate the "terrain material semantic segmentation results" with the "laser point cloud spatial topology data" to extract four key terrain structural features (cliffs with a slope > 60°, valleys with continuous negative terrain, vegetation canopy with NDVI > 0.7, and bare rock with spectral reflectance > 0.4), providing a basis for structural dimension judgment for subsequent flight mode matching. This step requires a detailed explanation of the data fusion method, feature extraction principles, and enhancement processing for special areas.

[0110] 5.1 Method for fusing semantic segmentation results with laser point clouds:

[0111] The laser point cloud spatial topology data contains the three-dimensional coordinates (x, y, z) and reflection intensity of each laser point, reflecting the elevation changes and spatial distribution of the terrain; the semantic segmentation results provide the material category of each pixel. The fusion of the two requires achieving "spatial correspondence between point cloud and pixels," and the specific process is as follows:

[0112] 1. Coordinate Mapping: The real-time position (latitude and longitude, elevation) and attitude (roll angle, pitch angle, yaw angle) of the UAV are obtained through the UAV's POS system (positioning and attitude system). The transformation relationship between the lidar coordinate system and the multispectral camera coordinate system is established (based on the calibration parameters during sensor installation, such as relative position offset and angle offset). Then, the three-dimensional coordinates (lidar coordinate system) of each laser point are converted into the image coordinate system (pixel coordinates) of the multispectral camera to achieve a one-to-one correspondence between "laser point and pixel". That is, each laser point can be matched with a pixel in the semantic segmentation map, thereby obtaining the terrain material category corresponding to the laser point.

[0113] 2. Data Association: The matched "laser point (3D coordinates, reflection intensity) + material label" data is stored in a structured data format (such as PCD format). Each laser point contains 7 attributes: x (eastward coordinates), y (northward coordinates), z (elevation), intensity (reflection intensity), material (material label), ndvi (NDVI value, calculated only for vegetation-related materials), and reflectivity (spectral reflectivity, calculated only for bare rock materials). This structured data preserves the spatial topological information of the laser point cloud while incorporating material categories and key spectral parameters, providing a complete data foundation for structural feature extraction.

[0114] 5.2 Principles for extracting key terrain structural features:

[0115] Based on the fused structured data, the following methods are used to extract four types of terrain structure features:

[0116] 1. Extraction from cliffs with a slope greater than 60°:

[0117] Slope calculation principle: Slope is the degree of inclination of any point on the terrain surface, defined as the rate of change of elevation at that point. The calculation formula is: ,in The elevation difference between adjacent laser points. The horizontal distance between adjacent laser points (calculated from x and y coordinates): .

[0118] Extraction process: First, the fused laser point cloud is divided into a 5m×5m grid (the grid size is set according to the terrain resolution requirements; a 5m grid can balance macroscopic slope and computational efficiency). The average elevation of all laser points in each grid is calculated. Then, adjacent grids are selected (e.g., four adjacent grids in the top, bottom, left, and right), and the average elevation difference between the current grid and its adjacent grids is calculated. and horizontal distance The slope value of the grid is obtained; finally, the grid with a slope value > 60° is marked as "cliff area", and the laser point cloud data of the area is recorded (for subsequent obstacle avoidance path generation).

[0119] Extraction basis: Areas with a slope greater than 60° are steep terrains. If drones fly at the usual ground-level altitude, they are prone to collisions due to sudden changes in terrain. Special flight modes (such as increasing altitude and generating obstacle avoidance paths) are required to cope with this. Therefore, it is necessary to accurately extract this feature.

[0120] 2. Extraction of valleys from continuous negative terrain:

[0121] Definition of negative terrain: Negative terrain refers to areas where the elevation is lower than that of the surrounding terrain. Valleys are typical examples of continuous negative terrain (characterized by multiple grids extending in a certain direction with elevations lower than those of the grids on either side).

[0122] Extraction process: First, calculate the elevation difference between each 5m×5m grid and the average elevation difference between the grids surrounding it and the 8 adjacent grids. ( );like If the grid is negative, then the grid is a negative terrain grid; then, the "region growing algorithm" is used, with the negative terrain grid as the seed point, and adjacent (up, down, left, right) and The grids were merged into negative terrain regions; finally, continuous negative terrain regions with a length > 50m (along the extension direction) and a width > 10m were selected and marked as "valley regions".

[0123] Extraction basis: The terrain of the valley area is complex and prone to obstacles such as low shrubs and rocks. In addition, the sag of power transmission lines in the valley area is usually large. It is necessary to increase the sampling density of point cloud to obtain more detailed terrain information and avoid the risk of collision.

[0124] 3. Extraction of vegetation canopy with NDVI > 0.7:

[0125] NDVI Calculation Principle: NDVI (Normalized Difference Vegetation Index) is a core indicator reflecting vegetation growth and density. The calculation formula is as follows: ,in The spectral reflectance is in the near-infrared band. The spectral reflectance is in the red band (all data were collected and calibrated by a multispectral camera). The NDVI value ranges from [-1, 1], with higher values ​​indicating denser vegetation.

[0126] Extraction process: First, select laser points with the material tag "tree crown" from the fused data, and obtain the corresponding data for these points. and Then, the average NDVI value of the canopy points in each grid is calculated according to a 5m×5m grid. Finally, the grids with an average NDVI value > 0.7 are marked as "dense vegetation canopy areas".

[0127] Extraction criteria: NDVI > 0.7 indicates dense vegetation canopy (such as the pine forest in this example, where NDVI can reach 0.8-0.9 in summer). If the drone flies too low, it is easy to collide with the top of the canopy. At the same time, dense canopy requires the acquisition of the underlying terrain information through the penetration scanning mode, so this feature needs to be extracted accurately.

[0128] 4. Extraction from bare rocks with a spectral reflectance > 0.4:

[0129] Spectral reflectance acquisition: Reflectance data of bare rock areas collected by multispectral cameras (after radiometric calibration to eliminate the influence of factors such as light and atmosphere) were selected. The average reflectance in the visible light band (450-690nm) was selected as the spectral reflectance index of bare rock (the reflectance of bare rock in the visible light band is significantly higher than that of vegetation and water surface).

[0130] Extraction process: Select laser points with the material label "bare rock" from the fused data, calculate the average spectral reflectance of bare rock points in each 5m×5m grid, and mark grids with an average reflectance > 0.4 as "high reflectance bare rock areas".

[0131] Extraction criteria: Bare rocks with a spectral reflectance > 0.4 are usually exposed rock surfaces without vegetation cover, and the slope of such bare rocks is often large (such as the cliff section in this embodiment). It is necessary to obtain their terrain details through a high-precision reconstruction mode to ensure the safety of drone obstacle avoidance.

[0132] 5.3 Enhancement processing for special areas;

[0133] For high-risk areas such as valleys, cliffs, and sag sections of power transmission lines, laser point cloud data needs to be enhanced to improve feature extraction accuracy and the reliability of obstacle avoidance path generation. The specific processing methods are as follows:

[0134] 1. Increased point cloud sampling density in valley and cliff areas:

[0135] Processing principle: The terrain in valleys and cliff areas is complex. The laser point cloud with a conventional sampling frequency of 10Hz may have sparse data areas (such as the low density of point cloud at the edge of the cliff, which cannot accurately reflect the terrain outline), resulting in deviations in slope calculation or missed obstacles.

[0136] Processing procedure: After extracting the valley or cliff area, the embedded platform sends a "density increase command" to the LiDAR, increasing the sampling density of the laser point cloud in that area by 50% (i.e., increasing the number of point clouds per frame from 100,000 to 150,000); at the same time, the point cloud data acquisition time in that area is extended (one frame is acquired every 0.2 seconds instead of the usual 0.1 seconds) to ensure that sufficiently dense point cloud data is obtained.

[0137] Subsequent processing: Based on the increased density of the point cloud data, the continuous range of the valley and the slope of the cliff are recalculated to ensure the accuracy of feature extraction; at the same time, a 3D obstacle avoidance path generation is triggered (using the RRT algorithm to search for collision-free paths in the point cloud data, with the safe distance between the path and the terrain set to 3m) to prevent the drone from flying directly over complex areas.

[0138] 2. Prediction of the lowest point of transmission line sag and generation of the loop path:

[0139] The principle behind predicting the lowest point of sag: The sag of a power transmission line is affected by temperature and load, exhibiting a curved distribution along the line direction that is "lower in the middle and higher at both ends." The lowest point of sag is the area with the highest risk of drone collision (closest to the ground). Based on the semantic distribution of the power transmission line (the "power transmission line" label in the semantic segmentation map) and laser point cloud elevation data, the location of the lowest point of sag can be predicted.

[0140] Prediction process: First, extract all laser points with the material label "power transmission line" from the fused data. Divide these points into multiple segments (each segment corresponds to a 50m long power transmission line) according to the route (based on the UAV heading angle and the spatial distribution of the power transmission line). Then, perform curve fitting on the elevation data of the laser points of each segment (using quadratic function fitting, with the sag of the power transmission line approximating a parabola). Calculate the three-dimensional coordinates of the lowest point of the fitted curve (i.e., the lowest point of the sag).

[0141] Path generation: For the predicted lowest point of the sag, a counter-clockwise loop path with a radius of 10m is generated. The reason for choosing counter-clockwise looping is that the electromagnetic field generated by the power line is distributed in a clockwise rotation. Counter-clockwise looping can avoid the UAV entering the electromagnetic field blind zone (the electromagnetic field blind zone will cause signal interference of ultrasonic radar and lidar, affecting data acquisition). The design basis of 10m radius is that the electromagnetic influence range of the power line is about 8m. A 10m radius can ensure that the UAV loops within a safe distance, while clearly collecting detailed data of the lowest point of the sag (such as line wear and insulator condition).

[0142] The implementation of flight mode matching, smooth switching and closed-loop control (corresponding to steps S400-S600) is as follows;

[0143] This stage is the core decision-making and control stage for UAV terrain-following flight. It matches preset flight modes based on a combination of "terrain material classification labels + terrain structure features" to generate dynamic control commands; it achieves smooth switching between modes through dual-ended queue constraints; and it constructs closed-loop control based on sensor feedback data to correct decision-making deviations and ensure flight safety and inspection accuracy. This stage requires a detailed explanation of the specific implementation of the mode matching logic, smooth switching algorithm, and closed-loop control mechanism.

[0144] 6.1 Flight mode matching logic and control command generation (S400):

[0145] This embodiment pre-defines four core flight modes (low-disturbance mode, penetration scan mode, high-precision reconstruction mode, and line-following obstacle avoidance mode). Each mode corresponds to a specific "material-structure" combination. The matching logic is designed based on the principle that "terrain attributes determine the optimal flight strategy," as detailed below:

[0146] 1. Matching and control of low-disturbance modes:

[0147] Matching conditions: The terrain material label is "water surface," and the structural features meet the requirements of "flat water surface area (slope < 5°) and spectral reflectance higher than the preset threshold." The reflectance threshold is set to 0.6, based on the following design principle: the reflectance of water surface in the visible light band is typically 0.5-0.8. When the reflectance > 0.6, the TOF signal of the lidar is easily interfered with by specular reflection, leading to point cloud data distortion (misjudging the water surface as "high-elevation ground"). A low-disturbance mode is needed to avoid data distortion and water surface disturbance.

[0148] Control command generation: When the matching conditions are met, the embedded platform sends the following control commands to the flight control system:

[0149] Altitude control: The drone's flight altitude is reduced to 3m (above the water surface). The design basis for the 3m altitude is to avoid both the propeller airflow disturbing the water surface (affecting water quality inspection data if water quality monitoring is carried out in the area at the same time) and the lidar point cloud acquisition accuracy of water surface details (such as floating objects) being reduced.

[0150] Sensor control: Turn off the downward-looking lidar (only keep the forward-looking lidar for terrain detection ahead) because the data from the downward-looking lidar is invalid when reflected by the water surface. Turning it off reduces data redundancy and computational load.

[0151] Speed ​​control: Reducing the flight speed from the usual 5m / s to 3m / s increases the time for the multispectral camera to capture details of the water surface, improving the accuracy of water quality monitoring (if equipped with a water quality sensor) or the identification of obstacles on the water surface (such as floating branches).

[0152] 2. Matching and control of penetration scanning modes:

[0153] Matching conditions: The terrain material label is "tree canopy," and the structural features meet the criteria of "NDVI > 0.7 (dense canopy) and canopy thickness exceeding the set standard" (canopy thickness is calculated using LiDAR point cloud computing, i.e., the elevation difference between the top and bottom of the canopy > 5m). The design basis for these conditions is: a dense canopy with NDVI > 0.7 makes it easy for drones to collide with the top of the canopy in normal ground-hugging mode; a canopy thickness > 5m indicates that there may be terrain undulations in the lower layer, requiring penetration scanning to obtain information about the lower layer.

[0154] Control command generation:

[0155] Altitude control: The drone's flight altitude is increased to 2m above the canopy. The design basis for the 2m altitude is that the short-wave infrared band of the multispectral camera can penetrate the canopy leaves within 2m to obtain the elevation data of the lower terrain, while avoiding the decrease in the acquisition accuracy of details (such as tree diseases) at the top of the canopy due to excessive altitude.

[0156] Sensor control: Activate the penetration imaging mode of the multispectral camera (increase the gain of the short-wave infrared band by 2 times to enhance penetration capability), and at the same time increase the vegetation filtering intensity of the lidar by 2 times (enhance the filtering of vegetation point clouds through algorithms to retain the lower terrain point clouds).

[0157] Gimbal control: Adjust the pitch angle of the multispectral camera gimbal to -30° (tilt down 30°) to ensure the camera lens is aimed at the lower canopy layer and obtain a clearer penetrating image.

[0158] 3. Matching and control of high-precision reconstruction modes:

[0159] Matching conditions: The terrain material label is "bare rock," and the structural feature meets the condition of "slope > 30°." The design basis for this condition is that bare rock areas with a slope > 30° have steep terrain and are prone to obstacles such as protruding rocks. High-precision terrain reconstruction is needed to obtain details and ensure obstacle avoidance safety. At the same time, power transmission line foundations (such as iron towers) in bare rock areas are easily affected by weathering, and high-precision reconstruction can be used for foundation defect detection.

[0160] Control command generation:

[0161] Gimbal control: Adjust the tilt angle of the lidar gimbal to -45° (tilt downwards by 45°). This angle allows the lidar's scanning range to better fit the surface of steep bare rock, acquiring denser vertical point cloud data and improving terrain reconstruction accuracy.

[0162] Sensor control: The scanning frequency of the LiDAR is increased from the conventional 10Hz to 100Hz. The 100Hz scanning frequency can increase the number of point clouds per frame to 1 million, and the point cloud density is increased by 10 times, ensuring that the resolution of the reconstructed terrain model reaches 0.1m (to meet the detection requirements of protruding rocks).

[0163] Speed ​​control: Reducing the flight speed to 2m / s and flying slowly ensures that the lidar acquires enough point cloud data at each location, avoiding sparse point clouds due to excessive flight speed.

[0164] 4. Matching and control of the simulated obstacle-encircling mode:

[0165] Matching conditions: The terrain material label is "Dense Power Line Area" (area where power line pixels account for >20%), and the structural feature meets the requirement of "Existence of Sag Gradient" (sag difference between adjacent 50m power lines >2m). The design basis for this condition is: in dense power line areas with large sag gradients, conventional flight paths parallel to the lines are prone to colliding with the lowest point of sag, so it is necessary to fly along the electromagnetic field strength gradient to dynamically track the line direction.

[0166] Control command generation:

[0167] Heading control: Adjust the flight heading angle along the direction of the electromagnetic field strength gradient. The direction of the electromagnetic field strength gradient is detected by the electromagnetic sensor on the UAV (the electromagnetic sensor is installed at the front of the fuselage and provides real-time feedback on the rate of change of the electromagnetic field strength; the gradient direction is the tangent direction of the route). This ensures that the UAV always flies along the tangent direction of the route and avoids deviation.

[0168] Distance control: The lateral obstacle avoidance distance is set to 5m (the horizontal distance between the drone and the power line is kept at 5m). The design basis for the 5m distance is: the safety distance standard for 220kV power lines is 3m, and 5m can reserve an error redundancy of 2m to avoid collisions caused by deviations in heading angle.

[0169] Altitude control: Based on the predicted elevation of the lowest point of the sag, the drone's flight altitude is set to "lowest point elevation of the sag + 3m" to ensure a safe vertical distance from the route.

[0170] 6.2 Dynamic switching logic of flight modes:

[0171] In addition to the basic matching logic mentioned above, this embodiment also incorporates dynamic switching logic based on environmental changes to address unexpected scenarios (such as changes in vegetation coverage or sudden gusts of wind), as detailed below:

[0172] 1. Tree Canopy Ratio-Triggered Switching: When the embedded platform calculates the tree canopy pixel ratio of the current region using semantic segmentation maps (every 0.5 seconds), it automatically switches from "Ground-Following Mode" (the default mode for normal terrain, with the height set to 5m above the ground) to "Penetrating Scan Mode." Simultaneously, the vegetation filtering intensity of the LiDAR is increased by 2 times. This is because a high tree canopy ratio significantly increases the number of vegetation points in the point cloud; enhanced filtering effectively preserves the point cloud data of the ground and power lines, preventing vegetation points from interfering with terrain feature extraction.

[0173] 2. Sudden Wind Speed ​​Trigger Switching: The wind speed sensor onboard the drone monitors the wind speed of the flight environment in real time (sampling frequency 10Hz). When a sudden increase in wind speed exceeding 8m / s is detected (sudden gust), the embedded platform immediately sends a command:

[0174] Inspection range adjustment: The current line-following inspection distance will be shortened from the usual 20km to 10km to reduce flight time and reduce the risk of continuous gusts.

[0175] Altitude hold mode switching: switch from "LiDAR altitude hold" (adjusting altitude based on LiDAR elevation data) to "backup barometric altitude hold" (adjusting altitude based on barometric pressure sensor elevation data). The reason is that strong winds can cause drone attitude instability, and LiDAR elevation data is easily affected by the shaking of the drone body, resulting in deviations. Barometric altitude hold is more stable (although the accuracy is slightly lower, stability is prioritized in strong wind scenarios).

[0176] 6.3 Smooth switching algorithm across modes (S500):

[0177] The altitude differences between different flight modes are significant (e.g., 3m in low-disturbance mode and 15m in penetration scanning mode). Directly switching modes would cause abrupt changes in the drone's altitude, leading to drastic fluctuations in its attitude and even loss of control. Therefore, this embodiment employs a smooth switching algorithm combining "dual-ended queue constraints + altitude change rate control," constraining the altitude deviation between adjacent frames using the following formula: ,in: The height change rate between adjacent frames is constrained by a threshold of 20%, meaning that the height change in each frame (0.1s) must not exceed 20% of the height of the previous frame. The design basis for this threshold is that the maximum height change rate of industrial-grade drones is usually 1m / s. If the height of the previous frame is 10m, a 20% change rate corresponds to a height change of 2m / s, which is within the controllable range of the drone and can avoid attitude fluctuations.

[0178] For the current moment ( The drone's flight altitude measurement (at any given time) is collected by lidar or barometric pressure sensor (based on the current altitude hold mode).

[0179] For the previous moment ( Time, and (Time interval 0.1s) UAV flight altitude measurement value.

[0180] The specific implementation process of smooth switching is as follows:

[0181] 1. Deque initialization: Create a deque of length 5 to store the flight altitude measurements of the most recent 5 frames. The purpose of the queue is to smooth out height measurement noise (such as height jumps caused by sensor fluctuations).

[0182] 2. Altitude Change Rate Calculation and Judgment: When it is necessary to switch flight modes (such as switching from low disturbance mode to penetration scan mode, with the target altitude increasing from 3m to 15m), the current altitude is first calculated. Altitude of the target The difference Then based on Given the constraints, calculate the maximum allowable height adjustment per frame. ;like Then only the current frame is adjusted. The height, and the remaining height difference is gradually adjusted in subsequent frames; if If so, adjust directly to the target height.

[0183] 3. Queue Update and Feedback: After adjusting the height of each frame, the new height value is updated. Add to a deque and remove the old value from the front of the queue. ); Calculate the average of the 5 height values ​​in the queue, and use it as the reference height for the next frame. ,like and If the deviation exceeds 5%, the height adjustment amount in the next frame will be corrected to further smooth the height change.

[0184] For example, from Switch to The process:

[0185] Frame 1 ( ), After adjustment , It complies with the constraints.

[0186] Frame 2 ( ), After adjustment , .

[0187] This process continues, with the height increasing by a maximum rate of 20% per frame until the target height of 15m is reached around the 10th frame. The height change is stable throughout the process, and the aircraft's attitude fluctuation is less than ±2° (verified by actual testing).

[0188] 6.4 Closed-loop control feedback mechanism (S600):

[0189] To ensure that the control commands for the flight modes are executed as expected and to avoid decision-making deviations due to sensor errors or environmental interference, this embodiment constructs a closed-loop control mechanism based on sensor feedback, specifically including the following two core components:

[0190] 1. Obstacle avoidance deviation correction for ultrasonic radar:

[0191] Feedback data acquisition: Four ultrasonic radars monitor the actual distance between the drone and surrounding obstacles in real time (sampling frequency 20Hz) and transmit the distance data to the embedded platform.

[0192] Deviation Judgment: The embedded platform compares the actual distance with the preset obstacle avoidance threshold for the current flight mode (e.g., the obstacle avoidance threshold is set to 3m for low-disturbance mode and 5m for penetration scanning mode), and calculates the deviation rate. ,in This is the actual distance. This is a preset threshold.

[0193] Corrective measures: When When the actual distance deviates from the threshold by more than 10%, it is judged as "obstacle avoidance deviation," and the embedded platform re-matches the flight mode.

[0194] like If the actual distance is too close, the flight altitude will be increased by 50%, and the flight will be switched to "temporary obstacle avoidance mode" (the flight speed will be reduced to 2m / s and the lidar scanning frequency will be increased to 50Hz) until the actual distance returns to the threshold range.

[0195] like If the actual distance is too far, reduce the flight altitude by 30% to ensure the accuracy of the inspection data collection (too far will cause the details to be blurred).

[0196] 2. Dynamic calibration of heading angle in areas with dense power transmission lines:

[0197] Feedback Data Acquisition: The electromagnetic sensors on the UAV detect the electromagnetic field gradient direction of the power transmission line in real time (sampling frequency 10Hz), while the lidar acquires the three-dimensional coordinates of the power transmission line in real time, calculating the deviation between the UAV's current heading angle and the direction of the power transmission line (horizontal error). ).

[0198] Deviation judgment: The allowable range of horizontal error is set to ±0.5m. The design basis for this range is: the inspection of power transmission lines must ensure that the drone flies along the tangent of the line. A horizontal error exceeding 0.5m may cause collision with adjacent lines (if the line spacing is small).

[0199] Calibration measures: when the horizontal error or At that time, the embedded platform sends a heading angle calibration command to the flight control system:

[0200] like (If the drone veers to the right of the path), adjust the heading angle to the left. ( The proportionality coefficient is set to 0.1° / m, meaning that for every 1m deviation, an adjustment of 0.1° is made to avoid excessive adjustment that could cause heading fluctuations.

[0201] like (If the drone veers to the left of the path), adjust the heading angle to the right, with the adjustment amount also following the same principle. calculate.

[0202] After calibration, the horizontal error is recalculated using the transmission line coordinates fed back by the lidar, until... This forms a closed-loop calibration.

[0203] In summary, this embodiment, by deploying a complete technical process of "multi-source sensor fusion - semantic segmentation - structural feature extraction - dynamic modal matching - smooth switching - closed-loop control" in a 220kV mountain power transmission line inspection scenario, verifies in detail the feasibility and effectiveness of the UAV autonomous inspection terrain-following flight method of this invention. During implementation, the parameter design of each technical step (such as sensor accuracy, model optimization indicators, and flight modal altitude) was determined based on actual inspection needs and hardware constraints, ensuring the practicality of the technical solution. The FocalLoss function was used to address the class imbalance problem, improving the segmentation accuracy of a few material classes (such as power transmission lines). A smooth switching algorithm with double-ended queue constraints was used to avoid fuselage fluctuations caused by modal switching. A closed-loop control mechanism was used to correct decision-making biases caused by sensor errors and environmental interference in real time. This embodiment, through specific technical implementation and parameter design, fully demonstrates the superiority of the method of the present invention in complex terrain inspection scenarios. It effectively solves the problem of inaccurate obstacle avoidance judgment caused by existing technologies relying solely on elevation data. It provides a feasible technical solution for autonomous inspection and terrain-following flight of UAVs and has the value for large-scale application in fields such as power, photovoltaics, and mountain monitoring.

[0204] Example 2: Figure 1 As shown in Example 1, this example elaborates on the specific steps of a method for autonomous inspection and terrain-following flight of a UAV during operation. The specific steps are as follows:

[0205] 1. Multi-source sensor data acquisition:

[0206] The drone simultaneously collects terrain data using a lightweight lidar, multispectral camera, and ultrasonic radar. The lidar acquires the three-dimensional coordinates and reflection intensity of the terrain, the multispectral camera captures the five-band spectral reflectance, and the ultrasonic radar monitors nearby obstacles in real time.

[0207] 2. Asynchronous data fusion and alignment:

[0208] Timestamp alignment is performed on sensor data with different sampling frequencies (10Hz for lidar, 30Hz for multispectral camera, and 20Hz for ultrasonic radar). Spatiotemporally consistent data frames are generated through linear interpolation and mean filtering, reducing computational redundancy by 35%.

[0209] 3. Semantic segmentation of terrain and material:

[0210] A lightweight PSPNet-MobileNetV2 model is used to perform pixel-level semantic segmentation on the fused data, outputting 6 types of terrain material labels (water surface, sand, tree canopy, bare rock, power line, and building). The model is optimized by channel pruning and INT8 quantization, with a size of 0.8MB and a single-frame inference time of <50ms.

[0211] 4. Extraction of terrain structure features:

[0212] By integrating semantic labels and laser point cloud topology data, four types of key features are calculated.

[0213] Cliffs with a slope greater than 60° (calculated based on grid elevation difference);

[0214] Valleys with continuous negative topography (identified by region growing algorithm).

[0215] Vegetation canopy with NDVI > 0.7 (combined with red light / near-infrared reflectance).

[0216] Bare rock with spectral reflectance > 0.4 (mean analysis of visible light band).

[0217] 5. Dynamic decision-making based on flight modes:

[0218] Match preset flight modes based on material-feature combinations;

[0219] Water surface + high reflectivity → Enable low disturbance mode (height 3m, turn off downward laser);

[0220] Tree canopy + NDVI > 0.7 → Enable penetration scanning mode (2m above the canopy, start multispectral imaging);

[0221] Bare rock with a slope greater than 30° → Activate high-precision reconstruction mode (pan-tilt-zoom -45°, laser frequency 100Hz).

[0222] Dense power line area + sag gradient → Activate line-following obstacle avoidance mode (lateral obstacle avoidance 5m, flying along electromagnetic gradient).

[0223] 6. Smooth cross-modal switching:

[0224] Five consecutive frames of height data are stored using a double-ended queue. to To constrain the height change rate between adjacent frames to ≤20%, a gradual height adjustment strategy is adopted to avoid abrupt changes in posture.

[0225] 7. Closed-loop control and real-time correction:

[0226] Based on ultrasonic radar monitoring of actual obstacle avoidance distance, the flight mode is rematched when the deviation exceeds ±10%; in areas with dense power lines, the heading angle is dynamically calibrated based on the electromagnetic field strength gradient, and the horizontal error is controlled within ±0.5m.

[0227] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV), characterized in that, Includes the following steps: S100: Real-time terrain data is collected by the onboard multi-source sensors, including a lightweight lidar, a multispectral camera, and an ultrasonic radar. The ranging accuracy of the lightweight lidar is ±3cm, the multispectral camera covers 5 spectral bands, and the obstacle avoidance accuracy of the ultrasonic radar is ±5cm. S200, based on a lightweight PSPNet-MobileNetV2 hybrid architecture model, performs pixel-level semantic segmentation on real-time collected terrain data and outputs terrain material classification labels, which include water surface, sand, tree canopy, bare rock, power transmission line and building; S300. By fusing semantic segmentation results with laser point cloud spatial topology data, terrain structure features are extracted. These features include cliffs with slopes greater than 60°, valleys with continuous negative terrain, vegetation canopies with NDVI greater than 0.7, and bare rocks with spectral reflectance greater than 0.

4. S400: Based on the combination of terrain material classification labels and terrain structure features, a preset flight mode is matched to generate dynamic control commands; the flight mode includes: When a water surface is detected and the reflectivity is higher than the threshold, the low-disturbance mode is activated, the drone's altitude is reduced to 3m and the downward-looking laser is turned off; When the tree canopy is identified and the NDVI is greater than 0.7, the penetration scanning mode is activated, the height is raised to 2m above the canopy, and multispectral penetration imaging is initiated. When bare rock is identified and the slope is greater than 30°, the high-precision reconstruction mode is activated, the gimbal tilt is adjusted to -45° and the laser scanning frequency is increased to 100Hz. When a dense area of ​​power transmission lines is identified and there is a sag gradient, the line-following obstacle avoidance mode is activated, and the plane flies along the electromagnetic field strength gradient with a lateral obstacle avoidance distance of 5m. The S500 achieves smooth switching across flight modes by constraining the height deviation between adjacent frames to no more than 20% through a double-ended queue. S600 evaluates the effectiveness of control command execution in real time during flight and corrects decision deviations based on sensor feedback data to form closed-loop control. The dynamic switching logic of flight modes in step S400 includes: When the tree canopy accounts for more than 40%, it automatically switches from ground-level mode to penetration scanning mode, and simultaneously increases the vegetation filtering intensity of the lidar by 2 times; When a sudden increase in wind speed exceeding 8m / s is detected, shorten the simulated line inspection distance to 10km and switch to the standby pressure constant altitude mode. The flight modal decision matrix is ​​constructed based on the following logic: When the terrain semantic type is identified as water surface, and its structural features are flat and its reflectivity is higher than a preset threshold, the low-disturbance flight mode is enabled. When the terrain semantic type is identified as tree canopy, and its structural features meet the requirements of NDVI value greater than 0.7 and canopy thickness exceeding the set standard, the penetration scan flight mode is enabled. When the terrain semantic type is identified as bare rock and its structural feature is a slope greater than 30°, high-precision reconstruction of flight modes is enabled. When the terrain semantic type is identified as a dense area of ​​power transmission lines and its structural features have sag gradients, the line-following obstacle-circling flight mode is activated.

2. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, In step S200, the semantic segmentation model removes 20% of redundant layers through channel pruning and compresses the model size to 0.8MB using INT8 quantization. The model introduces the FocalLoss function to address dynamic disturbances, and its expression is: ,in, This represents the predicted probability value that the current pixel belongs to a specific terrain material category. This represents the weight adjustment coefficient for the current terrain material category. This indicates a regulating factor that focuses on difficult-to-distinguish samples.

3. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, The extraction of terrain structure features in step S300 includes: The point cloud sampling density is forcibly increased by 50% in valley and cliff areas, and a three-dimensional obstacle avoidance path is generated. Based on the semantic distribution of the transmission line, the lowest point of the sag is predicted, and a counterclockwise loop path with a radius of 10m is generated to avoid electromagnetic blind spots.

4. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, The smooth switching algorithm across flight modes in step S500 constrains altitude changes using the following formula: ,in, Indicates the rate of change of height between adjacent frames. This indicates the current altitude measurement of the drone. This indicates the measured altitude of the drone at the previous moment.

5. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Asynchronous fusion methods for multi-source sensors include: The LiDAR data sampling frequency is 10Hz, and the visual data sampling frequency is 30Hz. Timestamp alignment reduces computational overhead by 35%.

6. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, The closed-loop control feedback mechanism in step S600 includes: The actual distance to obstacles is monitored in real time by ultrasonic radar. When the deviation from the preset obstacle avoidance threshold exceeds ±10%, the flight mode is rematched. In areas with dense power transmission lines, the flight heading angle is dynamically calibrated based on the electromagnetic field strength gradient direction, and the horizontal error is controlled within ±0.5m.

7. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, The joint output of terrain material classification and structural features includes: Simultaneously generate 6 types of material labels and 4 types of structural feature labels; The model inference time is less than 50ms and it is deployed on the NVIDIA Jetson TX2 embedded platform.

8. The method for autonomous inspection and terrain-following flight of an unmanned aerial vehicle according to claim 1, characterized in that, Optimized deployment in resource-constrained scenarios includes: LiDAR and visual data are asynchronously fused through timestamp alignment; Under the memory constraints of embedded devices, model pruning and quantization are used to reduce computational resource consumption.

Images