Unmanned aerial vehicle flight path planning method based on semantic guidance and visual perception

Abstract

The invention relates to an unmanned aerial vehicle flight path planning method based on semantic guidance and visual perception, and the method comprises the following steps: S1, obtaining original image information and a digital elevation model, and generating a standardized target region image, S2, inputting the standardized target region image into a trained improved U-Net semantic segmentation model, the improved U-Net semantic segmentation model outputs a pixel-level geographic feature classification result, S3, according to the geographic feature classification result, determining an obstacle and a passable area, and generating a binary grid map, the obstacle area being marked as 0, and the passable area being 1, and S4, generating an initial flight path based on the binary grid map, and if the initial flight path is blocked, executing the step S3. Triggering an online re-planning algorithm to generate an alternative path; the method has the advantages that lightweight calculation and high robustness are both considered, and the flight safety and the real-time decision-making capability in a complex scene are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) autonomous navigation, and particularly relates to a method for UAV flight path planning based on semantic guidance and visual perception. Background Art

[0002] With the continuous development of economy, technology and the needs of people's interests, the demand for autonomous flight of UAVs in scenarios such as logistics transportation and disaster rescue is becoming increasingly urgent. However, path planning in complex environments still faces severe challenges. Traditional methods mainly rely on geometric maps and fixed-rule algorithms to achieve path planning by predefining the positions of obstacles, but it is difficult to adapt to complex and dynamically changing environments. Although existing deep learning-based semantic segmentation technologies can identify some geographical features, there are still obvious defects in practical applications: conventional semantic segmentation models have insufficient ability to couple and process high-frequency details and low-frequency global information of images, resulting in blurred segmentation boundaries, which in turn affects the reliability of path planning. In addition, most model structures are complex and difficult to run in real time on UAV on-board embedded devices, severely restricting the actual deployment efficiency.

[0003] In response to the above problems, the proposed improvement solutions mainly focus on two aspects: one is to improve safety by integrating semantic segmentation and path planning algorithms. However, due to the insufficient ability of existing segmentation models to identify small targets from the top-down view of UAVs, the missed detection rate remains high; the other is to introduce multi-sensors to achieve dynamic obstacle avoidance, but semantic prior information is not effectively combined, resulting in repeated path planning and wasting time. In addition, the contradiction between the large computational amount of existing high-precision models and lightweight hardware has not been solved, and the response speed of dynamic replanning is also difficult to meet the real-time avoidance requirements of sudden obstacles. These limitations indicate that existing technologies still cannot balance accuracy, efficiency and dynamic adaptability. Therefore, how to simultaneously obtain high- and low-frequency semantic information of images with high fidelity; how to balance high efficiency with high precision has extremely high theoretical and practical value. Therefore, there is an urgent need for an innovative solution that deeply integrates semantic understanding and lightweight computing. Summary of the Invention

[0004] The object of the present invention is to overcome the deficiencies of the prior art and provide a method for UAV flight path planning based on semantic guidance and visual perception, which takes into account lightweight computing and high robustness, and significantly improves flight safety and real-time decision-making ability in complex scenarios.

[0005] The technical solution of the present invention is as follows: A method for UAV flight path planning based on semantic guidance and visual perception, comprising the following steps: S1: Obtain the original image information and digital elevation model, and generate a standardized target area image; S2: Input the standardized target area image into the trained improved U-Net semantic segmentation model, and the improved U-Net semantic segmentation model outputs the pixel-level geographical feature classification result; An adaptive frequency decomposition module is embedded in the improved U-Net semantic segmentation model. The adaptive frequency decomposition module can transform the multi-scale feature map into the frequency domain and adaptively divide the low-frequency component and the high-frequency component by using a learnable Gaussian filter, where the low-frequency component focuses on the global geographical structure and the high-frequency component strengthens the boundary details and texture features; S3: According to the geographical feature classification result, judge the obstacles and the passable area, and generate a binary grid map, where the obstacle area is marked as 0 and the passable area is 1; S4: Construct a binary three-dimensional grid map, generate an initial flight trajectory based on the binary three-dimensional grid map, and use the vision and lidar sensors carried by the unmanned aerial vehicle to collect environmental data in real time. Construct a local map through the positioning and mapping technology and detect dynamic obstacles. If the initial flight trajectory is blocked, trigger the online replanning algorithm to generate an alternative path.

[0006] Furthermore, the original images in step S1 include high-resolution satellite images and aerial photography images. The high-resolution satellite images and aerial photography images cover the terrain elevation, surface cover type, water body distribution, and building group geographical features of the target area. After geometric correction, radiometric correction, and multi-spectral fusion of the original images, a standardized target area image is generated.

[0007] Furthermore, the training of the improved U-Net semantic segmentation model in step S2 includes the following steps: S2.1: Design an encoder structure, including 5 levels of downsampling layers. Each level includes convolution, batch normalization, and ReLU activation functions, and the size of the feature map is gradually reduced layer by layer through pooling operations. After inputting the standardized target area image into the encoder structure, a multi-scale feature map is output; S2.2: Design a decoder structure, including multiple upsampling layers and convolution layers. After each level of upsampling, it is input into the adaptive frequency decomposition module; S2.3: Design an output layer after the adaptive frequency decomposition module, map the number of channels to the number of geographical categories, generate a semantic segmentation map, and output the pixel-level geographical feature classification result based on the semantic segmentation map.

[0008] Furthermore, the design process of the adaptive frequency decomposition module includes the following steps: S2.2.1: Perform a two-dimensional fast Fourier transform on the multi-scale feature map X after each level of upsampling to obtain the frequency domain of the multi-scale feature map , and construct a Gaussian filter with learnable parameters , and the process can be expressed as: , where r is the Euclidean distance from the center of the frequency domain to each frequency point; is adaptively adjusted by dynamic optimization through backpropagation to adjust the division threshold of low / high frequency components; S2.2.2: Low-frequency component extraction retains global semantic features through a low-pass filter: , where G is the frequency domain representation of the original image, and iFFTshift(*) is the inverse fast Fourier transform shift operation, which is used to move the low-frequency components at the center of the frequency domain to the corners of the matrix; S2.2.3: High-frequency component extraction enhances edge details through a high-pass filter: , where the frequency domain feature decomposition expression is: ; S2.2.4: The low-frequency feature components and high-frequency feature components after frequency domain decomposition are respectively subjected to two-dimensional inverse Fourier transform to reconstruct into spatial domain feature maps: , where represent the low-frequency global structure feature and the high-frequency detail feature respectively.

[0009] Furthermore, the design process of the adaptive frequency decomposition module further includes the following steps: S2.2.5: Optimize the low-frequency global structure feature and the high-frequency detail feature through two independent convolutional layers to obtain the optimized low-frequency feature and high-frequency feature , , where is a 3×3 convolutional kernel; S2.2.6: Fuse the optimized low-frequency feature and high-frequency feature in an element-wise addition manner: ; S2.2.7: Apply a 3×3 convolutional layer to the fused feature to eliminate local discontinuities caused by the frequency domain-spatial domain conversion: , to obtain the final output feature map as , where B is the batch size of training, C is the number of channels, and H, W are the spatial height and width of the feature map; S2.2.8: Design a loss function by using the weighted cross-entropy loss to train the network for the final output feature map, and combine the Dice coefficient loss To enhance boundary alignment, the loss function is designed as: ,in, , is the loss adjustment item.

[0010] Further, the initial flight trajectory generation in step S4 includes the following steps: S4.1: Fuse the binary raster map generated by S3 with the three-dimensional point cloud map and digital elevation model captured by the sensor to construct a binary three-dimensional raster map , and store the altitude value for calculating flight energy consumption.

[0011] S4.2: Using the A* algorithm, we search for the global optimal path in the traversable area of ​​the binary 3D grid map based on the cost function. The core mechanism is to define the comprehensive cost of the node. ; in represents the actual cost from the starting point to the current node n, is the heuristic function; When the node comprehensive cost is initialized, the starting point is added to the open list, and an empty closed list is created, and then the loop expansion process is entered; In each iteration, the A* algorithm selects from the open list The smallest node is used as the current expansion point. Its 8-neighborhood grid is traversed and the cost value of each neighboring node is calculated. If the neighboring node is not marked as an obstacle and does not exist in the closed list, its parent node pointer and cost value are updated and added to the open list. If it already exists in the open list but the new path has a lower cost, the cost value and parent node are updated synchronously.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention embeds an adaptive frequency decomposition module (AFDM) in the decoder of the improved U-Net semantic segmentation model, and uses a learnable Gaussian filter to dynamically separate low-frequency components and high-frequency components to enhance the accuracy of geographic obstacle recognition. It combines the binary raster map with the A* algorithm to generate a global optimal path, and designs a hierarchical replanning mechanism to perceive environmental changes in real time through airborne vision and lidar, and dynamically adjust the flight path to avoid sudden obstacles.

[0013] In summary, the present invention has the advantages of balancing lightweight computing and high robustness, significantly improving flight safety and real-time decision-making capabilities in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0015] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0016] As Figure 1 shown, the method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception includes the following steps: S1: Obtain high-resolution satellite images, aerial photography images, and digital elevation models. Among them, the high-resolution satellite images and aerial photography images cover various geographical features such as the terrain elevation, surface cover type, water body distribution, and building complex of the target area, such as buildings, dense forests, rivers, etc. Geometric correction, radiometric correction, and multispectral fusion are performed on the original images (high-resolution satellite images, aerial photography images) to generate a standardized target area image.

[0017] S2: Input the standardized target area image into the trained improved U-Net semantic segmentation model, and output the pixel-level geographical feature classification result.

[0018] S3: According to the geographical feature classification result, judge the obstacles and the passage area, and generate a binary grid map, where the obstacle area is marked as 0 and the passable area is 1.

[0019] S4: Construct a binary three-dimensional grid map. Based on the A* algorithm, for the formed binary three-dimensional grid map, calculate the global optimal path from the starting point to the ending point within the passable area, comprehensively evaluate the flight distance, terrain undulation degree, and energy consumption cost, generate an initial flight trajectory that takes into account both efficiency and safety. The unmanned aerial vehicle is equipped with vision and lidar sensors to collect environmental data in real time, and use the simultaneous localization and mapping technology SLAM to construct a local map and detect dynamic obstacles. If the original path is blocked, trigger the online replanning algorithm to generate an alternative path to ensure the continuity of the task.

[0020] Among them, the online replanning algorithm is to return to step S5 to recalculate the path when the original path is blocked, and mark the blocked location as 0 (obstacle area); In this embodiment, the training of the improved U-Net semantic segmentation model in step S2 includes the following steps: S2.1: Design an encoder structure, which includes 5 levels of downsampling layers. Each level includes a convolution ( ), batch normalization (BN), and ReLU activation function, and gradually reduce the size of the feature map through pooling operations. After inputting the standardized target area image into the encoder structure, output a multi-scale feature map; S2.2: Design the decoder structure, which includes multiple upsampling layers and convolutional layers. After each level of upsampling, an Adaptive Frequency Decomposition Module (AFDM) is connected to enhance the feature boundaries and global semantic information. The Adaptive Frequency Decomposition Module (AFDM) enhances the feature representation ability through a frequency-domain dynamic decomposition and fusion mechanism. Its core function is: after converting the multi-scale feature map to the frequency domain, it adaptively divides the low-frequency and high-frequency components using a learnable Gaussian filter. The low-frequency components focus on the global geographical structure and suppress background noise; the high-frequency components strengthen the boundary details and texture features, solving the problem that it is difficult to balance global and local features in the spatial-domain method.

[0021] The design process of the AFDM module is as follows: First, perform a two-dimensional fast Fourier transform on the multi-scale feature map X after each level of upsampling to obtain the frequency domain of the multi-scale feature map and construct a Gaussian filter with learnable parameters . The process can be expressed as: , where r is the Euclidean distance from the frequency-domain center to each frequency point, is dynamically optimized through backpropagation to adaptively adjust the division threshold of low / high-frequency components.

[0022] The frequency band separation step can be divided into low-frequency component extraction and high-frequency component extraction. Among them, low-frequency component extraction retains the global semantic features through a low-pass filter: , where G is the frequency-domain representation of the original image (a complex matrix after Fourier transform), and iFFTshift(*) is the inverse fast Fourier transform shift operation to move the low-frequency component at the frequency-domain center to the corner of the matrix; while high-frequency component extraction strengthens the edge details through a high-pass filter: , where the frequency-domain feature decomposition expression is: , Secondly, perform two-dimensional inverse Fourier transform (iFFT2) on the low / high-frequency feature components and after frequency-domain decomposition respectively to reconstruct them into spatial-domain feature maps: , where represent the low-frequency global structure feature and the high-frequency detail feature respectively.

[0023] Furthermore, optimize the two-way features through two independent convolutional layers to obtain the optimized low-frequency feature and high-frequency feature : , in, It is a 3×3 convolution kernel, which is used to suppress the artifacts introduced by frequency band separation and enhance the semantic consistency of low-frequency areas and the boundary sharpness of high-frequency areas.

[0024] The optimized low / high frequency features are fused by element-by-element addition: , This operation retains the global context information of low-frequency features, while introducing the fine texture of high-frequency features to achieve complementary feature expression, and applies a 3×3 convolution layer to the fused features to eliminate local discontinuities caused by frequency-spatial conversion: , The final output feature map is , where B is the batch size of the training, C is the number of channels, and H and W are the spatial height and width of the feature map. This operation enables the output feature map to have both low-frequency advantages, high-frequency advantages, and feature compatibility, that is, it retains the overall shape of the geographic target while strengthening edge details (such as road boundaries and vegetation contours), reducing blur and mis-segmentation, and aligning with the U-Net jump connection feature scale to support multi-level semantic fusion.

[0025] By adopting weighted cross entropy loss The loss function is designed to train the network on the final output feature map to alleviate the category imbalance problem and combine the Dice loss To enhance boundary alignment, the loss function is designed as: , in, , is the loss adjustment item.

[0026] S2.3: After the AFDM module is designed, the output layer is designed to map the number of channels to the number of geographic categories and generate a semantic segmentation map.

[0027] Further, the path planning in step S5 includes the following steps: S4.1: Fuse the binary raster map generated by S3 with the three-dimensional point cloud map and digital elevation model captured by the sensor to construct a binary three-dimensional raster map , and store the elevation value for calculating flight energy consumption, where: the obstacle grid is marked as 0, including buildings, dense forests, water areas and other no-fly areas. The passable grid is marked as 1, and the elevation value is stored for calculating flight energy consumption.

[0028] S4.2: The A* algorithm is adopted. As one of the heuristic search algorithms, the A* search algorithm is an algorithm that finds the lowest passing cost for a path with multiple nodes in a map or space, and searches for the global optimal path in the binary raster map based on the cost function. Its core mechanism is to define the comprehensive cost of the node , where represents the actual cost from the starting point to the current node n (including the energy consumption caused by the flight distance and terrain undulation), is the heuristic function (usually using the Euclidean distance to estimate the remaining cost from the node to the end point). When the algorithm is initialized, the starting point is added to the open list (the pool of nodes to be expanded), and at the same time, an empty closed list (the pool of processed nodes) is created, and then the loop expansion process is entered.

[0029] In each iteration, the algorithm selects the node with the smallest cost in the open list as the current expansion point, traverses its 8-neighborhood raster, and calculates the cost values of each neighboring node. If the neighboring node is not marked as an obstacle and does not exist in the closed list, then update its parent node pointer and cost value, and add it to the open list; if it already exists in the open list but the new path cost is lower, then synchronously update the cost value and the parent node. In this process, the introduction of the heuristic function significantly reduces the search scope, avoids brute-force traversal of all feasible nodes, and balances efficiency and optimality.

[0030] When the expanded node reaches the end point or the open list is exhausted, the algorithm terminates. If it successfully reaches the end point, the global optimal path is generated by backtracking the parent node chain; if the open list is emptied in advance, it is determined that there is no feasible path. The finally output path strictly avoids all preset obstacles, and comprehensively considers factors such as flight distance, terrain complexity, and energy consumption to ensure the safe and efficient navigation of the UAV in a complex geographical environment.

[0031] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception, characterized in that: It includes the following steps: S1: Obtain the original image information and digital elevation model, and generate a standardized target area image; S2: Input the standardized target area image into the trained improved U-Net semantic segmentation model, and the improved U-Net semantic segmentation model outputs the pixel-level geographical feature classification result; An adaptive frequency decomposition module is embedded in the improved U-Net semantic segmentation model. The adaptive frequency decomposition module can transform the multi-scale feature map to the frequency domain and adaptively divide the low-frequency component and the high-frequency component by using a learnable Gaussian filter. Among them, the low-frequency component focuses on the global geographical structure, and the high-frequency component strengthens the boundary details and texture features; S3: According to the geographical feature classification result, judge the obstacles and the passable area, and generate a binary grid map. The obstacle area is marked as 0, and the passable area is 1; S4: Construct a binary three-dimensional grid map, generate an initial flight trajectory based on the binary three-dimensional grid map, and use the vision and lidar sensors carried by the unmanned aerial vehicle to collect environmental data in real time. Construct a local map through the positioning and mapping technology and detect dynamic obstacles. If the initial flight trajectory is blocked, trigger the online replanning algorithm to generate an alternative path.

2. The method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception according to claim 1, wherein: The original images in step S1 include high-resolution satellite images and aerial photography images. The high-resolution satellite images and aerial photography images cover the terrain elevation, surface cover type, water body distribution, and geographical features of the building group in the target area. After geometric correction, radiometric correction, and multispectral fusion of the original images, a standardized target area image is generated.

3. The method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception according to claim 1, wherein: The training of the improved U-Net semantic segmentation model includes the following steps: S2.1: Design the encoder structure, including 5 levels of downsampling layers. Each level includes convolution, batch normalization, and ReLU activation functions, and the size of the feature map is gradually reduced through pooling operations. After inputting the standardized target area image into the encoder structure, a multi-scale feature map is output; S2.2: Design the decoder structure, including multiple upsampling layers and convolution layers. After each level of upsampling, it is input into the adaptive frequency decomposition module; S2.3: Design an output layer after the adaptive frequency decomposition module, map the number of channels to the number of geographical categories, generate a semantic segmentation map, and output the pixel-level geographical feature classification result based on the semantic segmentation map.

4. The method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception according to claim 1, wherein: The design process of the adaptive frequency decomposition module includes the following steps: S2.2.1: Perform a two-dimensional fast Fourier transform on the multi-scale feature map X after each level of upsampling to obtain the frequency domain of the multi-scale feature map , and construct a Gaussian filter with learnable parameters . The process can be expressed as: ， where r is the Euclidean distance from the center of the frequency domain to each frequency point; is to dynamically optimize through backpropagation and adaptively adjust the division threshold of low / high frequency components; S2.2.2: The extraction of low-frequency components retains the global semantic features through a low-pass filter: , Where G is the frequency domain representation of the original image, and iFFTshift(*) is the inverse fast Fourier transform shift operation, which is used to move the center low-frequency component of the frequency domain to the corner of the matrix; S2.2.3: High-frequency component extraction enhances edge details through a high-pass filter: , Among them, the frequency-domain feature decomposition expression is: ; S2.2.4: Perform two-dimensional inverse Fourier transforms on the low-frequency feature components and high-frequency feature components after frequency-domain decomposition respectively to reconstruct them into spatial-domain feature maps: and high-frequency feature components respectively, and reconstruct them into spatial-domain feature maps: ， Among them, respectively represent low-frequency global structural features and high-frequency detail features.

5. The method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception according to claim 4, characterized in that: The design process of the adaptive frequency decomposition module also includes the following steps: S2.2.5: Optimize the low-frequency global structure features and high-frequency detail features through two independent convolutional layers to obtain the optimized low-frequency features and high-frequency features : ， Among them, is a 3×3 convolution kernel; S2.2.6: Fuse the optimized low-frequency features and high-frequency features in an element-wise addition manner: ; S2.2.7: Apply a 3×3 convolutional layer to the fused features to eliminate local discontinuities caused by the frequency-domain to spatial-domain conversion: ; The final output feature map obtained is , where B is the training batch size, C is the number of channels, and H and W are the spatial height and width of the feature map; S2.2.8: The network training of the final output feature map is carried out by adopting the loss function design of weighted cross-entropy loss, and the Dice coefficient loss is combined to enhance the boundary alignment. The loss function is designed as: ​ ， Among them, , is the loss adjustment term.

6. The method for unmanned aerial vehicle flight path planning based on semantic guidance and visual perception according to claim 1, wherein: The generation of the initial flight trajectory in step S4 includes the following steps: S4.1: Fuse the binary raster map generated in S3 with the three-dimensional point cloud map and digital elevation model captured by the sensor to construct a binary three-dimensional raster map , and store the elevation value for calculating flight energy consumption; S4.2: Use the A* algorithm to search for the global optimal path within the passable area of the binary three-dimensional grid map based on the cost function. Its core mechanism is to define the comprehensive cost of nodes ; Among them represents the actual cost from the starting point to the current node n is the heuristic function When initializing the comprehensive cost of the node, add the starting point to the open list, and at the same time create an empty closed list, and then enter the loop expansion process; In each iteration, the A* algorithm selects the node with the smallest value from the open list as the current expansion point, traverses its 8-neighborhood grid, and calculates the cost values of each neighboring node. If the neighboring node is not marked as an obstacle and does not exist in the closed list, then update its parent node pointer and cost value, and add it to the open list; if it already exists in the open list but the new path cost is lower, then synchronously update the cost value and the parent node.

Images