UAV flight path planning method based on semantic guidance and visual perception

Abstract

The present invention relates to a UAV flight path planning method based on semantic guidance and visual perception, comprising the following steps S1: obtaining original image information and a digital elevation model, and generating a standardized target area image; S2: inputting the standardized target area image into a trained improved U-Net semantic segmentation model, and the improved U-Net semantic segmentation model outputting pixel-level geographic feature classification results. ， S3: Based on the results of the geographic feature classification, obstacles and passable areas are determined, and a binary raster map is generated, with obstacle areas marked as 0 and passable areas as 1. S4: An initial flight trajectory is generated based on the binary raster map. If the initial flight trajectory is blocked, an online replanning algorithm is triggered to generate an alternative path. The present invention has the advantages of taking into account lightweight computing and high robustness, significantly improving flight safety and real-time decision-making capabilities in complex scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous navigation of unmanned aerial vehicles (UAVs), and in particular relates to a flight path planning method for UAVs based on semantic guidance and visual perception. Background Art

[0002] With the continuous development of the economy, technology, and public interests, the demand for autonomous drone flight in scenarios such as logistics and disaster relief is becoming increasingly urgent, but their path planning in complex environments still faces severe challenges. Traditional methods mainly rely on geometric maps and fixed-rule algorithms to achieve path planning by pre-defining obstacle positions, but they are difficult to adapt to complex and dynamically changing environments. Although existing semantic segmentation technology based on deep learning can identify some geographic features, it still has significant shortcomings in practical applications: conventional semantic segmentation models lack the ability to couple high-frequency image details with low-frequency global information, resulting in blurred segmentation boundaries and affecting the reliability of path planning. In addition, the complex structure of most models makes it difficult to run in real time on the embedded devices onboard drones, which seriously restricts the efficiency of actual deployment.

[0003] To address these issues, proposed improvements focus on two key areas: First, safety is enhanced by integrating semantic segmentation with path planning algorithms. However, existing segmentation models are limited in their ability to recognize small objects from a drone's bird's-eye view, resulting in a high rate of missed detections. Second, dynamic obstacle avoidance employs multiple sensors, but this fails to effectively incorporate semantic prior information, leading to repeated path planning and wasted time. Furthermore, the computational complexity of existing high-precision models and the inherent inefficiency of lightweight hardware remain unresolved, while the response speed of dynamic replanning is insufficient to meet the demands of real-time avoidance of unexpected obstacles. These limitations demonstrate that existing technologies still struggle to balance accuracy, efficiency, and dynamic adaptability. Therefore, simultaneously acquiring high- and low-frequency semantic information from images with high fidelity, and achieving high efficiency while maintaining high accuracy, are of great theoretical and practical value. Therefore, an innovative solution that deeply integrates semantic understanding with lightweight computing is urgently needed. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a UAV flight path planning method based on semantic guidance and visual perception, which has both lightweight computing and high robustness, and significantly improves flight safety and real-time decision-making capabilities in complex scenarios.

[0005] The technical solutions of the present invention are as follows:

[0006] The UAV flight path planning method based on semantic guidance and visual perception includes the following steps:

[0007] S1: Obtain original image information and digital elevation model, and generate standardized target area image;

[0008] S2: Input the standardized target area image into the trained improved U-Net semantic segmentation model, and the improved U-Net semantic segmentation model outputs the pixel-level geographic feature classification results;

[0009] The improved U-Net semantic segmentation model is embedded with an adaptive frequency decomposition module, which can convert multi-scale feature maps into the frequency domain and adaptively separate low-frequency components and high-frequency components using a learnable Gaussian filter. The low-frequency components focus on global geographical structures, while the high-frequency components enhance boundary details and texture features.

[0010] S3: Based on the results of geographic feature classification, obstacles and passable areas are determined and a binary raster map is generated, with obstacle areas marked as 0 and passable areas as 1;

[0011] S4: Construct a binary 3D grid map, generate an initial flight trajectory based on the map, and use the UAV-mounted vision and lidar sensors to collect environmental data in real time. Use positioning and mapping technology to build a local map and detect dynamic obstacles. If the initial flight trajectory is blocked, trigger the online replanning algorithm to generate an alternative path.

[0012] Furthermore, the original images in step S1 include high-resolution satellite images and aerial photography images, wherein the high-resolution satellite images and aerial photography images cover the terrain elevation, surface cover type, water body distribution, and geographical features of the building complex of the target area, and the original images are subjected to geometric correction, radiation correction, and multispectral fusion to generate a standardized target area image.

[0013] Furthermore, the improved U-Net semantic segmentation model training in step S2 includes the following steps:

[0014] S2.1: Design an encoder structure with five downsampling layers. Each layer includes convolution, batch normalization, and ReLU activation functions. Pooling is used to reduce the size of the feature map layer by layer. The normalized target region image is fed into the encoder structure, and the output is a multi-scale feature map.

[0015] S2.2: Design a decoder structure consisting of multiple upsampling layers and convolutional layers. Each upsampling layer is fed into an adaptive frequency decomposition module.

[0016] S2.3: Design the output layer after the adaptive frequency decomposition module, map the number of channels to the number of geographic categories, generate a semantic segmentation map, and output the pixel-level geographic feature classification results based on the semantic segmentation map.

[0017] Furthermore, the design process of the adaptive frequency decomposition module includes the following steps:

[0018] S2.2.1: Perform a two-dimensional fast Fourier transform on the multi-scale feature map X after each level of upsampling to obtain the frequency domain of the multi-scale feature map , and construct the learnable parameters as The Gaussian filter process can be expressed as:

[0019] ,

[0020] Where r is the Euclidean distance from the center of the frequency domain to each frequency point; To dynamically optimize through back-propagation, the threshold for dividing low / high frequency components is adaptively adjusted;

[0021] S2.2.2: Low-frequency component extraction preserves global semantic features through low-pass filtering: ,

[0022] Where G is the frequency domain representation of the original image, iFFTshift(*) is the inverse fast Fourier transform shift operation, which is used to move the low-frequency components in the center of the frequency domain to the corners of the matrix;

[0023] S2.2.3: High-frequency component extraction enhances edge details through high-pass filtering: ,

[0024] Among them, the frequency domain characteristic decomposition expression is: ;

[0025] S2.2.4: Low-frequency characteristic components after frequency domain decomposition and high-frequency characteristic components Perform two-dimensional inverse Fourier transform respectively and reconstruct it into spatial domain feature map:

[0026] ,

[0027] in, They represent low-frequency global structural features and high-frequency detail features respectively.

[0028] Furthermore, the design process of the adaptive frequency decomposition module further includes the following steps:

[0029] S2.2.5: Optimize the low-frequency global structural features and high-frequency detail features through two independent convolutional layers to obtain the optimized low-frequency features and high-frequency features , ,

[0030] in, is a 3×3 convolution kernel;

[0031] S2.2.6: The optimized low-frequency features and high-frequency features Perform the fusion by element-wise addition: ;

[0032] S2.2.7: Apply a 3×3 convolutional layer to the fused features to eliminate local discontinuities caused by the frequency-to-spatial conversion: ,

[0033] The final output feature map is , where B is the batch size of training, C is the number of channels, H and W are the spatial height and width of the feature map;

[0034] S2.2.8: By using weighted cross entropy loss The loss function is designed to train the network on the final output feature map and combine the Dice coefficient loss To enhance boundary alignment, the loss function is designed as:

[0035] ,in, 、 is the loss adjustment item.

[0036] Furthermore, the initial flight trajectory generation in step S4 includes the following steps:

[0037] S4.1: Fuse the binary raster map generated by S3 with the 3D point cloud map and digital elevation model captured by the sensor to construct a binary 3D raster map. , and store the altitude value for calculating flight energy consumption.

[0038] S4.2: Using the A* algorithm, we search for the global optimal path in the traversable area of the binary 3D grid map based on the cost function. The core mechanism is to define the comprehensive cost of the node. ;

[0039] in represents the actual cost from the starting point to the current node n, is the heuristic function;

[0040] When the node comprehensive cost is initialized, the starting point is added to the open list, and an empty closed list is created at the same time, and then the loop expansion process is entered;

[0041] In each iteration, the A* algorithm selects from the open list The smallest node is used as the current expansion point. Its 8-neighborhood grid is traversed and the cost value of each neighboring node is calculated. If the neighboring node is not marked as an obstacle and does not exist in the closed list, its parent node pointer and cost value are updated and added to the open list. If the neighboring node already exists in the open list but the new path has a lower cost, the cost value and parent node are updated synchronously.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. This invention embeds an adaptive frequency decomposition module (AFDM) in the decoder of the improved U-Net semantic segmentation model, using a learnable Gaussian filter to dynamically separate low-frequency and high-frequency components to enhance the accuracy of geographic obstacle recognition. It also combines a binary raster map with the A* algorithm to generate a global optimal path, and designs a hierarchical replanning mechanism. Using onboard vision and lidar to perceive environmental changes in real time, the flight path is dynamically adjusted to avoid unexpected obstacles.

[0044] In summary, the present invention has the advantages of balancing lightweight computing with high robustness, significantly improving flight safety and real-time decision-making capabilities in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] like Figure 1 As shown in the figure, the UAV flight path planning method based on semantic guidance and visual perception includes the following steps:

[0048] S1: Acquire high-resolution satellite images, aerial photography images, and digital elevation models. The high-resolution satellite images and aerial photography images cover the target area's terrain elevation, surface cover type, water distribution, building complexes, and other geographical features, such as buildings, dense forests, and rivers. Perform geometric correction, radiometric correction, and multispectral fusion on the original images (high-resolution satellite images and aerial photography images) to generate a standardized image of the target area.

[0049] S2: Input the standardized target area image into the trained improved U-Net semantic segmentation model to output the pixel-level geographic feature classification results.

[0050] S3: Based on the results of geographic feature classification, obstacles and passable areas are determined, and a binary raster map is generated, with obstacle areas marked as 0 and passable areas as 1.

[0051] S4: Construct a binary three-dimensional grid map. Based on the A* algorithm, calculate the global optimal path from the starting point to the end point within the traversable area for the constructed binary three-dimensional grid map. Comprehensively evaluate the flight distance, terrain undulation and energy consumption cost to generate an initial flight trajectory that takes into account both efficiency and safety. Use the UAV-mounted vision and lidar sensors to collect environmental data in real time. Use the SLAM positioning and mapping technology to build a local map and detect dynamic obstacles. If the original path is blocked, trigger the online replanning algorithm to generate an alternative path to ensure mission continuity.

[0052] Among them, the online replanning algorithm returns to step S5 to recalculate the path when the original path is blocked, and marks the blocked location as 0 (obstacle area);

[0053] In this embodiment, the improved U-Net semantic segmentation model training in step S2 includes the following steps:

[0054] S2.1: Design the encoder structure, which includes 5 levels of downsampling layers, each level contains convolution ( ), batch normalization (BN) and ReLU activation function, and reduce the size of the feature map layer by layer through pooling operation, input the standardized target area image into the encoder structure and output a multi-scale feature map;

[0055] S2.2: Design a decoder architecture consisting of multiple upsampling and convolutional layers. Each upsampling level is followed by an adaptive frequency decomposition module (AFDM) to enhance feature boundaries and global semantic information.

[0056] The Adaptive Frequency Decomposition Module (AFDM) enhances feature expression capabilities through dynamic decomposition and fusion mechanisms in the frequency domain. Its core functions are: after converting multi-scale feature maps to the frequency domain, it uses a learnable Gaussian filter to adaptively divide low-frequency and high-frequency components. The low-frequency components focus on the global geographic structure and suppress background noise; the high-frequency components enhance boundary details and texture features, solving the problem of balancing global and local features in spatial domain methods.

[0057] The design process of the AFDM module is as follows:

[0058] First, the multi-scale feature map X after each level of upsampling is subjected to a two-dimensional fast Fourier transform to obtain the frequency domain of the multi-scale feature map , and construct the learnable parameters as The Gaussian filter process can be expressed as:

[0059] ,

[0060] Where r is the Euclidean distance from the center of the frequency domain to each frequency point, The threshold for segmenting low / high frequency components is adaptively adjusted through dynamic optimization via back-propagation.

[0061] The frequency band separation step can be divided into low-frequency component extraction and high-frequency component extraction, where low-frequency component extraction retains global semantic features through a low-pass filter:

[0062] ,

[0063] G is the frequency domain representation of the original image (the complex matrix after Fourier transform), and iFFTshift(*) is the inverse fast Fourier transform shift operation, which is used to move the low-frequency components in the center of the frequency domain to the corners of the matrix;

[0064] The high-frequency component extraction enhances edge details through a high-pass filter: ,

[0065] Among them, the frequency domain characteristic decomposition expression is: ,

[0066] Secondly, the low / high frequency characteristic components after frequency domain decomposition and Perform two-dimensional inverse Fourier transform (iFFT2) respectively and reconstruct it into spatial domain feature map:

[0067] ,

[0068] in, They represent low-frequency global structural features and high-frequency detail features respectively.

[0069] Furthermore, the two-way features are optimized through two independent convolutional layers to obtain the optimized low-frequency features and high-frequency features : ,

[0070] in, It is a 3×3 convolution kernel, which is used to suppress the artifacts introduced by frequency band separation and enhance the semantic consistency of low-frequency areas and the boundary sharpness of high-frequency areas.

[0071] The optimized low / high frequency features are fused by element-by-element addition: ,

[0072] This operation retains the global context information of low-frequency features while introducing the fine texture of high-frequency features to achieve complementary feature expression. A 3×3 convolution layer is applied to the fused features to eliminate local discontinuities caused by the frequency-spatial domain conversion: ,

[0073] The final output feature map is , where B is the training batch size, C is the number of channels, and H and W are the spatial height and width of the feature map. This operation ensures that the output feature map has both low-frequency and high-frequency advantages and feature compatibility. This means that it preserves the overall shape of the geographic object while enhancing edge details (such as road boundaries and vegetation outlines), reducing blur and missegmentation. Furthermore, it aligns with the U-Net skip connection feature scale and supports multi-level semantic fusion.

[0074] By adopting weighted cross entropy loss The loss function is designed to train the network on the final output feature map to alleviate the category imbalance problem and combine the Dice loss To enhance boundary alignment, the loss function is designed as: ,

[0075] in, 、 is the loss adjustment item.

[0076] S2.3: After the AFDM module is designed, the output layer is designed to map the number of channels to the number of geographic categories and generate a semantic segmentation map.

[0077] Furthermore, the path planning in step S5 includes the following steps:

[0078] S4.1: Fuse the binary raster map generated by S3 with the 3D point cloud map and digital elevation model captured by the sensor to construct a binary 3D raster map. , and stores the elevation value for calculating flight energy consumption. Among them, the obstacle grid is marked as 0, including no-fly areas such as buildings, dense forests, and water areas. The passable grid is marked as 1, and the elevation value is stored for calculating flight energy consumption.

[0079] S4.2: Use the A* algorithm, which is a heuristic search algorithm. It is an algorithm that finds the lowest cost path for a path with multiple nodes in a map or space. It searches for the global optimal path in a binary grid map based on a cost function. Its core mechanism is to define the node comprehensive cost. ,in represents the actual cost from the starting point to the current node n (including the flight distance and energy consumption caused by terrain undulations), is a heuristic function (usually using Euclidean distance to estimate the remaining cost from a node to the end point). When the algorithm is initialized, the starting point is added to the open list (the pool of nodes to be expanded), and an empty closed list (the pool of processed nodes) is created at the same time. Then, the cyclic expansion process begins.

[0080] In each iteration, the algorithm selects from the open list The smallest node is used as the current expansion point, and its 8-neighborhood grid is traversed to calculate the cost value of each neighboring node. If the neighboring node is not marked as an obstacle and does not exist in the closed list, its parent node pointer and cost value are updated and added to the open list; if it already exists in the open list but the new path has a lower cost, the cost value and parent node are updated simultaneously. In this process, the heuristic function The introduction of significantly narrows the search scope, avoids brute force traversal of all feasible nodes, and takes into account both efficiency and optimality.

[0081] The algorithm terminates when the expanded node reaches its endpoint or the open list is exhausted. If the endpoint is successfully reached, a globally optimal path is generated by backtracking through the parent node chain. If the open list is cleared prematurely, no feasible path is considered. The resulting path strictly avoids all pre-set obstacles and takes into account flight distance, terrain complexity, and energy consumption, ensuring safe and efficient navigation in complex geographical environments.

[0082] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A UAV flight path planning method based on semantic guidance and visual perception, characterized by: The steps include: S1: Obtain original image information and digital elevation model, and generate standardized target area image; S2: Input the standardized target area image into the trained improved U-Net semantic segmentation model, and the improved U-Net semantic segmentation model outputs the pixel-level geographic feature classification results; The improved U-Net semantic segmentation model is embedded with an adaptive frequency decomposition module, which can convert multi-scale feature maps into the frequency domain and adaptively separate low-frequency components and high-frequency components using a learnable Gaussian filter. The low-frequency components focus on global geographical structures, while the high-frequency components enhance boundary details and texture features. S3: Based on the results of geographic feature classification, obstacles and passable areas are determined and a binary raster map is generated, with obstacle areas marked as 0 and passable areas as 1; S4: Construct a binary 3D grid map, generate an initial flight trajectory based on the map, and use the UAV-mounted vision and lidar sensors to collect environmental data in real time. Use positioning and mapping technology to build a local map and detect dynamic obstacles. If the initial flight trajectory is blocked, trigger the online replanning algorithm to generate an alternative path.

2. The method for UAV flight path planning based on semantic guidance and visual perception according to claim 1 is characterized in that: The original images in step S1 include high-resolution satellite images and aerial photography images, wherein the high-resolution satellite images and aerial photography images cover the terrain elevation, surface cover type, water body distribution, and geographical features of the building complex of the target area, and the original images are subjected to geometric correction, radiation correction, and multispectral fusion to generate a standardized target area image.

3. The method for UAV flight path planning based on semantic guidance and visual perception according to claim 1, characterized in that: The training of the improved U-Net semantic segmentation model includes the following steps: S2.1: Design an encoder structure with five downsampling layers. Each layer includes convolution, batch normalization, and ReLU activation functions. Pooling is used to reduce the size of the feature map layer by layer. The normalized target region image is fed into the encoder structure, and the output is a multi-scale feature map. S2.2: Design a decoder structure consisting of multiple upsampling layers and convolutional layers. Each upsampling layer is fed into an adaptive frequency decomposition module. S2.3: Design the output layer after the adaptive frequency decomposition module, map the number of channels to the number of geographic categories, generate a semantic segmentation map, and output the pixel-level geographic feature classification results based on the semantic segmentation map.

4. The method for UAV flight path planning based on semantic guidance and visual perception according to claim 1, characterized in that: The design process of the adaptive frequency decomposition module includes the following steps: S2.2.1: Perform a two-dimensional fast Fourier transform on the multi-scale feature map X after each level of upsampling to obtain the frequency domain of the multi-scale feature map , and construct the learnable parameters as The Gaussian filter process can be expressed as: ， Where r is the Euclidean distance from the center of the frequency domain to each frequency point; To dynamically optimize through back-propagation, the threshold for dividing low / high frequency components is adaptively adjusted; S2.2.2: Low-frequency component extraction preserves global semantic features through low-pass filtering: , Where G is the frequency domain representation of the original image, iFFTshift(*) is the inverse fast Fourier transform shift operation, which is used to move the low-frequency components in the center of the frequency domain to the corners of the matrix; S2.2.3: High-frequency component extraction enhances edge details through high-pass filtering: , Among them, the frequency domain characteristic decomposition expression is: ; S2.2.4: Low-frequency characteristic components after frequency domain decomposition and high-frequency characteristic components Perform two-dimensional inverse Fourier transform respectively and reconstruct it into spatial domain feature map: ， in, They represent low-frequency global structural features and high-frequency detail features respectively.

5. The method for UAV flight path planning based on semantic guidance and visual perception according to claim 4, characterized in that: The design process of the adaptive frequency decomposition module also includes the following steps: S2.2.5: Optimize the low-frequency global structural features and high-frequency detail features through two independent convolutional layers to obtain the optimized low-frequency features and high-frequency features : ， in, is a 3×3 convolution kernel; S2.2.6: The optimized low-frequency features and high-frequency features Perform the fusion by element-wise addition: ; S2.2.7: Apply a 3×3 convolutional layer to the fused features to eliminate local discontinuities caused by the frequency-to-spatial conversion: ; The final output feature map is , where B is the batch size of training, C is the number of channels, H and W are the spatial height and width of the feature map; S2.2.8: By using weighted cross entropy loss The loss function is designed to train the network on the final output feature map and combine the Dice coefficient loss To enhance boundary alignment, the loss function is designed as: ， in, 、 is the loss adjustment item.

6. The method for UAV flight path planning based on semantic guidance and visual perception according to claim 1, characterized in that: The initial flight trajectory generation in step S4 includes the following steps: S4.1: Fuse the binary raster map generated by S3 with the 3D point cloud map and digital elevation model captured by the sensor to construct a binary 3D raster map. , and store the altitude value for calculating flight energy consumption; S4.2: Using the A* algorithm, we search for the global optimal path in the traversable area of the binary 3D grid map based on the cost function. The core mechanism is to define the comprehensive cost of the node. ; in represents the actual cost from the starting point to the current node n, is the heuristic function; When the node comprehensive cost is initialized, the starting point is added to the open list, and an empty closed list is created at the same time, and then the loop expansion process is entered; In each iteration, the A* algorithm selects from the open list The smallest node is used as the current expansion point. Its 8-neighborhood grid is traversed and the cost value of each neighboring node is calculated. If the neighboring node is not marked as an obstacle and does not exist in the closed list, its parent node pointer and cost value are updated and added to the open list. If the neighboring node already exists in the open list but the new path has a lower cost, the cost value and parent node are updated synchronously.

Images