Unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in non-visual state

Abstract

The invention belongs to the technical field of unmanned aerial vehicle path adjustment, and particularly relates to an unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in a non-visual state, and the method comprises the steps: generating a field three-dimensional basic model; acquiring three-dimensional space data in a task area of the unmanned aerial vehicle and point cloud data in front of a flight path; acquiring a flight path feature sequence table; respectively obtaining an optimal reference path vector and an instant avoidance path vector; outputting a flight path vector for guiding the unmanned aerial vehicle to execute path adjustment by using the path decision model; simulating a non-visual flight scene on the on-site three-dimensional basic model, and analyzing the flight path vector into a flight control instruction for driving the unmanned aerial vehicle to execute; calculating a path adjustment amplitude based on the optimal reference path vector and the flight path vector; and updating the field three-dimensional basic model, and displaying the flight path of the unmanned aerial vehicle in a three-dimensional form. According to the invention, the flight safety and path adaptive ability of the unmanned aerial vehicle in a complex non-visual environment are significantly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) path adjustment, and in particular relates to a UAV flight path adjustment method based on obstacle avoidance in a non-visual state. Background Art

[0002] Drone flight path adjustment refers to the real-time correction and adjustment of the drone's planned flight path through a dynamic route optimization algorithm based on the actual environment of the infrastructure construction site in the uninhabited area (the main application scenario of this invention). This process uses multi-sensor fusion data and combines SLAM technology to achieve precise positioning and map construction of the drone, ensuring that the drone can still fly safely and efficiently in complex and low-visibility environments. By real-time monitoring and adjusting the flight path, the drone can better adapt to on-site changes and improve inspection efficiency and safety.

[0003] To address the issue of ensuring the safety and efficiency of automated drone inspections in uninhabited areas with poor communications and complex terrain, existing technologies pre-build a three-dimensional model of the site based on satellite imagery and drone data, use it to plan fixed flight routes, and utilize sensors for real-time environmental perception during flight. However, when a drone strictly follows a preset global optimal route, it cannot effectively avoid sudden and dynamic obstacles not included in the three-dimensional model. When a drone relies solely on real-time sensors for local, reactive obstacle avoidance, it can easily deviate from the established optimal path. Lacking the guidance of a global plan, the flight path becomes disjointed and obstacle avoidance decisions become simplistic. This leads to increased safety risks for drones, reduced overall inspection efficiency, and the possibility of missing key inspection points due to frequent and drastic adjustments to the route, ultimately impacting operational quality and mission integrity. Summary of the Invention

[0004] The purpose of the present invention is to provide a UAV flight path adjustment method based on obstacle avoidance in non-visual conditions. By combining global optimal path planning with local dynamic obstacle avoidance, the flight safety and path adaptation capability of the UAV in complex non-visual environments are significantly improved.

[0005] To achieve the above object, the technical solution adopted by the present invention is: The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state includes the following steps: Using satellite imagery and drones, a 3D base model of the site is generated through simultaneous positioning and mapping technology; Using the on-site 3D basic model and LiDAR sensors, the 3D spatial data within the UAV mission area and the point cloud data ahead of the flight path are acquired, and the point cloud data is pre-processed. Obtaining a flight path feature sequence list from the three-dimensional spatial data and the pre-processed point cloud data; The flight path feature sequence table is input into the global path prediction model and the local path response model to obtain the optimal reference path vector and the immediate avoidance path vector respectively; The path decision model is used to perform weighted adjustment on the optimal reference path vector and the immediate avoidance path vector, and the flight path vector is output to guide the UAV to perform path adjustment. Simulate non-visual flight scenarios on a 3D base model of the site, train a path decision model offline, and interpret the flight path vectors into flight control instructions to drive the UAV; Calculating the path adjustment amplitude based on the optimal reference path vector and the flight path vector; Update the 3D base model of the scene and display the drone flight path in 3D.

[0006] Preferably, the process of generating the on-site three-dimensional basic model includes: Obtain satellite images covering the mission area through the data processing workstation to generate a two-dimensional base map; The drone uses the onboard visual sensors, lidar sensors, and inertial measurement units to obtain ground image data, three-dimensional point cloud data, and the drone's own motion posture data. The drone takes off from a mobile drone nest and flies at low altitude over the mission area according to a preset route. The ground object image data, 3D point cloud data, and motion posture data are input into the data processing workstation. The data processing workstation uses simultaneous positioning and mapping technology to inversely solve the UAV's six-degree-of-freedom posture at each moment, synchronously project the ground object image data and 3D point cloud data into the global coordinate system, and construct a 3D point cloud map of the mission area. Closed-loop detection and global optimization technology are then used to eliminate accumulated errors. The two-dimensional base map is geo-referenced and aligned with the three-dimensional point cloud map. Based on the two-dimensional base map, the texture information of the three-dimensional point cloud map is rendered and mapped using the ground feature image data to generate a three-dimensional base model of the site.

[0007] Preferably, the process of obtaining the three-dimensional spatial data within the UAV mission area and the point cloud data ahead of the flight path, and preprocessing the point cloud data includes: Based on the planned flight path, the 3D spatial data of the flight path and surrounding areas are retrieved and loaded from the on-site 3D basic model stored in the data processing workstation. The 3D spatial data includes terrain elevation data within the mission area, and the location and contour data of fixed obstacles. The LiDAR sensor is used to obtain three-dimensional point cloud data in real time, with the LiDAR sensor as the coordinate origin. The three-dimensional point cloud data is obtained by the LiDAR sensor continuously emitting a laser beam toward a fan-shaped area in front of the flight path and receiving the reflected signal. The three-dimensional point cloud data is projected into the global coordinate system, and the projected three-dimensional point cloud data is filtered to remove noise points. The filtered three-dimensional point cloud data is segmented into point cloud clusters representing the contours of independent obstacles through a density-based spatial clustering algorithm to obtain the preprocessed point cloud data.

[0008] Preferably, the flight path feature sequence table includes slope features along the flight path and in the surrounding area, position features and envelope volume features of fixed obstacles, centroid position features of point cloud clusters, and closest distance features relative to the flight direction of the UAV; The slope characteristic is represented by the terrain slope angle by calculating the ratio of the vertical elevation difference between the two sampling points and the horizontal distance based on the terrain elevation data in the flight path and the surrounding area; The position feature is obtained by using the three-dimensional coordinates of the center point of the fixed obstacle in the global coordinate system, and the envelope volume feature is represented by calculating the length, width and height of the minimum orthogonal bounding box that encloses the fixed obstacle; The centroid position feature is obtained by summing the three-dimensional coordinate values ​​of the points in the point cloud cluster and taking the average value; The closest distance feature is obtained by traversing the points in the point cloud cluster, calculating the Euclidean distance of each point to the current three-dimensional coordinate point of the drone, and taking the minimum value of the Euclidean distance.

[0009] Preferably, the method of inputting the flight path feature sequence table into the global path prediction model and outputting the optimal reference path vector is to establish a global path prediction model based on a genetic algorithm using the slope feature, position feature, and envelope volume feature as input, and output the optimal reference path vector using the global path prediction model, the process comprising: Initial path set generation stage: A set of ordered three-dimensional path point sequences is used to represent the feasible flight path from the starting point to the end point. Based on the position characteristics and envelope volume characteristics, a set number of feasible flight paths are generated through obstacle collision detection. The feasible flight paths constitute the initial path set; Fitness function construction phase: Use the fitness function to quantitatively evaluate each feasible flight path and calculate the fitness value. The fitness function is composed of the normalized total path length, path energy consumption, and obstacle collision penalty. Selection phase: Based on the fitness value, a tournament selection method is used to extract a set number of feasible flight paths from the initial path set. The feasible flight path with the lowest fitness value is selected as the optimal path from the extracted feasible flight paths, and a new path set is generated based on the optimal path. Crossover mutation phase: Two feasible flight paths are selected in pairs from the new path set. A path point sequence number is set as the intersection point. The 3D path point sequences of the two feasible flight paths after the intersection point are exchanged to form two new feasible flight paths. 3D path points in the new feasible flight paths are selected, and the 3D coordinates of the 3D path points are randomly offset and obstacle collision detected to select new feasible flight paths that do not collide with fixed obstacles. Termination output phase: Iteratively execute the fitness function construction phase, selection phase, and crossover mutation phase. When the fitness value of the optimal path does not decrease during the iteration process, the iteration is terminated, and the optimal path is selected from the path set formed by the last iteration. The three-dimensional path point sequence corresponding to the optimal path is used as the optimal reference path vector.

[0010] Preferably, the method of inputting the flight path feature sequence table into the local path reaction model and outputting the immediate avoidance path vector is to establish a local path reaction model based on the A* algorithm using the center of mass position feature and the closest distance feature as input, and output the immediate avoidance path vector using the local path reaction model, the process comprising: 3D search space construction stage: With the current 3D coordinate position of the UAV as the center, a 3D search space is constructed, the current 3D coordinate position is defined as the starting node of the path search, and the path point with a preset forward distance from the current 3D coordinate position on the optimal reference path vector is defined as the target node. The centroid position feature and the closest distance feature are mapped to the 3D search space. When the distance between a node in the 3D search space grid and the centroid of the point cloud cluster is less than the preset safety radius, and the node is located inside the point cloud cluster, the node is marked as an obstacle node and screened out. Define the path cost function stage: along the optimal reference path vector, calculate the cumulative sum of the spatial straight-line distances of each road segment from the starting node to the current node. Use this cumulative sum as the first actual path cost of moving from the starting node to the current node. Use the Euclidean straight-line distance between the current node and the target node as the estimated cost of moving from the current node to the target node. Combine the first actual path cost and the estimated cost to determine the cost function. Use the cost function to evaluate the path cost from the starting node through the current node to the target node. Path search iteration phase: set an open set for storing nodes to be visited and a closed set for storing visited nodes, put the starting node into the open set, define the closed set to be empty initially, select the node with the lowest path cost from the open set as the current processing node, remove the current processing node from the open set and put it into the closed set, determine whether the current processing node is the target node, if the current processing node is the target node, then end the path search iteration phase, or if the current processing node is not the target node, obtain the passable adjacent nodes of the current processing node, calculate the second actual path cost of each adjacent node to the adjacent node via the current processing node, if the first actual path cost is lower than the second actual path cost recorded by the adjacent node, then update the second actual path cost of the adjacent node, and add the adjacent node to the open set, and continue the path search iteration phase until the target node is found; Output immediate avoidance path vector stage: When the path search iteration phase ends and the target node is successfully found, a local shortest feasible path consisting of an optimal node sequence is reconstructed by tracing back from the target node to the starting node. The ordered three-dimensional path point coordinate sequence corresponding to the local shortest feasible path is output as an immediate avoidance path vector.

[0011] Preferably, the process of using the path decision model to perform weighted adjustment on the optimal reference path vector and the immediate avoidance path vector to output a flight path vector for guiding the UAV to perform the path adjustment includes: A safety distance threshold and a warning distance threshold are preset for the path decision model, and the safety distance threshold is greater than the warning distance threshold; When the minimum value of the Euclidean distance corresponding to the closest distance feature is greater than the safety distance threshold, it is considered that there is no immediate collision risk. The weight of the optimal reference path vector is set to be close to 1, and the weight of the immediate avoidance path vector is set to be close to 0. When the minimum Euclidean distance does not exceed the warning distance threshold, it is determined that there is a high collision risk, and the weight of the optimal reference path vector is set to be close to 0, and the weight of the immediate avoidance path vector is set to be close to 1; When the minimum Euclidean distance is between the warning distance threshold and the safety distance threshold, it is considered a transitional state for path decision making. The weights of the optimal reference path vector and the immediate avoidance path vector are linearly interpolated based on the minimum Euclidean distance, and the sum of the weights of the optimal reference path vector and the immediate avoidance path vector is 1. Using the weight of the optimal reference path vector and the weight of the immediate avoidance path vector, the three-dimensional coordinates of the path points with corresponding serial numbers in the optimal reference path vector and the immediate avoidance path vector are weighted averaged to generate a new path point sequence, which constitutes the flight path vector.

[0012] Preferably, the process of simulating a non-visual flight scene on a three-dimensional base model, training a path decision model offline, and resolving a flight path vector into flight control instructions for driving the UAV to execute includes: Based on the on-site 3D basic model, a 3D simulation environment is constructed, and non-visual flight test cases are generated in the 3D simulation environment. Each non-visual flight test case includes a set of UAV start and end points. Obstacles not included in the 3D base model of the site are placed in the 3D simulation environment. Using the optimization solver, the optimal flight path is analyzed to ensure that the drone meets safety constraints and completes the mission in the shortest time and with the lowest energy consumption in each non-visual flight test scenario. The safety distance threshold and warning distance threshold corresponding to the optimal flight path are used as the ideal solution of the non-visual flight test case. The non-visual flight test case and the corresponding ideal solution constitute the evaluation data set for path decision model optimization. An objective function is set to quantitatively evaluate the performance of the path decision model. The objective function uses the decision parameters to be tested as input. The path decision model is run on non-visual flight test cases in the evaluation dataset. The decision parameters to be tested include the safety distance threshold and the warning distance threshold. The comprehensive performance indicators of the non-visual flight test cases are calculated, including the average path length, average flight time, average minimum distance to obstacles, and the number of collisions. A grid search method is used to search within the preset value range of the decision parameters to be tested. For each combination of decision parameters to be tested, the comprehensive performance score is calculated using the objective function. After traversing all the decision parameter combinations to be tested, the decision parameter to be tested that gives the highest comprehensive performance score is selected as the calibration parameter of the path decision model. The flight path vector is received by the flight controller, and the flight controller decomposes the flight path vector into path segments connecting adjacent path points; Using the flight controller to calculate the flight velocity vector based on the decomposed path segments and analyze the flight attitude corresponding to the flight velocity vector, including roll angle, pitch angle and yaw angle; The flight velocity vector and flight attitude are input as set points into the attitude control system of the UAV. A proportional-integral-differential controller is built into the attitude control system. The proportional-integral-differential controller compares the motion attitude data measured by the inertial measurement unit with the flight attitude. Based on the error between the motion attitude data and the flight attitude, the controller calculates and outputs flight control instructions for adjusting the speed of the UAV motor to drive the UAV to fly along the flight path vector.

[0013] Preferably, the process of calculating the path adjustment amplitude based on the optimal reference path vector and the flight path vector includes: Calculating the difference vector between the flight path vector and the optimal reference path vector, and quantizing the difference vector into a path adjustment amplitude by obtaining the Euclidean norm of the difference vector; Set the amplitude adjustment threshold and the number of calculation cycles, and compare the path adjustment amplitude with the amplitude adjustment threshold. If the path adjustment amplitude exceeds the amplitude adjustment threshold continuously within the number of calculation cycles, it is determined that the drone is in a complex environment and a complex environment warning signal is automatically generated. If the path adjustment amplitude does not exceed the amplitude adjustment threshold continuously within the number of calculation cycles, the drone executes the flight control command.

[0014] Preferably, the process of updating the on-site three-dimensional basic model and displaying the UAV flight path in three-dimensional form includes: Through time series analysis, newly added static obstacles whose point cloud cluster positions remain unchanged in the global coordinate system are identified from the real-time detected point cloud clusters. The point cloud data of the point cloud clusters and the corresponding position features and envelope volume features of the point cloud clusters are then integrated into the 3D basic model of the site. Using the fused on-site 3D basic model as the 3D digital environment background, the monitoring terminal renders and displays the drone entity, optimal reference path vectors and actual flight path vectors in different colors, as well as known and real-time detected obstacles in real time, and provides visual prompts when complex environment warnings are triggered.

[0015] The beneficial effects of the present invention are: The present invention combines macro-control of the global optimal path with rapid response to local sudden obstacles by constructing a global path prediction model and a local path response model, and dynamically weighting the path vectors output by the two. This enables the drone to flexibly avoid unknown risks while following the optimal route, greatly improving flight safety and path adaptability in complex and changing environments.

[0016] The present invention uses a trained path decision model to perform intelligent and continuous weighted adjustments to paths under different strategies, avoiding the inefficiency and path discontinuity caused by abrupt switching between different flight modes. At the same time, by quantifying the path adjustment amplitude for early warning, the situational awareness capability of environmental complexity is enhanced, ensuring that the UAV maintains maximum flight stability and efficiency while avoiding obstacles, thereby guaranteeing the consistency and quality of mission execution.

[0017] The present invention constructs a closed-loop self-optimization system by feeding back and updating the real-time detected and unrecorded static obstacle information into the on-site three-dimensional basic model. The accuracy of the on-site three-dimensional basic model is continuously improved with the execution of the flight mission, providing higher-quality prior data for subsequent path planning, fundamentally achieving iterative improvement, and enhancing the long-term reliability and sustainability of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a simulation test of the UAV flight path adjustment during the verification process of the present invention. Figure 1 ; Figure 3 This is a simulation test of the UAV flight path adjustment during the verification process of the present invention. Figure 2 ; Figure 4 This is a simulation test of the UAV flight path adjustment during the verification process of the present invention. Figure 3 ; Figure 5 This is a simulation test of the UAV flight path adjustment during the verification process of the present invention. Figure 4 . DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention are described clearly and completely below with reference to the accompanying drawings.

[0020] Example 1: Figure 1 As shown, the method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state includes the following steps: Using satellite imagery and drones, a 3D base model of the site is generated through simultaneous positioning and mapping technology; Using the on-site 3D basic model and LiDAR sensors, the 3D spatial data within the UAV mission area and the point cloud data ahead of the flight path are acquired, and the point cloud data is pre-processed. Obtaining a flight path feature sequence list from the three-dimensional spatial data and the pre-processed point cloud data; The flight path feature sequence table is input into the global path prediction model and the local path response model to obtain the optimal reference path vector and the immediate avoidance path vector respectively; The path decision model is used to perform weighted adjustment on the optimal reference path vector and the immediate avoidance path vector, and the flight path vector is output to guide the UAV to perform path adjustment. Simulate non-visual flight scenarios on a 3D base model of the site, train a path decision model offline, and interpret the flight path vectors into flight control instructions to drive the UAV; Calculating the path adjustment amplitude based on the optimal reference path vector and the flight path vector; Update the 3D base model of the scene and display the drone flight path in 3D.

[0021] The uninhabited area infrastructure construction site includes drones and mobile drone nests deployed at the infrastructure construction site, as well as data processing workstations. The drones are integrated with visual sensors, lidar sensors and inertial measurement units.

[0022] The process of generating a 3D base model of the site includes: The data processing workstation acquires satellite images covering the mission area and generates a two-dimensional base map. The data processing workstation performs denoising, color balancing, geometric correction, and radiometric correction on the satellite images. The drone uses the onboard visual sensors, lidar sensors, and inertial measurement units to obtain ground image data, three-dimensional point cloud data, and the drone's own motion posture data. The drone takes off from a mobile drone nest and flies at low altitude over the mission area according to a preset route. The ground object image data, 3D point cloud data, and motion posture data are input into the data processing workstation. The data processing workstation uses simultaneous positioning and mapping technology to inversely solve the UAV's six-degree-of-freedom posture at each moment, synchronously project the ground object image data and 3D point cloud data into the global coordinate system, and construct a 3D point cloud map of the mission area. Closed-loop detection and global optimization technology are then used to eliminate accumulated errors. Cumulative error refers to the deviation in the pose estimation that increases over time due to measurement uncertainty during the continuous movement of the drone. Closed-loop detection is to identify the flight path feature sequence list of a previously visited mission area when the drone moves to the area, thereby forming a constraint loop in the path trajectory. Global optimization technology uses the constraint loop to perform overall mathematical optimization of all pose nodes of the flight trajectory and 3D point cloud map, redistribute and correct the cumulative error to ensure the global consistency of the 3D point cloud map.

[0023] The two-dimensional base map is geo-referenced and aligned with the three-dimensional point cloud map. Based on the two-dimensional base map, the texture information of the three-dimensional point cloud map is rendered and mapped using the ground feature image data to generate a three-dimensional base model of the site.

[0024] The process of obtaining the three-dimensional spatial data within the UAV mission area and the point cloud data ahead of the flight path, and preprocessing the point cloud data includes: Based on the planned flight path, the 3D spatial data of the flight path and surrounding areas are retrieved and loaded from the on-site 3D basic model stored in the data processing workstation. The 3D spatial data includes terrain elevation data within the mission area, and the location and contour data of fixed obstacles. The LiDAR sensor is used to obtain three-dimensional point cloud data in real time, with the LiDAR sensor as the coordinate origin. The three-dimensional point cloud data is obtained by the LiDAR sensor continuously emitting a laser beam toward a fan-shaped area in front of the flight path and receiving the reflected signal. The three-dimensional point cloud data is projected into the global coordinate system, and the projected three-dimensional point cloud data is filtered to remove noise points. The filtered three-dimensional point cloud data is segmented into point cloud clusters representing the contours of independent obstacles through a density-based spatial clustering algorithm to obtain the preprocessed point cloud data.

[0025] The flight path feature sequence table includes the slope characteristics along the flight path and the surrounding area, the position characteristics and envelope volume characteristics of fixed obstacles, the centroid position characteristics of the point cloud cluster, and the closest distance characteristics relative to the flight direction of the UAV; The slope characteristic is represented by the terrain slope angle by calculating the ratio of the vertical elevation difference between the two sampling points and the horizontal distance based on the terrain elevation data in the flight path and the surrounding area; The position feature is obtained by using the three-dimensional coordinates of the center point of the fixed obstacle in the global coordinate system, and the envelope volume feature is represented by calculating the length, width and height of the minimum orthogonal bounding box that encloses the fixed obstacle; The centroid position feature is obtained by summing the three-dimensional coordinate values ​​of the points in the point cloud cluster and taking the average value; The closest distance feature is obtained by traversing the points in the point cloud cluster, calculating the Euclidean distance of each point to the current three-dimensional coordinate point of the drone, and taking the minimum value of the Euclidean distance.

[0026] The flight path feature sequence table is input into the global path prediction model, and the optimal reference path vector is outputted by establishing a global path prediction model based on a genetic algorithm using the slope feature, position feature, and envelope volume feature as inputs. The optimal reference path vector is outputted using the global path prediction model. The process includes: Initial path set generation stage: A set of ordered three-dimensional path point sequences is used to represent the feasible flight path from the starting point to the end point. Based on the position characteristics and envelope volume characteristics, a set number of feasible flight paths are generated through obstacle collision detection. The feasible flight paths constitute the initial path set; Fitness function construction phase: Use the fitness function to quantitatively evaluate each feasible flight path and calculate the fitness value F. The fitness function is composed of the normalized total path length L, path energy consumption E, and obstacle collision penalty O: ; Where, 、 、 Represent the weight coefficients of L, E, and O respectively, and .

[0027] Selection phase: Based on the fitness value, a tournament selection method is used to extract a set number of feasible flight paths from the initial path set. The feasible flight path with the lowest fitness value is selected as the optimal path from the extracted feasible flight paths, and a new path set is generated based on the optimal path. Crossover mutation phase: Two feasible flight paths are selected in pairs from the new path set. A path point sequence number is set as the intersection point. The 3D path point sequences of the two feasible flight paths after the intersection point are exchanged to form two new feasible flight paths. 3D path points in the new feasible flight paths are selected, and the 3D coordinates of the 3D path points are randomly offset and obstacle collision detected to select new feasible flight paths that do not collide with fixed obstacles. Termination output phase: Iteratively execute the fitness function construction phase, selection phase, and crossover mutation phase. When the fitness value of the optimal path does not decrease during the iteration process, the iteration is terminated, and the optimal path is selected from the path set formed by the last iteration. The three-dimensional path point sequence corresponding to the optimal path is used as the optimal reference path vector.

[0028] The flight path feature sequence table is input into the local path reaction model, and the method of outputting the immediate avoidance path vector is to establish an A-based * The algorithm's local path reaction model uses the local path reaction model to output an immediate avoidance path vector. The process includes: 3D search space construction stage: With the current 3D coordinate position of the UAV as the center, a 3D search space is constructed, the current 3D coordinate position is defined as the starting node of the path search, and the path point with a preset forward distance from the current 3D coordinate position on the optimal reference path vector is defined as the target node. The centroid position feature and the closest distance feature are mapped to the 3D search space. When the distance between a node in the 3D search space grid and the centroid of the point cloud cluster is less than the preset safety radius, and the node is located inside the point cloud cluster, the node is marked as an obstacle node and screened out. Define the path cost function stage: along the optimal reference path vector, calculate the cumulative sum of the spatial straight-line distances of each section from the starting node to the current node, and use the cumulative sum as the first actual path cost of moving from the starting node to the current node , the Euclidean straight-line distance between the current node and the target node is used as the estimated cost of moving the current node to the target node , combined with the first actual path cost and estimated costs Determine the cost function, and use the cost function to evaluate the path cost from the starting node through the current node to the target node : ; Where n represents the node to be evaluated in the three-dimensional search space grid.

[0029] Path search iteration phase: set an open set for storing nodes to be visited and a closed set for storing visited nodes, put the starting node into the open set, define the closed set to be empty initially, select the node with the lowest path cost from the open set as the current processing node, remove the current processing node from the open set and put it into the closed set, determine whether the current processing node is the target node, if the current processing node is the target node, then end the path search iteration phase, or if the current processing node is not the target node, obtain the passable adjacent nodes of the current processing node, calculate the second actual path cost of each adjacent node to the adjacent node via the current processing node, if the first actual path cost is lower than the second actual path cost recorded by the adjacent node, then update the second actual path cost of the adjacent node, and add the adjacent node to the open set, and continue the path search iteration phase until the target node is found; Output immediate avoidance path vector stage: When the path search iteration phase ends and the target node is successfully found, a local shortest feasible path consisting of an optimal node sequence is reconstructed by tracing back from the target node to the starting node. The ordered three-dimensional path point coordinate sequence corresponding to the local shortest feasible path is output as an immediate avoidance path vector.

[0030] The process of using the path decision model to perform weighted adjustment on the optimal reference path vector and the immediate avoidance path vector and outputting a flight path vector for guiding the UAV to perform path adjustment includes: A safety distance threshold and a warning distance threshold are preset for the path decision model, and the safety distance threshold is greater than the warning distance threshold; When the minimum value of the Euclidean distance corresponding to the closest distance feature is greater than the safety distance threshold, it is considered that there is no immediate collision risk. The weight of the optimal reference path vector is set to be close to 1, and the weight of the immediate avoidance path vector is set to be close to 0. When the minimum Euclidean distance does not exceed the warning distance threshold, it is determined that there is a high collision risk, and the weight of the optimal reference path vector is set to be close to 0, and the weight of the immediate avoidance path vector is set to be close to 1; When the minimum value of the Euclidean distance is between the alarm distance threshold Safety distance threshold When the time is between , it is considered as the transition state of path decision, and the weight of the optimal reference path vector and the weight of the immediate avoidance path vector Linear interpolation is performed based on the minimum value d of the Euclidean distance, and the sum of the weights of the optimal reference path vector and the immediate avoidance path vector is 1: ; Using the weight of the optimal reference path vector and the weight of the immediate avoidance path vector, the three-dimensional coordinates of the path points with corresponding serial numbers in the optimal reference path vector and the immediate avoidance path vector are weighted averaged to generate a new path point sequence, which constitutes the flight path vector.

[0031] The process of simulating non-visual flight scenarios on a 3D base model of the site, training the path decision model offline, and parsing the flight path vectors into flight control instructions for the UAV includes: Based on the on-site 3D basic model, a 3D simulation environment is constructed, and non-visual flight test cases are generated in the 3D simulation environment. Each non-visual flight test case includes a set of UAV start and end points. Obstacles not included in the 3D base model of the site are placed in the 3D simulation environment. Using the optimization solver, the optimal flight path is analyzed to ensure that the drone meets safety constraints and completes the mission in the shortest time and with the lowest energy consumption in each non-visual flight test scenario. The safety distance threshold and warning distance threshold corresponding to the optimal flight path are used as the ideal solution of the non-visual flight test case. The non-visual flight test case and the corresponding ideal solution constitute the evaluation data set for path decision model optimization. An objective function is set to quantitatively evaluate the performance of the path decision model. The objective function uses the decision parameters to be tested as input. The path decision model is run on non-visual flight test cases in the evaluation dataset. The decision parameters to be tested include the safety distance threshold and the warning distance threshold. The comprehensive performance indicators of the non-visual flight test cases are calculated, including the average path length, average flight time, average minimum distance to obstacles, and the number of collisions. A grid search method is used to search within the preset value range of the decision parameters to be tested. For each combination of decision parameters to be tested, the comprehensive performance score is calculated using the objective function. After traversing all the decision parameter combinations to be tested, the decision parameter to be tested that gives the highest comprehensive performance score is selected as the calibration parameter of the path decision model. The flight path vector is received by the flight controller, and the flight controller decomposes the flight path vector into path segments connecting adjacent path points; Using the flight controller to calculate the flight velocity vector based on the decomposed path segments and analyze the flight attitude corresponding to the flight velocity vector, including roll angle, pitch angle and yaw angle; The flight velocity vector and flight attitude are input as set points into the attitude control system of the UAV. A proportional-integral-differential controller is built into the attitude control system. The proportional-integral-differential controller compares the motion attitude data measured by the inertial measurement unit with the flight attitude. Based on the error between the motion attitude data and the flight attitude, the controller calculates and outputs flight control instructions for adjusting the speed of the UAV motor to drive the UAV to fly along the flight path vector.

[0032] The process of calculating the path adjustment amplitude based on the optimal reference path vector and the flight path vector includes: Calculating the difference vector between the flight path vector and the optimal reference path vector, and quantizing the difference vector into a path adjustment amplitude by obtaining the Euclidean norm of the difference vector; Set the amplitude adjustment threshold and the number of calculation cycles, and compare the path adjustment amplitude with the amplitude adjustment threshold. If the path adjustment amplitude exceeds the amplitude adjustment threshold continuously within the number of calculation cycles, it is determined that the drone is in a complex environment and a complex environment warning signal is automatically generated. If the path adjustment amplitude does not exceed the amplitude adjustment threshold continuously within the number of calculation cycles, the drone executes the flight control command.

[0033] The process of updating the 3D base model of the site and displaying the drone's flight path in 3D includes: Through time series analysis, newly added static obstacles whose point cloud cluster positions remain unchanged in the global coordinate system are identified from the real-time detected point cloud clusters. The point cloud data of the point cloud clusters and the corresponding position features and envelope volume features of the point cloud clusters are then integrated into the 3D basic model of the site. Using the fused on-site 3D basic model as the 3D digital environment background, the monitoring terminal renders and displays the drone entity, optimal reference path vectors and actual flight path vectors in different colors, as well as known and real-time detected obstacles in real time, and provides visual prompts when complex environment warnings are triggered.

[0034] The verification process is: Through simulation experiments, the ability of the method of this embodiment to combine global planning and local response in a dynamic non-visual environment is verified.

[0035] With the grid map as the background, planning time, response cycle and path smoothness as evaluation indicators, the simulation experiment is shown in Figure 2. Figure 2-Figure 5 shown.

[0036] Figure 2-Figure 5 In each sub-graph, the lower left corner is the starting point (starting node), the upper right corner is the end point (target node), the curve from the starting point to the end point is the planned path, and the black square part is the fixed obstacle position.

[0037] Figure 2The figure shows the global path planning stage of the method of this embodiment. The flight path feature sequence table is input into the global path prediction model. The global path prediction model is based on a genetic algorithm and outputs an optimal reference path (the dotted line starting from the starting node and ending at the target node in the figure). This path successfully avoids all fixed obstacles.

[0038] Figure 3 The figure shows the stage where the drone (the hexagon in the figure) flies along the optimal path without the risk of immediate collision. According to the path decision model in this embodiment, when there is no threat of dynamic obstacles (the dots on the dotted line in the figure), the drone's flight path (the solid line in the figure) completely overlaps with the optimal reference path (the dotted line in the figure starting from the starting node and ending at the target node).

[0039] Figure 4 As shown, when the UAV flies to the front of the predetermined path, the simulated lidar sensor detects a dynamic obstacle in real time (the dot on the dotted line in the figure). At this time, the local path reaction model based on the A* algorithm is activated to generate an immediate avoidance path for bypassing the dynamic obstacle (the dotted line in the figure does not start from the starting node). At the same time, the path decision model smoothly adjusts the flight path based on the weighted adjustment strategy, causing the flight path to shift toward the immediate avoidance path.

[0040] Figure 5 The figure shows the stage where the drone successfully avoids an obstacle. As the distance between the drone and the obstacle decreases, the weight of the immediate avoidance path approaches 1, causing the drone's actual flight trajectory (solid line in the figure) to detour along the immediate avoidance path. After successfully crossing the obstacle, the immediate avoidance path disappears, and the flight path smoothly returns to the global optimal reference path and continues to fly towards the destination.

[0041] This simulation example verifies that the method of this embodiment effectively resolves the conflict between the preset route and real-time obstacle avoidance by combining global optimal path planning (genetic algorithm) with local dynamic obstacle avoidance (A* algorithm). The outputs of the path decision model and the local path response model are dynamically weighted through the path decision model, ensuring the continuity and smoothness of the UAV flight path.

[0042] Example 2: A device for adjusting the flight path of a drone based on obstacle avoidance in a non-visual state, comprising: one or more processors; a memory for storing one or more computer programs; When one or more programs are executed by one or more processors, the one or more processors execute the method in Example 1.

[0043] Example 3: A computer-readable storage medium stores executable instructions, which implement the method in Example 1 when the instructions are executed by a processor.

Claims

1. A method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state, characterized in that: The following steps are involved: Using satellite imagery and drones, a 3D base model of the site is generated through simultaneous positioning and mapping technology; Using the on-site 3D basic model and LiDAR sensors, the 3D spatial data within the UAV mission area and the point cloud data ahead of the flight path are acquired, and the point cloud data is pre-processed. Obtaining a flight path feature sequence list from the three-dimensional spatial data and the pre-processed point cloud data; The flight path feature sequence table is input into the global path prediction model and the local path response model to obtain the optimal reference path vector and the immediate avoidance path vector respectively; The path decision model is used to perform weighted adjustment on the optimal reference path vector and the immediate avoidance path vector, and the flight path vector is output to guide the UAV to perform path adjustment. Simulate non-visual flight scenarios on a 3D base model of the site, train a path decision model offline, and interpret the flight path vectors into flight control instructions to drive the UAV; Calculating the path adjustment amplitude based on the optimal reference path vector and the flight path vector; Update the 3D base model of the scene and display the drone flight path in 3D.

2. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 1, characterized in that: The process of generating the on-site three-dimensional basic model includes: Obtain satellite images covering the mission area through the data processing workstation to generate a two-dimensional base map; The drone uses the onboard visual sensors, lidar sensors, and inertial measurement units to obtain ground image data, three-dimensional point cloud data, and the drone's own motion posture data. The drone takes off from a mobile drone nest and flies at low altitude over the mission area according to a preset route. The ground object image data, 3D point cloud data, and motion posture data are input into the data processing workstation. The data processing workstation uses simultaneous positioning and mapping technology to inversely solve the UAV's six-degree-of-freedom posture at each moment, synchronously project the ground object image data and 3D point cloud data into the global coordinate system, and construct a 3D point cloud map of the mission area. Closed-loop detection and global optimization technology are then used to eliminate accumulated errors. The two-dimensional base map is geo-referenced and aligned with the three-dimensional point cloud map. Based on the two-dimensional base map, the texture information of the three-dimensional point cloud map is rendered and mapped using the ground feature image data to generate a three-dimensional base model of the site.

3. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 2, characterized in that: The process of obtaining the three-dimensional spatial data within the UAV mission area and the point cloud data ahead of the flight path, and preprocessing the point cloud data includes: Based on the planned flight path, the 3D spatial data of the flight path and surrounding areas are retrieved and loaded from the on-site 3D basic model stored in the data processing workstation. The 3D spatial data includes terrain elevation data within the mission area, and the location and contour data of fixed obstacles. The LiDAR sensor is used to obtain three-dimensional point cloud data in real time, with the LiDAR sensor as the coordinate origin. The three-dimensional point cloud data is obtained by the LiDAR sensor continuously emitting a laser beam toward a fan-shaped area in front of the flight path and receiving the reflected signal. The three-dimensional point cloud data is projected into the global coordinate system, and the projected three-dimensional point cloud data is filtered to remove noise points. The filtered three-dimensional point cloud data is segmented into point cloud clusters representing the contours of independent obstacles through a density-based spatial clustering algorithm to obtain the preprocessed point cloud data.

4. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 3 is characterized in that: The flight path feature sequence table includes the slope features along the flight path and the surrounding area, the position features and envelope volume features of fixed obstacles, the centroid position features of the point cloud cluster, and the closest distance features relative to the flight direction of the UAV; The slope characteristic is represented by the terrain slope angle by calculating the ratio of the vertical elevation difference between the two sampling points and the horizontal distance based on the terrain elevation data in the flight path and the surrounding area; The position feature is obtained by using the three-dimensional coordinates of the center point of the fixed obstacle in the global coordinate system, and the envelope volume feature is represented by calculating the length, width and height of the minimum orthogonal bounding box that encloses the fixed obstacle; The centroid position feature is obtained by summing the three-dimensional coordinate values ​​of the points in the point cloud cluster and taking the average value; The closest distance feature is obtained by traversing the points in the point cloud cluster, calculating the Euclidean distance of each point to the current three-dimensional coordinate point of the drone, and taking the minimum value of the Euclidean distance.

5. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that: The method of inputting the flight path feature sequence table into the global path prediction model and outputting the optimal reference path vector is to establish a global path prediction model based on a genetic algorithm using the slope feature, position feature, and envelope volume feature as input, and output the optimal reference path vector using the global path prediction model. The process includes: Initial path set generation stage: A set of ordered three-dimensional path point sequences is used to represent the feasible flight path from the starting point to the end point. Based on the position characteristics and envelope volume characteristics, a set number of feasible flight paths are generated through obstacle collision detection. The feasible flight paths constitute the initial path set; Fitness function construction phase: Use the fitness function to quantitatively evaluate each feasible flight path and calculate the fitness value. The fitness function is composed of the normalized total path length, path energy consumption, and obstacle collision penalty. Selection phase: Based on the fitness value, a tournament selection method is used to extract a set number of feasible flight paths from the initial path set. The feasible flight path with the lowest fitness value is selected as the optimal path from the extracted feasible flight paths, and a new path set is generated based on the optimal path. Crossover mutation phase: Two feasible flight paths are selected in pairs from the new path set. A path point sequence number is set as the intersection point. The 3D path point sequences of the two feasible flight paths after the intersection point are exchanged to form two new feasible flight paths. 3D path points in the new feasible flight paths are selected, and the 3D coordinates of the 3D path points are randomly offset and obstacle collision detected to select new feasible flight paths that do not collide with fixed obstacles. Termination output phase: Iteratively execute the fitness function construction phase, selection phase, and crossover mutation phase. When the fitness value of the optimal path does not decrease during the iteration process, the iteration is terminated, and the optimal path is selected from the path set formed by the last iteration. The three-dimensional path point sequence corresponding to the optimal path is used as the optimal reference path vector.

6. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that: The method of inputting the flight path feature sequence table into the local path reaction model and outputting the immediate avoidance path vector is to establish a method based on A with the center of mass position feature and the closest distance feature as input. * The algorithm's local path reaction model uses the local path reaction model to output an immediate avoidance path vector. The process includes: 3D search space construction stage: With the current 3D coordinate position of the UAV as the center, a 3D search space is constructed, the current 3D coordinate position is defined as the starting node of the path search, and the path point with a preset forward distance from the current 3D coordinate position on the optimal reference path vector is defined as the target node. The centroid position feature and the closest distance feature are mapped to the 3D search space. When the distance between a node in the 3D search space grid and the centroid of the point cloud cluster is less than the preset safety radius, and the node is located inside the point cloud cluster, the node is marked as an obstacle node and screened out. Define the path cost function stage: along the optimal reference path vector, calculate the cumulative sum of the spatial straight-line distances of each road segment from the starting node to the current node. Use this cumulative sum as the first actual path cost of moving from the starting node to the current node. Use the Euclidean straight-line distance between the current node and the target node as the estimated cost of moving from the current node to the target node. Combine the first actual path cost and the estimated cost to determine the cost function. Use the cost function to evaluate the path cost from the starting node through the current node to the target node. Path search iteration phase: set an open set for storing nodes to be visited and a closed set for storing visited nodes, put the starting node into the open set, define the closed set to be empty initially, select the node with the lowest path cost from the open set as the current processing node, remove the current processing node from the open set and put it into the closed set, determine whether the current processing node is the target node, if the current processing node is the target node, then end the path search iteration phase, or if the current processing node is not the target node, obtain the passable adjacent nodes of the current processing node, calculate the second actual path cost of each adjacent node to the adjacent node via the current processing node, if the first actual path cost is lower than the second actual path cost recorded by the adjacent node, then update the second actual path cost of the adjacent node, and add the adjacent node to the open set, and continue the path search iteration phase until the target node is found; Output immediate avoidance path vector stage: When the path search iteration phase ends and the target node is successfully found, a local shortest feasible path consisting of an optimal node sequence is reconstructed by tracing back from the target node to the starting node. The ordered three-dimensional path point coordinate sequence corresponding to the local shortest feasible path is output as an immediate avoidance path vector.

7. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that: The process of using the path decision model to perform weighted adjustment on the optimal reference path vector and the immediate avoidance path vector to output a flight path vector for guiding the UAV to perform path adjustment includes: A safety distance threshold and a warning distance threshold are preset for the path decision model, and the safety distance threshold is greater than the warning distance threshold; When the minimum value of the Euclidean distance corresponding to the closest distance feature is greater than the safety distance threshold, it is considered that there is no immediate collision risk. The weight of the optimal reference path vector is set to be close to 1, and the weight of the immediate avoidance path vector is set to be close to 0. When the minimum Euclidean distance does not exceed the warning distance threshold, it is determined that there is a high collision risk, and the weight of the optimal reference path vector is set to be close to 0, and the weight of the immediate avoidance path vector is set to be close to 1; When the minimum Euclidean distance is between the warning distance threshold and the safety distance threshold, it is considered a transitional state for path decision making. The weights of the optimal reference path vector and the immediate avoidance path vector are linearly interpolated based on the minimum Euclidean distance, and the sum of the weights of the optimal reference path vector and the immediate avoidance path vector is 1. Using the weight of the optimal reference path vector and the weight of the immediate avoidance path vector, the three-dimensional coordinates of the path points with corresponding serial numbers in the optimal reference path vector and the immediate avoidance path vector are weighted averaged to generate a new path point sequence, which constitutes the flight path vector.

8. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 7, characterized in that: The process of simulating a non-visual flight scene on a three-dimensional base model, training a path decision model offline, and resolving the flight path vector into flight control instructions for driving the UAV includes: Based on the on-site 3D basic model, a 3D simulation environment is constructed, and non-visual flight test cases are generated in the 3D simulation environment. Each non-visual flight test case includes a set of UAV start and end points. Obstacles not included in the 3D base model of the site are placed in the 3D simulation environment. Using the optimization solver, the optimal flight path is analyzed to ensure that the drone meets safety constraints and completes the mission in the shortest time and with the lowest energy consumption in each non-visual flight test scenario. The safety distance threshold and warning distance threshold corresponding to the optimal flight path are used as the ideal solution of the non-visual flight test case. The non-visual flight test case and the corresponding ideal solution constitute the evaluation data set for path decision model optimization. An objective function is set to quantitatively evaluate the performance of the path decision model. The objective function uses the decision parameters to be tested as input. The path decision model is run on non-visual flight test cases in the evaluation dataset. The decision parameters to be tested include the safety distance threshold and the warning distance threshold. The comprehensive performance indicators of the non-visual flight test cases are calculated, including the average path length, average flight time, average minimum distance to obstacles, and the number of collisions. A grid search method is used to search within the preset value range of the decision parameters to be tested. For each combination of decision parameters to be tested, the comprehensive performance score is calculated using the objective function. After traversing all the decision parameter combinations to be tested, the decision parameter to be tested that gives the highest comprehensive performance score is selected as the calibration parameter of the path decision model. The flight path vector is received by the flight controller, and the flight controller decomposes the flight path vector into path segments connecting adjacent path points; The flight controller is used to calculate the flight velocity vector based on the decomposed path segments and analyze the flight attitude corresponding to the flight velocity vector, including the roll angle, pitch angle, and yaw angle. The flight velocity vector and flight attitude are input as set points into the attitude control system of the UAV. A proportional-integral-differential controller is built into the attitude control system. The proportional-integral-differential controller compares the motion attitude data measured by the inertial measurement unit with the flight attitude. Based on the error between the motion attitude data and the flight attitude, the controller calculates and outputs flight control instructions for adjusting the speed of the UAV motor to drive the UAV to fly along the flight path vector.

9. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 1, characterized in that: The process of calculating the path adjustment amplitude based on the optimal reference path vector and the flight path vector includes: Calculating the difference vector between the flight path vector and the optimal reference path vector, and quantizing the difference vector into a path adjustment amplitude by obtaining the Euclidean norm of the difference vector; Set the amplitude adjustment threshold and the number of calculation cycles, and compare the path adjustment amplitude with the amplitude adjustment threshold. If the path adjustment amplitude exceeds the amplitude adjustment threshold continuously within the number of calculation cycles, it is determined that the drone is in a complex environment and a complex environment warning signal is automatically generated. If the path adjustment amplitude does not exceed the amplitude adjustment threshold continuously within the number of calculation cycles, the drone executes the flight control command.

10. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that: The process of updating the three-dimensional base model of the site and displaying the flight path of the drone in three-dimensional form includes: Through time series analysis, newly added static obstacles whose point cloud cluster positions remain unchanged in the global coordinate system are identified from the real-time detected point cloud clusters. The point cloud data of the point cloud clusters and the corresponding position features and envelope volume features of the point cloud clusters are then integrated into the 3D basic model of the site. Using the fused on-site 3D basic model as the 3D digital environment background, the monitoring terminal renders and displays the drone entity, optimal reference path vectors and actual flight path vectors in different colors, as well as known and real-time detected obstacles in real time, and provides visual prompts when complex environment warnings are triggered.

Images