Unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in non-visual state

Abstract

The present application belongs to the technical field of unmanned aerial vehicle path adjustment, and particularly relates to an unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in a non-visible state, which comprises the following steps: generating a three-dimensional basic model of a scene; obtaining three-dimensional space data in a task area of the unmanned aerial vehicle and point cloud data in front of a flight path; obtaining a feature sequence list of the flight path; obtaining an optimal reference path vector and an instant avoidance path vector respectively; outputting a flight path vector for guiding the unmanned aerial vehicle to perform path adjustment by using a path decision model; simulating a non-visible flight scene on the three-dimensional basic model of the scene, and analyzing the flight path vector into flight control instructions for driving the unmanned aerial vehicle to execute; calculating a path adjustment amplitude based on the optimal reference path vector and the flight path vector; updating the three-dimensional basic model of the scene, and displaying the flight path of the unmanned aerial vehicle in a three-dimensional form. The present application significantly improves the flight safety and path adaptive capability of the unmanned aerial vehicle in a complex non-visible environment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicle path adjustment, and particularly relates to an unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in a non-visual state. BACKGROUND

[0002] The unmanned aerial vehicle flight path adjustment refers to real-time correction and adjustment of a predetermined flight path of an unmanned aerial vehicle according to an actual environment of a no-man zone construction site (a main application scene of the application) through a dynamic route optimization algorithm. This process realizes accurate positioning and map construction of the unmanned aerial vehicle by using multi-sensor fusion data and combining SLAM technology, so as to ensure that the unmanned aerial vehicle can still fly safely and efficiently in a complex and poor visibility environment. Through real-time monitoring and adjustment of the flight path, the unmanned aerial vehicle better adapts to the changes on site, and the patrol efficiency and safety are improved.

[0003] In order to solve the problem of safety and operation efficiency of unmanned aerial vehicle automatic patrol operation in a no-man zone with poor communication and complex terrain, the prior art is to construct a three-dimensional model of the site in advance based on satellite images and unmanned aerial vehicle data, and to plan a fixed flight route based on the three-dimensional model, and to use sensors for real-time environment sensing during flight. However, when the unmanned aerial vehicle strictly follows the preset global optimal route, it cannot effectively avoid the sudden and dynamic obstacles not included in the three-dimensional model, and when the unmanned aerial vehicle only relies on real-time sensors for local reactive obstacle avoidance, it is easy to deviate from the established optimal path, lacks global planning guidance, and appears to be incoherent flight path and single obstacle avoidance decision, which further leads to high flight safety risk of the unmanned aerial vehicle, low overall patrol efficiency, and possible missing of key patrol points due to frequent and large-scale adjustment of the path, and finally affects the operation quality and task integrity. SUMMARY

[0004] The application aims to provide an unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in a non-visual state, which significantly improves the flight safety and path adaptive ability of the unmanned aerial vehicle in a complex non-visual environment by combining global optimal path planning and local dynamic obstacle avoidance.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the application is as follows:

[0006] The unmanned aerial vehicle flight path adjustment method based on obstacle avoidance in a non-visual state comprises the following steps:

[0007] A three-dimensional basic model of the site is generated by using satellite images and unmanned aerial vehicles through simultaneous localization and mapping technology;

[0008] The three-dimensional space data in the task area of the unmanned aerial vehicle and the point cloud data in front of the flight path are acquired through the field three-dimensional basic model and the laser radar sensor, and the point cloud data is preprocessed;

[0009] The flight path feature sequence list is acquired from the three-dimensional space data and the preprocessed point cloud data;

[0010] The flight path feature sequence list is input into the global path prediction model and the local path reaction model respectively, and the optimal reference path vector and the instant avoidance path vector are obtained respectively;

[0011] The optimal reference path vector and the instant avoidance path vector are weighted and adjusted by using the path decision model, and the flight path vector for guiding the unmanned aerial vehicle to execute path adjustment is output;

[0012] The non-visible flight scene on the field three-dimensional basic model is simulated, the path decision model is trained offline, and the flight path vector is analyzed into the flight control instruction for driving the unmanned aerial vehicle to execute;

[0013] The path adjustment amplitude is calculated based on the optimal reference path vector and the flight path vector;

[0014] The field three-dimensional basic model is updated, and the flight path of the unmanned aerial vehicle is displayed in three-dimensional form.

[0015] Preferably, the process of generating the field three-dimensional basic model comprises:

[0016] Satellite images covering the task area are acquired by the data processing workstation to generate a two-dimensional basic base map;

[0017] The ground feature image data, three-dimensional point cloud data and motion attitude data of the unmanned aerial vehicle are acquired by the visual sensor, laser radar sensor and inertial measurement unit carried by the unmanned aerial vehicle, wherein the unmanned aerial vehicle takes off from the mobile unmanned aerial vehicle nest and flies at low altitude according to the preset flight route over the task area;

[0018] The ground feature image data, three-dimensional point cloud data and motion attitude data are input into the data processing workstation, the data processing workstation inversely solves the six-degree-of-freedom pose of the unmanned aerial vehicle at each time through simultaneous localization and mapping technology, synchronously projects the ground feature image data and three-dimensional point cloud data to the global coordinate system, constructs a three-dimensional point cloud map of the task area, and eliminates the cumulative error by using the closed loop detection and global optimization technology;

[0019] The two-dimensional basic base map and the three-dimensional point cloud map are geographically registered and aligned, the texture information of the three-dimensional point cloud map is rendered and mapped with the ground feature image data as the reference, and the field three-dimensional basic model is generated.

[0020] Preferably, the process of obtaining the three-dimensional spatial data in the UAV task area and the point cloud data in front of the flight path, and preprocessing the point cloud data comprises:

[0021] According to the planned flight path, the three-dimensional spatial data of the flight path and the surrounding area is retrieved and loaded from the three-dimensional base model stored in the data processing workstation, and the three-dimensional spatial data includes the terrain elevation data in the task area, the position data and the contour data of the fixed obstacles;

[0022] The three-dimensional point cloud data is obtained in real time by the laser radar sensor, and the three-dimensional point cloud data is obtained by continuously emitting laser beams from the laser radar sensor to the fan-shaped area in front of the flight path and receiving the reflected signals.

[0023] The three-dimensional point cloud data is projected to the global coordinate system, the filtered three-dimensional point cloud data is segmented into point cloud clusters representing the contour of the independent obstacle by the spatial clustering algorithm based on density, and the preprocessed point cloud data is obtained.

[0024] Preferably, the flight path feature sequence list includes the slope feature of the flight path and the surrounding area, the position feature and the envelope volume feature of the fixed obstacle, the centroid position feature of the point cloud cluster, and the nearest distance feature relative to the flight direction of the UAV;

[0025] The slope feature is obtained by calculating the ratio of the vertical elevation difference and the horizontal distance between the front and rear sampling points based on the terrain elevation data in the flight path and the surrounding area, and the terrain slope angle is represented.

[0026] The position feature is obtained by using the three-dimensional coordinates of the center point of the fixed obstacle in the global coordinate system, and the envelope volume feature is represented by the length, width and height dimensions of the minimum orthogonal bounding box enclosing the fixed obstacle.

[0027] The centroid position feature is obtained by summing the three-dimensional coordinate values of the points in the point cloud cluster and taking the average value.

[0028] The nearest distance feature is obtained by calculating the Euclidean distance between each point in the point cloud cluster and the current three-dimensional coordinate point of the UAV, and taking the minimum value of the Euclidean distance.

[0029] Preferably, the method of inputting the flight path feature sequence list into the global path prediction model and outputting the optimal reference path vector is to input the slope feature, the position feature and the envelope volume feature into the global path prediction model based on genetic algorithm, and output the optimal reference path vector by using the global path prediction model, and the process comprises:

[0030] The initial path set generation stage: a set of ordered three-dimensional path point sequences is used to represent the feasible flight path from the starting point to the ending point, and a certain number of feasible flight paths are generated through obstacle collision detection based on position characteristics and envelope volume characteristics, and the feasible flight paths constitute the initial path set;

[0031] The fitness function construction stage: the fitness function is used to quantitatively evaluate each feasible flight path, and the fitness value is calculated, and the fitness function is composed of the normalized path total length, the path energy consumption and the obstacle collision penalty;

[0032] The selection stage: based on the fitness value, the tournament selection method is used to extract a certain number of feasible flight paths from the initial path set, and the feasible flight path with the lowest fitness value is selected from the extracted feasible flight paths as the optimal path, and a new path set is generated based on the optimal path;

[0033] The cross mutation stage: two feasible flight paths are selected from the new path set, a path point number is set as a cross point, the three-dimensional path point sequences after the cross point of the two feasible flight paths are exchanged with each other to form two new feasible flight paths, and the three-dimensional path points in the new feasible flight paths are selected, and the three-dimensional coordinates of the three-dimensional path points are randomly offset and obstacle collision detection is performed, and the new feasible flight path which does not collide with the fixed obstacle is selected;

[0034] The termination output stage: the fitness function construction stage, the selection stage and the cross mutation stage are iteratively executed, when the fitness value of the optimal path does not decrease in the iteration process, the iteration is terminated, the optimal path is selected from the path set formed in the last iteration, and the three-dimensional path point sequence corresponding to the optimal path is taken as the optimal reference path vector.

[0035] Preferably, the method of inputting the flight path feature sequence into the local path reaction model and outputting the immediate avoidance path vector is that the centroid position feature and the nearest distance feature are input, a local path reaction model based on A* algorithm is established, and the immediate avoidance path vector is output by using the local path reaction model, and the process includes:

[0036] The three-dimensional search space construction stage: the current three-dimensional coordinate position of the unmanned aerial vehicle is taken as the center to construct a three-dimensional search space, the current three-dimensional coordinate position is defined as the starting node of the path search, the path point on the optimal reference path vector with a preset forward distance from the current three-dimensional coordinate position is defined as the target node, the centroid position feature and the nearest distance feature are mapped into the three-dimensional search space, when the distance between the node in the three-dimensional search space grid and the point cloud cluster centroid is less than the preset safety radius, and the node is located inside the point cloud cluster, the node is marked as an obstacle node and is screened out;

[0037] In the path cost function definition stage: along the optimal reference path vector, calculate the sum of the spatial straight-line distances of each segment from the starting node to the current node. Use the sum as the first actual path cost from the starting node to the current node, and use the Euclidean straight-line distance between the current node and the target node as the estimated cost from the current node to the target node. Combine the first actual path cost and the estimated cost to determine the cost function. Use the cost function to evaluate the path cost from the starting node to the target node via the current node.

[0038] Path search iteration phase: Set up an open set to store nodes to be visited and a closed set to store visited nodes. Put the starting node into the open set. Define the closed set as empty initially. Select the node with the lowest path cost from the open set as the current processing node. Remove the current processing node from the open set and put it into the closed set. Determine if the current processing node is the target node. If the current processing node is the target node, end the path search iteration phase. If the current processing node is not the target node, obtain the neighboring nodes that the current processing node can reach. Calculate the second actual path cost of each neighboring node from the current processing node. If the first actual path cost is lower than the second actual path cost recorded by the neighboring node, update the second actual path cost of the neighboring node and add the neighboring node to the open set. Continue the path search iteration phase until the target node is found.

[0039] Outputting the immediate avoidance path vector stage: After the path search iteration stage ends and the target node is successfully found, a local shortest feasible path composed of the optimal node sequence is reconstructed by backtracking from the target node to the starting node. The ordered three-dimensional path point coordinate sequence corresponding to the local shortest feasible path is output as the immediate avoidance path vector.

[0040] Preferably, the process of using a path decision model to weight and adjust the optimal reference path vector and the instantaneous avoidance path vector, and outputting a flight path vector to guide the UAV in performing path adjustment, includes:

[0041] Preset a safe distance threshold and an alarm distance threshold for the path decision model, and the safe distance threshold is greater than the alarm distance threshold;

[0042] When the minimum value of the Euclidean distance corresponding to the nearest distance feature is greater than the safe distance threshold, it is considered to be a state with no immediate collision risk. The weight of the optimal reference path vector is set to approach 1, and the weight of the immediate avoidance path vector is set to approach 0.

[0043] When the minimum Euclidean distance does not exceed the alarm distance threshold, a high collision risk is identified, the weight of the optimal reference path vector is set to approach 0, and the weight of the immediate avoidance path vector is set to approach 1.

[0044] When the minimum value of the Euclidean distance is between the alarm distance threshold and the safe distance threshold, it is considered to be a transitional state of path decision. The weights of the optimal reference path vector and the immediate avoidance path vector are calculated by linear interpolation based on the minimum value of the Euclidean distance, and the sum of the weights of the optimal reference path vector and the immediate avoidance path vector is 1.

[0045] By using the weights of the optimal reference path vector and the instantaneous avoidance path vector, the three-dimensional coordinates of the path points with corresponding indices in the optimal reference path vector and the instantaneous avoidance path vector are weighted and averaged to generate a new path point sequence. The new path point sequence constitutes the flight path vector.

[0046] Preferably, the process of simulating a non-visual flight scenario on a 3D basic model of the simulated scene, training a path decision model offline, and resolving the flight path vector into flight control commands that drive the UAV includes:

[0047] Based on the on-site 3D basic model, a 3D simulation environment is constructed, and non-visual flight test cases are generated in the 3D simulation environment. Each non-visual flight test case contains a set of UAV start and end points.

[0048] In a 3D simulation environment, obstacles not included in the on-site 3D basic model are set up. By optimizing the solver, the optimal flight path is analyzed to enable the UAV to meet safety constraints and complete the task in the shortest time and with the lowest energy consumption in each non-visual flight test scenario.

[0049] The safe distance threshold and alarm distance threshold corresponding to the optimal flight path are used as the ideal solution of the non-visual flight test case. The non-visual flight test case and the corresponding ideal solution constitute the evaluation dataset for path decision model optimization.

[0050] A target function is defined to quantitatively evaluate the performance of the path decision model. The target function takes the decision parameters to be tested as input and runs the path decision model on non-visual flight test cases in the evaluation dataset. The decision parameters to be tested include the safe distance threshold and the alarm distance threshold. The comprehensive performance indicators of the non-visual flight test cases are statistically analyzed, including the average path length, average flight time, average minimum distance to obstacles, and number of collisions.

[0051] The grid search method is used to search within the preset range of values ​​of the decision parameters to be tested. For each combination of decision parameters to be tested, the comprehensive performance score is calculated through the objective function. After traversing all combinations of decision parameters to be tested, the decision parameters that result in the highest comprehensive performance score are selected as the calibration parameters of the path decision model.

[0052] The flight path vector is received by the flight controller, which then decomposes the flight path vector into path segments connecting adjacent path points.

[0053] Using the flight controller based on the decomposed path segments, the flight speed vector is calculated and the corresponding flight attitude, including roll angle, pitch angle and yaw angle, is analyzed.

[0054] The attitude control system of the UAV uses the flight speed vector and flight attitude as setpoints. The attitude control system has a built-in proportional-integral-derivative (PID) controller. The PID controller compares the motion attitude data measured by the inertial measurement unit with the flight attitude. Based on the error between the motion attitude data and the flight attitude, it calculates and outputs flight control commands to adjust the speed of the UAV motors, so as to drive the UAV to fly along the flight path vector.

[0055] Preferably, the process of calculating the path adjustment magnitude based on the optimal reference path vector and the flight path vector includes:

[0056] Calculate the difference vector between the flight path vector and the optimal reference path vector, and quantify the difference vector into the path adjustment magnitude by obtaining the Euclidean norm of the difference vector;

[0057] Set an amplitude adjustment threshold and a number of calculation cycles. Compare the path adjustment amplitude with the amplitude adjustment threshold. If the path adjustment amplitude exceeds the amplitude adjustment threshold continuously within the number of calculation cycles, it is determined that the UAV is in a complex environment and an automatic complex environment warning signal is generated. If the path adjustment amplitude does not meet the requirement of exceeding the amplitude adjustment threshold continuously within the number of calculation cycles, the UAV executes flight control commands.

[0058] Preferably, the process of updating the on-site 3D basic model and displaying the UAV flight path in 3D includes:

[0059] Through time series analysis, newly added static obstacles whose positions remain unchanged in the global coordinate system are identified from the real-time detected point cloud clusters. The point cloud data of the point cloud clusters and the corresponding position features and envelope volume features of the point cloud clusters are then integrated into the on-site three-dimensional basic model.

[0060] Using the fused on-site 3D basic model as the background of the 3D digital environment, the monitoring terminal renders and overlays the drone entity, the optimal reference path vector and the actual flight path vector of different colors, as well as the known and real-time detected obstacles, and provides visual prompts when a complex environment warning is triggered.

[0061] The beneficial effects of this invention are as follows:

[0062] This invention constructs a global path prediction model and a local path response model, and dynamically weights the path vectors output by the two models. This achieves a combination of macroscopic control of the global optimal path and rapid response to sudden local obstacles, enabling UAVs to flexibly avoid unknown risks while following the optimal route. This significantly improves flight safety and path adaptation capabilities in complex and ever-changing environments.

[0063] This invention utilizes a pre-trained path decision model to intelligently and continuously adjust paths under different strategies, avoiding the inefficiency and path inconsistency caused by abrupt switching between different flight modes. At the same time, by quantifying the path adjustment range for early warning, it enhances the situational awareness of environmental complexity, ensuring that the UAV maintains maximum flight stability and efficiency while avoiding obstacles, thus guaranteeing the continuity and quality of mission execution.

[0064] This invention constructs a closed-loop self-optimization system by feeding back and updating the on-site 3D basic model with real-time detected, unrecorded static obstacle information. This allows the accuracy of the on-site 3D basic model to continuously improve as the flight mission is executed, providing higher-quality prior data for subsequent path planning. It fundamentally achieves iterative improvement and enhances the long-term reliability and sustainability of the method. Attached Figure Description

[0065] Figure 1 This is a flowchart of the method of the present invention;

[0066] Figure 2 This is a simulation test of UAV flight path adjustment during the verification process of the present invention. Figure 1 ;

[0067] Figure 3 This is a simulation test of UAV flight path adjustment during the verification process of the present invention. Figure 2 ;

[0068] Figure 4 This is a simulation test of UAV flight path adjustment during the verification process of the present invention. Figure 3 ;

[0069] Figure 4 This is a simulation test of UAV flight path adjustment during the verification process of the present invention. Figure 1 . Detailed Implementation

[0070] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0071] Example 1: As Figures 2-5 As shown, the method for adjusting the flight path of a drone based on obstacle avoidance in non-visual conditions includes the following steps:

[0072] Using satellite imagery and drones, a three-dimensional basic model of the site is generated through simultaneous localization and mapping (SLAM) technology.

[0073] Using a 3D basic model and lidar sensors, the system acquires 3D spatial data within the UAV mission area and point cloud data ahead of the flight path, and preprocesses the point cloud data.

[0074] A flight path feature sequence list was obtained from the three-dimensional spatial data and the preprocessed point cloud data;

[0075] The flight path feature sequence list is input into the global path prediction model and the local path response model respectively to obtain the optimal reference path vector and the instantaneous avoidance path vector respectively.

[0076] The optimal reference path vector and the instantaneous avoidance path vector are weighted and adjusted using a path decision model, and the resulting flight path vector is used to guide the UAV to perform path adjustments.

[0077] The non-visual flight scenario is simulated on a 3D basic model of the scene, the path decision model is trained offline, and the flight path vector is parsed into flight control commands that drive the UAV to execute.

[0078] Calculate the path adjustment magnitude based on the optimal reference path vector and the flight path vector;

[0079] Update the on-site 3D basic model and display the drone's flight path in 3D.

[0080] The uninhabited infrastructure site includes drones and mobile drone nests deployed at the site, as well as data processing workstations. The drones integrate visual sensors, lidar sensors, and inertial measurement units.

[0081] The process of generating a basic 3D model of the site includes:

[0082] The satellite imagery covering the mission area is acquired by the data processing workstation, and a two-dimensional base map is generated. The satellite imagery is subjected to denoising, color balancing, geometric correction and radiometric correction operations by the data processing workstation.

[0083] The UAV acquires ground object image data, 3D point cloud data, and the UAV's own motion attitude data through onboard visual sensors, lidar sensors, and inertial measurement units. The UAV takes off from a mobile UAV pod and flies at low altitude over the mission area according to a preset route.

[0084] Ground feature image data, 3D point cloud data, and motion attitude data are input into the data processing workstation. The data processing workstation uses simultaneous localization and mapping (SMR) technology to reverse-engineer the six-degree-of-freedom pose of the UAV at each moment, and simultaneously projects the ground feature image data and 3D point cloud data onto the global coordinate system to construct a 3D point cloud map of the mission area. It also uses closed-loop detection and global optimization technology to eliminate accumulated errors.

[0085] Cumulative error refers to the pose estimation deviation that increases over time due to measurement uncertainties during continuous movement of a UAV. Loop closure detection identifies the flight path feature sequence list of the mission area when the UAV moves to a previously visited mission area, thereby forming a constraint loop in the path trajectory. Global optimization technology uses the constraint loop to perform overall mathematical optimization on all pose nodes of the flight trajectory and the 3D point cloud map, redistributing and correcting the cumulative error to ensure the global consistency of the 3D point cloud map.

[0086] The two-dimensional base map and the three-dimensional point cloud map are georeferenced and aligned. Using the two-dimensional base map as a reference, the texture information of the three-dimensional point cloud map is rendered and mapped using ground feature image data to generate a three-dimensional base model of the site.

[0087] The process of acquiring 3D spatial data within the UAV mission area and point cloud data ahead of the flight path, and preprocessing the point cloud data, includes:

[0088] According to the planned flight path, the three-dimensional spatial data of the flight path and surrounding areas are retrieved and loaded from the on-site three-dimensional basic model stored in the data processing workstation. The three-dimensional spatial data includes terrain elevation data, location data and contour data of fixed obstacles within the mission area.

[0089] The system acquires 3D point cloud data in real time with the lidar sensor as the origin of the coordinate system. The 3D point cloud data is obtained by the lidar sensor continuously emitting laser beams into a fan-shaped area in front of the flight path and receiving the reflected signals.

[0090] The 3D point cloud data is projected onto the global coordinate system, and the projected 3D point cloud data is filtered to remove noise points. The filtered 3D point cloud data is then segmented into point cloud clusters that represent the contours of independent obstacles using a density-based spatial clustering algorithm, resulting in preprocessed point cloud data.

[0091] The flight path feature sequence list includes the slope features along the flight path and surrounding areas, the position features and envelope volume features of fixed obstacles, the centroid position features of point cloud clusters, and the closest distance features relative to the UAV's flight direction.

[0092] The slope feature is obtained by calculating the ratio of the vertical elevation difference between two sampling points based on the terrain elevation data of the flight path and the surrounding area to the horizontal distance, thus representing the terrain slope angle.

[0093] The positional features are obtained by using the three-dimensional coordinates of the center point of the fixed obstacle in the global coordinate system, and the envelope volume features are characterized by the length, width and height dimensions of the minimum orthogonal bounding box that encloses the fixed obstacle.

[0094] The centroid location feature is obtained by summing the three-dimensional coordinates of each point within the point cloud cluster and then taking the average value.

[0095] The nearest distance feature is obtained by traversing the points within the point cloud cluster, calculating the Euclidean distance from each point to the current 3D coordinates of the UAV, and taking the minimum value of the Euclidean distance.

[0096] The method of inputting the flight path feature sequence list into the global path prediction model and outputting the optimal reference path vector is as follows: A global path prediction model based on a genetic algorithm is established using slope features, location features, and envelope volume features as inputs. The optimal reference path vector is then output using the global path prediction model. The process includes:

[0097] Initial path set generation stage: A set of ordered 3D path point sequences is used to represent the feasible flight path from the starting point to the ending point. Based on position features and envelope volume features, a set number of feasible flight paths are generated through obstacle collision detection. The feasible flight paths constitute the initial path set.

[0098] Fitness function construction phase: The fitness function is used to quantitatively evaluate each feasible flight path and calculate the fitness value F. The fitness function consists of the normalized total path length L, path energy consumption E, and obstacle collision penalty O.

[0099] ;

[0100] In the formula, , , Let L, E, and O represent the weighting coefficients, respectively. .

[0101] Selection phase: Based on fitness values, a tournament selection method is used to extract a set number of feasible flight paths from the initial path set. The feasible flight path with the lowest fitness value is selected from the extracted feasible flight paths as the optimal path, and a new path set is generated based on the optimal path.

[0102] Crossover and mutation phase: Select two feasible flight paths in pairs from the new path set, set a path point number as the crossover point, and exchange the three-dimensional path point sequences after the crossover point of the two feasible flight paths to form two new feasible flight paths. Select the three-dimensional path points in the new feasible flight paths, perform random offset and obstacle collision detection on the three-dimensional coordinates of the three-dimensional path points, and screen out new feasible flight paths that do not collide with fixed obstacles.

[0103] Termination of output phase: Iterative execution of the fitness function construction phase, selection phase and crossover mutation phase. When the fitness value of the optimal path does not decrease during the iteration process, the iteration is terminated. The optimal path is selected from the path set formed in the last iteration, and the three-dimensional path point sequence corresponding to the optimal path is used as the optimal reference path vector.

[0104] The method of inputting the flight path feature sequence list into the local path response model and outputting the instantaneous avoidance path vector is to establish an A-based model using centroid position features and nearest-neighbor features as input. * The algorithm's local path response model outputs an instantaneous avoidance path vector. The process includes:

[0105] The three-dimensional search space construction stage: With the current three-dimensional coordinate position of the UAV as the center, a three-dimensional search space is constructed. The current three-dimensional coordinate position is defined as the starting node of the path search. The path point on the optimal reference path vector that is a preset forward distance from the current three-dimensional coordinate position is defined as the target node. The centroid position feature and the nearest distance feature are mapped to the three-dimensional search space. When the distance between the node in the three-dimensional search space grid and the centroid of the point cloud cluster is less than the preset safety radius, or when the node is located inside the point cloud cluster, the node is marked as an obstacle node and filtered out.

[0106] In the path cost function definition phase: along the optimal reference path vector, calculate the sum of the spatial straight-line distances of each segment from the starting node to the current node, and use the sum as the first actual path cost to move from the starting node to the current node. The estimated cost of moving from the current node to the target node is the Euclidean distance between the current node and the target node. Combined with the first actual path cost and estimated costs Determine the cost function, and use the cost function to evaluate the path cost from the starting node to the target node via the current node. :

[0107] ;

[0108] In the formula, n represents the node to be evaluated in the constructed three-dimensional search space grid.

[0109] Path search iteration phase: Set up an open set to store nodes to be visited and a closed set to store visited nodes. Put the starting node into the open set. Define the closed set as empty initially. Select the node with the lowest path cost from the open set as the current processing node. Remove the current processing node from the open set and put it into the closed set. Determine if the current processing node is the target node. If the current processing node is the target node, end the path search iteration phase. If the current processing node is not the target node, obtain the neighboring nodes that the current processing node can reach. Calculate the second actual path cost of each neighboring node from the current processing node. If the first actual path cost is lower than the second actual path cost recorded by the neighboring node, update the second actual path cost of the neighboring node and add the neighboring node to the open set. Continue the path search iteration phase until the target node is found.

[0110] Outputting the immediate avoidance path vector stage: After the path search iteration stage ends and the target node is successfully found, a local shortest feasible path composed of the optimal node sequence is reconstructed by backtracking from the target node to the starting node. The ordered three-dimensional path point coordinate sequence corresponding to the local shortest feasible path is output as the immediate avoidance path vector.

[0111] The process of using a path decision model to weight and adjust the optimal reference path vector and the immediate avoidance path vector, and outputting a flight path vector to guide the UAV in performing path adjustments, includes:

[0112] Preset a safe distance threshold and an alarm distance threshold for the path decision model, and the safe distance threshold is greater than the alarm distance threshold;

[0113] When the minimum value of the Euclidean distance corresponding to the nearest distance feature is greater than the safe distance threshold, it is considered to be a state with no immediate collision risk. The weight of the optimal reference path vector is set to approach 1, and the weight of the immediate avoidance path vector is set to approach 0.

[0114] When the minimum Euclidean distance does not exceed the alarm distance threshold, a high collision risk is identified, the weight of the optimal reference path vector is set to approach 0, and the weight of the immediate avoidance path vector is set to approach 1.

[0115] When the minimum value of the Euclidean distance is within the alarm distance threshold With safe distance threshold During this period, it is considered a transitional state for path decision-making, and the weight of the optimal reference path vector is determined. and the weight of the immediate avoidance path vector Linear interpolation is performed based on the minimum Euclidean distance d, and the sum of the weights of the optimal reference path vector and the instantaneous avoidance path vector is 1:

[0116] ;

[0117] By using the weights of the optimal reference path vector and the instantaneous avoidance path vector, the three-dimensional coordinates of the path points with corresponding indices in the optimal reference path vector and the instantaneous avoidance path vector are weighted and averaged to generate a new path point sequence. The new path point sequence constitutes the flight path vector.

[0118] The process of simulating a non-visual flight scenario on a 3D basic model, training a path decision model offline, and resolving the flight path vector into flight control commands that drive the UAV includes:

[0119] Based on the on-site 3D basic model, a 3D simulation environment is constructed, and non-visual flight test cases are generated in the 3D simulation environment. Each non-visual flight test case contains a set of UAV start and end points.

[0120] In a 3D simulation environment, obstacles not included in the on-site 3D basic model are set up. By optimizing the solver, the optimal flight path is analyzed to enable the UAV to meet safety constraints and complete the task in the shortest time and with the lowest energy consumption in each non-visual flight test scenario.

[0121] The safe distance threshold and alarm distance threshold corresponding to the optimal flight path are used as the ideal solution of the non-visual flight test case. The non-visual flight test case and the corresponding ideal solution constitute the evaluation dataset for path decision model optimization.

[0122] A target function is defined to quantitatively evaluate the performance of the path decision model. The target function takes the decision parameters to be tested as input and runs the path decision model on non-visual flight test cases in the evaluation dataset. The decision parameters to be tested include the safe distance threshold and the alarm distance threshold. The comprehensive performance indicators of the non-visual flight test cases are statistically analyzed, including the average path length, average flight time, average minimum distance to obstacles, and number of collisions.

[0123] The grid search method is used to search within the preset range of values ​​of the decision parameters to be tested. For each combination of decision parameters to be tested, the comprehensive performance score is calculated through the objective function. After traversing all combinations of decision parameters to be tested, the decision parameters that result in the highest comprehensive performance score are selected as the calibration parameters of the path decision model.

[0124] The flight path vector is received by the flight controller, which then decomposes the flight path vector into path segments connecting adjacent path points.

[0125] Using the flight controller based on the decomposed path segments, the flight speed vector is calculated and the corresponding flight attitude, including roll angle, pitch angle and yaw angle, is analyzed.

[0126] The attitude control system of the UAV uses the flight speed vector and flight attitude as setpoints. The attitude control system has a built-in proportional-integral-derivative (PID) controller. The PID controller compares the motion attitude data measured by the inertial measurement unit with the flight attitude. Based on the error between the motion attitude data and the flight attitude, it calculates and outputs flight control commands to adjust the speed of the UAV motors, so as to drive the UAV to fly along the flight path vector.

[0127] The process of calculating the path adjustment magnitude based on the optimal reference path vector and the flight path vector includes:

[0128] Calculate the difference vector between the flight path vector and the optimal reference path vector, and quantify the difference vector into the path adjustment magnitude by obtaining the Euclidean norm of the difference vector;

[0129] Set an amplitude adjustment threshold and a number of calculation cycles. Compare the path adjustment amplitude with the amplitude adjustment threshold. If the path adjustment amplitude exceeds the amplitude adjustment threshold continuously within the number of calculation cycles, it is determined that the UAV is in a complex environment and an automatic complex environment warning signal is generated. If the path adjustment amplitude does not meet the requirement of exceeding the amplitude adjustment threshold continuously within the number of calculation cycles, the UAV executes flight control commands.

[0130] The process of updating the on-site 3D basic model and displaying the drone's flight path in 3D includes:

[0131] Through time series analysis, newly added static obstacles whose positions remain unchanged in the global coordinate system are identified from the real-time detected point cloud clusters. The point cloud data of the point cloud clusters and the corresponding position features and envelope volume features of the point cloud clusters are then integrated into the on-site three-dimensional basic model.

[0132] Using the fused on-site 3D basic model as the background of the 3D digital environment, the monitoring terminal renders and overlays the drone entity, the optimal reference path vector and the actual flight path vector of different colors, as well as the known and real-time detected obstacles, and provides visual prompts when a complex environment warning is triggered.

[0133] The verification process is as follows:

[0134] Simulation experiments were conducted to verify the ability of the method in this embodiment to combine global planning and local reaction in dynamic, non-visual environments.

[0135] Using a raster map as the background and planning time, response cycle, and path smoothness as evaluation metrics, the simulation experiment diagram is as follows: Figures 2-5 As shown.

[0136] Figure 2In the diagram, the bottom left corner of each sub-map is the starting point (starting node), the top right corner is the ending point (target node), the curve from the starting point to the ending point is the planned path, and the black squares represent the positions of fixed obstacles.

[0137] Figure 3 The diagram shows the global path planning stage of the method in this embodiment. The flight path feature sequence list is input into the global path prediction model. The global path prediction model is based on a genetic algorithm and outputs an optimal reference path (the dashed line from the starting node to the target node in the figure). This path successfully avoids all fixed obstacles.

[0138] Figure 4 The diagram shows the stage of a drone (hexagon in the figure) flying along the optimal path in the absence of immediate collision risk. According to the path decision model in this embodiment, when there is no threat of dynamic obstacles (dots on the dashed line in the figure), the drone's flight path (solid line in the figure) completely coincides with the optimal reference path (dashed line from the starting node to the target node in the figure).

[0139] Figure 5 As shown, when the UAV flies ahead of the predetermined path, the simulated lidar sensor detects a dynamic obstacle in real time (the dot on the dashed line in the figure). At this time, the local path response model based on the A* algorithm is activated to generate an instant avoidance path to bypass the dynamic obstacle (the dashed line in the figure that does not start from the starting node). At the same time, the path decision model smoothly adjusts the flight path according to the weighted adjustment strategy, so that the flight path shifts towards the instant avoidance path.

[0140] ​ The diagram shows the stage where the drone successfully avoids the obstacle. As the distance between the drone and the obstacle decreases, the weight of the instantaneous avoidance path approaches 1, causing the drone's actual flight trajectory (solid line in the diagram) to follow the instantaneous avoidance path. After successfully overcoming the obstacle, the instantaneous avoidance path disappears, and the flight path smoothly returns to the globally optimal reference path, continuing to fly towards the destination.

[0141] This simulation example verifies that the method in this embodiment effectively solves the conflict between the preset flight path and real-time obstacle avoidance by combining global optimal path planning (genetic algorithm) and local dynamic obstacle avoidance (A* algorithm). By dynamically weighting the outputs of the path decision model and the local path response model through the path decision model, the continuity and smoothness of the UAV flight path are guaranteed.

[0142] Example 2: A drone flight path adjustment device based on obstacle avoidance in non-visual conditions, comprising:

[0143] One or more processors;

[0144] Memory, used to store one or more computer programs;

[0145] When one or more programs are executed by one or more processors, the one or more processors execute the method in Example 1.

[0146] Example 3: A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, implement the method in Example 1.

Claims

1. A method for adjusting the flight path of a UAV based on obstacle avoidance in non-visual conditions, characterized in that, Includes the following steps: Using satellite imagery and drones, a three-dimensional basic model of the site is generated through simultaneous localization and mapping (SLAM) technology. Using a 3D basic model and lidar sensors, the system acquires 3D spatial data within the UAV mission area and point cloud data ahead of the flight path, and preprocesses the point cloud data. The flight path feature sequence table is obtained from the three-dimensional spatial data and the preprocessed point cloud data. The flight path feature sequence table includes the closest distance features of point cloud clusters relative to the UAV's flight direction. The flight path feature sequence list is input into the global path prediction model and the local path response model respectively to obtain the optimal reference path vector and the instantaneous avoidance path vector respectively. The optimal reference path vector and the immediate avoidance path vector are weighted and adjusted using a path decision model to output a flight path vector that guides the UAV in performing path adjustments. Specifically, this includes: Preset a safe distance threshold and an alarm distance threshold for the path decision model, and the safe distance threshold is greater than the alarm distance threshold; When the minimum value of the Euclidean distance corresponding to the nearest distance feature is greater than the safe distance threshold, it is considered to be a state with no immediate collision risk. The weight of the optimal reference path vector is set to approach 1, and the weight of the immediate avoidance path vector is set to approach 0. When the minimum Euclidean distance does not exceed the alarm distance threshold, a high collision risk is identified, the weight of the optimal reference path vector is set to approach 0, and the weight of the immediate avoidance path vector is set to approach 1. When the minimum value of the Euclidean distance is within the alarm distance threshold With safe distance threshold During this period, it is considered a transitional state for path decision-making, and the weight of the optimal reference path vector is determined. and the weight of the immediate avoidance path vector Linear interpolation is performed based on the minimum Euclidean distance d, and the sum of the weights of the optimal reference path vector and the instantaneous avoidance path vector is 1: ； By using the weights of the optimal reference path vector and the instantaneous avoidance path vector, the three-dimensional coordinates of the path points with corresponding indices in the optimal reference path vector and the instantaneous avoidance path vector are weighted and averaged to generate a new path point sequence. The new path point sequence constitutes the flight path vector. The non-visual flight scenario is simulated on a 3D basic model of the scene, the path decision model is trained offline, and the flight path vector is parsed into flight control commands that drive the UAV to execute. Calculate the path adjustment magnitude based on the optimal reference path vector and the flight path vector; Update the on-site 3D basic model and display the drone's flight path in 3D.

2. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 1, characterized in that, The process of generating the on-site 3D basic model includes: The satellite imagery covering the mission area is acquired through a data processing workstation, and a two-dimensional base map is generated. The UAV acquires ground object image data, 3D point cloud data, and the UAV's own motion attitude data through onboard visual sensors, lidar sensors, and inertial measurement units. The UAV takes off from a mobile UAV pod and flies at low altitude over the mission area according to a preset route. Ground feature image data, 3D point cloud data, and motion attitude data are input into the data processing workstation. The data processing workstation uses simultaneous localization and mapping (SMR) technology to reverse-engineer the six-degree-of-freedom pose of the UAV at each moment, and simultaneously projects the ground feature image data and 3D point cloud data onto the global coordinate system to construct a 3D point cloud map of the mission area. It also uses closed-loop detection and global optimization technology to eliminate accumulated errors. The two-dimensional base map and the three-dimensional point cloud map are georeferenced and aligned. Using the two-dimensional base map as a reference, the texture information of the three-dimensional point cloud map is rendered and mapped using ground feature image data to generate a three-dimensional base model of the site.

3. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 2, characterized in that, The process of acquiring three-dimensional spatial data within the UAV mission area and point cloud data ahead of the flight path, and preprocessing the point cloud data, includes: According to the planned flight path, the three-dimensional spatial data of the flight path and surrounding areas are retrieved and loaded from the on-site three-dimensional basic model stored in the data processing workstation. The three-dimensional spatial data includes terrain elevation data, location data and contour data of fixed obstacles within the mission area. The system acquires 3D point cloud data in real time with the lidar sensor as the origin of the coordinate system. The 3D point cloud data is obtained by the lidar sensor continuously emitting laser beams into a fan-shaped area in front of the flight path and receiving the reflected signals. The 3D point cloud data is projected onto the global coordinate system, and the projected 3D point cloud data is filtered to remove noise points. The filtered 3D point cloud data is then segmented into point cloud clusters that represent the contours of independent obstacles using a density-based spatial clustering algorithm, resulting in preprocessed point cloud data.

4. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 3, characterized in that, The flight path feature sequence table includes the slope features along the flight path and surrounding areas, the position features and envelope volume features of fixed obstacles, the centroid position features of point cloud clusters, and the closest distance features relative to the UAV's flight direction. The slope feature is obtained by calculating the ratio of the vertical elevation difference between two sampling points based on the terrain elevation data of the flight path and the surrounding area to the horizontal distance, thus representing the terrain slope angle. The positional features are obtained by using the three-dimensional coordinates of the center point of the fixed obstacle in the global coordinate system, and the envelope volume features are characterized by the length, width and height dimensions of the minimum orthogonal bounding box that encloses the fixed obstacle. The centroid location feature is obtained by summing the three-dimensional coordinates of each point within the point cloud cluster and then taking the average value. The nearest distance feature is obtained by traversing the points within the point cloud cluster, calculating the Euclidean distance from each point to the current 3D coordinates of the UAV, and taking the minimum value of the Euclidean distance.

5. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that, The method of inputting the flight path feature sequence list into the global path prediction model and outputting the optimal reference path vector is as follows: A global path prediction model based on a genetic algorithm is established using slope features, location features, and envelope volume features as inputs. The optimal reference path vector is then output using the global path prediction model. The process includes: Initial path set generation stage: A set of ordered 3D path point sequences is used to represent the feasible flight path from the starting point to the ending point. Based on position features and envelope volume features, a set number of feasible flight paths are generated through obstacle collision detection. The feasible flight paths constitute the initial path set. Fitness function construction phase: The fitness function is used to quantitatively evaluate each feasible flight path and calculate the fitness value. The fitness function consists of the normalized total path length, path energy consumption and obstacle collision penalty. Selection phase: Based on fitness values, a tournament selection method is used to extract a set number of feasible flight paths from the initial path set. The feasible flight path with the lowest fitness value is selected from the extracted feasible flight paths as the optimal path, and a new path set is generated based on the optimal path. Crossover and mutation phase: Select two feasible flight paths in pairs from the new path set, set a path point number as the crossover point, and exchange the three-dimensional path point sequences after the crossover point of the two feasible flight paths to form two new feasible flight paths. Select the three-dimensional path points in the new feasible flight paths, perform random offset and obstacle collision detection on the three-dimensional coordinates of the three-dimensional path points, and screen out new feasible flight paths that do not collide with fixed obstacles. Termination of output phase: Iterative execution of the fitness function construction phase, selection phase and crossover mutation phase. When the fitness value of the optimal path does not decrease during the iteration process, the iteration is terminated. The optimal path is selected from the path set formed in the last iteration, and the three-dimensional path point sequence corresponding to the optimal path is used as the optimal reference path vector.

6. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that, The method of inputting the flight path feature sequence list into the local path response model and outputting an instantaneous avoidance path vector is as follows: using centroid position features and nearest distance features as input, a model is established based on A... * The algorithm's local path response model outputs an instantaneous avoidance path vector. The process includes: The three-dimensional search space construction stage: With the current three-dimensional coordinate position of the UAV as the center, a three-dimensional search space is constructed. The current three-dimensional coordinate position is defined as the starting node of the path search. The path point on the optimal reference path vector that is a preset forward distance from the current three-dimensional coordinate position is defined as the target node. The centroid position feature and the nearest distance feature are mapped to the three-dimensional search space. When the distance between the node in the three-dimensional search space grid and the centroid of the point cloud cluster is less than the preset safety radius, or when the node is located inside the point cloud cluster, the node is marked as an obstacle node and filtered out. In the path cost function definition stage: along the optimal reference path vector, calculate the sum of the spatial straight-line distances of each segment from the starting node to the current node. Use the sum as the first actual path cost from the starting node to the current node, and use the Euclidean straight-line distance between the current node and the target node as the estimated cost from the current node to the target node. Combine the first actual path cost and the estimated cost to determine the cost function. Use the cost function to evaluate the path cost from the starting node to the target node via the current node. Path search iteration phase: Set up an open set to store nodes to be visited and a closed set to store visited nodes. Put the starting node into the open set. Define the closed set as empty initially. Select the node with the lowest path cost from the open set as the current processing node. Remove the current processing node from the open set and put it into the closed set. Determine if the current processing node is the target node. If the current processing node is the target node, end the path search iteration phase. If the current processing node is not the target node, obtain the neighboring nodes that the current processing node can reach. Calculate the second actual path cost of each neighboring node from the current processing node. If the first actual path cost is lower than the second actual path cost recorded by the neighboring node, update the second actual path cost of the neighboring node and add the neighboring node to the open set. Continue the path search iteration phase until the target node is found. Outputting the immediate avoidance path vector stage: After the path search iteration stage ends and the target node is successfully found, a local shortest feasible path composed of the optimal node sequence is reconstructed by backtracking from the target node to the starting node. The ordered three-dimensional path point coordinate sequence corresponding to the local shortest feasible path is output as the immediate avoidance path vector.

7. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 1, characterized in that, The process of simulating a non-visual flight scenario on a 3D basic model, training a path decision model offline, and resolving the flight path vector into flight control commands that drive the UAV includes: Based on the on-site 3D basic model, a 3D simulation environment is constructed, and non-visual flight test cases are generated in the 3D simulation environment. Each non-visual flight test case contains a set of UAV start and end points. In a 3D simulation environment, obstacles not included in the on-site 3D basic model are set up. By optimizing the solver, the optimal flight path is analyzed to enable the UAV to meet safety constraints and complete the task in the shortest time and with the lowest energy consumption in each non-visual flight test scenario. The safe distance threshold and alarm distance threshold corresponding to the optimal flight path are used as the ideal solution of the non-visual flight test case. The non-visual flight test case and the corresponding ideal solution constitute the evaluation dataset for path decision model optimization. A target function is defined to quantitatively evaluate the performance of the path decision model. The target function takes the decision parameters to be tested as input and runs the path decision model on non-visual flight test cases in the evaluation dataset. The decision parameters to be tested include the safe distance threshold and the alarm distance threshold. The comprehensive performance indicators of the non-visual flight test cases are statistically analyzed, including the average path length, average flight time, average minimum distance to obstacles, and number of collisions. The grid search method is used to search within the preset range of values ​​of the decision parameters to be tested. For each combination of decision parameters to be tested, the comprehensive performance score is calculated through the objective function. After traversing all combinations of decision parameters to be tested, the decision parameters that result in the highest comprehensive performance score are selected as the calibration parameters of the path decision model. The flight path vector is received by the flight controller, which then decomposes the flight path vector into path segments connecting adjacent path points. Using the flight controller based on the decomposed path segments, the flight speed vector is calculated and the corresponding flight attitude, including roll angle, pitch angle and yaw angle, is analyzed. The attitude control system of the UAV uses the flight speed vector and flight attitude as setpoints. The attitude control system has a built-in proportional-integral-derivative (PID) controller. The PID controller compares the motion attitude data measured by the inertial measurement unit with the flight attitude. Based on the error between the motion attitude data and the flight attitude, it calculates and outputs flight control commands to adjust the speed of the UAV motors, so as to drive the UAV to fly along the flight path vector.

8. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 1, characterized in that, The process of calculating the path adjustment magnitude based on the optimal reference path vector and the flight path vector includes: Calculate the difference vector between the flight path vector and the optimal reference path vector, and quantify the difference vector into the path adjustment magnitude by obtaining the Euclidean norm of the difference vector; Set an amplitude adjustment threshold and a number of calculation cycles. Compare the path adjustment amplitude with the amplitude adjustment threshold. If the path adjustment amplitude exceeds the amplitude adjustment threshold continuously within the number of calculation cycles, it is determined that the UAV is in a complex environment and an automatic complex environment warning signal is generated. If the path adjustment amplitude does not meet the requirement of exceeding the amplitude adjustment threshold continuously within the number of calculation cycles, the UAV executes flight control commands.

9. The method for adjusting the flight path of a UAV based on obstacle avoidance in a non-visual state according to claim 4, characterized in that, The process of updating the on-site 3D basic model and displaying the drone's flight path in 3D includes: Through time series analysis, newly added static obstacles whose positions remain unchanged in the global coordinate system are identified from the real-time detected point cloud clusters. The point cloud data of the point cloud clusters and the corresponding position features and envelope volume features of the point cloud clusters are then integrated into the on-site three-dimensional basic model. Using the fused on-site 3D basic model as the background of the 3D digital environment, the monitoring terminal renders and overlays the drone entity, the optimal reference path vector and the actual flight path vector of different colors, as well as the known and real-time detected obstacles, and provides visual prompts when a complex environment warning is triggered.

Images