Obstacle avoidance method, device and storage medium for autonomous flight of UAV power inspection based on laser radar

Abstract

The present invention discloses an obstacle avoidance method, device and storage medium for autonomous flight of unmanned aerial vehicle power inspection based on laser radar, which relates to the technical field of unmanned aerial vehicle obstacle avoidance, including configuring laser radar and visual sensors on unmanned aerial vehicles, collecting three-dimensional point cloud data and image data of the flight environment in real time, and generating a three-dimensional environment model after optimizing the point cloud data; classifying obstacles into static and dynamic based on the three-dimensional environment model, if the obstacle is a static obstacle, identifying the obstacle type and locating it through the CNN model, if the obstacle is a dynamic obstacle, clustering the motion trajectory through the K-Medoids clustering method. The present invention can not only identify and locate static obstacles, but also predict the motion trajectory of dynamic obstacles such as flying birds through dynamic obstacle prediction technology, thereby generating a preliminary flight path, and sensing and avoiding surrounding obstacles in real time during the flight, to ensure that the unmanned aerial vehicle can smoothly perform in the power inspection task.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle obstacle avoidance, and in particular to an obstacle avoidance method, device and storage medium for autonomous flight of unmanned aerial vehicle power inspection based on laser radar. Background Art

[0002] With the rapid development of UAV technology, especially in the field of power inspection, autonomous flight obstacle avoidance technology of UAV has gradually become a research focus. When UAVs are performing power inspection tasks, they need to perform tasks in complex environments and usually encounter various static and dynamic obstacles, such as power lines, tree barriers, electric poles, flying birds, etc. In order to ensure the flight safety of UAVs, the accuracy and real-time performance of the obstacle avoidance system have become the key factors in determining whether the UAV can successfully complete the inspection task. At present, the research on obstacle avoidance technology of UAVs mainly focuses on three aspects: obstacle detection, path planning and dynamic obstacle avoidance. As a high-precision sensor, LiDAR is widely used in UAV obstacle avoidance systems. It helps UAVs achieve environmental perception by acquiring three-dimensional point cloud data of the environment in real time.

[0003] However, the existing obstacle avoidance technology for drones still has some shortcomings. First, although lidar can provide high-precision environmental data, its data processing and obstacle classification still face challenges, especially in complex environments. How to efficiently process and fuse data from different sensors is a difficult point. Secondly, most of the existing obstacle avoidance systems rely on the identification of static obstacles, and have weak prediction and avoidance capabilities for dynamic obstacles (such as flying birds or flying wires), and lack sufficient intelligent real-time response mechanisms. In addition, most of the existing path planning methods focus on the avoidance of static obstacles, and it is difficult to cope with dynamically changing flight environments, especially unpredictable obstacles or aircraft that appear during flight. Therefore, how to perceive and avoid obstacles, especially dynamic obstacles, in real time during flight is a problem that needs to be solved in the existing technology. Summary of the invention

[0004] In view of the problems existing in the above-mentioned existing obstacle avoidance methods, equipment and storage media for autonomous flight of UAV power inspection based on laser radar, the present invention is proposed.

[0005] Therefore, the present invention provides a laser radar-based obstacle avoidance method, device and storage medium for autonomous flight of UAV power inspection to solve how to perceive and avoid obstacles, especially dynamic obstacles, in real time during flight, which is a problem that needs to be urgently solved in the prior art.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar, which includes configuring a laser radar and a visual sensor on the UAV, collecting three-dimensional point cloud data and image data of the flight environment in real time, and generating a three-dimensional environment model after optimizing the point cloud data;

[0008] Based on the three-dimensional environment model, obstacles are classified into static and dynamic. If the obstacle is a static obstacle, the obstacle type is identified and located through the CNN model. If the obstacle is a dynamic obstacle, the motion trajectory is clustered through the K-Medoids clustering method, and the data features are extracted through the TFBNet model. The motion trajectory is predicted based on the UKF algorithm, the BIGRU model and the MHA.

[0009] A preliminary flight path is generated based on obstacle recognition and dynamic prediction results, and the surrounding obstacles are perceived in real time during the flight of the UAV, the obstacle avoidance path is calculated, and the flight path of the UAV is dynamically adjusted.

[0010] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar of the present invention, the laser radar and visual sensor are configured on the UAV to collect three-dimensional point cloud data and image data of the flight environment in real time, and generate a three-dimensional environment model after optimizing the point cloud data, which includes the following steps:

[0011] The laser radar collects three-dimensional point cloud data of the flight environment;

[0012] The visual sensor collects obstacle images of the flight environment through an RGB camera;

[0013] Use multi-zone planar point cloud denoising technology to denoise the point cloud data collected by the lidar, further filter the denoised point cloud data to remove redundant points, use the GICP algorithm to minimize the difference between point clouds, calculate the rotation matrix, and align the point clouds to obtain the processed point cloud data;

[0014] Preprocess the data collected by the visual sensor, extract the depth information and features in the point cloud data and visual images through the convolution layer of CNN, generate depth maps and feature maps through multiple convolution layers, and use regression methods to calculate the disparity of each point in the depth map and feature map;

[0015] When performing depth matching between visual images and point cloud data, the depth information is optimized by calculating the disparity of each pixel. The disparity value of each pixel reflects the depth information of the pixel in three-dimensional space. The candidate visual values ​​are selected through softmax, and the predicted disparity is optimized using the minimum square method to minimize the depth error between the lidar and the visual sensor.

[0016] Use the spatial propagation network to propagate the depth information from the high-density area to the low-density area to obtain the optimized point cloud data, and use the Chamfer distance to evaluate the accuracy of the optimized point cloud data to obtain the final optimized point cloud data;

[0017] Time-synchronize the collected and optimized data and mark them with timestamps;

[0018] A weighted fusion algorithm is used to integrate data from different sensors to generate a complete three-dimensional environment model that includes the spatial position and appearance characteristics of obstacles.

[0019] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar of the present invention, if the obstacle is a dynamic obstacle, the motion trajectory is clustered by K-Medoids clustering method, data features are extracted by TFBNet model, and motion trajectory prediction is performed based on UKF algorithm, BIGRU model and MHA, including the following steps:

[0020] Collect obstacle data at each time point to construct the obstacle state vector S t， The state vectors collected from multiple time steps are organized in chronological order to form a complete trajectory data set. Each trajectory data set consists of the state vectors of the obstacle at multiple time steps, representing the movement trajectory of the obstacle T i ;

[0021] Perform feature extraction on each trajectory and standardize all trajectory data;

[0022] The number of clusters is selected using the elbow rule, the K-Medoids clustering method is used to cluster the movement trajectories of obstacles, the Hausdorff distance formula is used to measure the similarity between the clustered trajectories, and the trajectories of obstacles are grouped according to the movement mode to obtain a result containing multiple trajectory categories;

[0023] The clustered trajectory data is used as the input of the TFBNet model, the spatial features in the trajectory data are extracted through the convolution layer, and the trajectory features are mapped to a high-dimensional space;

[0024] The data mapped to the high-dimensional space is input into the UKF algorithm to make a preliminary state estimate of the current position, velocity and acceleration of the obstacle. The state is expanded through the sigma point, and the current state is propagated to the future moment using nonlinear transformation. The future trajectory of the obstacle is predicted to obtain the preliminary future trajectory of the obstacle.

[0025] The obtained preliminary future trajectory data is converted into time series data, and the preliminary trajectory prediction data is input into the BiGRU model for time series modeling to obtain the optimized trajectory features;

[0026] The time series modeling results generated by BiGRU are used as input and passed to MHA for attention weighting. MHA generates attention weights by calculating query Q, key K and value V, and further weights each time step in the trajectory. The weight of each time step is calculated by using softmax, and the weighted sum is performed according to the calculated attention weights. The weighted trajectory features are obtained as the input of the Gaussian process regression model to predict future trajectories and generate multiple prediction targets.

[0027] Calculate the minimum distance between each trajectory and the current actual position of the obstacle, and sort the trajectories in descending order according to the distance to generate a distance sorting table;

[0028] Calculate the acceleration and velocity of each trajectory and compare the results with the historical motion patterns of the collected obstacles. If they are inconsistent, exclude the trajectory from the distance sorting table.

[0029] The first trajectory in the distance sorting table after exclusion is selected as the predicted trajectory of the dynamic obstacle.

[0030] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar described in the present invention, wherein: the classification of obstacles into static and dynamic based on the three-dimensional environment model refers to obtaining the initial position of the obstacle based on the obstacle spatial positioning information in the three-dimensional environment model, tracking the spatial position of the obstacle through real-time data, comparing the obstacle positions collected at different time points, calculating the position change amplitude of each obstacle through the Euclidean distance formula, setting a threshold A, if the change amplitude is greater than or equal to the threshold A, it is a dynamic obstacle, otherwise it is a static obstacle.

[0031] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar of the present invention, if the obstacle is a static obstacle, identifying the obstacle type and locating it by using the CNN model includes the following steps:

[0032] Obtain the precise spatial position of the obstacle through the point cloud data of the laser radar and obtain the three-dimensional coordinates (x, y, z) of the obstacle;

[0033] Use convolutional neural networks to build object classification models, including input layers, convolution layers, pooling layers, and output layers;

[0034] The historical image data is used as a training set to input the object classification model for iterative training, and the loss function and Adam optimizer are defined to iteratively optimize the model parameters. When the loss of the object classification model no longer decreases significantly during the continuous iteration process, the iteration is stopped and the model parameters are output to update the object classification model.

[0035] Input the real-time obstacle image into the object classification model to obtain the static obstacle category;

[0036] The static obstacle categories include power lines, tree barriers, and utility poles.

[0037] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar of the present invention, wherein: the generating of the preliminary flight path based on obstacle identification and dynamic prediction results refers to marking the state and classification results of the obstacle into the three-dimensional environment model, using the grid map method to divide the environment into multiple grids, each grid unit represents the corresponding spatial area, and marking the obstacle area on the map according to the position of the obstacle;

[0038] According to the position of the flight target, the Dijkstra algorithm is used to plan the preliminary flight path of the UAV in the grid map. If the predicted trajectory of the dynamic obstacle and the flight path of the UAV have an intersection, the speed obstacle method is used to calculate the relative speed of the dynamic obstacle and determine a geometric area of ​​relative motion. If the trajectory of the UAV is on the trajectory of the geometric area, there is a risk of collision and the flight path of the UAV is replanned.

[0039] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar described in the present invention, wherein: the real-time perception of surrounding obstacles during the flight of the UAV, calculation of obstacle avoidance path, and dynamic adjustment of the UAV flight path means that the UAV performs tasks in the power inspection area according to the planned path, perceives surrounding obstacles in real time through sensors, generates the obstacle avoidance path of the UAV based on the position of the obstacle and the relative motion of the UAV through the artificial potential field method, and adjusts it in real time, and stores the real-time data of the UAV in the database.

[0040] As a preferred solution of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar described in the present invention, the real-time data of the UAV is stored in the database, which means storing the flight path and obstacle avoidance data of the UAV in the database, and updating the obstacle data viewed during the real-time flight of the UAV to the three-dimensional environment model for marking, storing the updated three-dimensional environment model data in the database and performing real-time access control.

[0041] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the obstacle avoidance method for autonomous flight of unmanned aerial vehicle power inspection based on laser radar as described in the first aspect of the present invention is implemented.

[0042] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the obstacle avoidance method for autonomous flight of unmanned aerial vehicle power inspection based on laser radar as described in the first aspect of the present invention.

[0043] The beneficial effects of the present invention are as follows: by integrating laser radar, visual sensors and other auxiliary sensors, three-dimensional point cloud data and image data of the flight environment are collected in real time to generate a high-precision three-dimensional environmental model, which can not only identify and locate static obstacles, but also predict the motion trajectory of dynamic obstacles such as flying birds through dynamic obstacle prediction technology, thereby generating a preliminary flight path, and sensing and avoiding surrounding obstacles in real time during flight. Through path adjustment based on obstacle identification and dynamic prediction results, the obstacle avoidance path can be dynamically calculated, and the flight path can be adjusted in real time to ensure that the UAV can smoothly perform in the power inspection task. Through three-dimensional environmental modeling and data fusion, the environmental perception ability and the accuracy of path planning are improved, and it has a higher real-time response capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0045] Figure 1 Schematic diagram of the process of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar in Example 1. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0049] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and provides an obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar includes the following steps:

[0050] S1. Configure laser radar and visual sensors on the drone to collect 3D point cloud data and image data of the flight environment in real time, and generate a 3D environment model after optimizing the point cloud data;

[0051] Specifically, the laser radar collects three-dimensional point cloud data of the flight environment, including the x, y, z coordinates, distance and reflection intensity of obstacles, to form spatial positioning data of obstacles;

[0052] The visual sensor collects obstacle images of the flight environment through an RGB camera and analyzes the color, texture and shape in the image;

[0053] Use multi-zone planar point cloud denoising technology to denoise the point cloud data collected by the lidar, further filter the denoised point cloud data to remove redundant points, use the GICP algorithm to minimize the difference between point clouds, calculate the rotation matrix, and align the point clouds to obtain the processed point cloud data;

[0054] Through multi-zone planar point cloud denoising technology, the point cloud data collected by the lidar is denoised, which can eliminate noise and redundant points from the sensor and improve the accuracy and clarity of the data. The use of the GICP algorithm further optimizes the alignment accuracy of the point cloud and ensures that the point cloud data from different sensors can be accurately fused.

[0055] The data collected by the visual sensor is preprocessed, and the depth information and features in the point cloud data and visual images are extracted through the convolution layer of CNN. The depth map and feature map are generated through multiple convolution layers, and the regression method is used to calculate the disparity of each point in the depth map and feature map:

[0056]

[0057] Where bd is the predicted disparity value, indicating the change in depth information, d is the candidate disparity value, is the maximum candidate disparity value, indicating the maximum range of disparity, is a softmax function that calculates the probability of each disparity candidate value and is used to generate the disparity regression result. is the matching generated value, which indicates the matching degree of each disparity candidate value and is used to determine the most suitable depth match;

[0058] When performing depth matching between visual images and point cloud data, the depth information is optimized by calculating the disparity of each pixel. The disparity value of each pixel reflects the depth information of the pixel in three-dimensional space. The candidate visual values ​​are selected through softmax, and the predicted disparity is optimized using the minimum square method to minimize the depth error between the lidar and the visual sensor:

[0059]

[0060] Where R is the rotation matrix, which is used to describe the rotation from one coordinate system to another. and is the normal vector of the lidar and visual sensor data points, N is the total number of data points, i is the index, and min() represents the operation of finding the minimum value;

[0061] The depth information is extracted from the visual image through a convolutional neural network (CNN) and matched with the point cloud data of the LiDAR to optimize the depth information of the point cloud data. The Softmax function is used to optimize the depth matching process, effectively reducing the depth error between the LiDAR and the visual sensor, thereby enhancing the accuracy of environmental perception.

[0062] Use the spatial propagation network to propagate the depth information from the high-density area to the low-density area to obtain the optimized point cloud data, and use the Chamfer distance to evaluate the accuracy of the optimized point cloud data to obtain the final optimized point cloud data;

[0063] The spatial propagation network is used to propagate the depth information of high-density areas to low-density areas, improving the integrity and accuracy of point cloud data. The accuracy of the optimized point cloud data is evaluated by Chamfer distance, ensuring that the final 3D environment model is more accurate and the spatial position and appearance characteristics of obstacles are clearly depicted.

[0064] Time-synchronize the collected and optimized data and mark them with timestamps;

[0065] The weighted fusion algorithm is used to integrate the data of different sensors to generate a complete 3D environment model that includes the spatial position and appearance characteristics of obstacles. By integrating the lidar, visual sensor and ADS-B data through the weighted fusion algorithm, a complete and accurate 3D environment model can be generated, so that the spatial position and appearance characteristics of obstacles can be fully presented. This method can solve the limitations of single sensor data and achieve efficient fusion of multi-sensor data.

[0066] By collecting 3D point cloud data in real time through LiDAR, the system can accurately build a spatial model of the surrounding environment. This provides the drone with high-precision obstacle spatial positioning capabilities, helping to identify and track obstacles in the flight path, especially static obstacles such as power lines and poles. By integrating LiDAR and visual sensors, a high-precision 3D environment modeling and dynamic obstacle avoidance method is provided. Through precise 3D point cloud data collection, depth information optimization, point cloud denoising and fusion and other technical steps, the system can efficiently identify and avoid static and dynamic obstacles.

[0067] S2. Based on the three-dimensional environment model, obstacles are classified into static and dynamic. If the obstacle is a static obstacle, the obstacle type is identified and located through the CNN model. If the obstacle is a dynamic obstacle, the motion trajectory is clustered through the K-Medoids clustering method, and the data features are extracted through the TFBNet model. The motion trajectory is predicted based on the UKF algorithm, the BIGRU model and the MHA.

[0068] Specifically, the classification of obstacles into static and dynamic based on the three-dimensional environment model refers to obtaining the initial position of the obstacle based on the spatial positioning information of the obstacle in the three-dimensional environment model, tracking the spatial position of the obstacle through real-time data (from the lidar and the visual sensor), comparing the obstacle positions collected at different time points, calculating the position change amplitude of each obstacle through the Euclidean distance formula, setting the threshold A by performing statistical analysis on the historical position data, and if the change amplitude is greater than or equal to the threshold A, it is a dynamic obstacle, otherwise it is a static obstacle.

[0069] By tracking the spatial position of obstacles in real time and calculating the magnitude of their position changes, the system can classify obstacles as static or dynamic based on the speed and magnitude of the changes. This classification process uses the Euclidean distance formula to calculate the position change of each obstacle and compare it with the set threshold A. This method can efficiently distinguish between static and dynamic obstacles. For dynamic obstacles, the system can predict their movement trajectory and adjust the flight path in a timely manner, effectively reducing the risk of collision during flight. For static obstacles, traditional path planning strategies can be used to avoid them, greatly improving the flight safety and path planning efficiency of drones.

[0070] Furthermore, if the obstacle is a static obstacle, identifying the obstacle type and locating it by using a CNN model includes the following steps:

[0071] Obtain the precise spatial position of the obstacle through the point cloud data of the laser radar and obtain the three-dimensional coordinates (x, y, z) of the obstacle;

[0072] Use convolutional neural networks to build object classification models, including input layers, convolution layers, pooling layers, and output layers;

[0073] The historical image data is used as a training set to input the object classification model for iterative training, and the loss function and Adam optimizer are defined to iteratively optimize the model parameters. When the loss of the object classification model no longer decreases significantly during the continuous iteration process, the iteration is stopped and the model parameters are output to update the object classification model.

[0074] Input the real-time obstacle image into the object classification model to obtain the static obstacle category;

[0075] The static obstacle categories include power lines, tree barriers, and utility poles.

[0076] By integrating LiDAR, visual sensors and convolutional neural networks (CNN), efficient recognition and classification of static obstacles are achieved. By combining historical image data with real-time image data, the system can generate a three-dimensional spatial model of obstacles in real time and classify them, thereby providing data support for flight path planning. This method has high-precision obstacle recognition capabilities, can effectively avoid obstacles in complex environments, reduce manual intervention, and improve the autonomy and safety of drones in tasks such as power inspections.

[0077] Furthermore, if the obstacle is a dynamic obstacle, the motion trajectory is clustered by the K-Medoids clustering method, data features are extracted by the TFBNet model, and the motion trajectory prediction is performed based on the UKF algorithm, the BIGRU model and the MHA, including the following steps:

[0078] Collect the position coordinates, velocity and acceleration of the obstacle at each time point to construct the state vector S of the obstacle t， The state vectors collected from multiple time steps are organized in chronological order to form a complete trajectory data set. Each trajectory data set consists of the state vectors of the obstacle at multiple time steps, representing the movement trajectory of the obstacle T i ;

[0079] Perform feature extraction on each trajectory and standardize all trajectory data;

[0080] The elbow rule is used to select the number of clusters, that is, the number of obstacle trajectory groups. The K-Medoids clustering method is used to cluster the obstacle trajectories. The Hausdorff distance formula is used to measure the similarity between the clustered trajectories. The obstacle trajectories are grouped according to the movement mode to obtain a result containing multiple trajectory categories. Through K-Medoids clustering, the system can effectively group the obstacle trajectories according to the movement mode. For example, one type of trajectory may represent the movement mode of a flying bird, and another type of trajectory may represent the movement mode of a telephone pole. These clustering results provide a data basis for subsequent trajectory prediction and collision risk assessment; this grouping can help understand the behavior patterns of different types of obstacles and provide structured input for subsequent trajectory prediction. In this way, obstacles can be accurately identified in a dynamic environment, and corresponding processing strategies can be adopted according to different categories. The biggest advantage of using K-Medoids is its ability to resist outliers, which can effectively avoid the influence of a few abnormal data points during the clustering process.

[0081] The clustered trajectory data is used as the input of the TFBNet model. The spatial features (speed changes, turns, acceleration fluctuations) in the trajectory data are extracted through the convolution layer, and the trajectory features are mapped to a high-dimensional space.

[0082] The data mapped to the high-dimensional space is input into the UKF algorithm to make a preliminary state estimate of the current position, velocity and acceleration of the obstacle. The state is expanded through the sigma point, and the current state is propagated to the future moment using nonlinear transformation. The future trajectory of the obstacle is predicted to obtain the preliminary future trajectory of the obstacle.

[0083] The TFBNet model extracts spatial features from obstacle trajectories through a deep convolutional network to generate a high-dimensional feature map. This process can accurately capture the changing laws of obstacle motion, especially information such as speed, acceleration, and steering. These features are crucial in dynamic obstacle avoidance systems because they can effectively predict the future position of obstacles. Through high-dimensional feature mapping, TFBNet enhances the system's perception of complex trajectories, especially when planning flight paths, and can provide a more accurate obstacle motion model.

[0084] The UKF algorithm accurately predicts the future position of obstacles through an extended nonlinear state estimation method. In the present invention, UKF not only estimates the current state of the obstacle, but also predicts the future trajectory through nonlinear transformation. This is especially important for dynamic obstacles, such as fast-moving targets such as birds. By predicting the trajectory of obstacles, UKF can identify potential collision points in advance during the flight of the drone and adjust the obstacle avoidance path, thus avoiding the limitation of traditional methods that can only deal with static obstacles.

[0085] The obtained preliminary future trajectory data is converted into time series data, and the preliminary trajectory prediction data is input into the BiGRU model for time series modeling to obtain the optimized trajectory features;

[0086] The application of the BiGRU model makes the prediction of obstacle trajectories more accurate. Since BiGRU can capture temporal dependencies in both forward and reverse directions, the system can model the movement of obstacles more comprehensively, improving the accuracy of predictions. When facing rapidly changing obstacles, BiGRU can better adapt to their nonlinear dynamics, enhancing the safety and stability of the flight path.

[0087] The timing modeling results generated by BiGRU are used as input and passed to MHA (multi-head attention mechanism) for attention weighting. MHA generates attention weights by calculating query Q, key K and value V, and further weights each time step in the trajectory;

[0088] By using softmax to calculate the weight of each time step, the weighted sum is performed according to the calculated attention weight to obtain the weighted trajectory feature;

[0089] MHA further optimizes the accuracy of trajectory prediction by weighting the trajectory features of each time step. By assigning different weights to each time step, the system can focus on those time steps that have the greatest impact on future trajectory predictions, while suppressing unimportant parts. The final weighted trajectory features are used as input to the Gaussian process regression model to generate multiple future prediction targets. By using MHA, the system not only improves the accuracy of trajectory prediction, but also reduces redundant calculations, enhancing the real-time performance and reliability of the model.

[0090] The weighted trajectory features are used as the input of the Gaussian process regression model to predict future trajectories and generate multiple prediction targets.

[0091] Calculate the minimum distance between each trajectory and the current actual position of the obstacle, and sort the trajectories in descending order according to the distance to generate a distance sorting table;

[0092] Calculating the minimum distance between each trajectory and the current actual position of the obstacle means calculating the Euclidean distance between each point on the obstacle trajectory and the actual position of the obstacle at each moment, and then finding the minimum value as the minimum safety distance;

[0093] Calculate the acceleration and velocity of each trajectory and compare the results with the historical motion patterns of the collected obstacles. If they are inconsistent, exclude the trajectory from the distance sorting table.

[0094] The first trajectory in the distance sorting table after exclusion is selected as the predicted trajectory of the dynamic obstacle.

[0095] K-Medoids clustering provides a powerful basis for obstacle classification, while the TFBNet model further enhances the expressiveness of these trajectory data, improves the prediction accuracy of dynamic obstacles, avoids data redundancy, and reduces computational complexity. K-Medoids clustering alone cannot deeply extract complex features in the trajectory, while the TFBNet model can extract rich spatial features from the motion trajectory, such as speed changes, acceleration fluctuations, etc., and provide more detailed obstacle motion information. The combination of the two enables the system to efficiently classify and accurately extract the motion features of obstacles, providing reliable data for subsequent predictions and path adjustments. The UKF algorithm provides preliminary trajectory estimates, and the BiGRU model further optimizes these estimates. The combination of the two makes the prediction of dynamic obstacles more robust, and can accurately predict the future position of obstacles during flight, ensuring that the drone can make timely obstacle avoidance decisions. MHA (multi-head attention mechanism) applies attention weights to the trajectory features generated by BiGRU, further enhancing the accuracy of trajectory prediction. By weighting the trajectory features generated by BiGRU, MHA not only improves the sensitivity to dynamic obstacles, but also optimizes the accuracy and stability of trajectory prediction. Through this weighting, the system can more effectively identify and respond to potential collision risks, improving the responsiveness of the entire obstacle avoidance system. K-Medoids clustering, TFBNet model, UKF and BiGRU models work together to classify, extract features and predict trajectories of different movement modes of obstacles, providing sufficient information support for flight path planning. MHA optimizes this information by weighting to improve the reliability and accuracy of path prediction.

[0096] S3, generate a preliminary flight path based on obstacle recognition and dynamic prediction results, and perceive surrounding obstacles in real time during the flight of the drone, calculate the obstacle avoidance path, and dynamically adjust the flight path of the drone;

[0097] Specifically, the generation of a preliminary flight path based on obstacle identification and dynamic prediction results refers to marking the state and classification results of obstacles into a three-dimensional environment model, using a grid map method to divide the environment into multiple grids, each grid unit represents a corresponding spatial area, and marking the obstacle area on the map according to the location of the obstacle, which can clearly express the obstacle distribution and open areas within the UAV flight area. This method provides a concise and efficient environmental representation for path planning, which facilitates subsequent flight path calculation and adjustment;

[0098] According to the location of the flight target, the Dijkstra algorithm is used to plan the initial flight path of the UAV in the grid map. The Dijkstra algorithm can provide the UAV with an optimal flight path from the starting point to the target by calculating the shortest path in the grid map. This path not only takes into account the location of obstacles, but also ensures that the flight time and energy consumption are minimized. The accuracy and efficiency of the Dijkstra algorithm ensure the real-time and accuracy of path planning, and provide a reliable path basis for subsequent obstacle avoidance decisions. If the predicted trajectory of the dynamic obstacle and the flight path of the UAV have an intersection, the speed obstacle method (VO) is used to calculate the relative speed of the dynamic obstacle and determine a relative motion geometric area. If the trajectory of the UAV is on the trajectory of the geometric area, there is a collision risk. The flight path of the UAV is replanned, and the speed obstacle method (VO) is used to calculate the relative speed of the dynamic obstacle, which can accurately predict the collision risk between the dynamic obstacle and the UAV. This process can timely detect possible collision risks by determining the relative motion geometric area, and dynamically adjust the path. The VO method effectively improves the system's ability to cope with dynamic obstacles and can respond to changes in obstacles in flight in real time.

[0099] When the predicted trajectory of a dynamic obstacle intersects with the flight path of the drone, the system will use the VO algorithm to recalculate the obstacle avoidance path to avoid collision with the obstacle. At this time, the system not only adjusts the direction of the flight path, but also replans the flight route to avoid areas where collisions may occur. Through this dynamic path adjustment, the system can respond quickly to emergencies during flight. The present invention integrates multiple sensor data such as lidar and visual sensors, combines three-dimensional environmental models, grid map methods, Dijkstra algorithms and speed obstacle methods, and provides drones with an efficient and accurate obstacle avoidance and path planning method.

[0100] Furthermore, the real-time perception of surrounding obstacles, calculation of obstacle avoidance paths, and dynamic adjustment of the UAV flight path during the flight of the UAV means that the UAV performs tasks in the power inspection area according to the planned path, perceives surrounding obstacles in real time through sensors, and generates and adjusts the UAV obstacle avoidance path in real time based on the position of the obstacles and the relative motion of the UAV through the artificial potential field method, and stores the real-time data of the UAV in a database.

[0101] When a drone performs a power inspection mission, there may be a variety of static and dynamic obstacles (such as power lines, tree barriers, flying birds, aircraft, etc.) in the flight path. By sensing the surrounding obstacles in real time, the drone can detect these obstacles in time and respond to avoid collisions. This is crucial to ensure the safety of inspection missions, especially in complex environments, where real-time perception can significantly improve the autonomy of drones and reduce dependence on human intervention. The artificial potential field method provides an efficient and simple solution for obstacle avoidance calculations. By simulating the relationship between gravity and repulsion between drones and obstacles, the artificial potential field method can plan a path for drones to avoid obstacles without adding too much computational burden. This is crucial for responding quickly and adjusting the flight path in real-time flight, especially in power inspection missions, where complex environments may cause sudden obstacles to appear, and traditional path planning algorithms may not be able to make adjustments in time. The use of the artificial potential field method can ensure that even if the drone encounters new obstacles during flight, it can quickly adjust the path to avoid large errors or safety hazards during flight.

[0102] Furthermore, storing the real-time data of the drone in a database means storing the flight path and obstacle avoidance data of the drone in the database, updating the obstacle data viewed during the real-time flight of the drone into the three-dimensional environment model for annotation, storing the updated three-dimensional environment model data in the database and performing real-time access control.

[0103] By storing the flight path and obstacle avoidance data of the drone in real time in the database and dynamically updating it in combination with the three-dimensional environmental model, not only the efficiency of the flight mission is improved, but also the safety during the flight is enhanced. Especially in power inspection tasks, facing complex flight environments and changing obstacles, the path planning method based on real-time perception and data update significantly improves the obstacle avoidance ability of drones. Through precise data storage and management, real-time environmental model updates and strict access control, the efficiency, stability and safety of drones in performing tasks are ensured. This method can be applied in a variety of flight environments and provides strong technical support for autonomous flight of drones in complex tasks.

[0104] This embodiment also provides a computer device, which is suitable for the obstacle avoidance method of autonomous flight of UAV power inspection based on laser radar, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the obstacle avoidance method of autonomous flight of UAV power inspection based on laser radar as proposed in the above embodiment.

[0105] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0106] The present embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the obstacle avoidance method for autonomous flight of unmanned aerial vehicle power inspection based on laser radar as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0107] In summary, the present invention can not only identify and locate static obstacles, but also predict the motion trajectory of dynamic obstacles such as flying birds through dynamic obstacle prediction technology, thereby generating a preliminary flight path, and sensing and avoiding surrounding obstacles in real time during flight, ensuring that the UAV can smoothly perform power inspection tasks.

[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar, characterized by: include, Configure laser radar and visual sensors on the drone to collect 3D point cloud data and image data of the flight environment in real time, and generate a 3D environment model after optimizing the point cloud data; Based on the three-dimensional environment model, obstacles are classified into static and dynamic. If the obstacle is a static obstacle, the obstacle type is identified and located through the CNN model. If the obstacle is a dynamic obstacle, the motion trajectory is clustered through the K-Medoids clustering method, and the data features are extracted through the TFBNet model. The motion trajectory is predicted based on the UKF algorithm, the BIGRU model and the MHA. Generate a preliminary flight path based on obstacle recognition and dynamic prediction results, and sense surrounding obstacles in real time during the flight of the drone, calculate the obstacle avoidance path, and dynamically adjust the flight path of the drone; If the obstacle is a dynamic obstacle, the motion trajectory is clustered by the K-Medoids clustering method, the data features are extracted by the TFBNet model, and the motion trajectory prediction is performed based on the UKF algorithm, the BIGRU model and the MHA, including the following steps: Collect obstacle data at each time point to construct the obstacle state vector S t， The state vectors collected from multiple time steps are organized in chronological order to form a complete trajectory data set. Each trajectory data set consists of the state vectors of the obstacle at multiple time steps, representing the movement trajectory of the obstacle T i ; Perform feature extraction on each trajectory and standardize all trajectory data; The number of clusters is selected using the elbow rule, the K-Medoids clustering method is used to cluster the movement trajectories of obstacles, the Hausdorff distance formula is used to measure the similarity between the clustered trajectories, and the trajectories of obstacles are grouped according to the movement mode to obtain a result containing multiple trajectory categories; The clustered trajectory data is used as the input of the TFBNet model, the spatial features in the trajectory data are extracted through the convolution layer, and the trajectory features are mapped to a high-dimensional space; The data mapped to the high-dimensional space is input into the UKF algorithm to make a preliminary state estimate of the current position, velocity and acceleration of the obstacle. The state is expanded through the sigma point, and the current state is propagated to the future moment using nonlinear transformation. The future trajectory of the obstacle is predicted to obtain the preliminary future trajectory of the obstacle. The obtained preliminary future trajectory data is converted into time series data, and the preliminary trajectory prediction data is input into the BiGRU model for time series modeling to obtain the optimized trajectory features; The time series modeling results generated by BiGRU are used as input and passed to MHA for attention weighting. MHA generates attention weights by calculating query Q, key K and value V, and further weights each time step in the trajectory. The weight of each time step is calculated by using softmax, and the weighted sum is performed according to the calculated attention weights. The weighted trajectory features are obtained as the input of the Gaussian process regression model to predict future trajectories and generate multiple prediction targets. Calculate the minimum distance between each trajectory and the current actual position of the obstacle, and sort the trajectories in descending order according to the distance to generate a distance sorting table; Calculate the acceleration and velocity of each trajectory and compare the results with the historical motion patterns of the collected obstacles. If they are inconsistent, exclude the trajectory from the distance sorting table. The first trajectory in the distance sorting table after exclusion is selected as the predicted trajectory of the dynamic obstacle.

2. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar as claimed in claim 1, characterized in that: The process of configuring a laser radar and a visual sensor on a drone to collect three-dimensional point cloud data and image data of a flight environment in real time and generating a three-dimensional environment model after optimizing the point cloud data includes the following steps: The laser radar collects three-dimensional point cloud data of the flight environment; The visual sensor collects obstacle images of the flight environment through an RGB camera; Use multi-zone planar point cloud denoising technology to denoise the point cloud data collected by the lidar, further filter the denoised point cloud data to remove redundant points, use the GICP algorithm to minimize the difference between point clouds, calculate the rotation matrix, and align the point clouds to obtain the processed point cloud data; Preprocess the data collected by the visual sensor, extract the depth information and features in the point cloud data and visual images through the convolution layer of CNN, generate depth maps and feature maps through multiple convolution layers, and use regression methods to calculate the disparity of each point in the depth map and feature map; When performing depth matching between visual images and point cloud data, the depth information is optimized by calculating the disparity of each pixel. The disparity value of each pixel reflects the depth information of the pixel in three-dimensional space. The candidate visual values ​​are selected through softmax, and the predicted disparity is optimized using the minimum square method to minimize the depth error between the lidar and the visual sensor. Use the spatial propagation network to propagate the depth information from the high-density area to the low-density area to obtain the optimized point cloud data, and use the Chamfer distance to evaluate the accuracy of the optimized point cloud data to obtain the final optimized point cloud data; Time-synchronize the collected and optimized data and mark them with timestamps; A weighted fusion algorithm is used to integrate data from different sensors to generate a complete three-dimensional environment model that includes the spatial position and appearance characteristics of obstacles.

3. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar as claimed in claim 2, characterized in that: The classification of obstacles into static and dynamic based on the three-dimensional environment model refers to obtaining the initial position of the obstacle based on the spatial positioning information of the obstacle in the three-dimensional environment model, tracking the spatial position of the obstacle through real-time data, comparing the obstacle positions collected at different time points, calculating the position change amplitude of each obstacle through the Euclidean distance formula, setting a threshold A, and if the change amplitude is greater than or equal to the threshold A, it is a dynamic obstacle, otherwise it is a static obstacle.

4. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar as claimed in claim 3 is characterized by: If the obstacle is a static obstacle, identifying the obstacle type and locating it through the CNN model includes the following steps: Obtain the precise spatial position of the obstacle through the point cloud data of the laser radar and obtain the three-dimensional coordinates (x, y, z) of the obstacle; Use convolutional neural networks to build object classification models, including input layers, convolution layers, pooling layers, and output layers; The historical image data is used as a training set to input the object classification model for iterative training, and the loss function and Adam optimizer are defined to iteratively optimize the model parameters. When the loss of the object classification model no longer decreases significantly during the continuous iteration process, the iteration is stopped and the model parameters are output to update the object classification model. Input the real-time obstacle image into the object classification model to obtain the static obstacle category; The static obstacle categories include power lines, tree barriers, and utility poles.

5. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar as claimed in claim 4 is characterized by: Generating a preliminary flight path based on obstacle identification and dynamic prediction results refers to marking the state and classification results of obstacles into a three-dimensional environment model, dividing the environment into multiple grids using a grid map method, each grid unit represents a corresponding spatial area, and marking the obstacle area on the map according to the location of the obstacle; According to the position of the flight target, the Dijkstra algorithm is used to plan the preliminary flight path of the UAV in the grid map. If the predicted trajectory of the dynamic obstacle and the flight path of the UAV have an intersection, the speed obstacle method is used to calculate the relative speed of the dynamic obstacle and determine a geometric area of ​​relative motion. If the trajectory of the UAV is on the trajectory of the geometric area, there is a risk of collision and the flight path of the UAV is replanned.

6. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar as claimed in claim 5, characterized in that: The real-time perception of surrounding obstacles, calculation of obstacle avoidance paths, and dynamic adjustment of the flight path of the drone during flight of the drone means that the drone performs tasks in the power inspection area according to the planned path, perceives surrounding obstacles in real time through sensors, generates the obstacle avoidance path of the drone and adjusts it in real time based on the position of the obstacles and the relative motion of the drone through the artificial potential field method, and stores the real-time data of the drone in a database.

7. The obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar as claimed in claim 6, characterized in that: Storing the real-time data of the drone in a database refers to storing the flight path and obstacle avoidance data of the drone in the database, updating the obstacle data viewed during the real-time flight of the drone into the three-dimensional environment model for annotation, storing the updated three-dimensional environment model data in the database and performing real-time access control.

8. A computer device comprising: Memory and processor; The memory stores a computer program, which is characterized in that when the processor executes the computer program, the steps of the obstacle avoidance method for autonomous flight of unmanned aerial vehicle power inspection based on laser radar described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the obstacle avoidance method for autonomous flight of UAV power inspection based on laser radar are implemented as described in any one of claims 1 to 7.

Images