Unmanned aerial vehicle autonomous obstacle avoidance method for electric power inspection complex environment

Abstract

The invention relates to an unmanned aerial vehicle autonomous obstacle avoidance method for an electric power inspection complex environment. The method comprises the following steps: S1, acquiring an image, LiDAR, an optical flow sensor and IMU data on an inspection path through an airborne camera and a sensor; s2, fusing the multi-source data through an EKF (Extended Kalman Filter) algorithm; s3, real-time obstacle detection and recognition are carried out by using a Tiny YOLO model, the Tiny YOLO model is introduced into a double-flow network, and the detection capability of the Tiny YOLO model in a resource limited scene is enhanced; s4, performing semantic segmentation on the image by using deep learning, obtaining positions and states of different types of objects, combining with a geometric map generated by LiDAR, updating a global map through an ICP algorithm, performing continuous pose map optimization by using g2o, reducing drift in path estimation, and obtaining a semantic map; and S5, calculating a new safety path by using a D * Lite algorithm, and dynamically adjusting the flight strategy of the unmanned aerial vehicle. According to the invention, the obstacle avoidance capability and the path planning efficiency of the unmanned aerial vehicle in a power inspection complex environment are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of electric power inspection, and in particular to an autonomous obstacle avoidance method for an unmanned aerial vehicle in a complex electric power inspection environment. Background Art

[0002] In modern power transmission systems, power line inspection is a key link in maintaining the stability and reliability of the power grid. Traditional manual inspection methods are inefficient, costly, and accompanied by certain safety risks. In recent years, with the development of unmanned driving technology and artificial intelligence, drones have gradually become one of the effective tools in the field of power inspection. However, power lines usually pass through various complex and changeable environments, including mountainous areas, forests, and urban areas, which poses severe challenges to the autonomous flight and obstacle avoidance capabilities of drones. Summary of the invention

[0003] In order to solve the above problems, the purpose of the present invention is to provide an autonomous obstacle avoidance method for UAVs in complex power inspection environments, which can effectively improve the obstacle avoidance capability and path planning efficiency of UAVs in complex power inspection environments.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] An autonomous obstacle avoidance method for a UAV in a complex power inspection environment, characterized by comprising the following steps:

[0006] S1: Obtain images, LiDAR, optical flow sensor and IMU data on the inspection path through the onboard camera and sensors, and use the high-frequency band synchronization mechanism to ensure the data time synchronization of the camera, LiDAR, optical flow sensor and IMU;

[0007] S2: The multi-source data is fused through the EKF filtering algorithm. The state vector after EKF fusion is the final UAV position, speed, and attitude estimation result;

[0008] S3: Use the Tiny YOLO model for real-time obstacle detection and recognition. The Tiny YOLO model introduces a two-stream network to enhance the detection capability of the Tiny YOLO model in resource-constrained scenarios.

[0009] S4: Use deep learning to perform semantic segmentation on the image, obtain the position and state of objects of different categories, and combine it with the geometric map generated by LiDAR, update the global map through the ICP algorithm, use g2o to perform continuous pose graph optimization, reduce the drift in path estimation, and obtain a semantic map;

[0010] S5: Based on the semantic map and the fused state vector, the D*Lite algorithm is used to calculate a new safe path and dynamically adjust the UAV flight strategy to adapt to the changing environment.

[0011] Furthermore, S2 is specifically:

[0012] Based on the acquired images, LiDAR, optical flow sensor, and IMU data, construct the state representation of the drone:

[0013]

[0014] Among them, P is the position vector, v is the velocity vector, and θ is the attitude angle;

[0015] State transfer equation:

[0016]

[0017] Where f is the nonlinear state transfer function, u k-1 is the control input, w k-1 is the process noise; is the prediction of the current state k; x k-1 is the state at the previous moment;

[0018] The error covariance matrix of state prediction is:

[0019]

[0020] Among them, F k-1 is the Jacobian matrix; P k-1 is the state estimation error covariance matrix of the previous moment; Q k-1 is the process noise covariance matrix;

[0021] Compute the predicted measurements:

[0022]

[0023] Among them, h is the measurement function;

[0024] Get the Kalman gain:

[0025]

[0026] Among them, H k is the measurement function h for the state The Jacobian matrix of R k is the measurement noise covariance matrix, representing the measurement uncertainty;

[0027] Based on the Kalman gain, the state is updated:

[0028]

[0029] Among them, z k is the actual measured value;

[0030] Covariance update:

[0031] P k|k =(IK k H k ) k|k-1 ;

[0032] Where I is the identity matrix;

[0033] The state vector x after EKF fusion k|k The final estimation results of the drone position, velocity, and attitude.

[0034] Furthermore, the TinyYOLO lightweight model is used for real-time obstacle detection and recognition, as follows:

[0035] Convert the image acquired by the camera into an RGB image; use the standard convolution architecture of TinyYOLO to process the input RGB image, embed the SE module after each convolution layer of TinyYOLO to weight the feature map to enhance the expressiveness of the feature map and obtain the feature map of the spatial stream;

[0036] In parallel, a shallow convolutional network is introduced to calculate the optical flow between the current frame and the previous frame, extract motion features, and obtain the feature map of the time flow;

[0037] Fuse the feature maps of the spatial stream and the temporal stream;

[0038]

[0039] Among them, α is the fusion weight; is the feature map of the spatial stream; F time is the feature map of the time flow; F fused is the fused feature map;

[0040] Use the fused feature map to continue reasoning, and perform bounding box regression and category prediction based on the TinyYOLO output layer.

[0041] Furthermore, the SE module weights the feature maps output by each convolutional layer to highlight the information in the key areas, and the SE module includes a Squeeze operation, an Excitation operation, and a Scale operation;

[0042] Squeeze operation performs global average pooling on each channel to generate a channel description vector that captures the global spatial information of each channel:

[0043]

[0044] Among them, z c is the generated channel description vector; F(i,j,c) is the pixel value of the input feature map at position (i,j) and channel c; H,W are the height and width of the feature map;

[0045] Excitation operation generates the weight of each channel, models the global information of the channel through the fully connected layer and assigns importance:

[0046] s c =σ(W 2 δ(W 1 z c ));

[0047] Among them, s c is the weight of each channel, indicating the importance of the channel; W 1 ,W 2 is the weight matrix of the fully connected layer, where W 1 Reduce the channel vector to a smaller dimension, W 2 Restore the reduced vector to the original number of channels; δ is the ReLU activation function; σ is the Sigmoid activation function;

[0048] The Scale operation uses the generated weights to adjust each channel of the input feature map, focusing on meaningful features and obtaining the feature map of the spatial stream:

[0049]

[0050] Furthermore, the fused feature map is used to continue reasoning, and bounding box regression and category prediction are performed based on the TinyYOLO output layer:

[0051] The feature map F after fusion of the feature maps of the spatial stream and the temporal stream fused , input to the TinyYOLO output module;

[0052] The output layer of Tiny YOLO consists of a convolutional layer to predict the bounding box information and class probability within each grid;

[0053] Convolution layer, using 1x1 convolution, adjusts the number of channels of the feature map to suit the output requirements:

[0054]

[0055] The output layer uses a set of convolutions to predict B bounding box information for each grid position on the S×S grid formed by the fused feature maps.

[0056] The information of each bounding box includes: 4 bounding box coordinates (b x ,b y ,b w ,b h ), 1 confidence level, and C class probabilities;

[0057]

[0058] Decode the real rectangular box based on the relative coordinates of the grid coordinates and the box:

[0059] B x =σ(t x )+c x

[0060] B y =σ(t y )+c y

[0061]

[0062] Among them, t x ,t y ,t w ,t h is the output of the network; c x ,c y is the grid cell coordinate, p w ,p h is the predefined width and height of the anchor box, σ is the sigmoid function;

[0063] Output the final target detection results, including the location of the object and the location and size of the bounding box.

[0064] Furthermore, S4 is specifically:

[0065] Use the deep neural network DeepLab to perform semantic segmentation on each frame of the image to obtain accurate object level and category information for each pixel;

[0066] The category information extracted from semantic segmentation and the detection results of Tiny YOLO are spatially aligned with the point cloud of LiDAR, and the category labels in the image are mapped to the corresponding point cloud through projection relationship to generate a local semantic geometry map;

[0067] Using the IMU to provide the initial pose for rough alignment, the newly acquired LiDAR point cloud is aligned with the existing global map based on the ICP algorithm;

[0068] Use nodes and edges to represent the motion relationship between consecutive frames, where nodes represent the device's posture and edges represent the estimated motion; use g2o optimization to minimize the difference between the error of each edge and the observation, and minimize the cumulative error in the posture graph;

[0069] Combine the conditioned LiDAR point cloud with category information to generate a complete 3D semantic map.

[0070] Furthermore, the IMU is used to provide an initial pose for rough alignment, and the newly acquired LiDAR point cloud is aligned with the existing global map based on the ICP algorithm, as follows:

[0071] The rotation matrix R provided by the IMU imu and displacement vector t imu Used to transform from a local coordinate system to a global coordinate system:

[0072] Apply the initial pose provided by the IMU to the newly acquired LiDAR point cloud to obtain a rough global alignment:

[0073] P aligned =R imu *P+ timu

[0074] Where P is the coordinate matrix of the original LiDAR point cloud, P aligned It is the point cloud after the initial pose estimation;

[0075] The ICP objective function is defined as minimizing the square error between the source point cloud P and the target map point cloud Q:

[0076] min R,t ∑||R·p i +tq i || 2

[0077] Among them, p i is the point of the source point cloud, q i is the target map with p i nearest point;

[0078] For each source point p i , find the nearest neighbor point q in the target map i， And use the kd tree data structure to speed up the nearest neighbor search;

[0079] Use the singular value decomposition method to solve the rotation matrix R and the translation vector t

[0080] Apply the calculated rotation matrix R and translation vector t to the entire source point cloud

[0081] P new =R·Paligned +t.

[0082] Furthermore, the singular value decomposition method is used to solve the rotation matrix R and the translation vector t, as follows:

[0083] First, calculate the centroid of the two point clouds:

[0084]

[0085] Where N is the number of point clouds;

[0086] And calculate the decentralized coordinates for each point:

[0087]

[0088] Calculate the covariance matrix H:

[0089]

[0090] Then, perform SVD decomposition on the covariance matrix:

[0091] U,∑,V T =SVD(H)

[0092] Among them, UU and V are orthogonal matrices, and Σ is a diagonal matrix;

[0093] The rotation matrix R is calculated by the following formula:

[0094] R=VU T If det(R)<0, correct it by reversing the direction of the minimum singular value:

[0095] V[:,-1]=-V[:,-1]

[0096] R=VU T ;

[0097] Calculate the translation vector t:

[0098]

[0099] Finally, the output rotation matrix R and translation vector tt are used to align the source point cloud to the target point cloud.

[0100] Furthermore, the D*Lite algorithm is specifically as follows:

[0101] Initialize the data structure, assuming that nodes represent grids in the grid map and edges represent paths between adjacent grids;

[0102] Initialize the cost, priority, and estimated cost of all nodes to the target node;

[0103] Starting from the starting point, continuously select the next node with the lowest estimated cost and move;

[0104] If a new obstacle or environmental change is detected, update the cost of the corresponding node:

[0105] g(s)=min(g(s),c(s,s′)+g(s′))

[0106] Where g(s) is the estimated cost from the current node to the starting point, and c(s,s′) is the path cost from node s to node s′;

[0107] Start the update path backward from the target node to ensure that the changes are propagated to all affected nodes;

[0108] Select the optimal path from the starting point to the target point and apply this path to guide the UAV's flight strategy.

[0109] Continuously update the semantic map through sensor fusion and detect significant changes in the environment, including new or moved obstacles;

[0110] According to the new map information, the update mechanism of the D*Lite algorithm is triggered to recalculate and update the safe path.

[0111] The present invention has the following beneficial effects:

[0112] 1. This invention targets the application scenarios where power inspection has high requirements for reliability and real-time performance. By utilizing the perception depth of semantic maps and the dynamics of the D*Lite algorithm, the drone can more intelligently navigate autonomously in complex environments and complete inspection tasks.

[0113] 2. The present invention uses a lightweight Tiny YOLO model to achieve real-time detection and identification of obstacles in the flight path, ensuring that the drone can respond quickly to emergencies. The dual-stream network is introduced to effectively combine spatial and temporal information, improving the target detection performance of Tiny YOLO in complex environments while maintaining computational efficiency. The SE module effectively enhances the feature expression capability while maintaining moderate computational complexity, enabling the network to better focus on key features and improving network accuracy without significantly increasing the amount of computation, which is very suitable for real-time and resource-limited drone scenarios.

[0114] 3. This paper realizes the combination and optimization of semantic information and geometric information, which can not only provide detailed location information for dynamic environments, but also supplement the semantic context in path estimation and improve the robustness and stability of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] Figure 1 is a flow chart of the method of the present invention;

[0116] Figure 2 Schematic diagram of obstacle avoidance in one embodiment of the present invention. DETAILED DESCRIPTION

[0117] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0118] refer to Figure 1-2 In this embodiment, a method for autonomous obstacle avoidance of a UAV in a complex power inspection environment is provided, which is characterized by comprising the following steps:

[0119] S1: Obtain images, LiDAR, optical flow sensor and IMU data on the inspection path through the onboard camera and sensors, and use the high-frequency band synchronization mechanism to ensure the data time synchronization of the camera, LiDAR, optical flow sensor and IMU;

[0120] In this embodiment, the drone is provided with: a camera that captures images for visual perception and feature extraction; a LiDAR that generates accurate environmental distance and contour data; an optical flow sensor that provides registration information to help identify motion states; and an IMU (inertial measurement unit) that provides acceleration and angular velocity for posture estimation.

[0121] S2: The multi-source data is fused through the EKF filtering algorithm. The state vector after EKF fusion is the final UAV position, speed, and attitude estimation result; improving the accuracy and robustness of environmental perception;

[0122] S3: Use the Tiny YOLO model for real-time obstacle detection and recognition. The Tiny YOLO model introduces a two-stream network to enhance the detection capability of the Tiny YOLO model in resource-constrained scenarios.

[0123] S4: Use deep learning to perform semantic segmentation on the image, obtain the position and state of objects of different categories, and combine it with the geometric map generated by LiDAR, update the global map through the ICP algorithm, use g2o to perform continuous pose graph optimization, reduce the drift in path estimation, and obtain a semantic map;

[0124] S5: Based on the semantic map and the fused state vector, the D*Lite algorithm is used to calculate a new safe path and dynamically adjust the UAV flight strategy to adapt to the changing environment.

[0125] In this embodiment, S2 is specifically:

[0126] Based on the acquired images, LiDAR, optical flow sensor, and IMU data, construct the state representation of the drone:

[0127]

[0128] Among them, P is the position vector, v is the velocity vector, and θ is the attitude angle;

[0129] State transfer equation:

[0130]

[0131] Where f is the nonlinear state transfer function, u k-1 is the control input, w k-1 is the process noise; is the prediction of the current state k; x k-1 is the state at the previous moment;

[0132] The error covariance matrix of state prediction is:

[0133]

[0134] Among them, F k-1 is the Jacobian matrix; P k-1 is the state estimation error covariance matrix of the previous moment; Q k-1 is the process noise covariance matrix;

[0135] Compute the predicted measurements:

[0136]

[0137] Among them, h is the measurement function;

[0138] Get the Kalman gain:

[0139]

[0140] Among them, H k is the measurement function h for the state The Jacobian matrix of R k is the measurement noise covariance matrix, representing the measurement uncertainty;

[0141] Based on the Kalman gain, the state is updated:

[0142]

[0143] Among them, z k is the actual measured value;

[0144] Covariance update:

[0145] P k|k =(IK k H k ) k|k-1 ;

[0146] Where I is the identity matrix;

[0147] The state vector x after EKF fusion k|k The final estimation results of the drone position, velocity, and attitude.

[0148] In this embodiment, the TinyYOLO lightweight model is used for real-time obstacle detection and recognition, as follows:

[0149] Convert the image acquired by the camera into an RGB image; use the standard convolution architecture of TinyYOLO to process the input RGB image, embed the SE module after each convolution layer of TinyYOLO to weight the feature map to enhance the expressiveness of the feature map and obtain the feature map of the spatial stream;

[0150] In parallel, a shallow convolutional network is introduced to calculate the optical flow between the current frame and the previous frame, extract motion features, and obtain the feature map of the time flow;

[0151] Fuse the feature maps of the spatial stream and the temporal stream;

[0152]

[0153] Among them, α is the fusion weight; is the feature map of the spatial stream; F time is the feature map of the time flow; F fused is the fused feature map;

[0154] Use the fused feature map to continue reasoning, and perform bounding box regression and category prediction based on the TinyYOLO output layer.

[0155] In this embodiment, the SE module weights the feature map output by each convolutional layer to highlight the information of the key area, and the SE module includes a Squeeze operation, an Excitation operation, and a Scale operation;

[0156] Squeeze operation performs global average pooling on each channel to generate a channel description vector that captures the global spatial information of each channel:

[0157]

[0158] Among them, z c is the generated channel description vector; F(i,j,c) is the pixel value of the input feature map at position (i,j) and channel c; H,W are the height and width of the feature map;

[0159] Excitation operation generates the weight of each channel, models the global information of the channel through the fully connected layer and assigns importance:

[0160] s c =σ(W 2 δ(W 1 z c ));

[0161] Among them, s c is the weight of each channel, indicating the importance of the channel; W 1 ,W 2 is the weight matrix of the fully connected layer, where W 1 Reduce the channel vector to a smaller dimension (embedding space), W 2 Restore the reduced vector to the original number of channels; δ is the ReLU activation function; σ is the Sigmoid activation function;

[0162] The Scale operation uses the generated weights to adjust each channel of the input feature map, focusing on meaningful features and obtaining the feature map of the spatial stream:

[0163]

[0164] In this embodiment, the fused feature map is used to continue reasoning, and bounding box regression and category prediction are performed based on the TinyYOLO output layer:

[0165] The feature map F after fusion of the feature maps of the spatial stream and the temporal stream fused , input to the TinyYOLO output module;

[0166] The output layer of TinyYOLO consists of a convolutional layer to predict the bounding box information and class probability within each grid;

[0167] Convolution layer, using 1x1 convolution, adjusts the number of channels of the feature map to suit the output requirements:

[0168]

[0169] The output layer uses a set of convolutions to predict B bounding box information for each grid position on the S×S grid formed by the fused feature maps.

[0170] The information of each bounding box includes: 4 bounding box coordinates (b x ,b y ,b w ,b h ), 1 confidence (whether it contains an object), and C category probabilities;

[0171]

[0172] Decode the real rectangular box based on the relative coordinates of the grid coordinates and the box:

[0173] B x =σ(t x )+c x

[0174] B y =σ(t y )+c y

[0175]

[0176] Among them, t x ,t y ,t w ,t h is the output of the network; c x ,c y is the grid cell coordinate, p w ,p h is the predefined width and height of the anchor box, σ is the sigmoid function;

[0177] Output the final target detection results, including the location of the object and the location and size of the bounding box.

[0178] In this embodiment, S4 is specifically:

[0179] Use the deep neural network DeepLab to perform semantic segmentation on each frame of the image to obtain accurate object level and category information for each pixel;

[0180] The category information extracted from semantic segmentation and the detection results of Tiny YOLO are spatially aligned with the point cloud of LiDAR, and the category labels in the image are mapped to the corresponding point cloud through projection relationship to generate a local semantic geometry map;

[0181] Using the IMU to provide the initial pose for rough alignment, the newly acquired LiDAR point cloud is aligned with the existing global map based on the ICP algorithm;

[0182] Use nodes and edges to represent the motion relationship between consecutive frames, where nodes represent the device's posture and edges represent the estimated motion; use g2o optimization to minimize the difference between the error of each edge and the observation, and minimize the cumulative error in the posture graph;

[0183] Combine the conditioned LiDAR point cloud with category information to generate a complete 3D semantic map.

[0184] In this embodiment, the IMU is used to provide an initial pose for rough alignment, and the newly acquired LiDAR point cloud is aligned with the existing global map based on the ICP algorithm, as follows:

[0185] The rotation matrix R provided by the IMUimu and displacement vector t imu Used to transform from a local coordinate system to a global coordinate system:

[0186] Apply the initial pose provided by the IMU to the newly acquired LiDAR point cloud to obtain a rough global alignment:

[0187] P aligned =R imu *P+ timu

[0188] Where P is the coordinate matrix of the original LiDAR point cloud, P aligned It is the point cloud after the initial pose estimation;

[0189] The ICP objective function is defined as minimizing the square error between the source point cloud P and the target map point cloud Q:

[0190] min R,t ∑‖R·p i +tq i || 2

[0191] Among them, p i is the point of the source point cloud, q i is the target map with p i nearest point;

[0192] For each source point p i , find the nearest neighbor point q in the target map i , and use the kd-tree data structure to speed up the nearest neighbor search;

[0193] Use the singular value decomposition method to solve the rotation matrix R and the translation vector t

[0194] Apply the calculated rotation matrix R and translation vector t to the entire source point cloud

[0195] P new =R·P aligned +t.

[0196] In this embodiment, the singular value decomposition method is used to solve the rotation matrix R and the translation vector t, as follows:

[0197] First, calculate the centroid of the two point clouds:

[0198]

[0199] Where N is the number of point clouds;

[0200] And calculate the decentralized coordinates for each point:

[0201]

[0202] Calculate the covariance matrix H:

[0203]

[0204] Then, perform SVD decomposition on the covariance matrix:

[0205] U,∑,V T =SVD(H)

[0206] Among them, UU and V are orthogonal matrices, and Σ is a diagonal matrix;

[0207] The rotation matrix R is calculated by the following formula:

[0208] R=VU T

[0209] If det(R)<0, correct it by reversing the direction of the minimum singular value:

[0210] V[:,-1]=-V[:,-1]

[0211] R=VU T ;

[0212] Calculate the translation vector t:

[0213]

[0214] Finally, the output rotation matrix R and translation vector tt are used to align the source point cloud to the target point cloud.

[0215] In this embodiment, the D*Lite algorithm is specifically:

[0216] Initialize the data structure, assuming that nodes represent grids in the grid map and edges represent paths between adjacent grids;

[0217] Initialize the cost, priority, and estimated cost of all nodes to the target node;

[0218] Starting from the starting point, continuously select the next node with the lowest estimated cost and move;

[0219] If a new obstacle or environmental change is detected, update the cost of the corresponding node:

[0220] g(s)=min(g(s),c(s,s′)+g(s′))

[0221] Where g(s) is the estimated cost from the current node to the starting point, and c(s,s′) is the path cost from node s to node s′;

[0222] Start the update path backward from the target node to ensure that the changes are propagated to all affected nodes;

[0223] Select the optimal path from the starting point to the target point and apply this path to guide the UAV's flight strategy.

[0224] Continuously update the semantic map through sensor fusion and detect significant changes in the environment, including new or moved obstacles;

[0225] According to the new map information, the update mechanism of the D*Lite algorithm is triggered to recalculate and update the safe path.

[0226] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0227] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0228] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0229] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0230] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for autonomous obstacle avoidance of a UAV in a complex power inspection environment, characterized in that: The following steps are involved: S1: Obtain images, LiDAR, optical flow sensor and IMU data on the inspection path through the onboard camera and sensors, and use the high-frequency band synchronization mechanism to ensure the data time synchronization of the camera, LiDAR, optical flow sensor and IMU; S2: The multi-source data is fused through the EKF filtering algorithm. The state vector after EKF fusion is the final UAV position, speed, and attitude estimation result; S3: Use the Tiny YOLO model for real-time obstacle detection and recognition. The Tiny YOLO model introduces a two-stream network to enhance the detection capability of the Tiny YOLO model in resource-constrained scenarios. S4: Use deep learning to perform semantic segmentation on the image, obtain the position and state of objects of different categories, and combine it with the geometric map generated by LiDAR, update the global map through the ICP algorithm, use g2o to perform continuous pose graph optimization, reduce the drift in path estimation, and obtain a semantic map; S5: Based on the semantic map and the fused state vector, the D*Lite algorithm is used to calculate a new safe path and dynamically adjust the UAV flight strategy to adapt to the changing environment.

2. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 1 is characterized in that: The S2 is specifically: Based on the acquired images, LiDAR, optical flow sensor, and IMU data, construct the state representation of the drone: Among them, P is the position vector, v is the velocity vector, and θ is the attitude angle; State transfer equation: Where f is the nonlinear state transfer function, u k-1 is the control input, w k-1 is the process noise; is the prediction of the current state k; x k-1 is the state at the previous moment; The error covariance matrix of state prediction is: Among them, F k-1 is the Jacobian matrix; P k-1 is the state estimation error covariance matrix of the previous moment; Q k-1 is the process noise covariance matrix; Compute the predicted measurements: Among them, h is the measurement function; Get the Kalman gain: Among them, H k is the measurement function h for the state The Jacobian matrix of R k is the measurement noise covariance matrix, representing the measurement uncertainty; Based on the Kalman gain, the state is updated: Among them, z k is the actual measured value; Covariance update: P kk =(I-K k H k )P kk-1 ; Where I is the identity matrix; The state vector x after EKF fusion kk The final estimation results of the drone position, velocity, and attitude.

3. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 1 is characterized in that: The Tiny YOLO lightweight model is used for real-time obstacle detection and recognition, as follows: Convert the image acquired by the camera into an RGB image; use the standard convolution architecture of Tiny YOLO to process the input RGB image, embed the SE module after each convolution layer of Tiny YOLO to weight the feature map to enhance the expressiveness of the feature map and obtain the feature map of the spatial stream; In parallel, a shallow convolutional network is introduced to calculate the optical flow between the current frame and the previous frame, extract motion features, and obtain the feature map of the time flow; Fuse the feature maps of the spatial stream and the temporal stream; Among them, α is the fusion weight; is the feature map of the spatial stream; F time is the feature map of the time flow; F fused is the fused feature map; Use the fused feature map to continue reasoning, and perform bounding box regression and category prediction based on the Tiny YOLO output layer.

4. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 3 is characterized in that: The SE module weights the feature map output by each convolutional layer to highlight the information in the key area. The SE module includes a Squeeze operation, an Excitation operation, and a Scale operation. Squeeze operation performs global average pooling on each channel to generate a channel description vector that captures the global spatial information of each channel: Among them, z c is the generated channel description vector; F(i,j,c) is the pixel value of the input feature map at position (i,j) and channel c; H,W are the height and width of the feature map; Excitation operation generates the weight of each channel, models the global information of the channel through the fully connected layer and assigns importance: s c =σ(W2δ(W1z c )); Among them, s c is the weight of each channel, indicating the importance of the channel; W1, W2 are the weight matrices of the fully connected layer, where W1 reduces the channel vector to a smaller dimension, and W2 restores the reduced vector to the original number of channels; δ is the ReLU activation function; σ is the Sigmoid activation function; The Scale operation uses the generated weights to adjust each channel of the input feature map, focusing on meaningful features and obtaining the feature map of the spatial stream:

5. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 3 is characterized in that: The fused feature map is used to continue reasoning, and bounding box regression and category prediction are performed based on the Tiny YOLO output layer: The feature map F after fusion of the feature maps of the spatial stream and the temporal stream fused , input to the Tiny YOLO output module; The output layer of Tiny YOLO consists of a convolutional layer to predict the bounding box information and class probability within each grid; Convolution layer, using 1x1 convolution, adjusts the number of channels of the feature map to suit the output requirements: The output layer uses a set of convolutions to predict B bounding box information for each grid position on the S×S grid formed by the fused feature maps. The information of each bounding box includes: 4 bounding box coordinates (b x ,b y ,b w ,b h ), 1 confidence level, and C class probabilities; Decode the real rectangular box based on the relative coordinates of the grid coordinates and the box: B x =σ(t x )+c x B y =σ(t y )+c y Among them, t x ,t y ,t w ,t h is the output of the network; c x ,c y is the grid unit coordinate, p w ,p h is the predefined width and height of the anchor box, σ is the sigmoid function; Output the final target detection results, including the location of the object and the location and size of the bounding box.

6. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 5 is characterized in that: The S4 is specifically: Use the deep neural network DeepLab to perform semantic segmentation on each frame of the image to obtain accurate object level and category information for each pixel; The category information extracted from semantic segmentation and the detection results of Tiny YOLO are spatially aligned with the point cloud of LiDAR, and the category labels in the image are mapped to the corresponding point cloud through projection relationship to generate a local semantic geometry map; Using the IMU to provide the initial pose for rough alignment, the newly acquired LiDAR point cloud is aligned with the existing global map based on the ICP algorithm; Use nodes and edges to represent the motion relationship between consecutive frames, where nodes represent the device's posture and edges represent the estimated motion; use g2o optimization to minimize the difference between the error of each edge and the observation, and minimize the cumulative error in the posture graph; Combine the conditioned LiDAR point cloud with category information to generate a complete 3D semantic map.

7. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 6 is characterized in that: The IMU is used to provide an initial pose for rough alignment, and the newly collected LiDAR point cloud is aligned with the existing global map based on the ICP algorithm, as follows: The rotation matrix R provided by the IMU imu and displacement vector t imu Used to transform from a local coordinate system to a global coordinate system: Apply the initial pose provided by the IMU to the newly acquired LiDAR point cloud to obtain a rough global alignment: P aligned =R imu *P+ timu Where P is the coordinate matrix of the original LiDAR point cloud, P aligned It is the point cloud after the initial pose estimation; The ICP objective function is defined as minimizing the square error between the source point cloud P and the target map point cloud Q: min R,t ∑||R·p i +t-q i || 2 Among them, p i is the point of the source point cloud, q i is the target map with p i nearest point; For each source point pi, find the nearest neighbor point qi in the target map and use the kd tree data structure to speed up the nearest neighbor search; Use the singular value decomposition method to solve the rotation matrix R and the translation vector t Apply the calculated rotation matrix R and translation vector t to the entire source point cloud P new =R·P aligned +t。 8. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 7 is characterized in that: The singular value decomposition method is used to solve the rotation matrix R and the translation vector t, as follows: First, calculate the centroid of the two point clouds: Where N is the number of point clouds; And calculate the decentralized coordinates for each point: Calculate the covariance matrix H: Then, perform SVD decomposition on the covariance matrix: U,∑,V T =SVD(H) Among them, UU and V are orthogonal matrices, and Σ is a diagonal matrix; The rotation matrix R is calculated by the following formula: R=VU T If det(R)<0, correct it by reversing the direction of the minimum singular value: V[:,-1]=-V[:,-1] R=VU T ; Calculate the translation vector t: Finally, the output rotation matrix R and translation vector tt are used to align the source point cloud to the target point cloud.

9. The autonomous obstacle avoidance method for a UAV in a complex power inspection environment according to claim 1 is characterized in that: The D*Lite algorithm is specifically: Initialize the data structure, assuming that nodes represent grids in the grid map and edges represent paths between adjacent grids; Initialize the cost, priority, and estimated cost of all nodes to the target node; Starting from the starting point, continuously select the next node with the lowest estimated cost and move; If a new obstacle or environmental change is detected, update the cost of the corresponding node: g(s)=min(g(s),c(s,s′)+g(s′)) Where g(s) is the estimated cost from the current node to the starting point, and c(s,s′) is the path cost from node s to node s′; Start the update path backward from the target node to ensure that the changes are propagated to all affected nodes; Select the optimal path from the starting point to the target point and apply this path to guide the UAV's flight strategy. Continuously update the semantic map through sensor fusion and detect significant changes in the environment, including new or moved obstacles; According to the new map information, the update mechanism of the D*Lite algorithm is triggered to recalculate and update the safe path.

Images