Unmanned aerial vehicle routing inspection positioning method, electronic equipment, storage medium and program product

Abstract

The invention provides an unmanned aerial vehicle routing inspection positioning method, electronic equipment, a storage medium and a program product, and relates to the technical field of unmanned aerial vehicle routing inspection. The method comprises the steps that motion state information and inspection environment images of an unmanned aerial vehicle in inspection work are acquired in real time, the motion state information is acquired through an IMU, and the inspection environment images are acquired through a camera; determining first pose information of the unmanned aerial vehicle based on the motion state information; correcting the first pose information according to the inspection environment image to obtain second pose information; and determining an inspection positioning result of the unmanned aerial vehicle according to the second pose information. According to the invention, high-precision positioning of the unmanned aerial vehicle is realized, and the inspection task completion efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of drone inspection, and particularly to a drone inspection positioning method, an electronic device, a storage medium, and a program product. Background Art

[0002] In the power industry, with the increasing demand for drone inspections, the requirements for the automation and intelligence of drones are also constantly improving, especially in unobstructed ground environments. The improvement of drone automation and intelligence depends on the high-precision positioning of drones. Therefore, how to ensure the high-precision positioning of drones has become a key issue.

[0003] Currently, drones usually rely on the Global Positioning System (GPS for short) to locate their positions and plan inspection routes according to pre-established models to complete inspection tasks. However, during the drone inspection process, the strong electromagnetic interference generated by high-voltage transmission lines may cause the GPS signal to be affected or even fail, making it impossible for the drone to achieve high-precision positioning, thus reducing the efficiency of completing the inspection task. Summary of the Invention

[0004] The present application provides a drone inspection positioning method, an electronic device, a storage medium, and a program product to achieve high-precision positioning of drones and improve the efficiency of completing inspection tasks.

[0005] In a first aspect, the present application provides a drone inspection positioning method, including:

[0006] Obtain in real time the motion state information of the drone during the inspection work and the inspection environment image. The motion state information is collected by an Inertial Measurement Unit (IMU for short), and the inspection environment image is collected by a camera;

[0007] Determine the first pose information of the drone based on the motion state information;

[0008] Correct the first pose information according to the inspection environment image to obtain the second pose information;

[0009] Determine the inspection positioning result of the drone according to the second pose information.

[0010] In a possible implementation manner, correcting the first pose information according to the inspection environment image to obtain the second pose information includes:

[0011] Extract feature points in the inspection environment image and determine the position information of the feature points based on a feature extraction algorithm;

[0012] Convert the position information of the feature points to the IMU coordinate system using the extrinsic parameters, which are determined by extrinsic parameter calibration;

[0013] Use the first pose information to convert the feature points from the IMU coordinate system to the world coordinate system to complete the calibration of the position information of the feature points;

[0014] Construct a local environment map based on the calibrated position information of the feature points;

[0015] Extract features with key information from the local environment map;

[0016] Based on the extracted features with key information, correct the first pose information to obtain the second pose information.

[0017] In a possible implementation, based on the extracted features with key information, correcting the first pose information to obtain the second pose information includes:

[0018] Multiply the extrinsic parameters by the first pose information to obtain the camera pose parameters;

[0019] Determine the pixel brightness error of the feature with key information relative to the reference feature, where the reference feature is the feature with key information corresponding to the reference frame;

[0020] Determine the sum of the squares of the pixel brightness errors as the photometric error function;

[0021] Through an optimization algorithm, based on the photometric error function, adjust the camera pose parameters to minimize the photometric error and obtain the corrected camera pose parameters;

[0022] Based on the corrected camera pose parameters, correct the first pose information to obtain the second pose information.

[0023] In a possible implementation, based on the corrected camera pose parameters, correcting the first pose information to obtain the second pose information includes:

[0024] Multiply the corrected camera pose parameters by the inverse matrix of the first pose information to obtain the pose transformation matrix;

[0025] Multiply the pose transformation matrix by the first pose information to obtain the second pose information.

[0026] In a possible implementation, extracting features with key information from the local environment map includes:

[0027] Perform data screening processing on the local environment map to extract features with key information. The data screening processing includes outlier exclusion and sparse direct visual alignment.

[0028] In a possible implementation manner, determining the first pose information of the unmanned aerial vehicle based on the motion state information includes:

[0029] Performing time integration processing on the acceleration data and angular velocity data in the motion state information to obtain the initial pose information of the unmanned aerial vehicle;

[0030] Determining whether the Global Positioning System (GPS) set on the unmanned aerial vehicle is available;

[0031] When the GPS is unavailable, the initial pose information is determined as the first pose information;

[0032] When the GPS is available, obtaining the global position information of the unmanned aerial vehicle collected by the GPS;

[0033] Determining the first residual between the global position information and the initial pose information;

[0034] Based on the first residual, using a Kalman filter to correct the initial pose information, and the corrected initial pose information is determined as the first pose information of the unmanned aerial vehicle.

[0035] In a possible implementation manner, obtaining the global position information of the unmanned aerial vehicle collected by the GPS includes:

[0036] Obtaining the initial global position information of the unmanned aerial vehicle collected by the GPS;

[0037] Performing coordinate system conversion on the initial global position information to obtain the global position information, and the global position information and the motion state information are based on the same coordinate system.

[0038] In a possible implementation manner, determining the inspection and positioning result of the unmanned aerial vehicle according to the second pose information includes:

[0039] Judging whether the deviation between the second pose information and the actual pose information of the unmanned aerial vehicle is greater than or equal to the error threshold;

[0040] When the deviation is less than the error threshold, determining the second pose information as the inspection and positioning result;

[0041] When the deviation is greater than or equal to the error threshold, determining the neighborhood plane of the points in the point cloud data collected by the lidar, and the lidar is deployed on the unmanned aerial vehicle;

[0042] Calculating the second residual between the points in the point cloud data and the corresponding neighborhood plane;

[0043] Based on the second residual, using a Kalman filter to correct the second pose information, and the corrected second pose information is determined as the inspection and positioning result.

[0044] In a second aspect, the present application provides a UAV inspection positioning device, including:

[0045] An acquisition module, configured to acquire in real time the motion state information of the UAV during the inspection work and the inspection environment image. The motion state information is collected by the IMU, and the inspection environment image is collected by the camera;

[0046] A first determination module, configured to determine the first pose information of the UAV based on the motion state information;

[0047] A correction module, configured to correct the first pose information according to the inspection environment image to obtain the second pose information;

[0048] A second determination module, configured to determine the inspection positioning result of the UAV according to the second pose information.

[0049] In a possible implementation manner, the correction module is specifically configured to:

[0050] Based on the feature extraction algorithm, extract the feature points in the inspection environment image and determine the position information of the feature points;

[0051] Use the external parameters to convert the position information of the feature points into the IMU coordinate system, and the external parameters are determined through external parameter calibration;

[0052] Use the first pose information to convert the feature points from the IMU coordinate system into the world coordinate system to complete the calibration of the position information of the feature points;

[0053] Based on the calibrated position information of the feature points, construct a local environment map;

[0054] Extract the features with key information from the local environment map;

[0055] Based on the extracted features with key information, correct the first pose information to obtain the second pose information.

[0056] In a possible implementation manner, the correction module is specifically configured to:

[0057] Multiply the external parameters by the first pose information to obtain the camera pose parameters;

[0058] Determine the pixel brightness error of the feature with key information relative to the reference feature, and the reference feature is the feature with key information corresponding to the reference frame;

[0059] Determine the sum of the squares of the pixel brightness errors as the photometric error function;

[0060] Through the optimization algorithm, based on the photometric error function, adjust the camera pose parameters to minimize the photometric error and obtain the corrected camera pose parameters;

[0061] Based on the corrected camera pose parameters, correct the first pose information to obtain the second pose information.

[0062] In a possible implementation manner, the correction module is specifically configured to:

[0063] Multiply the corrected camera pose parameters by the inverse matrix of the first pose information to obtain a pose transformation matrix;

[0064] Multiply the pose transformation matrix by the first pose information to obtain the second pose information.

[0065] In a possible implementation manner, the UAV inspection and positioning device further includes a processing module, and the processing module is specifically configured to:

[0066] Perform data screening processing on the local environment map to extract features with key information, and the data screening processing includes outlier exclusion and sparse direct visual alignment.

[0067] In a possible implementation manner, the first determination module is specifically configured to:

[0068] Perform time integration processing on the acceleration data and angular velocity data in the motion state information to obtain the initial pose information of the UAV;

[0069] Determine whether the GPS set on the UAV is available;

[0070] When the GPS is unavailable, the initial pose information is determined as the first pose information;

[0071] When the GPS is available, obtain the global position information of the UAV collected by the GPS;

[0072] Determine the first residual between the global position information and the initial pose information;

[0073] Based on the first residual, use the Kalman filter to correct the initial pose information, and the corrected initial pose information is determined as the first pose information of the UAV.

[0074] In a possible implementation manner, the acquisition module is specifically configured to:

[0075] Obtain the initial global position information of the UAV collected by the GPS;

[0076] Perform coordinate system conversion on the initial global position information to obtain the global position information, and the global position information and the motion state information are based on the same coordinate system.

[0077] In a possible implementation manner, the second determination module is specifically configured to:

[0078] Determine whether the deviation between the second pose information and the actual pose information of the drone is greater than or equal to the error threshold;

[0079] When the deviation is less than the error threshold, determine the second pose information as the inspection positioning result;

[0080] When the deviation is greater than or equal to the error threshold, determine the neighborhood plane of the points in the point cloud data collected by the lidar, and the lidar is deployed on the drone;

[0081] Calculate the second residual of the points in the point cloud data to the corresponding neighborhood plane;

[0082] According to the second residual, use the Kalman filter to correct the second pose information, and the corrected second pose information is determined as the inspection positioning result.

[0083] In a third aspect, the present application provides an electronic device, including: a memory, a processor;

[0084] The memory stores computer-executable instructions;

[0085] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0086] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0087] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0088] The drone inspection and positioning method, electronic device, storage medium, and program product provided by this application relate to the technical field of drone inspections. The method includes: obtaining in real time the motion state information of the drone during the inspection work and the inspection environment images. The motion state information is collected by the IMU, and the inspection environment images are collected by the camera; determining the first pose information of the drone based on the motion state information; correcting the first pose information according to the inspection environment images to obtain the second pose information; and determining the inspection and positioning result of the drone according to the second pose information. This application uses the IMU to obtain the real-time motion state information of the drone and initially determines the pose of the drone based on the motion state information, that is, determines the first pose information of the drone; combines the environmental images collected by the camera to correct the first pose information, effectively eliminating the cumulative error caused by a single sensor. This multi-sensor fusion positioning method improves the positioning accuracy of the drone. The high-precision positioning result enables the drone to complete the predetermined route more accurately, reducing the situation of repeated or missed inspections, thereby significantly improving the completion efficiency and quality of the inspection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The accompanying drawings herein are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0090] Figure 1 is a schematic flowchart of the drone inspection and positioning method provided by this application Figure 1 ;

[0091] Figure 2 is a schematic diagram of the system corresponding to the drone inspection and positioning method when there is GPS signal input provided by an embodiment of this application;

[0092] Figure 3 is a schematic diagram of the system corresponding to the drone inspection and positioning method when there is no GPS signal input provided by an embodiment of this application;

[0093] Figure 4 is a schematic diagram of the structure of the drone inspection and positioning device provided by an embodiment of this application;

[0094] Figure 5 is a schematic diagram of the structure of the electronic device provided by an embodiment of this application.

[0095] Through the above accompanying drawings, the clear embodiments of this application have been shown, and there will be more detailed descriptions later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to explain the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0096] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0097] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0098] At present, with the increasing demand for power drone inspections, the requirements for automated and intelligent operations are also constantly increasing, especially in unobstructed ground environments. In order to ensure high-precision positioning and stability, drones usually rely on GPS to determine their own positions, and plan routes based on pre-established models to achieve autonomous flight operations. Although drones are widely used in power inspections, they still face many challenges in actual operations. For example, due to the strong electromagnetic interference that may be generated by high-voltage transmission lines, satellite positioning signals may be affected or even fail, which requires drones to be able to achieve high-precision and high-robust positioning even in environments with weak GPS signals.

[0099] In response to the above problems, this application uses a multi-sensor fusion positioning method to improve the positioning accuracy of drones. High-precision positioning results enable drones to complete the scheduled route more accurately, reduce repeated or missed inspections, and thus significantly improve the efficiency and quality of inspection tasks.

[0100] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0101] Figure 1 Schematic diagram of the process of the drone inspection and positioning method provided for this application Figure 1 ,like Figure 1 As shown, the method includes:

[0102] S101. Obtain the motion state information of the drone during the inspection work and the inspection environment images in real time. The motion state information is collected by the IMU, and the inspection environment images are collected by the camera.

[0103] In this step, it can be understood that the motion state information refers to a series of parameters describing the motion characteristics of the drone in the air, including but not limited to acceleration data and angular velocity data. Acceleration data and angular velocity data are the core components of the drone's motion state information and are collected at high frequency by the accelerometer and gyroscope of the IMU. The acceleration data reflects the linear acceleration changes of the drone on three orthogonal axes, with the unit of m / s², and is used to estimate speed, position, and attitude. The angular velocity data describes the rotation rate of the drone around the three axes, with the unit of rad / s or ° / s, and is mainly used for attitude estimation and stability control.

[0104] The inspection environment images are visual data collected in real time by the camera installed on the drone and are the key information source for the drone to perceive and understand the surrounding environment. These images are usually high-resolution color or multispectral images and can capture the detailed visual features of the inspection area.

[0105] S102. Determine the first pose information of the drone based on the motion state information.

[0106] After obtaining the motion state information through S101, it is necessary to determine the first pose information of the drone, that is, the initial position and attitude, according to the motion state information. This means that the first pose information of the drone is directly derived from the motion state information. This step provides a data basis for subsequent multi-sensor fusion and high-precision positioning. However, due to the existence of IMU sensor noise, simply relying on the motion state information may lead to pose estimation drift. Therefore, after determining the first pose information, step S103 is executed, that is, the first pose information is calibrated and optimized by combining the image data of the camera.

[0107] S103. Correct the first pose information according to the inspection environment images to obtain the second pose information.

[0108] After determining the first pose information of the drone, in order to further improve the positioning accuracy, it is necessary to correct the first pose information by combining the inspection environment images in step S103, so as to obtain a more accurate second pose information. The core of this correction process lies in the cumulative errors generated by the IMU during the prediction stage, and these errors will gradually increase over time, resulting in pose estimation drift.

[0109] S104. Determine the inspection positioning result of the drone according to the second pose information.

[0110] The inspection and positioning result refers to the specific position and attitude of the UAV during the inspection mission. Based on the second pose information, determining the inspection and positioning result of the UAV is the final step of the entire positioning process and the key to achieving precise inspection. Among them, the second pose information has fused IMU data and visual information to provide high-precision position and attitude estimation.

[0111] Furthermore, the specific implementation method of determining the inspection and positioning result of the UAV based on the second pose information can be set according to actual needs.

[0112] Exemplarily, determining the inspection and positioning result of the UAV based on the second pose information includes: judging whether the deviation between the second pose information and the actual pose information of the UAV is greater than or equal to the error threshold; when the deviation is less than the error threshold, determining the second pose information as the inspection and positioning result; when the deviation is greater than or equal to the error threshold, determining the neighborhood plane of the points in the point cloud data collected by the lidar, where the lidar is deployed on the UAV; calculating the second residual of the points in the point cloud data to the corresponding neighborhood plane; according to the second residual, using the Kalman filter to correct the second pose information, and the corrected second pose information is determined as the inspection and positioning result.

[0113] In this example, it can be understood that to determine the inspection and positioning result of the UAV based on the second pose information, it is first necessary to judge whether the deviation between the second pose information and the actual pose information of the UAV is greater than or equal to the error threshold. Among them, the error threshold can be set according to actual needs. For example, the error threshold is set to 0.2.

[0114] Perform corresponding processing according to the judgment result. If the deviation is less than the error threshold, the second pose information is determined as the inspection and positioning result at this time; if the deviation is greater than or equal to the error threshold, determine the neighborhood plane of the points in the point cloud data collected by the lidar. Among them, determining the neighborhood plane of the points in the point cloud data collected by the lidar means that for each point in the point cloud data, by analyzing the point distribution in its surrounding neighborhood, a local plane model is fitted to describe the local geometric characteristics of the point.

[0115] Exemplarily, first select the neighborhood point set of the target point, then calculate the centroid of the neighborhood points and construct the covariance matrix, then perform eigenvalue decomposition on the covariance matrix through principal component analysis, and use the eigenvector corresponding to the smallest eigenvalue as the normal vector of the plane, and finally fit the plane equation. Furthermore, selecting the neighborhood point set of the target point can be achieved by radius search or K-nearest neighbor search.

[0116] Further, calculate the second residual of the points in the point cloud data to the corresponding neighborhood planes. Then, based on the second residual, use the Kalman filter to correct the second pose information, and the corrected second pose information is determined as the inspection and positioning result. It should be noted that before correcting the second pose information, the second residuals greater than or equal to the residual threshold need to be removed.

[0117] In the embodiment of the present application, the real-time motion state information of the UAV is obtained by using the IMU, and the pose of the UAV is initially determined according to the motion state information, that is, the first pose information of the UAV is determined; the first pose information is corrected by combining the environmental images collected by the camera, effectively eliminating the cumulative error caused by a single sensor. This multi-sensor fusion positioning method improves the positioning accuracy of the UAV. The high-precision positioning result enables the UAV to complete the predetermined flight path more accurately, reducing the situation of repeated or missed inspections, thereby significantly improving the efficiency and quality of the inspection task.

[0118] Based on the above embodiment, correcting the first pose information according to the inspection environment image to obtain the second pose information includes: extracting feature points in the inspection environment image and determining the position information of the feature points based on the feature extraction algorithm; converting the position information of the feature points to the IMU coordinate system by using the extrinsic parameters, and the extrinsic parameters are determined by extrinsic parameter calibration; using the first pose information to convert the feature points from the IMU coordinate system to the world coordinate system to complete the calibration of the position information of the feature points; constructing a local environment map based on the calibrated position information of the feature points; extracting features with key information from the local environment map; and correcting the first pose information based on the extracted features with key information to obtain the second pose information.

[0119] In this embodiment, it can be understood that key feature points and their image coordinates are identified from the inspection image by using the feature extraction algorithm, where the feature extraction algorithm can be selected according to actual needs, such as the Scale-Invariant Feature Transform (SIFT for short). Subsequently, these feature points are converted from the image coordinate system to the IMU coordinate system by using the pre-calibrated extrinsic parameters to achieve the initial fusion of visual data and inertial data. Finally, using the first pose information provided by the IMU, the feature points are converted from the IMU coordinate system to the world coordinate system to complete the final position calibration. This multi-step conversion method makes full use of the advantages of visual and inertial sensors to improve the accuracy of feature point positioning. By unifying all information into the world coordinate system, it provides a solid foundation for the high-precision navigation and target recognition of the UAV in complex environments.

[0120] After completing the calibration of the position information of the feature points, a local environmental map needs to be constructed based on the calibrated position information of the feature points. Among them, the specific implementation method of constructing the local map can be set according to actual needs. For example, based on the calibrated position information of the feature points, multiple frames of point cloud data are aligned to the same coordinate system through a point cloud registration algorithm to form a local environmental point cloud map.

[0121] Furthermore, based on the extracted features with key information, the first pose information is corrected to obtain the second pose information, including: multiplying the external parameters by the first pose information to obtain the camera pose parameters; determining the pixel brightness error of the features with key information relative to the reference features, where the reference features are the features with key information corresponding to the reference frame; determining the sum of the squares of the pixel brightness errors as the photometric error function; through an optimization algorithm, based on the photometric error function, adjusting the camera pose parameters to minimize the photometric error to obtain the corrected camera pose parameters; based on the corrected camera pose parameters, correcting the first pose information to obtain the second pose information.

[0122] Among them, the pixel brightness error of the features with key information is to convert the corresponding image into a grayscale image, and the pixel value of the grayscale image represents the brightness of the pixel.

[0123] Even further, the sum of the squares of the pixel brightness errors is determined as the photometric error function. And through an optimization algorithm, based on the photometric error function, the camera pose parameters are adjusted to minimize the photometric error to obtain the corrected camera pose parameters.

[0124] Among them, adjusting the camera pose parameters based on the photometric error function through an optimization algorithm is a common visual odometry technology. The core idea is to iteratively adjust the camera pose parameters by using the optimization algorithm to gradually reduce the photometric error until the photometric error converges to the minimum value, and then obtain the corrected camera pose parameters.

[0125] Immediately afterwards, the first pose information needs to be corrected based on the corrected camera pose parameters to obtain the second pose information. Specifically, based on the corrected camera pose parameters, the first pose information is corrected to obtain the second pose information, including: multiplying the corrected camera pose parameters by the inverse matrix of the first pose information to obtain the pose transformation matrix; multiplying the pose transformation matrix by the first pose information to obtain the second pose information.

[0126] By correcting the first pose information based on the corrected camera pose parameters in the embodiments of the present application to obtain the second pose information, the limitations of a single sensor are overcome, providing a more comprehensive, accurate, and reliable positioning basis for UAV patrol.

[0127] After constructing the local environment map, it is necessary to extract features with key information from the constructed local environment map. Specifically, extracting features with key information from the local environment map includes: performing data screening processing on the local environment map to extract features with key information, and the data screening processing includes outlier exclusion and sparse direct visual alignment. By performing data screening processing on the local environment map in the embodiments of the present application, not only can the quality and performance of the map be improved, but also more efficient and reliable environmental perception support can be provided for the precise positioning of the drone.

[0128] Based on the above embodiments, determining the first pose information of the drone according to the motion state information described in step S102 includes: performing time integration processing on the acceleration data and angular velocity data in the motion state information to obtain the initial pose information of the drone; determining whether the GPS set on the drone is available; when the GPS is unavailable, determining the initial pose information as the first pose information; when the GPS is available, obtaining the global position information of the drone collected by the GPS; determining the first residual between the global position information and the initial pose information; based on the first residual, using a Kalman filter to correct the initial pose information, and the corrected initial pose information is determined as the first pose information of the drone.

[0129] During the drone positioning process, the acceleration and angular velocity data collected by the IMU can be used to calculate the initial pose information of the drone through time integration. Specifically, integrating the acceleration data over time can obtain the velocity, and integrating the velocity again can obtain the displacement; integrating the angular velocity data over time can calculate the pose change. The above calculations provide a preliminary estimate of the position and attitude of the drone.

[0130] In addition, after determining the initial pose information of the drone, it is also necessary to determine whether the GPS set on the drone is available. The specific implementation method for determining whether the GPS set on the drone is available can be selected according to actual needs.

[0131] In one implementation, its availability is judged by monitoring the intensity and quality of the GPS signal. If the signal intensity is lower than a certain threshold, the GPS is considered unavailable.

[0132] In another implementation, the frequency of GPS position updates is monitored. If the update frequency is too low or unstable, it indicates that the GPS signal is unavailable.

[0133] When it is determined that GPS is available, the Kalman filter is used to correct the initial pose information. The Kalman filter corrects the initial pose information of the UAV through the following steps: First, define the state variables of the system, including position, velocity, and attitude, and predict the current state using the IMU data through the motion model, that is, determine the initial pose information. In the prediction step, update the state estimate and error covariance, considering the influence of process noise. Then, by obtaining the global position information provided by GPS, calculate the difference between the global position information and the initial pose information, that is, calculate the first residual. Next, calculate the Kalman gain, which is determined by the prediction error covariance and the measurement noise covariance, and is used to weighted average the predicted state and the measured value. Then, use the Kalman gain to correct the predicted state and update the error covariance to reduce uncertainty. This process is iterated, continuously updating and correcting the state estimate, so as to effectively fuse the IMU and GPS data, reduce the influence of noise, and significantly improve the accuracy and reliability of the UAV pose.

[0134] Further, obtaining the global position information of the UAV collected by GPS includes: obtaining the initial global position information of the UAV collected by GPS; performing coordinate system transformation on the initial global position information to obtain the global position information, and the global position information and the motion state information are based on the same coordinate system.

[0135] In the embodiment of the present application, the Kalman filter continuously updates and adjusts the state estimate, effectively integrating the IMU and GPS data, reducing noise interference, and thus significantly improving the accuracy of the UAV pose.

[0136] Next, an example will be given to illustrate how to use the UAV inspection and positioning method provided by the embodiment of the present application. The overall positioning technical route of this method can be divided into four modalities: camera, lidar, IMU, and GPS. Based on such a design idea, an Error State Iterative Kalman Filter (ESIKF) that tightly couples and fuses the measurement data of lidar, camera, and IMU is implemented to fuse the four different position information. At the same time, GPS is fused with the Simultaneous Localization and Mapping (SLAM) system in a loose coupling manner.

[0137] 1. Processing process when there is GPS signal input

[0138] Figure 2 This is the system schematic diagram corresponding to the UAV inspection and positioning method when there is GPS signal input provided by the embodiment of the present application. From Figure 2 it can be learned that the system is divided into four parts:

[0139] 1.1 Frame Transformation and Residual Calculation: After the GPS signal enters the system, frame transformation is performed, and at the same time, GPS-IMU residual calculation is carried out. In the whole process, GPS plays an important role and combines with other modules for data processing and state estimation;

[0140] 1.2 Sensor Data Input and Preprocessing: Observation data from cameras, Light Detection and Ranging (LiDAR for short), IMU, and GPS are input into the system simultaneously at their respective frequencies. First, corresponding preprocessing is performed on the raw data collected by various sensors. For the three sensors with different frequencies of IMU, LiDAR, and cameras, hardware clock synchronization is required for cameras, LiDAR, and IMU, but time synchronization is not required for GPS. After receiving GPS data and camera and LiDAR data, data inspection is carried out to ensure that the basic formats of each sensor are correct and to prevent timestamp backtracking;

[0141] 1.3 IMU Forward Propagation and State Estimation: When receiving IMU data, data accumulation is carried out until correct GPS data or point cloud data is received. After necessary inspections and preprocessing such as format, IMU-LiDAR-camera or IMU-GPS forward propagation is performed between the first-frame inertial data and the observation data during accumulation to obtain the predicted state;

[0142] 1.4 Subsystem State Estimation and Fusion Output: The overall system is divided into two subsystems to perform state estimation from coarse to fine, namely the camera-LiDAR-IMU subsystem and the GPS-IMU subsystem. The IMU predicted state will be used to achieve coarse state estimation under the GPS-IMU subsystem, adding global constraints to the system to improve stability. The coarse estimation result is not directly used as the system output but is temporarily cached. The state updated by GPS is used as the input of the camera-LiDAR-IMU subsystem to further refine the state. Finally, the system outputs the system state with GPS constraints in the form of a GPS-camera-LiDAR-IMU fusion mode at the LiDAR frequency, and at the same time constructs a dense three-dimensional point cloud map.

[0143] Furthermore, the positioning information determined by different sensors has its own world coordinate reference system, and the absolute position of each positioning information in the world reference coordinate system may show an increase in uncertainty and random drift over time. Therefore, the absolute positions of different positioning information cannot be compared with each other. These four types of positioning information will be used in the error state iterative Kalman filter through their respective relative pose differences. Due to the operation of the inspection UAV, the covariance of the four types of positioning information will also grow without bounds. Therefore, directly using the covariance of the four types of positioning information itself is of no use, but the covariance changes of the four types of positioning information over a period of time, such as velocity covariance, can be used.

[0144] The motion equation and observation equation for the inspection UAV carrying sensors and operating in an unknown environment are as follows:

[0145] (1)

[0146] (2)

[0147] where, \(x\) k is all the state variables at time \(k\); \(z\) k is all the observation data at time \(k\); \(u\) k is the sensor reading; \(f\) is the motion equation; \(h\) is the observation equation; \(w\) k is the noise; \(v\) k is the observation noise.

[0148] Assume that both noises follow a zero-mean Gaussian distribution:

[0149] (3)

[0150] (4)

[0151] where, \(R\) represents the covariance matrix of the observation noise; \(Q\) represents the covariance matrix of the process noise.

[0152] When the inspection UAV uses vision, IMU, GPS, and LiDAR, the motion equations and observation equations of these positioning information are usually non-linear functions. In a non-linear system, a non-Gaussian distribution needs to be approximated as a Gaussian distribution. Therefore, the linear results of the error state iterative Kalman filter need to be introduced into the non-linear system. The error state iterative Kalman filter is based on the Markov property, that is, the state at time \(k\) is only related to the state at time \(k - 1\) and has nothing to do with other states. Let the mean and covariance matrix at time \(k - 1\) be 、 . At time \(k\), the motion equation and observation equation are linearized at 、 Perform a first-order Taylor expansion at a certain point, retaining only the first-order terms, i.e., the linear part, and deriving the remaining part according to the linear system. The linearization operation is as follows:

[0153] (5)

[0154] (6)

[0155] Denote the partial derivatives at two places as:

[0156] (7)

[0157] (8)

[0158] During the prediction process, according to the motion equation, we have:

[0159] (9)

[0160] Denote the prior and covariance mean here as:

[0161] (10)

[0162] (11)

[0163] During the prediction process, according to the observation equation, we have:

[0164] (12)

[0165] During the update process, the Kalman gain K k is:

[0166] (13)

[0167] Based on the Kalman gain, the posterior probability form of x k is derived as:

[0168] (14)

[0169] (15)

[0170] In this non-linear system, the Kalman filter gives the maximum a posteriori estimate under a single linear approximation. The error-state iterative Kalman filter applied in the embodiments of this application synchronizes the update frequencies of all sensor information. It uses the relative poses output by cameras, IMUs, GPSs, and lidars, and finally outputs the multi-modal fusion pose of the ground robot.

[0171] 2. Processing process when there is no GPS signal input

[0172] Figure 3 This is a schematic diagram of the system corresponding to the UAV inspection and positioning method without GPS signal input provided by the embodiments of this application. It can be learned from Figure 3 that this system is divided into four parts:

[0173] 2.1 Independent operation of each subsystem: Thanks to the design architecture of the two subsystems, when the GPS signal is missing, the system can rely on the LiDAR-IMU and camera-IMU modes to continue to maintain the navigation function and ensure that the state estimation will not be interrupted. Among them, LiDAR data processing and odometer estimation: When LiDAR performs backpropagation, it will calculate the point-to-plane residuals. And LiDAR, camera, and IMU jointly perform odometer estimation, that is, LiDAR-camera-IMU odometer. LiDAR determines the relative position change of the UAV by comparing with the previous scan data and provides high-precision three-dimensional spatial information for the system;

[0174] 2.2 Image data processing and fusion output of the camera: Input the image data of the camera, and perform outlier exclusion, sparse direct visual alignment, and form a local environment map on it. The camera, LiDAR, and IMU jointly constitute a part of the system and participate in the overall data processing and state estimation through visual global map fusion output. The vision system uses the recognition and matching of feature points in the environment and combines image changes to estimate the motion state of the UAV, which complements the information of LiDAR and jointly constructs an environmental model;

[0175] 2.3 IMU data processing and state estimation: IMU performs forward propagation and state estimation, and there is also a process of state buffering. IMU also participates in odometer estimation and residual calculation in the system together with LiDAR and the camera. IMU plays an important role in state estimation and data fusion in the whole system. The data of IMU will accumulate errors over time, but through the information fusion with LiDAR and the camera, the position and attitude estimation of the UAV are continuously adjusted.

[0176] Furthermore, before performing state estimation, it is necessary to perform corresponding preprocessing on the raw data collected by various sensors. Among them, IMU, LiDAR, and the camera are three sensors with different frequencies. Therefore, this system needs to synchronize the hardware clocks of IMU, the camera, and LiDAR, and perform data checks after receiving the camera and LiDAR data to ensure that the basic formats of each sensor are correct and prevent timestamp backtracking.

[0177] To ensure that different subsystems have the ability of independent state estimation, a frame of point cloud data is set to directly trigger the algorithm for state estimation. That is, when the system receives IMU data, data accumulation is performed until correct point cloud data is received. After necessary preprocessing such as checking and formatting, IMU-LiDAR-camera will be performed between the first-frame inertial data and observation data during accumulation to obtain the predicted state.

[0178] It should be noted that when calculating the point-to-plane residual, the point cloud data collected by the lidar can be input into the IKDTree frame by frame. The IKDTree will organize the point cloud data into a tree structure according to the spatial position, which is beneficial to efficiently manage and query the point cloud data and speed up the calculation speed of the point-to-plane residual.

[0179] In summary, since the power transmission and distribution line belongs to a weak texture environment, and the multi-sensor fusion technology provides an efficient and reliable solution for improving the accuracy and robustness of pose estimation. Therefore, the embodiments of the present application combine the working scene characteristics and specific operation requirements of the unmanned aerial vehicle, and adopt multi-source sensor fusion based on cameras, LiDAR, IMU, and GPS of real-time kinematic (abbreviated as RTK) to achieve high-precision and stable positioning and navigation, and complete the inspection of the power transmission and distribution line corridor. Further, thanks to the design architecture of the two subsystems, when some sensors degenerate or fail, the system can continue to maintain the navigation function relying on the LiDAR-IMU and camera-IMU modes, ensuring that the state estimation will not be interrupted.

[0180] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For the details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0181] Figure 4 It is a schematic structural diagram of the unmanned aerial vehicle inspection and positioning device provided by the embodiment of the present application. As Figure 4 shown, the unmanned aerial vehicle inspection and positioning device 400 provided in this embodiment includes:

[0182] An acquisition module 401, configured to acquire in real time the motion state information of the unmanned aerial vehicle during the inspection work and the inspection environment image. The motion state information is collected by the IMU, and the inspection environment image is collected by the camera;

[0183] A first determination module 402, configured to determine the first pose information of the unmanned aerial vehicle based on the motion state information;

[0184] A correction module 403, configured to correct the first pose information according to the inspection environment image to obtain the second pose information;

[0185] The second determination module 404 is configured to determine the inspection and positioning result of the UAV according to the second pose information.

[0186] In a possible implementation, the correction module 403 is specifically configured to:

[0187] Extract feature points in the inspection environment image and determine the position information of the feature points based on the feature extraction algorithm;

[0188] Convert the position information of the feature points to the IMU coordinate system by using the external parameters, and the external parameters are determined through external parameter calibration;

[0189] Use the first pose information to convert the feature points from the IMU coordinate system to the world coordinate system to complete the calibration of the position information of the feature points;

[0190] Construct a local environment map based on the calibrated position information of the feature points;

[0191] Extract features with key information from the local environment map;

[0192] Based on the extracted features with key information, correct the first pose information to obtain the second pose information.

[0193] In a possible implementation, the correction module 403 is specifically configured to:

[0194] Multiply the external parameters by the first pose information to obtain the camera pose parameters;

[0195] Determine the pixel brightness error of the feature with key information relative to the reference feature, and the reference feature is the feature with key information corresponding to the reference frame;

[0196] Determine the sum of the squares of the pixel brightness errors as the photometric error function;

[0197] Through an optimization algorithm, based on the photometric error function, adjust the camera pose parameters to minimize the photometric error and obtain the corrected camera pose parameters;

[0198] Based on the corrected camera pose parameters, correct the first pose information to obtain the second pose information.

[0199] In a possible implementation, the correction module 403 is specifically configured to:

[0200] Multiply the corrected camera pose parameters by the inverse matrix of the first pose information to obtain the pose transformation matrix;

[0201] Multiply the pose transformation matrix by the first pose information to obtain the second pose information.

[0202] In a possible implementation, the UAV inspection and positioning device further includes a processing module (not shown), and the processing module is specifically configured to:

[0203] Perform data screening processing on the local environment map to extract features with key information. The data screening processing includes outlier exclusion and sparse direct visual alignment.

[0204] In a possible implementation, the first determination module 402 is specifically configured to:

[0205] Perform time integration processing on the acceleration data and angular velocity data in the motion state information to obtain the initial pose information of the UAV;

[0206] Determine whether the GPS set on the UAV is available;

[0207] When the GPS is unavailable, the initial pose information is determined as the first pose information;

[0208] When the GPS is available, obtain the global position information of the UAV collected by the GPS;

[0209] Determine the first residual between the global position information and the initial pose information;

[0210] Based on the first residual, use the Kalman filter to correct the initial pose information, and the corrected initial pose information is determined as the first pose information of the UAV.

[0211] In a possible implementation, the acquisition module 401 is specifically configured to:

[0212] Obtain the initial global position information of the UAV collected by the GPS;

[0213] Perform coordinate system conversion on the initial global position information to obtain the global position information, and the global position information and the motion state information are based on the same coordinate system.

[0214] In a possible implementation, the second determination module 404 is specifically configured to:

[0215] Judge whether the deviation between the second pose information and the actual pose information of the UAV is greater than or equal to the error threshold;

[0216] When the deviation is less than the error threshold, determine the second pose information as the inspection and positioning result;

[0217] When the deviation is greater than or equal to the error threshold, determine the neighborhood plane of the points in the point cloud data collected by the lidar. The lidar is deployed on the UAV;

[0218] Calculate the second residual between the points in the point cloud data and the corresponding neighborhood plane;

[0219] Based on the second residual, the pose information of the second position is corrected using a Kalman filter, and the corrected pose information of the second position is determined as the inspection and positioning result.

[0220] The drone inspection and positioning device provided by the embodiments of the present application can execute the method provided by the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0221] It should be noted that it should be understood that the division of each module of the above device is only a logical function division. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a certain processing element of the above device to perform the functions of the above processing module. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. Here, the processing element can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit in the hardware of the processor element or the instruction in the form of software.

[0222] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as: one or more Application Specific Integrated Circuits (ASICs), or, one or more Digital Signal Processors (DSPs), or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain module above is implemented in the form of a program code scheduled by a processing element, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a System-On-a-Chip (SOC).

[0223] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 5As shown, the electronic device 500 provided in the embodiment of the present application may include: a processor 501, and a memory 502 communicatively connected to the processor, where:

[0224] The memory stores computer-executable instructions;

[0225] The processor executes the computer-executable instructions stored in the memory to implement the method described in the foregoing method embodiment.

[0226] It should be understood that the processor 501 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, abbreviated as DSP), application-specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the application can be directly implemented by the execution of the hardware processor, or implemented by the combination of hardware and software modules in the processor. The memory 502 may include a high-speed random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile storage NVM (non-volatile memory), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disc, etc.

[0227] Optionally, the electronic device 500 may further include a communication interface 503. In a specific implementation, if the communication interface 503, the memory 502, and the processor 501 are independently implemented, the communication interface 503, the memory 502, and the processor 501 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0228] Optionally, in a specific implementation, if the communication interface 503, the memory 502, and the processor 501 are integrated on a chip, the communication interface 503, the memory 502, and the processor 501 may communicate through an internal interface.

[0229] An embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions that, when executed, are used to implement the method described in any of the foregoing embodiments.

[0230] It can be understood that the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0231] An exemplary computer-readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can be located in an ASIC. Of course, the processor and the computer-readable storage medium can also exist as discrete components in an electronic device.

[0232] The integrated modules implemented in the form of software functional modules described above can be stored in a computer-readable storage medium. The software functional modules stored in a computer-readable storage medium include several instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in various embodiments of the present application.

[0233] An embodiment of the present application further provides a computer program product including a computer program that, when executed, implements the method described in any of the foregoing embodiments.

[0234] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0235] Furthermore, it should be noted that although the steps in the flowchart are shown sequentially according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0236] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification.

[0237] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of this application. This application is intended to cover any variations, uses, or adaptations of this application, which follow the general principles of this application and include well-known common knowledge or conventional technical means in the technical field not disclosed in this application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of this application are pointed out by the following claims.

[0238] It should be understood that this application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

Claims

1. A method for positioning an unmanned aerial vehicle inspection, characterized in that: include: Real-time acquisition of the motion state information of the drone during inspection work and the inspection environment image, wherein the motion state information is collected by the inertial measurement unit IMU, and the inspection environment image is collected by the camera; Based on the motion state information, determining the first posture information of the drone; According to the inspection environment image, the first posture information is corrected to obtain second posture information; The inspection positioning result of the UAV is determined according to the second posture information.

2. The method according to claim 1, characterized in that The step of correcting the first posture information according to the inspection environment image to obtain second posture information includes: Based on a feature extraction algorithm, feature points in the inspection environment image are extracted and position information of the feature points is determined; The position information of the feature point is converted into an IMU coordinate system by using an external parameter, wherein the external parameter is determined by external parameter calibration; Using the first pose information, the feature point is converted from the IMU coordinate system to the world coordinate system to complete the calibration of the position information of the feature point; Based on the location information of the calibrated feature points, a local environment map is constructed; Extracting features with key information from the local environment map; Based on the extracted features with key information, the first posture information is corrected to obtain second posture information.

3. The method according to claim 2, characterized in that The step of correcting the first posture information based on the extracted features having key information to obtain second posture information includes: Multiplying the external parameter by the first pose information to obtain the camera pose parameter; Determine a pixel brightness error of the feature with key information relative to a reference feature, wherein the reference feature is a feature with key information corresponding to a reference frame; Determine the sum of squares of the pixel brightness errors as a photometric error function; By using an optimization algorithm, based on the photometric error function, the camera pose parameters are adjusted to minimize the photometric error, thereby obtaining corrected camera pose parameters; Based on the corrected camera pose parameters, the first pose information is corrected to obtain second pose information.

4. The method according to claim 3, characterized in that The step of correcting the first pose information based on the corrected camera pose parameters to obtain second pose information includes: Multiplying the corrected camera pose parameters by the inverse matrix of the first pose information to obtain a pose transformation matrix; The posture transformation matrix is ​​multiplied by the first posture information to obtain second posture information.

5. The method according to claim 2, characterized in that: The extracting features with key information from the local environment map includes: The local environment map is subjected to data screening processing to extract features with key information, wherein the data screening processing includes outlier exclusion and sparse direct visual alignment.

6. The method according to any one of claims 1 to 5, characterized in that The determining, based on the motion state information, the first posture information of the drone includes: Performing time integration processing on the acceleration data and angular velocity data in the motion state information to obtain initial position information of the UAV; Determine whether the global positioning system (GPS) set on the drone is available; When the GPS is unavailable, the initial position information is determined as the first position information; When the GPS is available, obtaining the global position information of the drone collected by the GPS; Determining a first residual between the global position information and the initial pose information; Based on the first residual, the initial posture information is corrected using a Kalman filter, and the corrected initial posture information is determined as the first posture information of the drone.

7. The method according to claim 6, characterized in that The obtaining of the global position information of the drone collected by the GPS includes: Obtaining the initial global position information of the drone collected by the GPS; The initial global position information is subjected to a coordinate system transformation to obtain the global position information, wherein the global position information and the motion state information are based on the same coordinate system.

8. The method according to any one of claims 1 to 5, characterized in that The step of determining the inspection positioning result of the UAV according to the second posture information includes: Determine whether a deviation between the second posture information and the actual posture information of the drone is greater than or equal to an error threshold; When the deviation is less than the error threshold, determining the second posture information as the inspection positioning result; When the deviation is greater than or equal to an error threshold, determining a neighborhood plane of a midpoint in the point cloud data collected by a laser radar, wherein the laser radar is deployed on the drone; Calculate a second residual from a midpoint of the point cloud data to a corresponding neighborhood plane; According to the second residual, the second posture information is corrected using a Kalman filter, and the corrected second posture information is determined as the inspection positioning result.

9. A drone inspection and positioning device, characterized in that: include: An acquisition module is used to acquire the motion state information and inspection environment image of the UAV in real time during the inspection work, wherein the motion state information is acquired by an inertial measurement unit IMU, and the inspection environment image is acquired by a camera; A first determination module, configured to determine the first posture information of the drone based on the motion state information; A correction module, used for correcting the first posture information according to the inspection environment image to obtain second posture information; The second determination module is used to determine the inspection positioning result of the UAV according to the second posture information.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed.

Images