Multi-sensor fusion navigation system and method of electric power inspection unmanned aerial vehicle and unmanned aerial vehicle

Abstract

The invention discloses a multi-sensor fusion navigation system and method of an electric power inspection unmanned aerial vehicle and the unmanned aerial vehicle, and relates to the technical field of unmanned aerial vehicle navigation, the multi-sensor fusion navigation system comprises a laser radar unit and an inertia measurement unit which are coupled to form a laser radar-inertia subsystem, and a vision unit and the inertia measurement unit which are coupled to form a vision-inertia subsystem; the laser radar-inertial subsystem is used for calculating an iterative nearest point error between the current frame and the global map; using the iterative nearest point error as an observed value of an error state iterative Kalman filter to execute a filtering updating process, and constructing and updating a global map; and the vision-inertia subsystem is used for constructing a hybrid reprojection error based on the projection points and the image feature points of the recovery depth, and performing state estimation by taking the hybrid reprojection error as an observation value of an error state iteration Kalman filter. The technical problem that in the prior art, stable and high-precision navigation cannot be achieved in an electric power scene is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle navigation, in particular to a multi-sensor fusion navigation system, method and unmanned aerial vehicle for power inspection. BACKGROUND

[0002] With the continuous improvement of the intelligent level of the power system, higher requirements for the autonomous inspection capability of unmanned aerial vehicles in scenarios such as power transmission lines and substations are proposed. Traditional navigation methods are mostly based on a single sensor, such as using only vision or laser radar for positioning, which has obvious defects in actual complex scenarios: the accuracy of laser radar is reduced in repeated structures or sparse point cloud areas, and the vision navigation system is easily affected by changes in light, occlusion and texture loss, resulting in tracking failure. Although the existing multi-sensor fusion method improves the system performance to a certain extent, it mostly adopts a loose coupling or weak fusion strategy, and does not fully utilize the complementarity of different modal data, resulting in that it is still unable to achieve stable and high-precision navigation in the power scene.

[0003] In addition, complex power environments are often accompanied by problems such as metal reflection, interference of occlusion, and sensor failure, requiring the navigation system to have stronger robustness and real-time performance. SUMMARY

[0004] The purpose of the present application is to provide a multi-sensor fusion navigation system, method and unmanned aerial vehicle for power inspection in order to solve at least one of the above technical problems.

[0005] In a first aspect, an embodiment of the present application provides a multi-sensor fusion navigation system of a power inspection unmanned aerial vehicle, comprising: a vision unit, an inertial measurement unit, and a laser radar unit; the laser radar unit and the inertial measurement unit are coupled to form a laser radar-inertial subsystem, the vision unit and the inertial measurement unit are coupled to form a vision-inertial subsystem, and the laser radar-inertial subsystem and the vision-inertial subsystem constitute a fusion navigation architecture; the laser radar-inertial subsystem is configured to: remove motion distortion in laser radar point cloud by using pre-integration information of the inertial measurement unit, calculate an iterative closest point error between a current frame and a global map, perform a filtering update process by taking the iterative closest point error as an observation value of an error state iterative Kalman filter, and construct and update the global map; and the vision-inertial subsystem is configured to: extract plane features from the global map based on an adaptive incremental method, project the plane features to a projection point based on a vision image frame obtained by the vision unit, and extract image feature points from the vision image frame; track the projection point and the image feature points between adjacent vision image frames by an optical flow method, and remove abnormal points in the tracking process; estimate depth values of the image feature points by a sliding window optimization method, and recover image feature points that fail to be tracked in a sliding window by epipolar geometry constraints; construct a hybrid reprojection error based on the projection point and the image feature points with recovered depth, and perform state estimation by taking the hybrid reprojection error as an observation value of the error state iterative Kalman filter.

[0006] In a second aspect, an embodiment of the present application further provides a multi-sensor fusion navigation method of a power inspection unmanned aerial vehicle, applied to the multi-sensor fusion navigation system of the power inspection unmanned aerial vehicle provided by the present application; the method comprises the following steps: removing motion distortion in laser radar point cloud by using pre-integration information of the inertial measurement unit, calculating an iterative closest point error between a current frame and a global map, performing a filtering update process by taking the iterative closest point error as an observation value of an error state iterative Kalman filter, and constructing and updating the global map; extracting plane features from the global map based on an adaptive incremental method, projecting the plane features to a projection point based on a vision image frame obtained by the vision unit, and extracting image feature points from the vision image frame; tracking the projection point and the image feature points between adjacent vision image frames by an optical flow method, and removing abnormal points in the tracking process; estimating depth values of the image feature points by a sliding window optimization method, and recovering image feature points that fail to be tracked in a sliding window by epipolar geometry constraints; constructing a hybrid reprojection error based on the projection point and the image feature points with recovered depth, and performing state estimation by taking the hybrid reprojection error as an observation value of the error state iterative Kalman filter.

[0007] Further, the pre-integration information of the inertial measurement unit is used to remove motion distortion in the laser radar point cloud, and an iterative closest point error between a current frame and a global map is calculated, including: acquiring laser radar point cloud data of the current frame, removing laser radar point cloud motion distortion by using the pre-integration result of the inertial measurement unit, and obtaining non-distorted laser radar point cloud; traversing the non-distorted laser radar point cloud of the current frame, and solving a three-dimensional distance error of any point in the current frame to a historical map point of the global map under the pre-integration state prior condition of the inertial measurement unit, and accumulating and summing to obtain the iterative closest point error.

[0008] Further, plane features are extracted from the global map based on an adaptive incremental method, including: selecting data in a preset range in the global map as a local map based on the pre-integration state of the inertial measurement unit, and rasterizing the local map to obtain a grid map; performing octree division on each grid in the grid map to extract plane features; and a division termination condition is that a number of points in the grid is less than a preset threshold or the points in the grid satisfy a preset plane constraint condition.

[0009] Further, a depth value of the image feature point is estimated by a sliding window optimization method, and a tracking failed image feature point in a sliding window is recovered by epipolar geometry constraint, including: selecting state estimation results and image feature points in a period of time to construct a sliding window, and recovering the tracking failed image feature point in the sliding window by epipolar geometry constraint; the depth initial value of the image feature point is recovered by triangulation method; a target function is constructed in the sliding window by re-projection error according to the depth initial value, and a maximum a posteriori value of the depth value of the image feature point is optimized and solved by Levenberg-Marquardt method.

[0010] Further, the image feature point includes a Harris corner point.

[0011] Further, abnormal points in the tracking process are removed, including: calculating a homography matrix by using the projection point and the image feature point, and removing error optical flow tracking points by RANSAC algorithm iteration.

[0012] In a third aspect, an electronic device is provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method provided by the embodiments of the present application when executing the computer program.

[0013] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions, and the computer instructions are executed by a processor to implement the method provided by the embodiments of the present application.

[0014] In the fifth aspect, the embodiment of the present application further provides an electric power inspection unmanned aerial vehicle, comprising the multi-sensor fusion navigation system of the electric power inspection unmanned aerial vehicle provided by the embodiment of the present application.

[0015] The present application provides a multi-sensor fusion navigation system, method and unmanned aerial vehicle of an electric power inspection unmanned aerial vehicle, which effectively overcomes the problems of unstable positioning and decreased precision of a single sensor in a complex electric power scene by constructing a navigation architecture of fusion of a laser radar-inertial subsystem and a vision-inertial subsystem, and combining a multi-source data deep fusion of an error state iterative Kalman filter; the efficient complement of image texture information and point cloud geometry structure is realized by introducing an adaptive plane feature extraction and projection mechanism, a hybrid feature point tracking strategy and a sliding window depth estimation method, the navigation robustness and continuity of the system in a strong light, shielding, sparse texture and other harsh environments are enhanced; and finally, the mixed re-projection error is uniformly integrated into a state optimization framework, which greatly improves the positioning precision and system stability of the unmanned aerial vehicle in the electric power inspection task. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the specific embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 The present application provides a multi-sensor fusion navigation system of an electric power inspection unmanned aerial vehicle; Figure 2 The present application provides a multi-sensor fusion navigation method of an electric power inspection unmanned aerial vehicle; Figure 3 The present application provides a multi-sensor fusion navigation method of an electric power inspection unmanned aerial vehicle; Figure 4 The present application provides a multi-sensor fusion navigation method of an electric power inspection unmanned aerial vehicle; Figure 5 The present application provides a multi-sensor fusion navigation method of an electric power inspection unmanned aerial vehicle. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] Embodiment one Figure 1 A multi-sensor fusion navigation system of a power inspection unmanned aerial vehicle is provided according to an embodiment of the present application. As shown in the figure, the system comprises a vision unit 10, an inertial measurement unit 20 and a laser radar unit 30; the laser radar unit 30 is coupled with the inertial measurement unit 20 to form a laser radar-inertial subsystem 200, the vision unit 10 and the inertial measurement unit 20 are coupled to form a vision-inertial subsystem 100, and the laser radar-inertial subsystem 200 and the vision-inertial subsystem 100 constitute a fusion navigation architecture. Figure 1

[0020] The vision unit 10 comprises a camera.

[0021] Specifically, the laser radar-inertial subsystem 200 is configured to: remove motion distortion in laser radar point cloud by using pre-integration information of the inertial measurement unit, calculate an iterative closest point error between a current frame and a global map, and perform a filtering update process by taking the iterative closest point (ICP) error as an observation value of an error-state iterated Kalman filter (ESIKF) to construct and update the global map. The vision-inertial subsystem 100 is configured to: extract plane features from the global map based on an adaptive incremental method, project the plane features to a vision image frame plane based on a vision unit to obtain projected points, and extract image feature points from the vision image frame; track the projected points and the image feature points between adjacent vision image frames by an optical flow method and remove abnormal points in the tracking process; estimate depth values of the image feature points by a sliding window optimization method and recover image feature points that fail to be tracked in the sliding window based on epipolar geometry constraints; construct a hybrid reprojection error based on the projected points and the image feature points with recovered depth, and perform state estimation by taking the hybrid reprojection error as an observation value of the error-state iterated Kalman filter.

[0022] Preferably, the image feature points comprise Harris corner points.

[0023] Specifically, as shown in the figure, Figure 1 ​As shown, the embodiment of the present application provides a fusion navigation architecture, for a laser radar-inertial subsystem (LiDAR-Inertial Subsystem, LIS), firstly, the pre-integration information of the inertial measurement unit (Inertial Measurement Unit, IMU) is used to remove the motion distortion in the laser radar point cloud; then, the ICP error between the current frame and the global map is calculated and used as the observation value of the ESIKF; finally, the ESIKF filtering update process is executed to realize state updating and global map construction.

[0024] For a visual-inertial subsystem (Visual-Inertial System, VIS), firstly, the latest global map is voxelized, and the plane information is extracted from the voxel through an adaptive incremental method, the plane center point is projected into the image to obtain the projection point, and the Harris corner point is extracted from the image; then, the projection point and the Harris corner point between adjacent image frames are tracked through the optical flow method, and the RANSAC is used to remove the abnormal points in the tracking process; next, the depth value of the Harris corner point is estimated through the sliding window optimization method, and the pixel value of the corner point that fails to be tracked in the sliding window is recovered through the epipolar geometry constraint; finally, the re-projection error is constructed by using the projection point and the Harris corner point, and the VIS state estimation is realized through the ESIKF.

[0025] The embodiment of the present application also provides an electric power inspection unmanned aerial vehicle, which comprises the multi-sensor fusion navigation system of the electric power inspection unmanned aerial vehicle provided by the embodiment of the present application.

[0026] Embodiment two Figure 2 It is a flow chart of a multi-sensor fusion navigation method of an electric power inspection unmanned aerial vehicle according to the embodiment of the present application, which is applied to the multi-sensor fusion navigation system of the electric power inspection unmanned aerial vehicle provided by the embodiment of the present application. As shown in the figure, Figure 2 The method specifically comprises the following steps: Step S202, the pre-integration information of the inertial measurement unit is used to remove the motion distortion in the laser radar point cloud, and the iterative closest point error between the current frame and the global map is calculated.

[0027] Step S204, the iterative closest point error is used as the observation value of the error state iterative Kalman filter to execute the filtering update process, and the global map is constructed and updated.

[0028] Step S206, the plane features are extracted from the global map based on the adaptive incremental method, the plane features are projected to the plane of the visual image frame based on the visual unit to obtain the projection point, and the image feature points are extracted from the visual image frame.

[0029] Step S208, track the projection points and image feature points between adjacent visual image frames by the optical flow method, and remove the abnormal points in the tracking process.

[0030] Step S210, estimate the depth value of the image feature points by the sliding window optimization method, and recover the image feature points that fail to track in the sliding window by the epipolar geometry constraint.

[0031] Step S212, construct a hybrid reprojection error based on the projection points and the image feature points with recovered depth, and use the hybrid reprojection error as the observation value of the error state iterative Kalman filter to perform state estimation.

[0032] Specifically, step S202 further includes the following steps: Step S2021, acquire the current frame of laser radar point cloud data, remove the motion distortion of the laser radar point cloud by using the pre-integration result of the inertial measurement unit, and obtain the non-distorted laser radar point cloud; Step S2022, traverse the current frame of non-distorted laser radar point cloud, solve the three-dimensional distance error of any point in the current frame to the historical map point of the global map under the pre-integration state prior condition of the inertial measurement unit, and accumulate and sum to obtain the iterative nearest point error.

[0033] Then, an error state propagation equation is constructed, an observation equation is constructed by using the ICP error, and the error state is estimated by using the error state iterative Kalman filter.

[0034] Specifically, step S206 further includes the following steps: Step S2061, select the data within a preset range in the global map as a local map based on the pre-integration state of the inertial measurement unit, and rasterize the local map to obtain a raster map.

[0035] For example, according to the IMU pre-integration state, the data within a range of 50 m is selected as a local map, and then the local map is rasterized according to 5 m.

[0036] Step S2062, perform octree division on each grid in the raster map to extract plane features; wherein the termination condition of the division is that the number of points in the grid is less than a preset threshold or the points in the grid satisfy a preset plane constraint condition.

[0037] Specifically, step S206 further includes incremental updating of the plane features: as the system runs, the point cloud in the grid changes, and only the newly added point cloud data is used to update the plane features, thereby reducing the calculation amount.

[0038] Specifically, Figure 3 is a schematic diagram of an adaptive plane feature extraction method according to an embodiment of the present application, Figure 4is a flow chart of an adaptive plane feature extraction method according to an embodiment of the present application. As shown in Figure 3 and Figure 4 , define the global map point representation as , where represents the world coordinate system, using the system pose information output by the LIS, the first laser radar point cloud data can be converted to the world coordinate system , that is, , and added to the global map. Next, the point cloud data in the world coordinate system is divided using a voxel grid , with a grid size of 1 m, and a hash table is used to manage the above divided grid data. Next, a three-layer octree structure will be used to perform adaptive plane feature extraction on the point cloud data in the voxel.

[0039] Specifically, plane feature initialization includes: when the point cloud data of the qth voxel grid clock exceeds a set threshold , the plane feature initialization process in the grid is started. First, the mean and covariance of the point cloud in the grid are calculated as follows: where N represents the number of points in the Qth grid, represents the kth point in the grid.

[0040] Next, the eigenvalues of the covariance matrix are obtained by eigenvalue decomposition, and the following plane determination criterion is introduced: where is a threshold, which is set to 0.01 m in the embodiment of the present application.

[0041] When the above plane determination criterion meets the requirements, it indicates that the point cloud in the voxel constitutes a plane. Otherwise, it indicates that the point cloud in the voxel does not belong to the same plane, then the point cloud data in the voxel is adaptively divided using a three-layer octree, and the plane determination is performed on the point cloud data in the octree child nodes through a loop. The loop termination conditions are as follows: (1) the number of point clouds in the child node is less than the threshold; (2) the point cloud in the child node meets the plane constraint; (3) the child node is already at the top of the octree.

[0042] ​​Specifically, the planar feature incremental update includes that when the initialization of the planar feature in the grid is completed, the point data added subsequently will be used to update the mean and covariance of the points in the grid, and the specific process is as follows: Specifically, in step S208, a homography matrix is calculated by using the projection points and the image feature points, and the RANSAC algorithm is used to remove the incorrect optical flow tracking points iteratively.

[0043] Figure 5 is a flow chart of a sliding window optimization method according to an embodiment of the present application. As shown in Figure 5 step S210 further includes the following steps: step S2101, state estimation results and image feature points in a period of time are selected to construct a sliding window, and the epipolar geometry constraint is used to recover the image feature points that fail to track in the sliding window; step S2102, the depth initial value of the image feature points is recovered by a triangulation method; step S2103, according to the depth initial value, a target function is constructed in the sliding window by a re-projection error, and the maximum a posteriori value of the depth value of the image feature points is solved by the Levenberg-Marquardt method.

[0044] Specifically, in order to ensure the accuracy of the depth estimation of the feature points in the sliding window optimization process, it is necessary to ensure that the feature points have sufficient parallax. However, in the case of fast system motion or repetitive texture scene, the feature points will face the problem of tracking failure, which leads to insufficient parallax of the feature points, and further affects the accuracy of the depth estimation of the feature points. In order to recover the tracking failed feature points, first, the rough pixel coordinates of the tracking failed feature points are recovered by the epipolar geometry constraint: wherein K represents the internal parameter of the camera, and represents the homogeneous coordinate form.

[0045] The above formula can be simplified as follows: wherein, By rearranging the above formula, the following can be obtained: wherein, .

[0046] Therefore, the following target function can be constructed: The above objective function can be solved by the Levenberg-Marquardt method.

[0047] Specifically, the method of triangulation is used to calculate the initial value of the depth of the feature point, and the image frame is selected as the reference frame in which the feature point is first observed , and then the disparity between and frames is calculated as follows: The pose transformation between the two image frames in the sliding window is and t, and the depth value of the feature point in the image frame can be expressed as: where and are the depth values of the feature point. By processing the formula, multiplying on the left of the equation on the left and right, we can get: According to the above formula, the initial value of the depth of the feature point can be obtained.

[0048] The sliding window optimization method provided by the embodiment of the application estimates the depth value of the feature point based on the initial depth value obtained by the above-mentioned triangulation. Specifically, in the sliding window, the first frame is set as the reference frame, and the depth value of the feature point in the frame is estimated. According to the re-projection error, the following error function can be constructed: At the same time, the following objective function can be constructed: The above objective function can be solved by the Levenberg-Marquardt method to obtain the maximum a posteriori probability estimate value.

[0049] When the disparity of the feature point in the newly added image frame in the sliding window exceeds the threshold value, the image frame will be added to the sliding window, and the first frame image in the sliding window will be removed from the sliding window. The depth value of the feature point to be estimated in the sliding window will be converted according to the following formula: With the continuous operation of the sliding window, the termination condition of the depth value of the feature point needs to be judged according to the following method: wherein is set to 0.1 m.​

[0050] Finally, in step S212: constructing hybrid reprojection error based on projected points and image feature points with recovered depth, the state is defined as follows: wherein is the image frame pose, is the velocity value of the image frame, is the zero offset value of the IMU.

[0051] In order to improve the navigation accuracy and stability in the scene of the decline of single sensor perception accuracy, the planar features extracted in the global map constructed by LIS and the image feature points with recovered depth are used to construct. First, the planar features and image features are projected into the image, and then the L-K optical flow tracking is used to track the projected point data. Since the image feature points have good luminosity gradient, the optical flow tracking result will be more accurate, while the luminosity gradient of the planar feature projected point may not be obvious, resulting in deviation of the optical flow tracking result. In order to reduce the abnormal results existing in the optical flow tracking result, the RANSAC algorithm is used to eliminate the abnormal values in the tracking result. Next, the state is updated by using the reprojection error , taking the kth feature point and the tth planar projected point as an example to introduce the construction method of the hybrid reprojection error, the following objective function can be obtained: wherein represents the state prior state estimation obtained by IMU integration, represents the noise covariance matrix of the prediction error, represents the covariance matrix of the feature point reprojection error, represents the reprojection covariance matrix of the planar projected point. In the formula, is the Jacobian matrix, and the specific calculation formula is as follows: According to the objective function, the following filtering update formula can be arranged: wherein, After the above filtering update equation, the mean value and the corresponding covariance of the VIS state estimation result can be obtained.

[0052] As can be seen from the above description, the application provides a multi-sensor fusion navigation system and method of a power inspection unmanned aerial vehicle and the unmanned aerial vehicle, a navigation architecture of fusion of a laser radar-inertial subsystem and a vision-inertial subsystem is constructed, and deep fusion of multi-source data is carried out in combination with an error state iterative Kalman filter, so that the problems of unstable positioning and decreased precision of a single sensor in a complex power scene are effectively overcome; through introduction of a self-adaptive plane feature extraction and projection mechanism, a hybrid feature point tracking strategy and a sliding window depth estimation method, efficient complementation of image texture information and point cloud geometric structure is realized, and the navigation robustness and continuity of the system in a severe environment such as strong light, occlusion and sparse texture are enhanced; finally, the mixed re-projection error is uniformly integrated into a state optimization framework, so that the positioning precision and system stability of the unmanned aerial vehicle in a power inspection task are greatly improved.

[0053] The application further provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method provided in the embodiments of the application when executing the computer program.

[0054] The application further provides a computer readable storage medium, which stores computer instructions, and the computer instructions are executed by the processor to implement the method provided in the embodiments of the application.

[0055] It is apparent for those skilled in the art that the application is not limited to the details of the above exemplary embodiments, and the application can be implemented in other specific forms without departing from the spirit or essential characteristics of the application. Therefore, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the application is defined by the appended claims rather than the above description, and all changes falling within the meaning and range of the equivalent elements of the claims are intended to be included in the application. Any reference signs in the claims should not be regarded as limiting the claims to which they relate.

[0056] In addition, it should be understood that, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can be properly combined to form other embodiments which can be understood by those skilled in the art.

Claims

1. A multi-sensor fusion navigation system for power inspection unmanned aerial vehicle, characterized in that, The method comprises the following steps: a visual unit, an inertial measurement unit, and a laser radar unit; the laser radar unit and the inertial measurement unit are coupled to form a laser radar-inertial subsystem, the visual unit and the inertial measurement unit are coupled to form a visual-inertial subsystem, and the laser radar-inertial subsystem and the visual-inertial subsystem constitute a fusion navigation architecture; the laser radar-inertial subsystem is used for: using pre-integration information of the inertial measurement unit to remove motion distortion in a laser radar point cloud, and calculating an iterative closest point error between a current frame and a global map; performing a filtering update process by taking the iterative closest point error as an observation value of an error state iterative Kalman filter, and constructing and updating the global map; the visual-inertial subsystem is used for: extracting plane features from the global map based on an adaptive incremental method, projecting the plane features to a projection point based on a visual image frame obtained by the visual unit, and extracting image feature points from the visual image frame; tracking the projection point and the image feature points between adjacent visual image frames by an optical flow method, and removing abnormal points in the tracking process; estimating depth values of the image feature points by a sliding window optimization method, and recovering image feature points that fail to be tracked in a sliding window by epipolar geometry constraints; constructing a hybrid reprojection error based on the projection point and the image feature points with recovered depth, and performing state estimation by taking the hybrid reprojection error as an observation value of an error state iterative Kalman filter.

2. A multi-sensor fusion navigation method for a power inspection unmanned aerial vehicle, characterized in that, The method is applied to the multi-sensor fusion navigation system of the power inspection unmanned aerial vehicle in claim 1, and the method comprises the following steps: using pre-integration information of the inertial measurement unit to remove motion distortion in a laser radar point cloud, and calculating an iterative closest point error between a current frame and a global map; performing a filtering update process by taking the iterative closest point error as an observation value of an error state iterative Kalman filter, and constructing and updating the global map; extracting plane features from the global map based on an adaptive incremental method, projecting the plane features to a projection point based on a visual image frame obtained by the visual unit, and extracting image feature points from the visual image frame; tracking the projection point and the image feature points between adjacent visual image frames by an optical flow method, and removing abnormal points in the tracking process; estimating depth values of the image feature points by a sliding window optimization method, and recovering image feature points that fail to be tracked in a sliding window by epipolar geometry constraints; constructing a hybrid reprojection error based on the projection point and the image feature points with recovered depth, and performing state estimation by taking the hybrid reprojection error as an observation value of an error state iterative Kalman filter.

3. The method of claim 2, wherein: The method comprises the following steps: acquiring current frame laser radar point cloud data, removing laser radar point cloud motion distortion by using pre-integration results of the inertial measurement unit, and obtaining non-distorted laser radar point cloud; The method comprises the following steps: traversing a current frame of distortion-free laser radar point cloud, solving a three-dimensional distance error of any point in the current frame to a historical map point of a global map under a pre-integration state prior condition of an inertial measurement unit, and accumulating and summing to obtain an iterative nearest point error.

4. The method of claim 2, wherein: The method comprises the following steps: Based on the pre-integration state of the inertial measurement unit, data in a preset range in the global map is selected as a local map, and the local map is rasterized to obtain a grid map. Each grid in the grid map is octree-divided to extract a plane feature; wherein a termination condition of the division is that a number of points in the grid is less than a preset threshold or the points in the grid satisfy a preset plane constraint condition.

5. The method of claim 2, wherein: A depth value of the image feature point is estimated by a sliding window optimization method, and a tracking-failed image feature point in the sliding window is recovered by an epipolar geometry constraint, comprising the following steps: State estimation results and image feature points in a period of time are selected to construct a sliding window, and a tracking-failed image feature point in the sliding window is recovered by an epipolar geometry constraint; A depth initial value of the image feature point is recovered by a triangulation method; According to the depth initial value, a target function is constructed in the sliding window by a re-projection error, and a maximum a posteriori value of the depth value of the image feature point is optimized and solved by a Levenberg-Marquardt method.

6. The method of claim 2 or 5, wherein: The image feature point comprises a Harris corner point.

7. The method of claim 2, wherein: Abnormal points in a tracking process are removed, comprising the following steps: a homography matrix is calculated by using the projection point and the image feature point, and an RANSAC algorithm is iterated to remove an error optical flow tracking point.

8. An electronic device, comprising: The method comprises the following steps: A memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to any one of claims 2-7 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are executed by the processor to implement the method according to any one of claims 2-7.

10. An electric power inspection unmanned aerial vehicle, characterized in that, The multi-sensor fusion navigation system of the power inspection unmanned aerial vehicle comprises the power inspection unmanned aerial vehicle according to claim 1.

Images