Monocular vision odometer method based on mine autonomous inspection robot

Abstract

The invention belongs to the technical field of mine automatic inspection, and relates to a monocular vision odometer method based on a mine autonomous inspection robot, which utilizes a prior dynamic object target detection model to detect and reject a dynamic object, and reduces the influence of the dynamic object in the pose estimation process; a depth feature extraction and matching network is utilized, so that the method has better robustness in a scene with weak texture and violent illumination change; a global dense point cloud map is obtained through monocular depth estimation; a scale recovery module is used for enabling the monocular visual odometer to obtain accurate scale information; according to the invention, the absolute trajectory error of the traditional visual odometer in a mine laneway scene can be reduced. According to the method, the feature points can be accurately matched in the mine laneway environment with weak wall texture, large visual angle change and violent illumination change, and the accuracy and robustness of positioning of the autonomous inspection robot are improved by eliminating the feature points of dynamic objects in an area with dense mine workers.

Description

Technical Field

[0001] The present invention relates to the technical field of mine automated inspection, and in particular to a monocular visual odometer method based on a mine autonomous inspection robot. Background Art

[0002] Smart mines are an important means to ensure the safe, efficient, economical and green development of the coal industry. In solving the safety problem of coal mine workers, it is particularly important for robots to conduct automated inspections in mines. Due to the poor GPS signal, complex terrain, and missing textures on the tunnel walls in the mine, the depth perception and positioning of robots in mines are affected. Depth perception and visual positioning are important components of the robot's autonomous inspection unmanned system. This technology allows the robot to perceive the surrounding environment and estimate the distance between the objects in the scene and itself, thereby achieving autonomous positioning and safe driving.

[0003] At present, most monocular visual odometry methods rely on feature point matching or optimizing pixel photometric errors to estimate the camera pose, thereby achieving robot depth perception and positioning. However, these methods do not work well in mine tunnel environments with weak wall textures, many dynamic objects, large viewing angle changes, and drastic lighting changes. In addition, monocular images lack scale information and cannot restore the absolute scale of the scene.

[0004] Therefore, using depth estimation, deep feature extraction and matching, and target detection technology to construct a monocular visual odometer method for an autonomous mine inspection robot is of great value in improving the depth perception and positioning accuracy of the autonomous mine inspection robot. Summary of the invention

[0005] In order to overcome the technical defects that the existing monocular visual odometer method has poor effect, the monocular image lacks scale information and cannot restore the absolute scale of the scene, the present invention provides a monocular visual odometer method based on a mine autonomous inspection robot.

[0006] The present invention provides a monocular visual odometer method based on a mine autonomous inspection robot, comprising the following steps:

[0007] Step 1, initialize the variables Cur_data, Ref_data, KeyPointList1, KeyPointList2, NewKeyPointList1, and NewKeyPointList2 to None respectively, and initialize the pose transformation matrix T to the unit matrix;

[0008] Step 2: The image data acquired by the camera of the autonomous inspection robot is used as Cur_data and input into the end-to-end model prediction network. The end-to-end model prediction network includes a depth estimation model, a priori dynamic object target detection model, and a feature point extraction and matching model. The depth estimation model is used to predict the dense depth map D of Cur_data, and the priori dynamic object target detection model is used to predict the priori dynamic object target detection map.

[0009] Step 3, determine whether Ref_data is None, if Ref_data is None, jump to step 11, if Ref_data is not None, continue to step 4;

[0010] Step 4, input the image frame data of Ref_data and Cur_data into the feature point extraction and matching model to obtain the matched feature point pairs KeyPointList1 and KeyPointList2; wherein the feature point extraction and matching model is pre-trained using the MSCOCO2014, Hpatch, and MegaDepth datasets;

[0011] Step 5, remove the feature point pairs in KeyPointList1 and KeyPointList2 on the prior dynamic object, that is, by traversing each feature point in the KeyPointList1 list, determine whether the feature point is within the two-dimensional bounding box coordinates of the prior dynamic object. If not, store the feature point in KeyPointList1 and the corresponding feature point in KeyPointList2 into NewKeyPointList1 and NewKeyPointList2 respectively;

[0012] Step 6: Calculate the camera's pose transformation matrix T using the epipolar geometry method. The sub-steps are:

[0013] Step 6.1, when the image feature extraction and matching obtains more than 10 feature point pairs, use the OPENCV library to calculate the function of the homography matrix H and the function of the basic matrix F, and use the Ransac method to calculate the homography matrix H and the basic matrix F between the two images of Ref_data and Cur_data;

[0014] Step 6.2, use the statistical model of the geometric robustness information criterion to calculate the scores of the homography matrix H and the basic matrix F, select the matrix with the lower score as the motion model and use the pose recovery function in the OPENCV library to estimate the pose transformation matrix T; the statistical model of the geometric robustness information criterion is as follows:

[0015]

[0016]

[0017] where n is the number of residuals to match, is the matching residual, is a robust function of the residual, is the standard deviation of the measurement error, r is the data dimension, k is the number of motion model parameters, and d is the dimension of the structure;

[0018] Step 7: Use the dense depth map D obtained in step 2 to restore the scale information of the camera pose transformation matrix;

[0019] Step 7.1, according to the new matching point pairs NewKeyPointList1 and NewKeyPointList2 obtained in step 5 and the camera pose transformation matrix T obtained in step 6, use the triangulation function in the OPENCV library to obtain the coordinates of the sparse map points in three-dimensional space, and then use the camera pose transformation matrix T between the two views of Ref_data and Cur_data to obtain the three-dimensional coordinates of the map points in different camera coordinate systems;

[0020] Step 7.2: Convert sparse map points to corresponding sparse depth maps It is aligned with the dense depth map D obtained by depth estimation, and then a mask matrix is ​​constructed to filter out areas with depth values ​​less than 0;

[0021] Step 7.3: Calculate sparse depth map The ratio relationship between the image and the dense depth map D is obtained, and the optimized depth value ratio information Scale is obtained by removing outliers using the Ransac algorithm. Then, the translation vector t in the pose transformation matrix T is multiplied by Scale to obtain the absolute pose information, thereby restoring the scale of the pose transformation matrix T.

[0022] Step 8. If the epipolar geometry method fails to solve the pose transformation matrix T in step 6, use the PNP method to estimate the camera pose transformation matrix T; use the feature points and depth estimation of NewKeyPointList1 to obtain the dense depth map D data of Ref_data, and use the camera intrinsic parameter matrix K to perform back projection to obtain the 3D points in the camera coordinate system; use the feature points and 3D points in NewKeyPointList2 to use the PNP method solution function in the OPENCV library to calculate the pose transformation matrix T;

[0023] Step 9: After camera tracking is completed, the global pose is updated;

[0024] Step 10: Use the OPENCV library to visualize the robot's movement trajectory map, feature point matching map, predicted dense depth map D, and prior dynamic object target detection map, and use the OPEN3D library to convert the single-frame depth map into a pseudo-lidar point cloud, and construct a point cloud map based on the global pose;

[0025] Step 11, update the data related to Ref_data, and update the dense depth map D, matching point pairs NewKeyPointList1 and NewKeyPointList2, and prior dynamic object target detection map related to Cur_data to the dense depth map D, matching point pairs NewKeyPointList1 and NewKeyPointList2, and prior dynamic object target detection map related to Ref_data;

[0026] Step 12: Determine whether the image data has been read. If the data has not been read, return to step 2. Otherwise, the system ends and a three-dimensional point cloud map of the scene is obtained using the global position of the camera.

[0027] Preferably, in step 2, the depth estimation network uses DepthAnythingv2, and the depth estimation model with absolute depth information is obtained after fine-tuning and training DepthAnythingv2 using the pre-collected mine tunnel dataset; the prior dynamic object target detection network uses YOLOv8, and the prior dynamic object target detection model is obtained after pre-training YOLOv8 using the COCO dataset; the feature point extraction and matching network uses the Superpoint algorithm and the LightGlue algorithm, and the feature point extraction and matching model is obtained after pre-training using the Superpoint algorithm and the LightGlue algorithm of the MSCOCO2014, Hpatch, MegaDepth datasets; the sub-steps of step 2 are:

[0028] Step 2.1: Use the depth estimation model to predict the depth of the current image frame data and obtain the initial depth map of a single channel. After that, the depth validity mask DepthMask is used together with the initial depth map Multiply, retain the depth value within the valid depth range, and the dense depth map after mask processing is D; the depth validity mask DepthMask is the product of the depth region mask RegionMask and the depth value range mask RangeMask; the size of the depth region mask RegionMask is the same as the initial depth map The size of the depth region mask RegionMask is the same as that of the depth region mask RegionMask in the width range [0,W] and the height range [0.3 H,H] is 1 in the region, otherwise it is 0; the size of the depth value range mask RangeMask is the same as the initial depth map The same size as the initial depth map If the depth value is between MinDepth and MaxDepth, the corresponding position of the depth value range mask RangeMask is 1, otherwise it is 0;

[0029] Step 2.2: Use the prior dynamic object target detection model to perform target detection on Cur_data and obtain a size of 4 A coordinate set of N prior dynamic objects, where the two-dimensional bounding box coordinates are composed of (X1, Y1, X2, Y2), and there are N prior dynamic objects in total.

[0030] Preferably, in step 6.2, the number of matched residuals n is the number of elements in the NewKeyPointList1 list, is 0.85, r is 2; for the basic matrix F, k is 7, d is 3; for the homography matrix H, k is 8, d is 2; the parameter value , , .

[0031] Compared with the prior art, the technical solution provided by the present invention has the following technical effects: the present invention uses a priori dynamic object target detection model to detect and eliminate dynamic objects, thereby reducing the influence of dynamic objects in the process of posture estimation; uses deep feature extraction and matching networks to have better robustness in scenes with weak textures and drastic changes in illumination; uses monocular depth estimation to obtain a global dense point cloud map; uses a scale recovery module to enable the monocular visual odometer to obtain accurate scale information; the present invention can reduce the absolute trajectory error of traditional visual odometer methods and deep learning-based visual odometers in mine tunnel scenes; the present invention can accurately match feature points in mine tunnel environments with weak wall textures, large changes in viewing angles, and drastic changes in illumination. In areas where mine workers are densely populated, the accuracy and robustness of the positioning of autonomous inspection robots are improved by excluding dynamic object feature points. In addition, by restoring the scale information of the scene and the three-dimensional reconstruction of the mine tunnel environment, the robot's autonomous navigation capability in a complex mine environment is further enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 It is a flow chart of a monocular visual odometer method based on a mine autonomous inspection robot described in an embodiment of the present invention;

[0035] Figure 2 The system structure diagram is obtained by a monocular visual odometer method based on a mine autonomous inspection robot described in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the scheme of the present invention will be further described below. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present invention, rather than all of the embodiments.

[0038] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0039] In one embodiment, Figure 1As shown in the figure, a monocular visual odometer method based on an autonomous mine inspection robot is implemented based on a visual odometer system, wherein the visual odometer system includes an end-to-end model prediction module, a tracking module, a scale recovery module, and a global point cloud dense reconstruction module; the end-to-end model prediction network is used to predict each frame of the image, and the end-to-end model prediction network includes a depth estimation model, a priori dynamic object target detection model, and a feature point extraction and matching model; the depth estimation network uses DepthAnythingv2, and DepthAnythingv2 is fine-tuned and trained using a pre-collected mine tunnel dataset to obtain a depth estimation model with absolute depth information; the priori dynamic object target detection network uses YOLOv8, and YOLOv8 is pre-trained using the COCO dataset to obtain a priori dynamic object target detection model; the feature point extraction and matching network uses Superpoint and LightGlue, and Superpoint and LightGlue are pre-trained using the MSCOCO2014, Hpatch, and MegaDepth datasets to obtain a feature point extraction and matching model. Matching model; storing the data obtained by the end-to-end model prediction module in the associated data of the current image frame data Cur_data for standby use; the function of the tracking module is to track the feature points of the two image frame data, so as to calculate the pose transformation matrix T between the multiple views. Specifically, the multi-view feature point matching results obtained by the feature point extraction and matching network are detected by a priori dynamic objects to remove dynamic feature points, and then the pose is estimated by the epipolar geometry method. When the epipolar geometry method fails, the depth map obtained by the deep model inference module and the matching points after removing the dynamic feature points are used to estimate the pose by the PNP method; the role of the scale recovery module is to solve the problem of scale uncertainty in the pose estimation of the monocular camera. Specifically, the depth map obtained by matching and sparsely triangulating the feature points is aligned with the depth map obtained by the deep model inference module, and the scale information of the camera pose estimation is determined by calculating the proportional relationship between the corresponding pixels; the role of the global point cloud dense reconstruction module is to convert the monocular depth map into a pseudo-lidar point cloud using the camera internal parameters, and then transform the point cloud coordinates according to the camera pose to realize the construction of a global dense point cloud map;

[0040] Specifically, the monocular visual odometer method of the present invention comprises the following steps:

[0041] Step 1, initialize the variables Cur_data, Ref_data, KeyPointList1, KeyPointList2, NewKeyPointList1, and NewKeyPointList2 to None respectively, and initialize the pose transformation matrix T to the unit matrix;

[0042] Step 2: The image data acquired by the camera of the autonomous inspection robot is used as Cur_data and input into the end-to-end model prediction network. The end-to-end model prediction network includes a depth estimation model, a priori dynamic object target detection model, and a feature point extraction and matching model. The depth estimation model is used to predict the dense depth map D of Cur_data, and the priori dynamic object target detection model is used to predict the priori dynamic object target detection map. The sub-steps of step 2 are:

[0043] Step 2.1: Use the depth estimation model to predict the depth of the current image frame data and obtain the initial depth map of a single channel. After that, the depth validity mask DepthMask is used together with the initial depth map Multiply, retain the depth value within the valid depth range, and the dense depth map after mask processing is D; the depth validity mask DepthMask is the product of the depth region mask RegionMask and the depth value range mask RangeMask; the size of the depth region mask RegionMask is the same as the initial depth map The size of the depth region mask RegionMask is the same as that of the depth region mask RegionMask in the width range [0,W] and the height range [0.3 H,H] is 1 in the region, otherwise it is 0; the size of the depth value range mask RangeMask is the same as the initial depth map The same size as the initial depth map If the depth value is between MinDepth and MaxDepth, the corresponding position of the depth value range mask RangeMask is 1, otherwise it is 0;

[0044] Step 2.2: Use the prior dynamic object target detection model to perform target detection on Cur_data and obtain a size of 4 A set of coordinates of N prior dynamic objects, where the 2D bounding box coordinates are composed of (X1, Y1, X2, Y2), and there are N prior dynamic objects in total;

[0045] Step 3, determine whether Ref_data is None, if Ref_data is None, jump to step 11, if Ref_data is not None, continue to step 4;

[0046] Step 4, input the image frame data of Ref_data and Cur_data into the feature point extraction and matching model to obtain the matched feature point pairs KeyPointList1 and KeyPointList2; wherein the feature point extraction and matching model is pre-trained using the MSCOCO2014, Hpatch, and MegaDepth datasets;

[0047] Step 5, remove the feature point pairs in KeyPointList1 and KeyPointList2 on the prior dynamic object, that is, by traversing each feature point in the KeyPointList1 list, determine whether the feature point is within the two-dimensional bounding box coordinates of the prior dynamic object. If not, store the feature point in KeyPointList1 and the corresponding feature point in KeyPointList2 into NewKeyPointList1 and NewKeyPointList2 respectively;

[0048] Step 6: Calculate the camera's pose transformation matrix T using the epipolar geometry method. The sub-steps are:

[0049] Step 6.1, when the image feature extraction and matching obtains more than 10 feature point pairs, use the function of calculating the homography matrix H and the function of the basic matrix F in the OPENCV library; use the Ransac method to calculate the homography matrix H and the basic matrix F between the two images of Ref_data and Cur_data;

[0050] Step 6.2, use the statistical model of the geometric robustness information criterion to calculate the scores of the homography matrix H and the basic matrix F, select the matrix with the lower score as the motion model and use the pose recovery function in the OPENCV library to estimate the pose transformation matrix T; the statistical model of the geometric robustness information criterion is as follows:

[0051]

[0052]

[0053] where n is the number of residuals to match, is the matching residual, is a robust function of the residual, is the standard deviation of the measurement error, r is the data dimension, k is the number of motion model parameters, and d is the dimension of the structure; the number of matched residuals n is the number of elements in the NewKeyPointList1 list, is 0.85, r is 2; for the basic matrix F, k is 7, d is 3; for the homography matrix H, k is 8, d is 2; the parameter value , , ;

[0054] Step 7: Use the dense depth map D obtained in step 2 to restore the scale information of the camera pose transformation matrix;

[0055] Step 7.1, according to the new matching point pairs NewKeyPointList1 and NewKeyPointList2 obtained in step 5 and the camera pose transformation matrix T obtained in step 6, use the triangulation function in the OPENCV library to obtain the coordinates of the sparse map points in three-dimensional space, and then use the camera pose transformation matrix T between the two views of Ref_data and Cur_data to obtain the three-dimensional coordinates of the map points in different camera coordinate systems;

[0056] Step 7.2: Convert sparse map points to corresponding sparse depth maps It is aligned with the dense depth map D obtained by depth estimation, and then a mask matrix is ​​constructed to filter out areas with depth values ​​less than 0;

[0057] Step 7.3: Calculate sparse depth map The ratio relationship between the image and the dense depth map D is obtained, and the optimized depth value ratio information Scale is obtained by removing outliers using the Ransac algorithm. Then, the translation vector t in the pose transformation matrix T is multiplied by Scale to obtain the absolute pose information, thereby restoring the scale of the pose transformation matrix T.

[0058] Step 8. If the epipolar geometry method fails to solve the pose transformation matrix T in step 6, use the PNP method to estimate the camera pose transformation matrix T; use the feature points and depth estimation of NewKeyPointList1 to obtain the dense depth map D data of Ref_data, and use the camera intrinsic parameter matrix K to perform back projection to obtain the 3D points in the camera coordinate system; use the feature points and 3D points in NewKeyPointList2 to use the PNP method solution function in the OPENCV library to calculate the pose transformation matrix T;

[0059] Step 9: After camera tracking is completed, the global pose is updated;

[0060] Step 10: Use the OPENCV library to visualize the robot's movement trajectory map, feature point matching map, predicted dense depth map D, and prior dynamic object target detection map, and use the OPEN3D library to convert the single-frame depth map into a pseudo-lidar point cloud, and construct a point cloud map based on the global pose;

[0061] Step 11, update the data related to Ref_data, and update the dense depth map D, matching point pairs NewKeyPointList1 and NewKeyPointList2, and prior dynamic object target detection map related to Cur_data to the dense depth map D, matching point pairs NewKeyPointList1 and NewKeyPointList2, and prior dynamic object target detection map related to Ref_data;

[0062] Step 12: Determine whether the image data has been read. If the data has not been read, return to step 2. Otherwise, the system ends and a three-dimensional point cloud map of the scene is obtained using the global position of the camera.

[0063] In the method described in the present invention, first, the robot collects image data in the mine tunnel through the monocular camera carried by it. Secondly, the DepthAnythingV2 algorithm is used to perform monocular depth estimation, and the image feature points are extracted by the Super Point algorithm, and the Light Glue algorithm matches the feature points of adjacent frame images. Then, the YOLOV8 algorithm is used to identify and remove the feature points of dynamic objects; then the epipolar geometry constraints and PNP algorithm in computer vision are used to solve the pose. In this process, the depth map obtained by depth estimation is aligned with the depth map of triangulation to restore the scale information of the scene and pose. Finally, the depth map predicted for each frame is converted into a pseudo-lidar point cloud, and the dense reconstruction of the three-dimensional point cloud map of the scene is achieved using the global pose of the camera.

[0064] The above is only a specific implementation of the present invention, which enables those skilled in the art to understand or implement the present invention. Although detailed descriptions are given with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments, and they should all be covered by the protection scope of the claims.

Claims

1. A monocular visual odometer method based on an autonomous mine inspection robot, characterized in that: The steps include: Step 1, initialize the variables Cur_data, Ref_data, KeyPointList1, KeyPointList2, NewKeyPointList1, and NewKeyPointList2 to None respectively, and initialize the pose transformation matrix T to the unit matrix; Step 2: The image data acquired by the camera of the autonomous inspection robot is used as Cur_data and input into the end-to-end model prediction network. The end-to-end model prediction network includes a depth estimation model, a priori dynamic object target detection model, and a feature point extraction and matching model. The depth estimation model is used to predict the dense depth map D of Cur_data, and the priori dynamic object target detection model is used to predict the priori dynamic object target detection map. Step 3, determine whether Ref_data is None, if Ref_data is None, jump to step 11, if Ref_data is not None, continue to step 4; Step 4, input the image frame data of Ref_data and Cur_data into the feature point extraction and matching model to obtain the matched feature point pairs KeyPointList1 and KeyPointList2; wherein the feature point extraction and matching model is pre-trained using the MSCOCO2014, Hpatch, and MegaDepth datasets; Step 5, remove the feature point pairs in KeyPointList1 and KeyPointList2 on the prior dynamic object, that is, by traversing each feature point in the KeyPointList1 list, determine whether the feature point is within the two-dimensional bounding box coordinates of the prior dynamic object. If not, store the feature point in KeyPointList1 and the corresponding feature point in KeyPointList2 into NewKeyPointList1 and NewKeyPointList2 respectively; Step 6: Calculate the camera's pose transformation matrix T using the epipolar geometry method. The sub-steps are: Step 6.1, when the image feature extraction and matching obtains more than 10 feature point pairs, use the function of calculating the homography matrix H and the function of calculating the basic matrix F in the OPENCV library, and use the Ransac method to calculate the homography matrix H and the basic matrix F between the two images of Ref_data and Cur_data; Step 6.2, use the statistical model of the geometric robustness information criterion to calculate the scores of the homography matrix H and the basic matrix F, select the matrix with the lower score as the motion model and use the pose recovery function in the OPENCV library to estimate the pose transformation matrix T; the statistical model of the geometric robustness information criterion is as follows: ， ， where n is the number of residuals to match, is the matching residual, is a robust function of the residual, is the standard deviation of the measurement error, r is the data dimension, k is the number of motion model parameters, and d is the dimension of the structure; Step 7: Use the dense depth map D obtained in step 2 to restore the scale information of the camera pose transformation matrix; Step 7.1, according to the new matching point pairs NewKeyPointList1 and NewKeyPointList2 obtained in step 5 and the camera pose transformation matrix T obtained in step 6, use the triangulation function in the OPENCV library to obtain the coordinates of the sparse map points in three-dimensional space, and then use the camera pose transformation matrix T between the two views of Ref_data and Cur_data to obtain the three-dimensional coordinates of the map points in different camera coordinate systems; Step 7.2: Convert sparse map points to corresponding sparse depth maps It is aligned with the dense depth map D obtained by depth estimation, and then a mask matrix is ​​constructed to filter out areas with depth values ​​less than 0; Step 7.3: Calculate sparse depth map The ratio relationship between the image and the dense depth map D is obtained, and the optimized depth value ratio information Scale is obtained by removing outliers using the Ransac algorithm. Then, the translation vector t in the pose transformation matrix T is multiplied by Scale to obtain the absolute pose information, thereby restoring the scale of the pose transformation matrix T. Step 8. If the epipolar geometry method fails to solve the pose transformation matrix T in step 6, use the PNP method to estimate the camera pose transformation matrix T; use the feature points and depth estimation of NewKeyPointList1 to obtain the dense depth map D data of Ref_data, and use the camera intrinsic parameter matrix K to perform back projection to obtain the 3D points in the camera coordinate system; use the feature points and 3D points in NewKeyPointList2 to use the PNP method solution function in the OPENCV library to calculate the pose transformation matrix T; Step 9: After camera tracking is completed, the global pose is updated; Step 10: Use the OPENCV library to visualize the robot's movement trajectory map, feature point matching map, predicted dense depth map D, and prior dynamic object target detection map, and use the OPEN3D library to convert the single-frame depth map into a pseudo-lidar point cloud, and construct a point cloud map based on the global pose; Step 11, update the data related to Ref_data, and update the dense depth map D, matching point pairs NewKeyPointList1 and NewKeyPointList2, and prior dynamic object target detection map related to Cur_data to the dense depth map D, matching point pairs NewKeyPointList1 and NewKeyPointList2, and prior dynamic object target detection map related to Ref_data; Step 12: Determine whether the image data has been read. If the data has not been read, return to step 2. Otherwise, the system ends and a three-dimensional point cloud map of the scene is obtained using the global position of the camera.

2. According to claim 1, a monocular visual odometer method based on a mine autonomous inspection robot is characterized in that: In step 2, the depth estimation network uses DepthAnythingv2, and the depth estimation model with absolute depth information is obtained by fine-tuning DepthAnythingv2 using the pre-collected mine tunnel dataset; the prior dynamic object target detection network uses YOLOv8, and the prior dynamic object target detection model is obtained by pre-training YOLOv8 using the COCO dataset; the feature point extraction and matching network uses the Superpoint algorithm and the LightGlue algorithm, and the feature point extraction and matching model is obtained by pre-training the Superpoint algorithm and the LightGlue algorithm using the MSCOCO2014, Hpatch, and MegaDepth datasets; The sub-steps of step 2 are: Step 2.1: Use the depth estimation model to predict the depth of the current image frame data and obtain the initial depth map of a single channel. After that, the depth validity mask DepthMask is used together with the initial depth map Multiply, retain the depth value within the valid depth range, and the dense depth map after mask processing is D; the depth validity mask DepthMask is the product of the depth region mask RegionMask and the depth value range mask RangeMask; the size of the depth region mask RegionMask is the same as the initial depth map The size of the depth region mask RegionMask is the same as that of the depth region mask RegionMask in the width range [0,W] and the height range [0.3 H,H] is 1 in the region, otherwise it is 0; the size of the depth value range mask RangeMask is the same as the initial depth map The same size as the initial depth map If the depth value is between MinDepth and MaxDepth, the corresponding position of the depth value range mask RangeMask is 1, otherwise it is 0; Step 2.2: Use the prior dynamic object target detection model to perform target detection on Cur_data and obtain a size of 4 A coordinate set of N prior dynamic objects, where the two-dimensional bounding box coordinates are composed of (X1, Y1, X2, Y2), and there are N prior dynamic objects in total.

3. The monocular visual odometer method based on a mine autonomous inspection robot according to claim 2 is characterized in that: In step 6.2, the number of matched residuals n is the number of elements in the NewKeyPointList1 list. is 0.85, r is 2; for the basic matrix F, k is 7, d is 3; for the homography matrix H, k is 8, d is 2; the parameter value , , .

Images