High-precision map reconstruction method and system based on monocular vision, medium and equipment

Abstract

The invention belongs to the technical field of robot positioning and three-dimensional mapping, and discloses a high-precision map reconstruction method and system based on monocular vision, a medium and equipment. Scene image data are acquired through a monocular image acquisition module, after feature extraction and matching are performed on each frame of image, attitude information acquired by an inertial measurement unit is fused, and pose calculation of a robot is completed through a sparse vision SLAM system; meanwhile, an image dense depth map is generated by a monocular depth estimation model based on an attention mechanism, and the image dense depth map is converted into a single-frame color dense point cloud in combination with image RGB color information. According to the invention, a loose coupling fusion strategy is adopted to carry out spatial registration on a single-frame colored dense point cloud and a robot pose at a corresponding moment, multi-frame data fusion is completed through point cloud splicing and optimization, and a globally consistent three-dimensional dense point cloud map is constructed to realize scene modeling. The method has the characteristics of high robustness and high reconstruction precision, and can be effectively applied to three-dimensional map construction in an outdoor complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot positioning and mapping, and particularly relates to a high-precision map reconstruction method and system based on monocular vision, a medium and equipment. BACKGROUND

[0002] The monocular visual inertial system (VIO) realizes estimation of the motion state of a camera by fusing image information from a monocular camera and acceleration and angular velocity information provided by an inertial measurement unit (IMU), and makes up for the defects of traditional monocular SLAM systems in feature extraction, scale uncertainty and poor real-time performance. Compared with binocular or laser SLAM systems, the monocular VIO has good real-time performance and robustness under the premise of simpler system hardware configuration and stronger adaptability, and has a wide application prospect, especially in mobile robots, autonomous unmanned systems and complex environment navigation.

[0003] However, the existing VIO system still faces many challenges in typical scenes such as complex, dynamic, weak texture, etc., such as incomplete depth information acquisition, motion estimation error accumulation, dynamic target interference and other problems, which limit its further application in large-scale and high-precision mapping tasks. In order to overcome these problems, in recent years, researchers have introduced deep learning technology to automatically extract and model deep semantic features in images using convolutional neural networks (CNN), attention mechanisms (Attention Mechanism) and Transformer architectures, etc., to improve the system's ability in depth estimation, feature matching and global consistent mapping. In particular, in the field of monocular depth estimation, deep learning has realized the direct prediction of dense depth maps from images, which, combined with the accurate pose information of the VIO system, is expected to generate a dense point cloud map with complete structure and rich details.

[0004] However, the current learning-based depth estimation method still faces problems such as large estimation error in occluded areas, serious dynamic object interference and strong dependence on training data, resulting in insufficient spatial continuity and geometric consistency of the generated point cloud map. In addition, due to the strong coupling relationship between depth estimation and motion estimation, optimizing any module alone often fails to significantly improve the overall system performance. Therefore, the collaborative optimization mechanism that integrates visual, inertial and deep learning information has become the key to improving the overall performance of the system.

[0005] Dense point cloud maps, as a core achievement of 3D reconstruction, are not only significant in academic research but also play a crucial role in fields such as autonomous driving, augmented reality (AR), virtual reality (VR), drone navigation, digital twins, and smart cities. High-quality dense point cloud maps require high spatial resolution, high geometric accuracy, and strong global consistency while ensuring real-time performance. Therefore, researching a dense point cloud construction method that integrates VIO (Virtual IoV) and deep learning for outdoor scenes not only helps improve the overall capabilities of 3D perception systems but also has significant theoretical and engineering value for promoting the practical application of intelligent perception technologies. Summary of the Invention

[0006] To address the aforementioned problems, the purpose of this invention is to provide a high-precision map reconstruction method, system, medium, and device based on monocular vision, which features good robustness and high reconstruction accuracy, and can be used for reconstruction in complex outdoor environments.

[0007] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: a high-precision map reconstruction method based on monocular vision, comprising: acquiring image data through a monocular image acquisition device, performing feature extraction and matching on each frame of image, and combining the pose information acquired by the IMU with the pose estimation using a sparse vision SLAM system to obtain the robot's pose information; performing monocular depth estimation on the current frame image using a depth estimation model based on an attention mechanism, and combining the RGB color information in the original image to obtain a single-frame color dense point cloud; using a loosely coupled fusion strategy to spatially register the single-frame color dense point cloud with the robot pose at the corresponding time, and completing the fusion of multiple frames of data through point cloud stitching and optimization to construct a globally consistent three-dimensional dense point cloud map, so as to realize the three-dimensional modeling of complex scenes.

[0008] Furthermore, feature extraction employs the ORB feature point extraction method, including: In the feature detection stage, the FAST corner detection algorithm is used to detect each pixel in the image. p Detection by pixel p Centered on, with radius r Within the circular neighborhood of a pixel m One sampling point; If there are consecutive [elements] in the neighborhood. n The grayscale values ​​of each sampling point are all higher than or below If the feature is a corner point, then that point is identified as a corner feature and used as a candidate feature point for subsequent matching; where, For pixels p grayscale value, h The set grayscale threshold.

[0009] Further, in feature matching, after matching all feature points, the RANSAC algorithm is used to remove false matches; specifically, the following process is included: In the two images to be registered, corner feature points are extracted, and their corresponding binary descriptors are calculated; in matching, the Hamming distance between the descriptors is calculated to quantify the similarity; When the number of feature points is less than the set number, brute force matching is used to compare each feature point with all feature points in the target image; when the number of feature points is greater than the set number, a search structure is constructed for fast retrieval; In the matching stage, the RANSAC algorithm is used to estimate the fundamental matrix or homography matrix between the two images to filter out false matching pairs caused by changes in illumination, motion blur, or repeated textures, ensuring that only reliable matching pairs that meet geometric consistency are retained.

[0010] Further, sparse visual SLAM system is used for pose estimation, and EPnP method based on RANSAC framework is used for pose solving, including: After obtaining the initial matching pairs, enter the robust solving process; First, randomly select 4 non-coplanar point pairs from all matching pairs as the minimum sample set; using the sample set, an initial camera pose hypothesis is directly calculated by the EPnP algorithm; Then, use the hypothesis pose to project all associated 3D map points to the current image plane, calculate the re-projection error between them and the corresponding 2D feature points, and determine the matching pairs with re-projection error less than the preset threshold as inliers, otherwise as outliers, and record the inlier set obtained in this iteration and its number; Repeat the above random sampling, EPnP solving and inlier discrimination process for multiple iterations, and after the iteration ends, select the model with the most inliers from all generated pose hypotheses as the optimal model; Finally, use all inliers corresponding to the optimal model to perform an accurate solution by the EPnP algorithm, and output the final optimal pose estimation matrix of the current frame relative to the world coordinate system T cw .

[0011] Further, the attention mechanism-based depth estimation model is used for monocular depth estimation of the current frame image, including: The single image is input into the attention mechanism-based depth estimation model, and the corresponding depth map is output; In the training process, a geometric invariance loss function is introduced, which constrains the consistency of features extracted after multiple random geometric transformations of the input image; The geometric invariance loss function is:

[0012] wherein, D i |E i denotes a camera prompt E i modulated depth features, i = 1, 2; denotes a stop gradient operation, for blocking the back propagation of the pseudo-supervised signal, is a feature consistency loss function.

[0013] Further, combined with the RGB color information in the original image, a single-frame color dense point cloud is obtained, including: The preset depth validity range is According to the depth validity range, the estimated depth map is filtered to retain the depth map falling within the depth validity range; The depth map is combined with the RGB color information of the original image, and each pixel is projected as a three-dimensional point with color; The three-dimensional point with color is subjected to radius filtering and statistical outlier removal, to remove outliers or noise points, and a dense color point cloud map with completeness and consistency is generated.

[0014] Further, a loose coupling fusion strategy is adopted to spatially register the single-frame color dense point cloud and the robot pose at the corresponding moment, to complete multi-frame data fusion through point cloud stitching and optimization, and to construct a globally consistent three-dimensional dense point cloud map, including: The accurate pose information of the current image frame output by the sparse visual SLAM system is obtained, and the corresponding depth image and RGB image at the same moment are obtained; then, according to the camera intrinsic parameter, the pixel color information in the RGB image is combined with the geometric information in the depth image to generate a color dense point cloud of the current frame; the single-frame point cloud is subjected to down-sampling and statistical filtering processing to remove noise and redundant points; The filtered single-frame color dense point cloud is transformed from the camera coordinate system to the global world coordinate system using the pose information of the current frame, to form a local map block; through the pose transformation between consecutive frames, the local map blocks obtained at different moments are sequentially registered and spliced, to realize incremental fusion of the point cloud, and gradually construct a global dense point cloud map covering the entire scene; During the mapping process, the map is optimized according to the loop detection result of the sparse visual SLAM system; when a loop is detected, the optimized key frame pose graph is obtained, and the spatial position of the corresponding point cloud block in the constructed global dense point cloud is adjusted according to the optimized pose, to realize global consistency optimization of the map, eliminate cumulative errors, and finally output a high-precision color three-dimensional dense point cloud map.

[0015] In a second aspect, the present application adopts the technical solution of a high-precision map reconstruction system based on monocular vision, which comprises: a feature point extraction and pose estimation module, which acquires image data through a monocular image acquisition device, extracts and matches features for each frame of image, combines with the attitude information acquired by an IMU, uses a sparse visual SLAM system to estimate pose, and obtains the pose information of the robot; a depth information extraction module, which uses a depth estimation model based on an attention mechanism to monocularly estimate the depth of the current frame of image, combines with the RGB color information in the original image, and obtains a single-frame color dense point cloud; and a three-dimensional dense map reconstruction module, which uses a loosely coupled fusion strategy to spatially register the single-frame color dense point cloud and the pose of the robot at the corresponding moment, fuses multiple frames of data through point cloud splicing and optimization, constructs a globally consistent three-dimensional dense point cloud map, and realizes three-dimensional modeling of a complex scene.

[0016] In a third aspect, the present application adopts the technical solution of a computer readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the above methods.

[0017] In a fourth aspect, the present application adopts the technical solution of a computing device, which comprises one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the above methods.

[0018] The present application has the following advantages due to the adoption of the above technical solutions: 1. The present application uses a monocular vision inertial system to perform dense three-dimensional reconstruction, which can avoid the difficulties of RGBD cameras in strong light and outdoor large depth scenes and the shortcomings of monocular cameras that cannot build a map when stationary. The monocular depth estimation method based on the attention mechanism directly acquires depth information through the input of monocular images. This image content-based method can obtain rich spatial structure information, and thus can effectively obtain spatial depth information indoors and outdoors.

[0019] 2. The present application uses a separate depth estimation method to infer picture depth information. Depth prediction takes a single input image as input and directly generates corresponding three-dimensional depth output. To improve the consistency and robustness of the estimation results, a geometric invariance loss is set. This loss imposes various geometric transformations (such as random rotation, scaling, and translation) on the input image, and establishes consistency constraints between the feature representations after different transformations, to promote the depth features to have better geometric invariance.

[0020] In conclusion, the present application can construct a dense three-dimensional map, and has certain practical value and research value in the fields of unmanned system navigation and unknown environment map construction. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a high-precision map reconstruction method based on monocular vision in the embodiment of the present application, which is a whole flowchart; Figure 2 is a detailed flowchart of the high-precision map reconstruction method based on monocular vision in the embodiment of the present application; Figure 3 is a FAST feature point extraction schematic diagram in the embodiment of the present application; Figure 4 is a pose estimation scene example diagram in the embodiment of the present application; Figure 5 is a monocular depth estimation network architecture diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0022] In view of the problems that the point cloud map generated by the prior art still has deficiencies in spatial continuity and geometric consistency, and it is difficult to achieve significant improvement in the overall system performance, the present application provides a high-precision map reconstruction method, system, medium and equipment based on monocular vision, which comprises the following steps: collecting visual images and inertial measurement unit (IMU) data, training a feature point detector and a descriptor by constructing a self-supervised learning framework, improving the robustness and accuracy of visual and inertial information fusion, and realizing visual-inertial pose optimization; introducing a cross-attention mechanism to design a depth estimation algorithm, fully fusing image region features and context information to obtain a high-precision depth map; combining the pose information provided by the visual-inertial SLAM system and the depth estimation result, using a loose coupling strategy to construct a dense map, and realizing three-dimensional modeling of complex scenes. The present application has the characteristics of high precision, strong robustness and good real-time performance, and can be widely applied in the fields of augmented reality, intelligent navigation, autonomous driving and robot three-dimensional perception.

[0023] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0024] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0025] In one embodiment of the present application, a high-precision map reconstruction method based on monocular vision is provided, specifically a dense point cloud construction method and system based on the fusion of monocular vision and inertial odometer, which realizes online positioning, motion pose estimation and three-dimensional map reconstruction in various environments where GPS is difficult to work. In the present embodiment, as shown in Figure 1 、 Figure 2 the method comprises the following steps: 1) Obtain image data through a monocular image acquisition device, extract and match features for each frame of image, and combine the pose information obtained by the IMU to perform pose estimation by using a sparse visual SLAM system to obtain the pose information of the robot. In the present embodiment, the monocular image acquisition device can be a monocular camera.

[0026] Wherein, the pose represents the position and the attitude, and any rigid body can be accurately and uniquely represented by the position and the attitude in the space coordinate system (OXYZ).

[0027] 2) Perform monocular depth estimation on the current frame of image based on a depth estimation model of attention mechanism, combine the RGB color information in the original image to obtain a single-frame color dense point cloud.

[0028] 3) Adopt a loosely coupled fusion strategy to spatially register the single-frame color dense point cloud with the pose of the robot at the corresponding moment, complete multi-frame data fusion through point cloud splicing and optimization, construct a globally consistent three-dimensional dense point cloud map, and realize three-dimensional modeling of a complex scene.

[0029] The present application directly generates the corresponding three-dimensional depth output by taking the data of the monocular camera sensor as the input, generates the color depth point cloud in combination with the image obtained by the camera, and performs three-dimensional dense map reconstruction in combination with the pose provided by the visual SLAM algorithm of the ORB-SLMA3, and this loosely coupled mode can realize flexible key frame selection.

[0030] In step 1) above, this embodiment uses the ORB-SLAM3 algorithm for a real-time SLAM library for a monocular camera, calculating the camera trajectory and performing loop closure detection and real-time camera repositioning. The main task of ORB-based front-end tracking visual odometry is to utilize data perceived by the camera in an unknown environment, extract and match feature points from the images obtained by the camera, and thus estimate the camera's motion trajectory through the relationships between image frame sequences. Therefore, one of the main tasks of SLAM visual odometry is feature point extraction and matching. In SLAM, the spatial points corresponding to image feature points are usually called landmarks. This invention selects ORB features as the front-end feature points, and their extraction is divided into two parts: using the FAST algorithm to extract feature points (corner points), and using BRIEF as a descriptor for subsequent feature point matching.

[0031] In this embodiment, feature points are selected from representative points in the image. These points typically remain stable after camera movement. For example, the corners of objects or the rim of a cup are often chosen as feature points. As the camera moves, its position and viewing angle change. The same landmark observed at this time might be mistaken for different feature points. Therefore, feature points must possess the property of remaining unchanged after rotation and translation. Based on these invariant properties, the system can identify the same object even if the same scene is captured from different angles.

[0032] Feature points have the following characteristics: (1) Repeatability: The same features can be found in other images within a certain range.

[0033] (2) Distinguishing features: Different feature points correspond to different descriptors.

[0034] (3) Efficiency: In an image, the number of feature points should be much smaller than the number of pixels.

[0035] (4) Locality: A feature point is only related to a small area around it.

[0036] Specifically, feature extraction uses the ORB feature point extraction method, which includes the following steps: 1.1.1) The image is rendered as a grayscale image, and an improved FAST corner detection algorithm is applied to extract initial feature points. For a candidate pixel p in the image, multiple consecutive sampling points are taken within a circular neighborhood centered on the pixel and with a radius equal to a preset pixel value. The grayscale values ​​of these sampling points are calculated relative to the center pixel. grayscale value The absolute difference; if there are several consecutive sampling points with gray values ​​all higher than or all below (where h is the set grayscale threshold), then the pixel point The corner point is determined.

[0037] 1.1.2) Calculate the FAST corner response value of the detected corner point, which is the sum of the absolute value of the gray difference between all sampling points in the circular neighborhood and the center point; then, in a certain neighborhood range of the feature point, perform a non-maximum suppression operation on all candidate feature points, specifically: only keep the feature point with the maximum response value in the neighborhood, and suppress other feature points with lower response values, so as to ensure that the most distinctive features are extracted in the local area and to avoid excessive aggregation of feature points. The specific formula is as follows. s s

[0038] (1) In the formula, V Score is represented by score, t Threshold is represented by threshold, Pixel value is represented by pixel value, and after non-maximum suppression, the feature points of each layer image are uniformly screened and distributed according to the quadtree structure, so as to ensure that the feature points are uniformly distributed on the image plane, and to provide a data basis for subsequent robust feature matching and high-precision pose estimation.

[0039] The FAST algorithm is simple and fast, and the extraction of ORB features can maintain the rotation invariance of the image by calculating the vector between the centroid of the image block and the center. The scale invariance of the image is ensured by using the image pyramid calculation principle.

[0040] In this embodiment, after feature extraction, feature matching is performed. Feature point matching is a core technology for realizing data association in a SLAM system, and its main task is to identify the same target in the image when the camera observes the same scene from different angles. This process is the basis for subsequent pose estimation and graph optimization.

[0041] After matching all the feature points, the RANSAC algorithm is used to remove false matches. Specifically, the specific implementation process of feature matching includes the following steps: 1.2.1) Extract the corner feature points from the two images to be registered, and calculate the corresponding binary descriptor; when matching, the similarity between the descriptors is quantified by calculating the Hamming distance between them; After extracting the feature points, the similarity of their descriptors needs to be evaluated. In the ORB algorithm, the feature point descriptor is a fixed-length binary string, so the Hamming distance is often used as a similarity measure. The smaller the Hamming distance between two descriptors, the higher the similarity, which means they are more likely to come from the same actual point.

[0042] Let the feature point sets extracted from two images be: ​​(2) (3) The feature points of image A are: (4) The feature points of image B are: (5) The calculation formula of Hamming distance is as follows: (6) wherein, represents the XOR operation; is the length of the descriptor; the Hamming distance In essence, it measures the number of corresponding bits that are different between two binary strings, usually implemented through bitwise operations.

[0043] 1.2.2) When the number of features is less than a set value, brute force matching is usually used, which compares each feature point with all feature points in the target image. Specifically, the Hamming distance is calculated between each feature point and all points in another frame, and the minimum Hamming distance is taken as the matching target.

[0044] 1.2.3) When the number of extracted feature points is greater than a set threshold, that is, in a dense feature point scene, a matching method based on FLANN (Fast Library for Approximate Nearest Neighbors) is used. The core of this method is as follows: first, a binary descriptor set generated based on the ORB description algorithm is constructed to represent the appearance information of the feature points; then, the descriptor of the query feature point is inputted, and the Hamming distance between it and the candidate descriptor is calculated to measure their similarity. Based on this distance, an index structure is used to quickly search for the nearest neighbor, so that the most likely matching point pair can be located in the feature space with less computational cost.

[0045] The application of Hamming distance provides an efficient and accurate measure of similarity for binary descriptors, which is the key to real-time and robust matching, significantly improving the feature matching efficiency and overall processing speed of the system in complex scenes.

[0046] 1.2.4) After matching all feature points, RANSAC algorithm is used to remove false matches.

[0047] In this embodiment, the feature point matching relies on the selected similarity measurement method. In practical applications, a large number of corresponding relationships containing errors will be generated in the preliminary matching of feature points, and no matter how accurate the matching method is, it is difficult to avoid mismatching. These false matches will introduce system drift and even cause localization failure. In order to eliminate these false matches and improve the reliability of matching, a robust error elimination algorithm needs to be introduced. In ORB-SLAM3, the RANSAC (Random Sample Consensus) algorithm is used for false match elimination: in the matching stage, the RANSAC algorithm is used to estimate the fundamental matrix or homography matrix between two images to filter out false matching pairs caused by changes in illumination, motion blur or repeated textures, and to ensure that only reliable matching that meets the geometric consistency is retained. The advantage of RANSAC is that even if there are a large number of outliers in the data, it can still extract relatively reliable model parameters. Compared with the least squares method, RANSAC is more robust when facing non-Gaussian noise.

[0048] The RANSAC algorithm includes the following steps: Initial assumption: the data contains "inliers" (points that meet the model) and "outliers" (points that do not meet the model), and the rest are noise.

[0049] Set parameters: maximum number of iterations N max , inlier error threshold delta , inlier number threshold T .

[0050] The iteration process is: a) randomly select the minimum number of samples; b) estimate the model parameters (such as the Homography matrix H ) based on this subset; c) use this model to evaluate all data and count the number of inliers; d) if the current number of inliers exceeds the previous optimal solution, update the model; e) repeat steps a) - d), and terminate when the number of inliers reaches the threshold or has been iterated N max times.

[0051] In step 1) above, the pose is estimated using a visual SLAM system, and the EPnP method based on the RANSAC framework is used for pose solving, including: After obtaining the initial matching pairs, the robust solving process is entered. To improve the robustness of the solution, the method performs EPnP calculation under the RANSAC framework. In each iteration, a minimum size matching point set (no less than 4 pairs) is randomly selected from the current matching pairs for EPnP solution. The inlier set is identified by calculating the re-projection error, and the pose information is iteratively updated. Finally, the solution with the most inliers is selected as the optimal pose of the current frame relative to the map T cw .

[0052] In this embodiment, the pose solving method is as follows: first, 4 point pairs that are not coplanar are randomly selected from all matching pairs as a minimum sample set; an initial camera pose hypothesis is directly calculated by the EPnP algorithm using the sample set; then all associated 3D map points are projected onto the current image plane using the hypothesis pose, and the re-projection error between the 3D map points and the corresponding 2D feature points is calculated. The matching pairs with a re-projection error less than a predetermined threshold are determined as inliers, and vice versa. The inlier set obtained in this iteration and the number of inliers are recorded; The above random sampling, EPnP solving and inlier discrimination process is repeated for multiple iterations. After the iterations end, the optimal model with the most inliers is selected from all generated pose hypotheses; finally, all inliers corresponding to the optimal model are used for accurate solving by the EPnP algorithm, and the final optimal pose estimation matrix of the current frame relative to the world coordinate system is output T cw .

[0053] In this embodiment, multi-view geometry modeling is used. After obtaining matching feature points in consecutive image frames, the relative motion relationship between cameras can be calculated using methods in multi-view geometry.

[0054] Specifically, as shown in the scene in Figure 4 , the spatial point P is projected onto the imaging planes of two cameras O 1 and O 2 to form image points p 1 and p 2. The line connecting O 1 and O 2 intersects the respective imaging planes at points e 1 and e 2, which are called epipoles. The intersection of the epipolar plane formed by P, O 1 、O 2 and the image plane is the epipolar line p 1 e 1 and p 2 e 2. Both epipolar lines must pass through the epipoles, which is a core constraint of epipolar geometry.

[0055] First, at the algebraic level, this geometric constraint can be expressed using a fundamental matrix. Assume the camera containing the first frame image... O If the coordinate system is taken as the world coordinate system, then the same point in space... P The image projections in the two cameras satisfy the following epipolar constraint: (7) in, F It is the fundamental matrix, reflecting the algebraic correspondence between image points.

[0056] If the camera's intrinsic parameter matrix is ​​further known... K Then the essential matrix can be calculated. E , E and F The relationship is: (8) By decomposing the essential matrix E It can recover the relative pose between two frames, including the rotation matrix and translation vector, thereby estimating the camera's motion.

[0057] Then, the modeling is optimized in the backend. Pose drift compensation and state estimation are affected by noise in the feature extraction and matching processes, which causes errors to accumulate over time, leading to pose estimation drift. To reduce this accumulated error, a backend graph optimization module is introduced into the SLAM system to maintain global consistency and correct errors.

[0058] In feature-based visual SLAM frameworks, system states are typically modeled using state transition equations and observation equations, with motion models as follows: (9) Observation model: (10) in, The robot's camera indicates the time. k The position, It is a control input. Indicates the first j The location of each road sign. These are observation results. and These represent process noise and observation noise, respectively.

[0059] This modeling method transforms the SLAM problem into a state estimation problem, namely, estimating the robot's camera pose at the current moment under noisy motion and observation data conditions. and various road signs in the environment The spatial position of the robot. The overall state variable of the robot can be represented as: (11) where m represents the total number of landmark points in the map.

[0060] Under the Bayesian framework, the goal of state estimation is to maximize the posterior probability: (12) where represents the set of all observation data from time 1 to time k ; and represents the set of all control input data from time 1 to time k .

[0061] Since the prior probability is usually unknown, the maximum a posteriori estimation (MAP) is often simplified as the maximum likelihood estimation (MLE). Under the assumption that the error terms follow a Gaussian distribution, this estimation problem can be transformed into a nonlinear least squares problem: (13) where represents the motion error term; represents the covariance matrix corresponding to the motion error; represents the observation error term; represents the covariance matrix corresponding to the observation error.

[0062] where the error terms are defined as follows: Motion error term: (14) Observation error term: (15) By minimizing these error terms, the system can simultaneously optimize the camera trajectory and the map point positions, achieving more robust and accurate pose estimation.

[0063] The modeling method of graph optimization based on the back-end BA is divided into two steps, the first step is to construct the edges and nodes of the graph, and the second step is to optimize the variable nodes by minimizing the constraints and

[0064] In visual SLAM, the front-end task constructs edges and variable nodes. For each frame, the pose and landmarks of the sequentially input images serve as graph nodes. Motion constraints between two frames constitute an edge, as do observation constraints between a frame and a landmark. Finally, keyframes with loop closure constraints form an edge. After constructing the nodes and edges, the back-end optimization of the visual SLAM system can use its optimization algorithm to adjust the node values ​​to minimize the error between the measured and estimated values ​​of each edge.

[0065] by Figure 4 For example, suppose TK1 in the figure represents the camera pose, and after a period of time, TK1 moves to TK2 with a length L1 of 2m, and then moves from TK2 to TK3 with an angle rotation of 180° and a length L2 of 1.8m. At this point, the conditions for a closed loop are formed, that is, the SLAM system detects that TK1 and TK2 are in the same position with a length of L3.

[0066] In step 2) above, monocular visual depth estimation aims to reconstruct the 3D geometry of a scene using only a single or consecutive images. A depth estimation framework based on a cross-attention feature fusion mechanism can establish cross-modal feature associations, adapting to depth estimation requirements under different environmental conditions, thereby improving the environmental perception capability of the visual-inertial SLAM system.

[0067] In this embodiment, the depth estimation model based on the attention mechanism performs monocular depth estimation on the current frame image, including the following steps: 2.1.1) Using a single image as input to the attention-based depth estimation model, the corresponding depth map is output.

[0068] To fully utilize the camera's pose information to guide depth estimation, dense features extracted from the camera are first received. The camera not only performs conventional feature encoding but also enhances the geometric consistency of the features by mapping the camera's azimuth and elevation angles onto a pseudo-spherical feature representation.

[0069] Specifically, the monocular depth estimation camera module employs a pseudo-spherical representation method, encoding the camera pose as azimuth, elevation, and log-depth, compared to traditional Cartesian coordinate representation. Contrast. The key advantage of this pseudo-spherical representation is that the decoupling of camera direction information and depth component is achieved by design, ensuring their orthogonality in the representation space, thus effectively reducing the information entanglement commonly seen in Cartesian coordinate systems. Monocular depth estimation architectures directly compute depth information through global matching, without relying on a predefined depth range. This enables the method to handle images of arbitrary resolution, providing stronger generalization capabilities, especially for applications such as visual SLAM, which can maintain good performance under different camera settings and image resolutions. The monocular depth estimation task is simplified to learning a task-agnostic strong feature representation for depth prediction, so the Transformer (attention mechanism network architecture) is used, especially the cross-attention mechanism, to achieve this, and the depth estimation network architecture is as shown in Figure 5 .

[0070] This framework dynamically injects global priors from the scene level into local depth features through cross-attention layers, enabling the network to rely not only on local texture but also on overall structural information for inference. This cross-modal guided design effectively alleviates the scale drift problem in traditional monocular depth estimation, especially in scale reasoning tasks, showing higher accuracy. In the depth feature decoding stage, the module adopts a structure of alternating stacking of multi-level self-attention mechanisms and learnable upsampling. The self-attention block is responsible for capturing long-range dependencies and modeling the context correlation across spatial regions, while the learnable upsampling module gradually restores the spatial resolution and preserves the depth details. With the restoration of the feature map, the depth tensor and the angle features output by the camera module are fused in the channel dimension, finally forming a high-quality depth representation that combines global geometric structure and local details.

[0071] Specifically, under the pseudo-spherical representation, the non-parametric dense output of the camera can be represented as a tensor : (16) where H and W are the height and width of the input image, respectively, and the last dimension corresponds to the azimuth and elevation values of each pixel.

[0072] In this representation space, the back-projection operation is no longer a simple uniform ray-based depth multiplication, but is achieved by concatenating the camera direction and depth features. Specifically, the uniform ray of the camera is defined as: (17) where is the intrinsic matrix, is the coordinate of the pixel on the image plane, and 1 is the constant term in the homogeneous coordinates.

[0073] To further enhance the expressiveness of angular information, Laplacian Spherical Harmonics Encoding (SHE) is introduced to embed angular features into a higher dimensional space. The camera embedding tensor is represented as: (18) where is the number of selected spherical harmonic bases. The Laplacian Spherical Harmonics Encoding process is to compute a series of spherical harmonics and concatenate along the channel dimension, which is formulated as: (19) where denote the elevation and azimuth angles, respectively, is the maximum spherical harmonic order. This combination of pseudo-sphere and spherical harmonic encoding allows the camera module to not only preserve local angular details but also capture more complex spatial structures through high-order frequency components, providing rich and geometrically consistent feature support for the subsequent depth estimation module.

[0074] For each image that needs to be depth-estimated, the module performs different geometric augmentations, denoted as . Each augmentation includes sampling a scaling factor from a uniform distribution and a translation amount along the axis direction, followed by cropping the augmented image to the standard input size.

[0075] This operation simulates the observation process from the same scene but with different extrinsic parameters, similar to image pairs captured by different cameras. Let be the camera representation obtained after augmentation , and be the camera-guided depth features extracted based on . Due to the observation changes caused by geometric transformations, there is usually: (20) 2.1.2) To maintain the consistency of depth features under different augmentation conditions, a geometric invariance loss function is introduced during the training process, which constrains the consistency of features extracted after multiple random geometric transformations on the input image.

[0076] where the geometric invariance loss function is: (21) where D i |Ei denotes a camera hint E i modulated depth features, i = 1, 2; denotes a stop-gradient operation, which blocks the backpropagation of the pseudo-supervision signal; is a feature consistency loss function, such as mean square error (MSE) or contrastive loss.

[0077] In the above step 2), the RGB color information in the original image is combined to obtain a single-frame color dense point cloud, including the following steps: 2.2.1) A preset depth validity range is According to the depth validity range, the estimated depth map is filtered to retain the depth map falling within the depth validity range.

[0078] In order to improve the reliability of the dense point cloud, the depth map is filtered. The depth validity range is set to Only the pixel points satisfying the condition are retained: (22) Generally, depend on the scale range of the application scenario, respectively; denotes the depth value corresponding to the pixel point.

[0079] 2.2.2) Combine the depth map with the RGB color information of the original image to project each pixel as a three-dimensional point with color.

[0080] Specifically, for each input image, its corresponding depth tensor and original color image are received, where H and W denote the height and width of the image, respectively.

[0081] (23) where, is the two-dimensional coordinate of the pixel point, is the depth value corresponding to the point, is the intrinsic matrix of the camera. Expanding can get the position of the three-dimensional point: (24) where, is the principal point coordinate, is the focal length.

[0082] 2.2.3) Radius filtering and statistical outlier removal are performed on the colored three-dimensional points to remove outliers or noise points, and a dense color point cloud map with integrity and consistency is generated.

[0083] Specifically, each valid three-dimensional point is assigned its corresponding pixel point in the color image RGB color value, forming a textured dense point cloud. Let the point cloud set be: (25) wherein, N is the total number of valid points, represents the color of each point.

[0084] In the above step 3), a loosely coupled fusion strategy is used to spatially register the single-frame color dense point cloud with the robot pose at the corresponding time, and multi-frame data fusion is completed through point cloud stitching and optimization to construct a globally consistent three-dimensional dense point cloud map, including the following steps: 3.1) Obtain the current frame accurate pose output by the sparse visual SLAM system, and combine the single-frame color dense point cloud generated by the corresponding depth image and RGB information at the same time. Then, according to the camera intrinsic parameters, the pixel color information in the RGB image is combined with the geometric information in the depth image to generate a color dense point cloud of the current frame; the single-frame point cloud is down-sampled and statistically filtered to remove noise and redundant points.

[0085] 3.2) Use the pose information of the current frame to transform the filtered single-frame color dense point cloud from the camera coordinate system to the global world coordinate system to form a local map block; by transforming the poses between consecutive frames, the local map blocks obtained at different times are sequentially registered and spliced to realize incremental fusion of the point cloud, and gradually build a global dense point cloud map covering the entire scene.

[0086] Specifically, the point cloud is down-sampled and statistically filtered to remove noise points and redundant points. Then, using the pose of the current frame, the filtered point cloud is transformed from the camera coordinate system to the global world coordinate system to form a local map block corresponding to the current view. This step is the basic unit for building a global map, as shown in FIG. 3. Figure 2 As shown in FIG. 3, a plurality of local map blocks are spliced and fused by subsequent steps to form the final global map.

[0087] By transforming the poses between consecutive frames, the local map blocks generated at different times are registered and incrementally spliced to gradually fuse into a global dense point cloud map covering the entire scene.

[0088] 3.3) During the mapping process, the loop detection result of the sparse visual SLAM module is used for back-end optimization: when loop detection is detected, the optimized key frame pose after mapping is obtained, and the spatial position of the corresponding point cloud block in the constructed global map is adjusted according to the optimized pose. This process realizes the global consistency optimization of the map, effectively eliminates the cumulative error, and finally outputs a high-precision dense three-dimensional point cloud map.

[0089] Specifically, the global map construction process is as shown in Figure 2 The coordinates of the image point cloud are relative to the camera. If the pose of the camera in the map is known, the point cloud can be converted into the map coordinate system through matrix transformation, and the matrix transformation is as follows: (26) In the above formula, is the camera coordinates of the point cloud, is the conversion relationship of the camera relative to the map world coordinate system, is the world coordinates of the point cloud.

[0090] In summary, the present application improves the deficiencies in the prior art. First, the traditional sparse visual SLAM system realizes positioning and mapping by extracting and tracking sparse feature points, but the constructed map only contains sparse feature points, lacking complete environmental geometric structure, surface information and texture details. If this kind of sparse mapping method is used alone, although real-time positioning can be achieved, the generated map has limited information, which is difficult to directly support downstream applications such as robot path planning, dynamic obstacle avoidance and other applications that require high-precision environmental modeling.

[0091] Secondly, some existing dense mapping methods usually adopt the strategy of simultaneously positioning and dense reconstruction, estimating camera motion and reconstructing the scene through dense image alignment. However, when using this kind of method alone, the dense image alignment process is easily disturbed by light changes, reflections and other photometric noise, resulting in unstable pose estimation; at the same time, due to the need to jointly optimize a large number of scene parameters, the computational complexity is high, and it is difficult to realize real-time and robust operation on resource-limited platforms. Therefore, the present application proposes an efficient combined dense mapping scheme to overcome the defects of using the existing method alone. The present application takes the image sequence collected by a monocular camera as input, directly generates corresponding depth information through a depth estimation network, and then fuses the RGB image to form a color depth point cloud; at the same time, a high-performance visual SLAM algorithm is used to provide accurate camera pose estimation. By combining the depth point cloud and the SLAM pose in a loosely coupled manner, the reconstruction of the three-dimensional dense map is realized. This combination avoids the sensitive dependence of dense image alignment on photometric consistency, reduces the computational burden, and at the same time ensures the geometric accuracy and texture integrity of the map.

[0092] The dense point cloud map constructed by the application completely retains the geometric shape and texture details of the environment, and overcomes the limitations of traditional sparse feature maps in robot navigation planning, three-dimensional scene immersive display (such as VR / AR), and fine three-dimensional modeling and other applications. By fusing the advantages of depth estimation and visual SLAM, the application provides reliable, rich and directly usable environment representation data basis for various downstream tasks under the premise of ensuring real-time.

[0093] In an embodiment of the application, a high-precision map reconstruction system based on monocular vision is provided, comprising: A feature point extraction and pose estimation module acquires image data through a monocular image acquisition device, extracts and matches features for each frame of image, combines with the attitude information acquired by the IMU, and uses a sparse visual SLAM system to estimate the pose to obtain the pose information of the robot; A depth information extraction module uses a depth estimation model based on an attention mechanism to perform monocular depth estimation on the current frame of image, combines with the RGB color information in the original image, and obtains a single-frame color dense point cloud; A three-dimensional dense map reconstruction module uses a loosely coupled fusion strategy to spatially register the single-frame color dense point cloud and the pose of the robot at the corresponding moment, performs multi-frame data fusion through point cloud stitching and optimization, constructs a globally consistent three-dimensional dense point cloud map, and realizes three-dimensional modeling of a complex scene.

[0094] In the above embodiment, the feature extraction uses an ORB feature point extraction method, which includes: In the feature detection stage, a FAST corner detection algorithm is used to detect, for each pixel point p in the image, a circular neighborhood with the pixel point p as the center and a radius of r pixels, and m sampling points in the neighborhood; If the gray values of the continuous n sampling points in the neighborhood are all higher than or lower than , the point is determined as a corner feature, which is used as a candidate feature point for subsequent matching; wherein, is the gray value of the pixel p , and h is a set gray threshold value.

[0095] In the above embodiment, after matching all the feature points, the RANSAC algorithm is used to remove the false matches; specifically, the following process is included: In the two images to be registered, corner feature points are extracted, and their corresponding binary descriptors are calculated; when matching, the similarity is quantified by calculating the Hamming distance between the descriptors; When the feature points are less than the set number, brute force matching is adopted to perform full traversal comparison between each feature point and all feature points in the target image; in the scene where the feature points are greater than the set number, fast retrieval is performed through the construction of a search structure; In the matching stage, the RANSAC algorithm is used to estimate the fundamental matrix or homography matrix between two images to filter out false matching pairs caused by illumination changes, motion blur or repeated textures, so that only reliable matching pairs that meet geometric consistency are retained.

[0096] In the above embodiment, sparse visual SLAM system is used for pose estimation, and EPnP method based on RANSAC framework is used for pose solving, including: After obtaining the initial matching pairs, the robust solving process is entered; First, 4 point pairs that are not coplanar are randomly selected from all matching pairs as a minimum sample set; using the sample set, an initial camera pose hypothesis is directly calculated by the EPnP algorithm; Then, all associated 3D map points are projected onto the current image plane using the assumed pose, and the re-projection error between them and the corresponding 2D feature points is calculated. The matching pairs with re-projection error less than the preset threshold are determined as inliers, otherwise as outliers, and the inlier set obtained in this iteration and its number are recorded; The above random sampling, EPnP solving and inlier discrimination process is repeated for multiple iterations. After the iteration ends, from all generated pose hypotheses, the model with the most supported inliers is selected as the optimal model; Finally, all inliers corresponding to the optimal model are used to perform an accurate solution by the EPnP algorithm, and the final optimal pose estimation matrix of the current frame relative to the world coordinate system is output T cw .

[0097] In the above embodiment, the attention mechanism-based depth estimation model performs monocular depth estimation on the current frame image, including: A single image is input into the attention mechanism-based depth estimation model, and the corresponding depth map is output; In the training process, a geometric invariance loss function is introduced. The loss function performs multiple random geometric transformations on the input image, and constrains the consistency of the features extracted after different transformations; The geometric invariance loss function is:

[0098] Wherein, D i |E i represents the camera prompt Ei modulated depth features, i = 1, 2; denotes a stop gradient operation, used to block the back propagation of the pseudo-supervised signal, is a feature consistency loss function.

[0099] In the above embodiment, the single-frame color dense point cloud is obtained by combining the RGB color information in the original image, including: The preset depth validity range is The estimated depth map is filtered according to the depth validity range, and the depth map falling within the depth validity range is retained; The depth map and the RGB color information of the original image are combined, and each pixel is projected as a three-dimensional point with color; The three-dimensional point with color is subjected to radius filtering and statistical outlier removal, and the outlier or noise point is removed, to generate a dense color point cloud map with completeness and consistency.

[0100] In the above embodiment, a loose coupling fusion strategy is used to spatially register the single-frame color dense point cloud and the robot pose at the corresponding moment, and multi-frame data fusion is completed through point cloud stitching and optimization to construct a globally consistent three-dimensional dense point cloud map, including: The accurate pose information of the current image frame output by the sparse visual SLAM system is obtained, and the corresponding depth image and RGB image at the same moment are obtained; then, according to the camera intrinsic parameter, the pixel color information in the RGB image is combined with the geometric information in the depth image to generate a color dense point cloud of the current frame; the single-frame point cloud is subjected to down-sampling and statistical filtering processing to remove noise and redundant points; The filtered single-frame color dense point cloud is transformed from the camera coordinate system to the global world coordinate system by using the pose information of the current frame, to form a local map block; the local map blocks obtained at different moments are sequentially registered and spliced by the pose transformation between consecutive frames, to realize incremental fusion of the point cloud, and gradually construct a global dense point cloud map covering the entire scene; During the mapping process, the map is optimized according to the loop detection result of the sparse visual SLAM system; when a loop is detected, the optimized key frame pose graph is obtained, and the spatial position of the corresponding point cloud block in the constructed global dense point cloud is re-adjusted according to the optimized pose, to realize global consistency optimization of the map, eliminate the cumulative error, and finally output a high-precision color three-dimensional dense point cloud map.

[0101] The system provided in the embodiment is used to execute the above-mentioned method embodiments, and the specific process and detailed content are referred to the above-mentioned embodiments, which will not be described here.

[0102] In an embodiment of the present application, a computing device, which can be a terminal, can include a processor, a communications interface, a memory, a display screen and an input device. The processor, the communications interface and the memory can communicate with each other through a communication bus. The processor is configured to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program is executed by the processor to implement the method of any of the above embodiments. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communications interface is configured to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, a management network, NFC (Near Field Communication) or other technologies. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the housing of the computing device, or an external keyboard, a touchpad or a mouse, etc. The processor can invoke the logical instructions in the memory.

[0103] In addition, the logical instructions in the memory described above can be implemented in the form of a software functional unit and sold or used as a standalone product, which can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application, in essence, or the parts that contribute to the prior art, or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0104] In an embodiment of the present application, a computer program product is provided, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, enable the computer to perform the methods provided by the various embodiments described above.

[0105] In an embodiment of the present application, a non-transitory computer-readable storage medium is provided, which stores server instructions. The computer instructions enable a computer to perform the methods provided by the various embodiments described above.

[0106] The computer readable storage medium provided by the above embodiment has similar implementation principles and technical effects to the method embodiment, and thus will not be described here.

[0107] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate one or more means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in one or more flows and / or blocks.

[0108] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in one or more flows and / or blocks.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in one or more flows and / or blocks.

[0110] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A high-precision map reconstruction method based on monocular vision, characterized in that, include: Image data is acquired through a monocular image acquisition device, features are extracted and matched for each frame of the image, and pose information is obtained by combining the pose information obtained by the IMU with a sparse visual SLAM system to estimate the pose information of the robot. The attention-based depth estimation model performs monocular depth estimation on the current frame image and combines it with the RGB color information in the original image to obtain a single-frame color dense point cloud. A loosely coupled fusion strategy is adopted to spatially register a single frame of dense color point cloud with the robot pose at the corresponding moment. Multi-frame data fusion is completed through point cloud stitching and optimization to construct a globally consistent 3D dense point cloud map, so as to realize 3D modeling of complex scenes.

2. The high-precision map reconstruction method based on monocular vision as described in claim 1, characterized in that, Feature extraction employs the ORB feature point extraction method, including: In the feature detection stage, the FAST corner detection algorithm is used to detect each pixel in the image. p Detection by pixel p Centered on, with radius r Within the circular neighborhood of a pixel m One sampling point; If there are consecutive [elements] in the neighborhood. n The grayscale values ​​of each sampling point are all higher than or below If the feature is a corner point, then that point is identified as a corner feature and used as a candidate feature point for subsequent matching; where, For pixels p grayscale value, h The set grayscale threshold.

3. The high-precision map reconstruction method based on monocular vision as described in claim 2, characterized in that, During feature matching, after matching all feature points, false matches are removed using the RANSAC algorithm; specifically, the process includes the following: Corner feature points are extracted from the two images to be registered, and their corresponding binary descriptors are calculated; during matching, the similarity is quantified by calculating the Hamming distance between the descriptors. When the number of feature points is less than the set number, brute-force matching is used, and each feature point is compared with all feature points in the target image. When the number of feature points is greater than the set number, a search structure is constructed for fast retrieval. During the matching phase, the fundamental matrix or homography matrix between two images is estimated using the RANSAC algorithm to filter out erroneous matching pairs caused by illumination changes, motion blur, or duplicate textures, ensuring that only reliable matches that meet geometric consistency are retained.

4. The high-precision map reconstruction method based on monocular vision as described in claim 1, characterized in that, Pose estimation is performed using a sparse visual SLAM system, and the pose solution is obtained using the EPnP method based on the RANSAC framework, including: After obtaining the initial matching pairs, the robust solution process begins. First, four non-coplanar point pairs are randomly selected from all matching pairs as the minimum sample set; using this sample set, an initial camera pose hypothesis is directly calculated using the EPnP algorithm. Then, using the assumed pose, all associated 3D map points are projected onto the current image plane, and the reprojection error between them and the corresponding 2D feature points is calculated. Matching pairs with reprojection errors less than a preset threshold are determined as interior points, and those with reprojection errors greater than a preset threshold are determined as exterior points. The set of interior points and their number obtained in this iteration are recorded. Repeat the above random sampling, EPnP solution and interior point discrimination process multiple times. After the iteration is completed, select the model with the most supported interior points from all generated pose assumptions as the optimal model. Finally, using all interior points corresponding to the optimal model, an exact solution is calculated using the EPnP algorithm, outputting the final optimal pose estimation matrix of the current frame relative to the world coordinate system. T cw .

5. The high-precision map reconstruction method based on monocular vision as described in claim 1, characterized in that, The attention-based depth estimation model performs monocular depth estimation on the current frame image, including: Using a single image as input to an attention-based depth estimation model, the corresponding depth map is output. During training, a geometric invariant loss function is introduced. This loss function performs various random geometric transformations on the input image and constrains the extracted features to maintain consistency after different transformations. The geometric invariance loss function is: in, D i |E i Indicates camera-based prompts E i Modulated depth features, i=1,2; This indicates that the gradient operation is stopped, which is used to block the backpropagation of false supervision signals. This is the feature consistency loss function.

6. The high-precision map reconstruction method based on monocular vision as described in claim 5, characterized in that, By combining the RGB color information from the original image, a single-frame color dense point cloud is obtained, including: The preset depth validity range is The estimated depth maps are filtered based on the depth validity range, and depth maps that fall within the depth validity range are retained. By combining the depth map with the RGB color information of the original image, each pixel is projected as a colored 3D point. Radius filtering and statistical outlier removal are performed on the colored 3D points to remove outliers or noise points and generate a dense color point cloud map with integrity and consistency.

7. The high-precision map reconstruction method based on monocular vision as described in claim 1, characterized in that, A loosely coupled fusion strategy is employed to spatially register a single frame of dense color point cloud with the robot's pose at the corresponding moment. Multi-frame data fusion is then completed through point cloud stitching and optimization to construct a globally consistent 3D dense point cloud map, including: The system acquires the precise pose information of the current image frame output by the sparse visual SLAM system, and acquires the corresponding depth image and RGB image at the same time. Then, based on the camera intrinsic parameters, it combines the pixel color information in the RGB image with the geometric information in the depth image to generate the color dense point cloud of the current frame. The single-frame point cloud is then downsampled and statistically filtered to remove noise and redundant points. Using the pose information of the current frame, the filtered single-frame color dense point cloud is transformed from the camera coordinate system to the global world coordinate system to form a local map patch; through pose transformation between consecutive frames, the local map patches acquired at different times are sequentially registered and stitched together to achieve incremental fusion of the point cloud and gradually build a global dense point cloud map covering the entire scene. During the mapping process, the map is optimized based on the loop closure detection results of the sparse visual SLAM system. When a loop closure is detected, the optimized keyframe pose map is obtained, and the spatial position of the corresponding point cloud block in the constructed global dense point cloud is readjusted according to the optimized pose to achieve global consistency optimization of the map, eliminate accumulated errors, and finally output a high-precision color 3D dense point cloud map.

8. A high-precision map reconstruction system based on monocular vision, characterized in that, include: The feature point extraction and pose estimation module acquires image data through a monocular image acquisition device, performs feature extraction and matching on each frame of the image, and combines the pose information acquired by the IMU with the pose estimation using a sparse visual SLAM system to obtain the robot's pose information. The depth information extraction module uses an attention-based depth estimation model to perform monocular depth estimation on the current frame image and combines it with the RGB color information in the original image to obtain a single-frame color dense point cloud. The 3D dense map reconstruction module employs a loosely coupled fusion strategy to spatially register a single frame of color dense point cloud with the robot's pose at the corresponding moment. Through point cloud stitching and optimization, it completes the fusion of multiple frames of data to construct a globally consistent 3D dense point cloud map, thereby enabling 3D modeling of complex scenes.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images