A mobile robot vision mapping method, system, device and medium

Abstract

The application discloses a kind of mobile robot vision mapping method, system, equipment and medium, wherein mapping method steps are as follows: image frame is collected, pose and feature point metric depth are obtained by feature extraction;Image frame is input monocular depth estimation network to obtain relative depth map and feature point relative depth;Scale grid is constructed by two levels of rough and fine;Sliding window containing Zhang image frame is constructed;Calculate re-projection residual and scale residual to obtain total residual function;The hessian matrix of metric depth, pose and scale grid is calculated, and laplace regularization is applied to the hessian matrix of scale grid;The diagonal dominant ratio of the hessian matrix of scale grid after regularization is calculated, and adaptive additional damping is applied accordingly to assemble the final hessian matrix;Two-stage iterative optimization is carried out on the optimization variables in the sliding window to obtain the optimized pose and fine grid;After processing all image frames, scale map and metric depth map are obtained by bilinear interpolation, and a visual map is constructed.

Description

Technical Field

[0001] This invention belongs to the field of visual mapping technology, specifically relating to a visual mapping method, system, electronic device, and storage medium for mobile robots. Background Technology

[0002] The widespread application of mobile robots in autonomous driving, warehousing and logistics, and home services has created an urgent need for accurate and efficient 3D mapping capabilities. Vision-based mapping technology, due to its low-cost sensors and rich information, has become one of the core solutions for autonomous navigation of mobile robots. However, traditional feature-point-based vision mapping methods can only provide sparse or semi-dense 3D maps, lacking depth information in many textureless areas, making it difficult to meet the needs of path planning and obstacle avoidance for dense maps.

[0003] In recent years, the rapid development of monocular depth estimation networks has provided a new approach to dense depth acquisition. These networks, through self-supervised learning on large-scale data, can predict dense relative depth maps with fine edges from a single color image, effectively compensating for depth holes in textureless regions caused by sparse feature points. However, the depth values ​​output by monocular depth estimation networks only represent the relative distances of points within a scene, lacking a true physical scale. How to reconstruct a scale-less relative depth map into a metric depth map with accurate units of measurement is a key challenge in achieving dense and accurate 3D reconstruction technology.

[0004] In existing technologies, some solutions attempt to perform global scale alignment by using the metric depth of sparse feature points and the relative depth at corresponding locations, uniformly scaling the relative depth map to the metric depth using a single global scale factor. However, due to the nonlinearity of camera imaging, the bias of the depth estimation network itself, and the scale variations of different regions within the scene, non-uniform scale drift often exists within a single image, and a single global scale factor cannot accurately describe the scale variations of each region within the entire image. Some solutions introduce complex depth completion or refined global optimization networks, which can improve local accuracy but significantly increase computational overhead, making it difficult to meet the real-time mapping needs of mobile robots. Furthermore, when using multi-frame observation fusion for mapping, if the scale recovery of each frame is inaccurate or the scales between frames are inconsistent, it will directly lead to surface misalignment, ghosting, or even mapping failure during multi-frame fusion.

[0005] In summary, there is an urgent need for a mobile robot visual mapping technology that can accurately describe pixel-by-pixel scale changes within an image in a lightweight parameterized manner and can be jointly optimized with multi-view geometric constraints to achieve dense and scale-consistent visual mapping. Summary of the Invention

[0006] The purpose of this invention is to overcome the problems in existing mobile robot visual mapping technologies, such as the lack of physical scale in the output of monocular depth estimation networks and the inability of a single global scale factor to describe non-uniform scale changes in local regions within an image, leading to inconsistent scale and low mapping accuracy during multi-frame fusion. To achieve the above objective, this invention adopts the following technical solution:

[0007] In a first aspect, the present invention discloses a visual mapping method for a mobile robot, the mapping method comprising the following steps:

[0008] S1. Image frames are acquired by a camera mounted on a mobile robot. Feature extraction and feature matching are performed on the acquired image frames to obtain the feature points and feature point depth values ​​of the image frames. The pose of the image frames is then obtained according to the Perspective-n-Point (PnP) algorithm. The PnP algorithm uses the sparse feature points and feature point depth values ​​of the image frames, establishes projection geometric constraints based on the calibrated camera intrinsic parameters, and solves the rotation and translation of the image frames relative to the world coordinate system to obtain the pose of the image frames.

[0009] S2. Input the image frame into the Depth Anything v2 monocular depth estimation network to obtain the relative depth map of the image frame. Based on the position of the feature points of the image frame in the relative depth map, query the value of the feature points at the corresponding positions in the relative depth map to obtain the relative depth value of the feature points of the image frame. After offline self-supervised learning on large-scale data, the Depth Anything v2 monocular depth estimation network can predict pixel-level relative depth maps with fine edges when image frames are input into the network.

[0010] S3. Construct the scale grid of the image frame and initialize the scale grid; the scale grid consists of two levels, the first level is a coarse grid; the second level is a fine grid;

[0011] S4. Construct a sliding window whose parameters include the latest data. Feature points of each image frame, feature point depth values, relative feature point depth values, relative depth maps, pose, and scale grid; construct optimization variables, which are derived from... The feature points of an image frame are composed of depth values, pose, and scale grids.

[0012] S5. Calculate the reprojection residual function and scaling residual function of the sliding window and sum them by weight to obtain the total residual function of the sliding window;

[0013] S6. Based on the total residual function, calculate the Hessian matrix of the optimization variables, including the metric depth Hessian matrix, pose Hessian matrix, and scale grid Hessian matrix; among them, apply Laplacian smoothing regularization to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix.

[0014] S7. Calculate the diagonal dominance ratio of the regularized scaled Hessian matrix, calculate the additional damping based on the diagonal dominance ratio, and apply the additional damping to the regularized scaled Hessian matrix; assemble the depth Hessian matrix, the pose Hessian matrix, and the regularized scaled Hessian matrix with the additional damping applied into the final Hessian matrix.

[0015] S8. Iterative optimization of the optimization variables, the iteration process is divided into two stages. In the first stage, a sliding window is constructed. The first-stage optimization variables, consisting of feature point depth values, pose, and a coarse mesh from each image frame, are used to extract the first-stage Hessian matrix from the final Hessian matrix. A first-stage incremental equation is then constructed, and iteratively solved to obtain the optimized coarse mesh. In the second stage, the optimized coarse mesh is assigned to the fine mesh, and a sliding window is used to construct a... The second-stage optimization variables consist of the feature point metric depth value, pose, and fine mesh of the image frame. The second-stage Hessian matrix is ​​extracted from the final Hessian matrix, and the second-stage incremental equation is constructed. The second-stage incremental equation is iteratively solved to obtain the pose and fine mesh of the optimized image frame.

[0016] S9. When a new image frame is acquired, repeat steps S1 to S8 until all image frames are processed to obtain the optimized pose and fine mesh of all image frames and the relative depth map of all image frames; based on the optimized fine mesh, obtain the scale map of all image frames; based on the scale map and relative depth map of all image frames, obtain the metric depth map of all image frames; based on the metric depth map of all image frames and the optimized pose, construct a 3D visual map by fusing the metric depth map of all image frames and the optimized pose.

[0017] Furthermore, step S3 is as follows:

[0018] S31. Construct the scale grid S of the image frame. The scale grid S of the image frame consists of two levels, the first level being a coarse grid. ,Include One parameter; the second level is a fine mesh. ,Include There are several parameters, and the relationship between the number of parameters satisfies... The image frame is uniformly divided according to the image resolution, the number of parameters of the coarse grid and the fine grid, and the grid nodes are determined. The parameters of the two levels of grid are located on the grid nodes of the image frame, and the parameters represent the scale factor of the grid node. By performing bilinear interpolation on the two levels of grid, the scale factor at any position of the image frame can be obtained, which is used to fit the mapping relationship between the feature point metric depth value and the feature point relative depth value.

[0019] S32, Apply coarse mesh Initialized as the mean of the ratio of feature point depth values ​​to relative feature point depth values ​​in an image frame, with a fine mesh. Initialize to 1.0; this step calculates the global average scale factor based on the ratio of the depth to the relative depth of the feature points in the image frame, providing an optimized initial value close to the true value for the coarse grid.

[0020] Furthermore, step S5 is as follows:

[0021] S51. Calculate the reprojection residual function based on the feature points, feature point depth values, and pose of the image frames within the sliding window. , represented as ;in, and These represent the first and second elements within the sliding window, respectively. The first image frame Each feature point and its depth measurement; Indicates the first [number]th ... The pose of the image frame; This represents the projection function, which utilizes the first... pose of the image frame Measure the depth value of feature points Project to the Zhang image frame; Indicates the first [number]th ... Number of feature points per image frame. Reprojection residual function. As a geometric consistency constraint, the pose, feature point depth measurement value, and pose of an image frame are associated through a projection function, guiding the subsequent iterative optimization of the pose and feature point depth measurement value of the image frame.

[0022] S52. Calculate the scale residual function based on the scale grid, relative depth values ​​of feature points, and depth measurement values ​​of feature points within the image frame of the sliding window. , represented as ,in, Represents scaled grid Feature points obtained by bilinear interpolation The scale factor, For the first The first image frame Feature points The corresponding feature point relative depth value. Scale residual function. By correlating the scale grid, the relative depth values ​​of feature points, and the metric depth values ​​of feature points, subsequent iterative optimization of the scale grid is guided, addressing the issue of the relative depth values ​​of feature points lacking a true physical scale. Using bilinear interpolation, the scale factor of the scale grid at any location can be obtained, enabling pixel-by-pixel scale factor calculation.

[0023] S53. The weighted summation of the reprojection residual function and the scaling residual function yields the total residual function. , represented as ,in, and These represent preset weighting coefficients used to balance the relative contributions of reprojection constraints and scale constraints. The weighted summation of the reprojection residual and the scale residual is used to obtain the total residual function, thereby unifying the mathematical forms of multi-view geometric constraints and single-frame scale constraints. This provides an objective function for subsequent iterative optimization to ensure positioning accuracy and scale consistency.

[0024] Furthermore, step S6 is as follows:

[0025] S61. Calculate the total residual function Regarding the depth value of feature points , pose and scale grid The first-order gradients are respectively expressed as: , and ;

[0026] The total residual function is further calculated based on the first-order gradient. Regarding the depth value of feature points , pose and scale grid The second-order gradients are expressed as follows: , and ,in Represents the matrix transpose operation;

[0027] according to , and The first-order gradient is used to calculate the depth-pose cross term matrix. Measure depth value - scale grid intersection matrix Pose-scale mesh cross term matrix ;

[0028] Using the second gradient of the optimization variable as the Hessian matrix, we obtain the metric depth Hessian matrix of the optimization variable. Pose Hessian matrix and scale grid Hessian matrix ;

[0029] Based on the second-order gradient of the total residual function with respect to the optimization variables, a Hessian matrix is ​​constructed to provide a precise numerical optimization direction for subsequent iterative optimization, ensuring correct convergence of the iterative optimization. Simultaneously, the curvature information implied by the second-order gradient can adaptively scale the update step size of the optimization variables, significantly improving optimization efficiency.

[0030] S62, Hessian matrix for scaled grids Applying Laplacian smoothing regularization yields the Hessian matrix of the regularized scale grid. , represented as ,in, This represents the first Laplacian smoothing regularization matrix applied to the coarse mesh; This represents the second Laplacian smoothing regularization matrix applied to the fine mesh; and These represent the preset first and second smoothing weights, respectively. Laplace smoothing regularization forces the scale grid to remain smooth during iterative optimization, suppresses local scale abrupt changes caused by sparse observation noise, enhances the spatial continuity of the scale grid, and improves the numerical stability of subsequent iterative optimization processes.

[0031] Furthermore, step S7 is as follows:

[0032] S71. Based on the Hessian matrix of the regularized scale grid Computational Scale Grid The Diagonal dominance of each parameter , Represented as: , where index satisfy ,index satisfy , express The first in the matrix Line 1 The diagonal element of the column, express The first in the matrix Line 1 The off-diagonal elements of the column, j≠n; This step represents the operation of finding the maximum absolute value. The technical problem this step addresses is that the scale grid depends on the feature point depth measurement values ​​of the image frame. When the feature point depth measurement values ​​are sparsely distributed, some parameters of the scale grid are weakly constrained, affecting the regularized scale grid Hessian matrix. The low ratio of diagonal to off-diagonal elements leads to numerical singularities during iterative optimization. This is addressed by calculating the Hessian matrix of the regularized scale grid. The diagonal dominance ratio quantifies the degree of constraint on the scaled grid, providing a numerical basis for subsequent calculations of additional damping.

[0033] S72. Calculate the degree of weak constraint of the scaled mesh based on the diagonal dominance ratio; calculate the median of the diagonal dominance ratio. Calculate the absolute deviation of the median Computational scale grid The The degree of weak constraint on each parameter , Represented as ,in The cutoff threshold for the degree of weak constraint; This represents the operation for finding the median; This represents the operation of finding the minimum value;

[0034] Computational Scale Grid The Additional damping for each parameter , Represented as ,in, This is the basic damping coefficient. The technical problem addressed in this step is that, based on the diagonal dominance ratio and utilizing the robust statistical properties of the median and the absolute deviation of the median, the degree of constraint of the scale grid can be adaptively quantified, and additional damping terms can be calculated accordingly. This solves the problem that fixed damping cannot distinguish between strong and weak constraints, making it difficult to balance optimization stability and accuracy.

[0035] S73, Hessian matrix for regularized scaled grid By applying additional damping, the scaled grid Hessian matrix after applying additional damping is obtained. , represented as ,in, The depth Hessian matrix will be used to measure this. Pose Hessian matrix The scaled grid Hessian matrix after applying additional damping Measure depth value-pose cross term matrix Pose-scale mesh cross term matrix And the depth value-scale grid intersection matrix Assembled into the final Hessian matrix , represented as ,in,() T This represents the matrix transpose operation. The final Hessian matrix is ​​obtained from this. Unifying the constraints between the pose, metric depth, and scale grid of the image frame provides second-order gradient information for subsequent iterative optimization, and provides the correct direction for iterative optimization of the pose, metric depth, and scale grid.

[0036] Furthermore, step S8 is as follows:

[0037] S81, Enter the first stage of iterative optimization; construct the depth measurement value of the image frames within the sliding window. Camera pose and coarse grid The first stage of optimization variables , represented as ;

[0038] In the final Hessian matrix Extracting the first-stage Hessian matrix The LM algorithm is used to solve the incremental equations. ,in It is the damping factor; The gradient for the first stage is denoted as... ,in The gradient is represented by a coarse grid. The steps of the LM algorithm are as follows: First, initialize the damping factor and optimization variables, calculate the actual objective function value, solve the incremental equation, and obtain the optimized variable values. Next, calculate the predicted objective function value and obtain the ratio of the predicted objective function value to the actual objective function value. Based on the ratio, decide whether to accept the optimized variable values. If the ratio is ideal, accept the value and decrease the damping factor to accelerate convergence; otherwise, reject the value and increase the damping factor to enhance the algorithm's stability. Iterate repeatedly until the preset iteration limit is reached, and output the optimal value of the optimization variables.

[0039] After the LM algorithm converges, the optimized coarse mesh is obtained. The first phase of iterative optimization has been completed.

[0040] The first stage of iterative optimization rapidly optimizes the coarse mesh, causing the coarse mesh parameters to converge from the initial global mean to a reasonable value, thus providing initial values ​​for the second stage of optimization.

[0041] S82, Enter the second stage of iterative optimization; adjust the optimized coarse mesh... Assign to fine mesh Construct a depth metric from the image frames within the sliding window. Camera pose and fine mesh The second-stage optimization variables consist of , represented as .

[0042] In the final Hessian matrix Extracting the second-stage Hessian matrix The LM algorithm is used to solve the incremental equations. ,in The second-stage gradient is represented as... ,in The gradient for the fine mesh;

[0043] After the LM algorithm converges iteratively, the optimized pose of the image frame is obtained. With fine mesh The second phase of iterative optimization has been completed.

[0044] The second stage only optimizes the fine mesh for local fine-tuning, while inheriting the scale structure of the first stage and efficiently capturing subtle fluctuations within the region, thus improving the accuracy of scale estimation.

[0045] Furthermore, step S9 is as follows:

[0046] S91. When a new image frame is acquired, repeat steps S1 to S8 until all images are processed. From the image frames, we can obtain all of them. Optimized pose of Zhang image frames All Fine mesh after image frame optimization and all Relative depth map after image frame optimization ,in, ,and It is far greater than w.

[0047] S92, for all The first image frame Zhang image frames, among which , define the first The fine grid after image frame optimization is , for the The fine grid of the image frame In pixel coordinates Perform bilinear interpolation at point 1 to obtain the first... Image frames at pixel coordinates scale at the location , represented as ,in This represents a bilinear interpolation operation; for the ... Perform bilinear interpolation on all pixel coordinates of the first image frame to obtain the second... Scale map of image frames Bilinear interpolation smoothly extends the scale factor of fine-grid nodes to the entire pixel domain, resulting in a spatially continuous and uninterrupted scale map;

[0048] S93, Definition For the first The relative depth map of the first image frame will be... Scale map of image frames With the Relative depth map of image frames Multiplying pixel by pixel, we get the first pixel. Measurement depth map of image frames , represented as For all Perform this step on each image frame to obtain all of them. Measurement depth map of image frames ;

[0049] S94, Integrate All Measurement depth map of image frames , pose A 3D visual map is constructed, completing the visual mapping. The fusion process is as follows: the 3D space is discretized into a voxel mesh, and each voxel stores its signed distance to the nearest object surface and its weight. When a depth map is input, the voxels are projected onto the image plane to obtain the observation distance, the truncation distance is calculated and weighted and fused with the current value for updating, and the zero isosurface is the reconstructed surface. This method, through truncation and weighted averaging, can effectively filter out single-frame noise and accumulate multi-view observations to achieve dense 3D scene visual mapping.

[0050] In a second aspect, the present invention provides a mobile robot visual mapping system for performing the above-described mapping method, the mapping system comprising:

[0051] The image frame acquisition module acquires image frames through a camera mounted on a mobile robot, and performs feature extraction and feature matching to obtain the feature points, feature point depth values, and pose of the image frame.

[0052] The monocular depth estimation module is used to perform monocular depth prediction on image frames to obtain the relative depth map of the image frames and the relative depth values ​​of feature points;

[0053] The scale grid module is used to construct the scale grid of the image frame and initialize the scale grid, which includes coarse and fine grids;

[0054] The sliding window module is used to construct a sliding window whose parameters include the most recently acquired data. The image frame contains feature points, feature point depth measurements, feature point relative depth values, relative depth maps, pose, and scale grids. W is the maximum number of image frames in the sliding window; construct the optimization variable, which is composed of... The feature points of an image frame are composed of depth values, pose, and scale grids.

[0055] The total residual function calculation module is used to calculate the reprojection residual function and the scaling residual function of the sliding window and then sum them by weight to obtain the total residual function of the sliding window.

[0056] The module for calculating the regularized scale grid Hessian matrix is ​​used to calculate the Hessian matrix of the optimization variables based on the total residual function. The Hessian matrix of the optimization variables includes the metric depth Hessian matrix, the pose Hessian matrix, and the scale grid Hessian matrix. The module applies Laplacian smoothing regularization to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix.

[0057] The final Hessian matrix calculation module assembles the depth Hessian matrix, the pose Hessian matrix, and the regularized scaled grid Hessian matrix after applying additional damping into the final Hessian matrix;

[0058] The iterative optimization module is used to iteratively optimize the optimization variables in two stages, obtaining the optimized coarse mesh, the pose of the image frame, and the fine mesh respectively.

[0059] The visual mapping module is used to obtain the scale map of all image frames after optimization, based on the optimized fine mesh; to obtain the metric depth map of all image frames based on the scale map and relative depth map of all image frames; and to construct a 3D visual map by fusing the metric depth map of all image frames with the optimized pose.

[0060] Thirdly, the present invention provides an electronic device, including a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the above-described mobile robot visual mapping method.

[0061] Fourthly, the present invention provides a storage medium storing a program, which, when executed by a processor, implements the aforementioned mobile robot visual mapping method.

[0062] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0063] (1) A dense scale recovery mechanism based on joint optimization of two-level scale grids and multi-source residuals is proposed to achieve accurate mapping from sparse metric depth to full-image metric depth. In the prior art, monocular depth estimation networks can only output relative depth maps without physical scale, and the metric depth of sparse feature points cannot be directly converted into dense pixel-by-pixel metric depth. In step S3, this invention constructs a two-level scale grid including coarse and fine grids, using the ratio of metric depth of sparse feature points to relative depth as the initial global scale, and in step S5, simultaneously constructs the reprojection residual function and the scale residual function and weighted sums them to obtain the total residual function. In terms of technical principle, the reprojection residual (see step S51) couples pose, metric depth and image observation into geometric consistency constraints, while the scale residual (see step S52) uses metric depth as supervision to drive the scale grid to convert relative depth into metric depth through bilinear interpolation, thereby realizing joint estimation of pose and dense scale in sliding window optimization. This mechanism solves the problem that sparse feature points alone cannot provide full-scale information, enabling regions without feature textures to obtain accurate physical scale.

[0064] (2) The introduction of Laplace smoothing regularization and an adaptive damping strategy based on the diagonal dominance ratio significantly improves the numerical stability and spatial continuity of scale-grid optimization. The constraints of scale-grid optimization come only from sparse feature points, which can easily lead to underdetermined optimization and singular Hessian matrix in some regions. In step S62, the present invention applies a first Laplace smoothing regularization and a second Laplace smoothing regularization to the scale-grid Hessian matrix to force a smooth transition of the scale factor of adjacent grid nodes; further, in step S71, the diagonal dominance ratio of the regularized scale-grid Hessian matrix is ​​calculated to quantify the constraint strength of each parameter; step S72 calculates the degree of weak constraint based on the median and the absolute deviation of the median and obtains additional damping; step S73 integrates the adaptive damping into the final Hessian matrix. In terms of technical principle, Laplace regularization provides a spatial smoothing prior and suppresses local mutations caused by sparse observations; the combination of the diagonal dominance ratio and robust statistics enables the weakly constrained region to obtain strong damping to suppress oscillations, while the strongly constrained region maintains low damping to ensure convergence accuracy. This differentiated processing effectively avoids optimization divergence, improves optimization efficiency, and ensures the continuity and reliability of the entire map-scale field.

[0065] (3) A two-stage relay optimization strategy from coarse to fine mesh is adopted to efficiently solve high-dimensional scale fields and improve the estimation accuracy of pose and fine mesh. Faced with the difficulty of full optimization convergence caused by the coupling of coarse and fine mesh parameters, this invention designs a two-stage relay optimization in step S8. In the first stage (see step S81), the fine mesh is fixed at 1.0, and only the coarse mesh is optimized to quickly capture the large-scale distribution trend from the initial global mean. In the second stage (see step S82), the optimized coarse mesh is fixed and used as the initial value for the fine mesh, and only the fine mesh is optimized for local fine-tuning. In terms of technical principle, this strategy decouples scale estimation into a "coarse-to-fine" optimization process, reducing the dimensionality of the parameter space and the ill-conditioned nature of the optimization problem. Simultaneously, both stages extract corresponding sub-blocks from the final Hessian matrix, preserving complete second-order gradient information and adaptive damping, ensuring rapid convergence of each stage's iteration. Ultimately, this relay strategy significantly improved optimization efficiency while maintaining accuracy, providing precise and spatially continuous fine-grid parameters for subsequent scale map generation and fusion mapping, thereby enhancing the accuracy of visual mapping. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a flowchart of the mobile robot visual mapping method proposed in this invention;

[0068] Figure 2 This is a diagram showing the relative depth estimation results of some image frames obtained in an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram of the scaled grid structure in an embodiment of the present invention;

[0070] Figure 4 This is a metric depth map of a portion of image frames obtained from the TUM dataset in an embodiment of the present invention;

[0071] Figure 5 These are the fine grid values ​​of a subset of image frames obtained from the TUM dataset in this embodiment of the invention;

[0072] Figure 6 This is a comparison chart of the estimated camera trajectory values ​​and the actual values ​​obtained in the embodiments of the present invention;

[0073] Figure 7 This is a 3D visual map obtained from the TUM dataset in an embodiment of the present invention;

[0074] Figure 8 This is a metric depth map of a portion of image frames obtained from the Replica dataset in an embodiment of the present invention;

[0075] Figure 9 This is a scale map of some image frames obtained in the embodiments of the present invention;

[0076] Figure 10 These are the fine grid values ​​of a subset of image frames obtained from the Replica dataset in this embodiment of the invention;

[0077] Figure 11 This is a 3D visual map obtained from the Replica dataset in an embodiment of the present invention;

[0078] Figure 12 This is a structural block diagram of a mobile robot visual mapping system according to Embodiment 3 of the present invention;

[0079] Figure 13 This is a structural block diagram of the electronic device in Embodiment 4 of the present invention. Detailed Implementation

[0080] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0081] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0082] Example 1

[0083] This embodiment discloses a visual mapping method for mobile robots. Taking the TUM dataset as an example, it details the specific implementation process and parameter configuration of each step in the visual mapping method for mobile robots.

[0084] Step S1: Image frames are acquired using a camera mounted on a mobile robot. Feature extraction and feature matching are performed on the acquired image frames to obtain feature points and feature point depth values. Then, the pose of the image frame is obtained based on the feature points and feature point depth values. The camera's image resolution is 640×480 pixels, and the camera intrinsic parameters are configured as follows: focal length fx=517.3, fy=516.5, principal point cx=318.6, cy=255.3. Feature correspondence is established through inter-frame descriptor matching to obtain feature point depth values. In this embodiment, the pose of the image frame is obtained according to the PnP algorithm. The PnP algorithm utilizes the sparse feature points and feature point depth values ​​of the image frame, establishes projection geometric constraints based on the calibrated camera intrinsic parameters, and solves the rotation and translation of the image frame relative to the world coordinate system to obtain the pose of the image frame.

[0085] Step S2, as follows Figure 2 As shown, the image frame is input into the Depth Anything v2 monocular depth estimation network to obtain the relative depth map of the image frame. Based on the position of the feature points in the relative depth map, the values ​​of the feature points at the corresponding positions in the relative depth map are queried to obtain the relative depth values ​​of the feature points in the image frame. After offline self-supervised learning on large-scale data, the Depth Anything v2 monocular depth estimation network can predict pixel-level relative depth maps with fine edges when image frames are input into the network. Step S3: Construct the scale grid of the image frame and initialize the scale grid; the scale grid consists of two levels: the first level is a coarse grid; the second level is a fine grid. The process is as follows:

[0086] Step S31, as follows Figure 3 As shown, the scale grid for constructing image frames is... The scale grid of the image frame It consists of two levels, the first level being a coarse grid. ,Include One parameter, in this embodiment The coarse grid contains 3×3=9 scale parameters; the second level is a fine grid. ,Include One parameter, in this embodiment The fine mesh contains 6×6=36 scale parameters, and the relationship between the number of parameters satisfies The two-level grid has a total of 45 scale parameters. The image frame is uniformly divided according to the image resolution of 640×480 pixels and the number of parameters for coarse and fine grids to determine the grid nodes. The parameters of the two-level grid are located on the grid nodes of the image frame, and the parameters represent the scale factor of that grid node. By performing bilinear interpolation on the two-level grid, the scale factor at any position in the image frame can be obtained, which is used to fit the mapping relationship between the feature point's depth measurement value and the feature point's relative depth value.

[0087] Step S32: Apply coarse mesh The parameters of the coarse grid are initialized to the mean of the ratio of the feature point metric depth value to the feature point relative depth value in the image frame. Taking a frame in the fr1_desk sequence as an example, the global mean of the ratio of the feature point metric depth to the relative depth value in that frame is 1.253. Therefore, all nine parameters of the coarse grid are initialized to 1.253; the fine grid... Initialize to 1.0; this step calculates the global average scale factor based on the ratio of the depth to the relative depth of the feature points in the image frame, providing an optimized initial value close to the true value for the coarse grid.

[0088] Step S4: Construct a sliding window whose parameters include the latest data. The image frames contain feature points, feature point depth measurements, feature point relative depth values, relative depth maps, poses, and scale grids. In this embodiment, a sliding window is used. An optimization variable is constructed, which consists of the feature point depth measurements, poses, and scale grids from the seven image frames. The sliding window moves forward with the arrival of new image frames, always maintaining the optimization variable from the latest seven frames.

[0089] Step S5: Calculate the reprojection residual function and scaling residual function of the sliding window, and sum them with weights to obtain the total residual function of the sliding window. The process is as follows:

[0090] Step S51: Calculate the reprojection residual function based on the feature points, feature point depth values, and pose of the image frames within the sliding window. , represented as ;in, and These represent the first and second elements within the sliding window, respectively. The first image frame Each feature point and its depth measurement; Indicates the first [number]th ... The pose of the image frame; This represents the projection function, which utilizes the first... pose of the image frame Measure the depth value of feature points Project to the Zhang image frame; Indicates the first [number]th ... Number of feature points per image frame. Reprojection residual function. As a geometric consistency constraint, the pose, feature point depth measurement value, and pose of an image frame are associated through a projection function, guiding the subsequent iterative optimization of the pose and feature point depth measurement value of the image frame.

[0091] Step S52: Calculate the scale residual function based on the scale grid, relative depth values ​​of feature points, and depth measurement values ​​of feature points within the image frame in the sliding window. , represented as ,in, Represents scaled grid Feature points obtained by bilinear interpolation The scale factor, For the first The first image frame Feature points The corresponding feature point relative depth value. Scale residual function. By correlating the scale grid, the relative depth values ​​of feature points, and the metric depth values ​​of feature points, subsequent iterative optimization of the scale grid is guided, addressing the issue of the relative depth values ​​of feature points lacking a true physical scale. Using bilinear interpolation, the scale factor of the scale grid at any location can be obtained, enabling pixel-by-pixel scale factor calculation.

[0092] S53. The weighted summation of the reprojection residual function and the scaling residual function yields the total residual function. , represented as ,in, and These represent preset weighting coefficients used to balance the relative contributions of reprojection constraints and scale constraints. The weighted summation of the reprojection residual and the scale residual is used to obtain the total residual function, thereby unifying the mathematical forms of multi-view geometric constraints and single-frame scale constraints. This provides an objective function for subsequent iterative optimization to ensure positioning accuracy and scale consistency.

[0093] Step S6: Calculate the Hessian matrices of the optimization variables based on the total residual function, including the metric depth Hessian matrix, pose Hessian matrix, and scale grid Hessian matrix; wherein, Laplacian smoothing regularization is applied to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix. The process is as follows:

[0094] Step S61: Calculate the total residual function Regarding the depth value of feature points , pose and scale grid The first-order gradients are respectively expressed as: , and ;

[0095] The total residual function is further calculated based on the first-order gradient. Regarding the depth value of feature points , pose and scale grid The second-order gradients are expressed as follows: , and ,in Represents the matrix transpose operation;

[0096] according to , and The first-order gradient is used to calculate the depth-pose cross term matrix. Measure depth value - scale grid intersection matrix Pose-scale mesh cross term matrix ;

[0097] Using the second gradient of the optimization variable as the Hessian matrix, we obtain the metric depth Hessian matrix of the optimization variable. Pose Hessian matrix and scale grid Hessian matrix ;

[0098] Step S62: Adjust the scale grid Hessian matrix. Applying Laplacian smoothing regularization yields the Hessian matrix of the regularized scale grid. , represented as ,in, This represents the first Laplacian smoothing regularization matrix applied to the coarse mesh; This represents the second Laplacian smoothing regularization matrix applied to the fine mesh; and These represent the preset first and second smoothing weights, respectively. In this embodiment, the first smoothing weight of the coarse grid... The second smoothing weight of the fine mesh Laplace smoothing regularization forces the scaled grid to remain smooth during iterative optimization, suppresses local scale abrupt changes caused by sparse observation noise, enhances the spatial continuity of the scaled grid, and improves the numerical stability of subsequent iterative optimization processes.

[0099] Step S7: Calculate the diagonal dominance ratio of the regularized scaled Hessian matrix, calculate the additional damping based on the diagonal dominance ratio, and apply this additional damping to the regularized scaled Hessian matrix; assemble the depth Hessian matrix, the pose Hessian matrix, and the regularized scaled Hessian matrix with the applied additional damping into the final Hessian matrix. The process is as follows:

[0100] Step S71: Based on the regularized scale grid Hessian matrix Computational Scale Grid The Diagonal dominance of each parameter , Represented as: , where index satisfy ,index satisfy , express The first in the matrix Line 1 The diagonal element of the column, express The first in the matrix Line 1 The off-diagonal elements of the column, j≠n; This indicates the operation of finding the maximum absolute value.

[0101] Step S72: Calculate the degree of weak constraint of the scale mesh based on the diagonal dominance ratio; calculate the median of the diagonal dominance ratio. Calculate the absolute deviation of the median Computational scale grid The The degree of weak constraint on each parameter , Represented as ,in The cutoff threshold for the degree of weak constraint; This represents the operation for finding the median; This represents the operation of finding the minimum value;

[0102] Computational Scale Grid The Additional damping for each parameter , Represented as ,in, It is the basic damping coefficient.

[0103] Step S73: Adjust the Hessian matrix of the regularized scale grid. By applying additional damping, the scaled grid Hessian matrix after applying additional damping is obtained. , represented as ,in, The depth Hessian matrix will be used to measure this. Pose Hessian matrix The scaled grid Hessian matrix after applying additional damping Measure depth value-pose cross term matrix Pose-scale mesh cross term matrix And the depth value-scale grid intersection matrix Assembled into the final Hessian matrix , represented as ,in,() T This represents the matrix transpose operation.

[0104] Step S8: Iteratively optimize the optimization variables. The iteration process is divided into two stages. In the first stage, construct the first-stage optimization variables consisting of the feature point depth measurement values, pose, and coarse mesh of the 7 image frames of the sliding window. Extract the first-stage Hessian matrix from the final Hessian matrix, construct the first-stage incremental equation using the LM algorithm, and iteratively solve the first-stage incremental equation to obtain the optimized coarse mesh. In the second stage, assign the optimized coarse mesh to the fine mesh, construct the second-stage optimization variables consisting of the feature point depth measurement values, pose, and fine mesh of the 7 image frames of the sliding window, extract the second-stage Hessian matrix from the final Hessian matrix, construct the second-stage incremental equation, and iteratively solve the second-stage incremental equation to obtain the optimized pose and fine mesh of the image frames. The process is as follows:

[0105] Step S81: Enter the first stage of iterative optimization; construct the depth measurement value of the image frames within the sliding window. Camera pose and coarse grid The first stage of optimization variables , represented as ;

[0106] In the final Hessian matrix Extracting the first-stage Hessian matrix The LM algorithm is used to solve the incremental equations. ,in It is the damping factor; The gradient for the first stage is denoted as... ,in The gradient for the coarse grid;

[0107] After 10 iterations of the LM algorithm, an optimized coarse mesh is obtained. The first phase of iterative optimization has been completed.

[0108] Step S82: Enter the second stage of iterative optimization; the optimized coarse mesh... Assign to fine mesh Construct a depth metric from the image frames within the sliding window. Camera pose and fine mesh The second-stage optimization variables consist of , represented as

[0109] In the final Hessian matrix Extracting the second-stage Hessian matrix The LM algorithm is used to solve the incremental equations. ,in The second-stage gradient is represented as... ,in The gradient for the fine mesh;

[0110] After 10 iterations of the LM algorithm, the optimized pose of the image frame is obtained. With fine mesh The second phase of iterative optimization has been completed.

[0111] Step S9: When a new image frame is acquired, repeat steps S1 to S8 until all image frames are processed, obtaining the optimized pose and fine mesh of all image frames, as well as the relative depth map of all image frames; based on the optimized fine mesh, obtain the scale map of all image frames; based on the scale map and relative depth map of all image frames, obtain the metric depth map of all image frames; based on the metric depth map of all image frames and the optimized pose, construct a 3D visual map by fusing the metric depth map and optimized pose of all image frames. The process is as follows:

[0112] Step S91: When a new image frame is acquired, repeat steps S1 to S8. Taking the fr1_desk sequence as an example, this experimental sequence contains approximately 613 images, until all images are processed. From the image frames, we can obtain all of them. Optimized pose of Zhang image frames All Fine mesh after image frame optimization and all Relative depth map after image frame optimization ,in, ,and Far greater than .

[0113] Step S92, for all The first image frame Zhang image frames, among which , define the first The fine grid after image frame optimization is , for the The fine grid of the image frame In pixel coordinates Perform bilinear interpolation at point 1 to obtain the first... Image frames at pixel coordinates scale at the location , represented as ,in This represents a bilinear interpolation operation; for the ... Perform bilinear interpolation on all pixel coordinates of the first image frame to obtain the second... Scale map of image frames Bilinear interpolation smoothly extends the scale factor of fine-grid nodes to the entire pixel domain, resulting in a spatially continuous and uninterrupted scale map;

[0114] Step S93, Definition For the first The relative depth map of the first image frame will be... Scale map of image frames With the Relative depth map of image frames Multiplying pixel by pixel, we get the first pixel. Measurement depth map of image frames , represented as For all Perform this step on each image frame to obtain all of them. Measurement depth map of image frames ;

[0115] Step S94: Merge all Measurement depth map of image frames , pose A 3D visual map is constructed, completing the visual mapping. The fusion process is as follows: The 3D space is discretized into a voxel mesh with a voxel resolution of 0.008 m. Each voxel stores its signed distance to the nearest object surface and its weight, with a truncation distance of 0.032 m. When a depth map is input, the voxels are projected onto the image plane to obtain the observation distance. The truncation distance is calculated and weighted, and then fused with the current truncation distance value for updating. The zero isosurface is the reconstructed surface. After fusion, isosurfaces are extracted with a weight threshold of 2.0. This method, through truncation and weighted averaging, can effectively filter out single-frame noise and accumulate multi-view observations, achieving dense 3D scene visual mapping.

[0116] To further verify the effectiveness and advancement of the mobile robot visual mapping method proposed in this invention, this embodiment performs quantitative depth estimation accuracy evaluation and trajectory accuracy evaluation on 5 sequences of the TUM dataset.

[0117] 1. Depth Estimation Accuracy Evaluation: On five sequences (fr1_desk, fr1_room, fr1_xyz, fr2_desk, and fr2_xyz) in the TUM dataset, using the real depth map provided by the dataset as a reference, the absolute relative error (AbsRel), squared relative error (SqRel), root mean square error (RMSE), logarithmic root mean square error (RMSElog), and threshold accuracy (δ<1.25) of the recovered metric depth map are calculated. 3 Five indicators. The depth estimation indicators for each sequence are shown in Table 1:

[0118] Table 1. Depth estimation metrics for 5 sequences in the TUM RGBD dataset

[0119]

[0120] As shown in Table 1, the method of this invention achieved good depth estimation accuracy on all five sequences, with δ < 1.25. 3 All indicators exceeded 0.99, with δ < 1.25 for the fr1_room sequence. 3 The accuracy is as high as 0.998. This verifies that the method of the present invention can accurately recover from relative depth to metric depth through two-level scale mesh optimization in different indoor scenarios.

[0121] 2. Trajectory Accuracy Evaluation: The absolute trajectory error (ATE) between the estimated trajectory and the true trajectory on the same 5 sequences is shown in Table 2:

[0122] Table 2. Pose estimation metrics for 5 sequences in the TUM RGBD dataset.

[0123]

[0124] As shown in Table 2, the method of this invention achieved centimeter-level trajectory accuracy on all five sequences, with an ATE of only 0.012 m for the fr1_xyz sequence. This result demonstrates that the present invention, through joint optimization of reprojection residuals and scale residuals within a sliding window, effectively improves the accuracy of pose estimation while restoring accurate depth measurements.

[0125] 3. Visualization Results and Analysis:

[0126] Figure 4 The image shows the metric depth map of some image frames. By applying Laplacian smoothing regularization to the scale grid, the scale factor of adjacent grid nodes is forced to transition smoothly, which effectively suppresses the local scale jump that may be caused by insufficient feature point sparsity constraints and ensures the fine edges of the metric depth map.

[0127] Figure 5 The figure shows the parameter values ​​of some image frames after fine-grid optimization. Each grid point is labeled with a specific scale factor value, reflecting the spatial variation trend of the scale factor within the same image frame. It can be observed from the figure that the scale factor within the same image frame is not a global constant, but rather exhibits a smooth and non-uniform distribution trend in space. This verifies the core innovation of this invention: by using a two-level scale grid, it accurately depicts the non-uniform scale drift of a single-frame image caused by factors such as camera imaging nonlinearity and depth estimation network bias. Compared with existing technologies that use a single global scale factor, this method can more accurately restore the correct physical scale for different regions within the image, thereby avoiding surface misalignment and ghosting problems caused by inconsistent scales in different regions during multi-frame fusion.

[0128] Figure 6The image shows a comparison between the estimated and true trajectories of the two sequences fr1_desk and fr1_room. The results show that the estimated trajectories are highly consistent with the true trajectories. This accuracy is attributed to the joint optimization of reprojection residuals and scale residuals within a sliding window, as described in this invention. On one hand, the reprojection residuals refine the pose and feature point metric depth through multi-view geometric constraints; on the other hand, the scale residuals drive the relative depth to align with the metric depth through a scale grid. The two constraints are solved within a unified optimization framework, enabling pose estimation and scale recovery to mutually promote each other and jointly improve accuracy.

[0129] Figure 7 The image shows the visual reconstruction results of the fr1_desk, fr1_room, fr2_desk, and fr2_xyz sequences. The results show that the reconstructed 3D model accurately reproduces the geometric structure of objects such as the desktop, monitor, and keyboard, verifying the complete mapping capability of the method of this invention from image sequences to dense 3D maps, and demonstrating the dense 3D reconstruction effect of indoor scenes.

[0130] Example 2

[0131] In this embodiment, the same experimental steps as in Embodiment 1 are used, but the experimental dataset is changed to the Replica dataset, and the parameters of the scale grid, the base damping coefficient of the adaptive additional damping, and the Laplace smoothing regularization weight parameters are changed. The specific implementation process of each step is described below.

[0132] Step S1: Image frames are acquired using a camera mounted on a mobile robot. Feature extraction and feature matching are performed on the acquired image frames to obtain feature points and feature point depth values. Then, the pose of the image frame is obtained based on the feature points and feature point depth values. In this embodiment, the Replica dataset is used, with camera intrinsics of fx=fy=600.0, principal point cx=599.5, cy=339.5, and image resolution of 1200×680 pixels. Feature extraction uses a corner detection algorithm, and feature correspondence is established through inter-frame descriptor matching to obtain feature point depth values. The pose of the image frame is further obtained according to the PnP algorithm. The PnP algorithm utilizes the sparse feature points and feature point depth values ​​of the image frame, establishes projection geometric constraints based on the calibrated camera intrinsics, and solves the rotation and translation of the image frame relative to the world coordinate system to obtain the pose of the image frame.

[0133] Step S2: Input the image frame into the Depth Anything v2 monocular depth estimation network to obtain the relative depth map of the image frame. Based on the position of the feature points of the image frame in the relative depth map, query the value of the corresponding position of the feature points in the relative depth map to obtain the relative depth value of the feature points of the image frame. After offline self-supervised learning on large-scale data, the Depth Anything v2 monocular depth estimation network can predict pixel-level relative depth maps with fine edges when image frames are input into the network.

[0134] Step S3: Construct the scale grid of the image frame and initialize the scale grid; this scale grid consists of two levels: the first level is a coarse grid; the second level is a fine grid. The process is as follows:

[0135] Step S31: Construct the scale grid S of the image frame. The scale grid S of the image frame consists of two levels, with the first level being a coarse grid. ,Include One parameter, in this embodiment The coarse grid contains 2×2=4 scale parameters; the second level is a fine grid. ,Include One parameter, in this embodiment The fine mesh contains 5×5=25 scale parameters, and the relationship between the number of parameters satisfies The two-level grid has a total of 29 scale parameters. The image frame is uniformly divided according to the image resolution of 1200×680 pixels and the number of parameters for coarse and fine grids to determine the grid nodes. The parameters of the two-level grid are located on the grid nodes of the image frame, and the parameters represent the scale factor of that grid node. By performing bilinear interpolation on the two-level grid, the scale factor at any position in the image frame can be obtained, which is used to fit the mapping relationship between the feature point's depth measurement value and the feature point's relative depth value.

[0136] Step S32: Apply coarse mesh The parameters of the coarse grid are initialized to the mean of the ratio of the feature point metric depth value to the feature point relative depth value in the image frame. Taking a frame in the office0 scene as an example, the global mean of the ratio of the feature point metric depth to the relative depth value in this frame is 2.317. Therefore, all four parameters of the coarse grid are initialized to 2.317; the fine grid... Initialize to 1.0; this step calculates the global average scale factor based on the ratio of the depth to the relative depth of the feature points in the image frame, providing an optimized initial value close to the true value for the coarse grid.

[0137] Step S4: Construct a sliding window whose parameters include the latest data. The image frames contain feature points, feature point depth measurements, feature point relative depth values, relative depth maps, poses, and scale grids. In this embodiment, a sliding window is used. An optimization variable is constructed, which consists of the feature point depth measurements, poses, and scale grids from the seven image frames. The sliding window moves forward with the arrival of new image frames, always maintaining the optimization variable from the latest seven frames.

[0138] Step S5: Calculate the reprojection residual function and scaling residual function of the sliding window, and sum them with weights to obtain the total residual function of the sliding window. The process is as follows:

[0139] Step S51: Calculate the reprojection residual function based on the feature points, feature point depth values, and pose of the image frames within the sliding window. , represented as ;in, and These represent the first and second elements within the sliding window, respectively. The first image frame Each feature point and its depth measurement; Indicates the first [number]th ... The pose of the image frame; This represents the projection function, which utilizes the first... pose of the image frame Measure the depth value of feature points Project to the Zhang image frame; Indicates the first [number]th ... Number of feature points per image frame. Reprojection residual function. As a geometric consistency constraint, the pose, feature point depth measurement value, and pose of an image frame are associated through a projection function, guiding the subsequent iterative optimization of the pose and feature point depth measurement value of the image frame.

[0140] Step S52: Calculate the scale residual function based on the scale grid, relative depth values ​​of feature points, and depth measurement values ​​of feature points within the image frame in the sliding window. , represented as ,in, Represents scaled grid Feature points obtained by bilinear interpolation The scale factor, For the first The first image frame Feature points The corresponding feature point relative depth value. Scale residual function. By correlating the scale grid, the relative depth values ​​of feature points, and the metric depth values ​​of feature points, subsequent iterative optimization of the scale grid is guided, addressing the issue of the relative depth values ​​of feature points lacking a true physical scale. Using bilinear interpolation, the scale factor of the scale grid at any location can be obtained, enabling pixel-by-pixel scale factor calculation.

[0141] S53. The weighted summation of the reprojection residual function and the scaling residual function yields the total residual function. , represented as ,in, and These represent preset weighting coefficients used to balance the relative contributions of reprojection constraints and scale constraints. The weighted summation of the reprojection residual and the scale residual is used to obtain the total residual function, thereby unifying the mathematical forms of multi-view geometric constraints and single-frame scale constraints. This provides an objective function for subsequent iterative optimization to ensure positioning accuracy and scale consistency.

[0142] Step S6: Calculate the Hessian matrices of the optimization variables based on the total residual function, including the metric depth Hessian matrix, pose Hessian matrix, and scale grid Hessian matrix; wherein, Laplacian smoothing regularization is applied to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix. The process is as follows:

[0143] Step S61: Calculate the total residual function Regarding the depth value of feature points , pose and scale grid The first-order gradients are respectively expressed as: , and ;

[0144] The total residual function is further calculated based on the first-order gradient. Regarding the depth value of feature points , pose and scale grid The second-order gradients are expressed as follows: , and ,in Represents the matrix transpose operation;

[0145] according to , and The first-order gradient is used to calculate the depth-pose cross term matrix. Measure depth value - scale grid intersection matrix Pose-scale mesh cross term matrix ;

[0146] Using the second gradient of the optimization variable as the Hessian matrix, we obtain the metric depth Hessian matrix of the optimization variable. Pose Hessian matrix and scale grid Hessian matrix ;

[0147] Step S62: Adjust the scale grid Hessian matrix. Applying Laplacian smoothing regularization yields the Hessian matrix of the regularized scale grid. , represented as ,in, This represents the first Laplacian smoothing regularization matrix applied to the coarse mesh; This represents the second Laplacian smoothing regularization matrix applied to the fine mesh; and These represent the preset first and second smoothing weights, respectively. In this embodiment, the first smoothing weight is applied to the coarse grid. The second smoothing weight for fine mesh Laplace smoothing regularization forces the scaled grid to remain smooth during iterative optimization, suppresses local scale abrupt changes caused by sparse observation noise, enhances the spatial continuity of the scaled grid, and improves the numerical stability of subsequent iterative optimization processes.

[0148] Step S7: Calculate the diagonal dominance ratio of the regularized scaled Hessian matrix, calculate the additional damping based on the diagonal dominance ratio, and apply this additional damping to the regularized scaled Hessian matrix; assemble the depth Hessian matrix, the pose Hessian matrix, and the regularized scaled Hessian matrix with the applied additional damping into the final Hessian matrix. The process is as follows:

[0149] Step S71: Based on the regularized scale grid Hessian matrix Computational Scale Grid The Diagonal dominance of each parameter , Represented as: , where index satisfy ,index satisfy , express The first in the matrix Line 1 The diagonal element of the column, express The first in the matrix Line 1 The off-diagonal elements of the column, j≠n; This indicates the operation of finding the maximum absolute value.

[0150] Step S72: Calculate the degree of weak constraint of the scale mesh based on the diagonal dominance ratio; calculate the median of the diagonal dominance ratio. Calculate the absolute deviation of the median Computational scale grid The The degree of weak constraint on each parameter , Represented as ,in The cutoff threshold for the degree of weak constraint; This represents the operation for finding the median; This represents the operation of finding the minimum value;

[0151] Computational Scale Grid The Additional damping for each parameter , Represented as ,in, It is the basic damping coefficient.

[0152] Step S73: Adjust the Hessian matrix of the regularized scale grid. By applying additional damping, the scaled grid Hessian matrix after applying additional damping is obtained. , represented as ,in, The depth Hessian matrix will be used to measure this. Pose Hessian matrix The scaled grid Hessian matrix after applying additional damping Measure depth value-pose cross term matrix Pose-scale mesh cross term matrix And the depth value-scale grid intersection matrix Assembled into the final Hessian matrix , represented as ,in,() T This represents the matrix transpose operation.

[0153] Step S8: Iteratively optimize the optimization variables. The iteration process is divided into two stages. In the first stage, a one-stage optimization variable is constructed, consisting of the feature point depth measurement values, pose, and coarse mesh of the seven image frames of the sliding window. A one-stage Hessian matrix is ​​extracted from the final Hessian matrix, and a one-stage incremental equation is constructed using the LM algorithm. This one-stage incremental equation is iteratively solved to obtain the optimized coarse mesh. In the second stage, the optimized coarse mesh is assigned to the fine mesh, and a two-stage optimization variable is constructed, consisting of the feature point depth measurement values, pose, and fine mesh of the seven image frames of the sliding window. A two-stage Hessian matrix is ​​extracted from the final Hessian matrix, and a two-stage incremental equation is constructed. This two-stage incremental equation is iteratively solved to obtain the optimized pose and fine mesh of the image frames. The process is as follows:

[0154] Step S81: Enter the first stage of iterative optimization; construct the depth measurement value of the image frames within the sliding window. Camera pose and coarse grid The first stage of optimization variables , represented as ;

[0155] In the final Hessian matrix Extracting the first-stage Hessian matrix The LM algorithm is used to solve the incremental equations. ,in It is the damping factor; The gradient for the first stage is denoted as... ,in The gradient for the coarse grid;

[0156] After 10 iterations of the LM algorithm, an optimized coarse mesh is obtained. The first phase of iterative optimization has been completed.

[0157] Step S82: Enter the second stage of iterative optimization; the optimized coarse mesh... Assign to fine mesh Construct a depth metric from the image frames within the sliding window. Camera pose and fine mesh The second-stage optimization variables consist of , represented as

[0158] In the final Hessian matrix Extracting the second-stage Hessian matrix The LM algorithm is used to solve the incremental equations. ,in The second-stage gradient is represented as... ,in The gradient for the fine mesh;

[0159] After 10 iterations of the LM algorithm, the optimized pose of the image frame is obtained. With fine mesh The second phase of iterative optimization has been completed.

[0160] Step S9: When a new image frame is acquired, repeat steps S1 to S8 until all image frames are processed, obtaining the optimized pose and fine mesh of all image frames, as well as the relative depth map of all image frames; based on the optimized fine mesh, obtain the scale map of all image frames; based on the scale map and relative depth map of all image frames, obtain the metric depth map of all image frames; based on the metric depth map of all image frames and the optimized pose, construct a 3D visual map by fusing the metric depth map and optimized pose of all image frames. The process is as follows:

[0161] Step S91: When a new image frame is acquired, repeat steps S1 to S8. Taking the office0 scene as an example, this experimental sequence contains a total of 2000 images until all images are processed. From the image frames, we can obtain all of them. Optimized pose of Zhang image frames All Fine mesh after image frame optimization and all Relative depth map after image frame optimization ,in, ,and Far greater than .

[0162] Step S92, for all The first image frame Zhang image frames, among which , define the first The fine grid after image frame optimization is , for the The fine grid of the image frame In pixel coordinates Perform bilinear interpolation at point 1 to obtain the first... Image frames at pixel coordinates scale at the location , represented as ,in This represents a bilinear interpolation operation; for the ... Perform bilinear interpolation on all pixel coordinates of the first image frame to obtain the second... Scale map of image frames Bilinear interpolation smoothly extends the scale factor of fine-grid nodes to the entire pixel domain, resulting in a spatially continuous and uninterrupted scale map;

[0163] Step S93, Definition For the first The relative depth map of the first image frame will be... Scale map of image frames With the Relative depth map of image frames Multiplying pixel by pixel, we get the first pixel. Measurement depth map of image frames , represented as For all Perform this step on each image frame to obtain all of them. Measurement depth map of image frames ;

[0164] Step S94: Merge all Measurement depth map of image frames , pose A 3D visual map is constructed, completing the visual mapping. The fusion process is as follows: The 3D space is discretized into a voxel mesh with a voxel resolution of 0.008 m. Each voxel stores its signed distance to the nearest object surface and its weight, with a truncation distance of 0.032 m. When a depth map is input, the voxels are projected onto the image plane to obtain the observation distance. The truncation distance is calculated and weighted, and then fused with the current truncation distance value for updating. The zero isosurface is the reconstructed surface. After fusion, isosurfaces are extracted with a weight threshold of 2.0. This method, through truncation and weighted averaging, can effectively filter out single-frame noise and accumulate multi-view observations, achieving dense 3D scene visual mapping.

[0165] To verify the effectiveness of this embodiment in mapping under scaled grid size reduction and Laplacian smoothing and damping parameter adjustment, the depth estimation accuracy was evaluated on three scenes of the Replica dataset: office0, office1, and room0.

[0166] 1. Depth Estimation Accuracy Evaluation: Using the real depth map provided by the dataset as a reference, six metrics were calculated to measure the recovered depth map. Evaluation results for each scenario are shown in Table 3.

[0167] Table 3. Depth estimation metrics for the three sequences in the Replica dataset

[0168]

[0169] As shown in Table 3, under the parameter configuration of Example 2, the method of the present invention still achieved stable depth recovery results in the office0, office1 and room0 scenarios of the Replica dataset, verifying the adaptability of the method to different dataset scenarios and different scale grid parameter configurations.

[0170] 2. Visualization Results and Analysis:

[0171] Figure 8The paper presents four representative image frames from the office0 scene and their corresponding depth maps. Compared to Example 1, this example reduces the number of scale grid parameters from 45 to 29. The depth maps show that even with a significant reduction in parameter size, the recovered depth maps effectively capture the main geometric structures in the office scene (such as walls, floors, tables, and chairs), and the depth values ​​have the correct physical scale. This demonstrates that the two-level scale grid of this invention has good parameter efficiency and generalization.

[0172] Figure 9 The diagram shows four representative image frames and their scale maps. The scale map was initially optimized using a 2×2 coarse grid, then upsampled to a 5×5 fine grid, and further optimized in a second stage. Finally, it was extended to a full-image resolution of 1200×680 pixels using bilinear interpolation. The scale map shows that the fine grid, with 5×5=25 node parameters, achieves a spatially continuous but non-uniform scale field distribution. Although the number of parameters is reduced compared to the 6×6=36 nodes in Example 1, the scale field maintains a smooth transition between nodes through Laplacian smoothing regularization, avoiding local oscillations in large-scale scenes. Simultaneously, reducing the base damping coefficient from 100 to 50 decreases the additional damping of weakly constrained nodes, allowing scale estimation in weakly textured regions to respond more flexibly to observation information and facilitating faster convergence to a reasonable scale value.

[0173] Figure 10 The image shows 25 parameter values ​​optimized using a 5×5 fine grid for two representative image frames, with each grid point labeled with a specific scale factor value. Figure 10 Compared with Example 1 Figure 5 By comparing the 6×6 fine mesh, two points can be observed: (1) The 5×5 mesh can also depict the spatial variation trend of the scale factor, indicating that the reasonable reduction of the parameter size will not lead to a significant degradation in scale modeling ability; (2) The scale values ​​are not uniform within the same frame, reflecting the coupling relationship between the camera viewpoint and the scene depth in the Replica scene. This further verifies that the present invention can still effectively solve high-dimensional scale fields under the condition of a significant reduction in the parameter space dimension, taking into account both modeling accuracy and computational efficiency.

[0174] Figure 11 The image shows the reconstruction results obtained by using a fusion algorithm to reconstruct the metric depth maps of the office0 and office1 scenes into 3D. The reconstruction results show that the reconstructed 3D model accurately reproduces the main structures of the office, such as walls, floors, desks, chairs, and shelves, verifying that the method of this invention has stable 3D visual mapping capabilities.

[0175] Example 3

[0176] Reference Figure 12This embodiment provides a mobile robot visual mapping system for executing a mobile robot visual mapping method disclosed in the above embodiment. The mobile robot visual mapping system includes, in sequence, an image frame acquisition module 1201, a monocular depth estimation module 1202, a scale grid module 1203, a sliding window module 1204, a total residual function calculation module 1205, a regularized scale grid Hessian matrix calculation module 1206, a final Hessian matrix calculation module 1207, an iterative optimization module 1208, and a visual mapping module 1209, wherein:

[0177] The image frame acquisition module 1201 acquires image frames through a camera mounted on a mobile robot, and performs feature extraction and feature matching to obtain the feature points, feature point depth values, and pose of the image frame.

[0178] The monocular depth estimation module 1202 is used to perform monocular depth prediction on the image frame to obtain the relative depth map of the image frame and the relative depth values ​​of the feature points.

[0179] The scale grid module 1203 is used to construct the scale grid of the image frame and initialize the scale grid, wherein the scale grid includes a coarse grid and a fine grid;

[0180] The sliding window module 1204 is used to construct a sliding window whose parameters include the most recently acquired data. The image frame contains feature points, feature point depth measurements, feature point relative depth values, relative depth maps, pose, and scale grids. W is the maximum number of image frames in the sliding window; construct the optimization variable, which is composed of... The feature points of an image frame are composed of depth values, pose, and scale grids.

[0181] The total residual function calculation module 1205 is used to calculate the reprojection residual function and the scaling residual function of the sliding window and then perform a weighted summation to obtain the total residual function of the sliding window.

[0182] The regularized scale grid Hessian matrix calculation module 1206 is used to calculate the Hessian matrix of the optimization variables based on the total residual function. The Hessian matrix of the optimization variables includes the metric depth Hessian matrix, the pose Hessian matrix, and the scale grid Hessian matrix. Laplacian smoothing regularization is applied to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix.

[0183] The final Hessian matrix calculation module 1207 assembles the depth Hessian matrix, the pose Hessian matrix, and the regularized scaled grid Hessian matrix after applying additional damping into the final Hessian matrix;

[0184] The iterative optimization module 1208 is used to iteratively optimize the optimization variables in two stages to obtain the optimized coarse mesh, the pose of the image frame, and the fine mesh, respectively.

[0185] The visual mapping module 1209 is used to obtain the scale map of all image frames after all image frames have been optimized, based on the optimized fine mesh; to obtain the metric depth map of all image frames based on the scale map and relative depth map of all image frames; and to construct a 3D visual map by fusing the metric depth map of all image frames with the optimized pose.

[0186] Example 4

[0187] This embodiment provides an electronic device, which can be a computer, such as... Figure 13 As shown, the system bus 1301 connects a processor 1302, a memory, an input device 1303, a display 1304, and a network interface 1305. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 1306 and internal memory 1307. The non-volatile storage medium 1306 stores an operating system, computer programs, and a database. The internal memory 1307 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor 1302 executes the computer programs stored in the memory, it implements the mobile robot visual mapping method proposed in Embodiment 1. The mobile robot visual mapping method includes the following steps:

[0188] S1. Image frames are acquired by a camera mounted on a mobile robot, and feature extraction and feature matching are performed to obtain the feature points, feature point depth values ​​and pose of the image frames.

[0189] S2. Perform monocular depth prediction on the image frame to obtain the relative depth map of the image frame and the relative depth values ​​of the feature points;

[0190] S3. Construct the scale grid of the image frame and initialize the scale grid, which includes coarse and fine grids;

[0191] S4. Construct a sliding window whose parameters include the latest data. Feature points of each image frame, feature point depth values, relative feature point depth values, relative depth maps, pose, and scale grid; construct optimization variables, which are derived from... The feature points of an image frame are composed of depth values, pose, and scale grids.

[0192] S5. Calculate the reprojection residual function and scaling residual function of the sliding window and sum them by weight to obtain the total residual function of the sliding window;

[0193] S6. Based on the total residual function, calculate the Hessian matrix of the optimization variables, which includes the metric depth Hessian matrix, the pose Hessian matrix, and the scale grid Hessian matrix; apply Laplacian smoothing regularization to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix.

[0194] S7. Assemble the depth Hessian matrix, pose Hessian matrix, and regularized scaled grid Hessian matrix after applying additional damping into the final Hessian matrix.

[0195] S8. Iteratively optimize the optimization variables in two stages to obtain the optimized coarse mesh, the pose of the image frame, and the fine mesh, respectively.

[0196] S9. After all image frames are optimized, the scale map of all image frames is obtained based on the optimized fine mesh; the metric depth map of all image frames is obtained based on the scale map and relative depth map of all image frames; and a 3D visual map is constructed by fusing the metric depth map of all image frames with the optimized pose.

[0197] Example 5

[0198] This embodiment provides a storable medium, which is a computer-readable and storable medium, storing a computer program. When the computer program is executed by a processor, it implements a mobile robot visual mapping method proposed in Embodiment 1 above. The mobile robot visual mapping method includes the following steps:

[0199] S1. Image frames are acquired by a camera mounted on a mobile robot, and feature extraction and feature matching are performed to obtain the feature points, feature point depth values ​​and pose of the image frames.

[0200] S2. Perform monocular depth prediction on the image frame to obtain the relative depth map of the image frame and the relative depth values ​​of the feature points;

[0201] S3. Construct the scale grid of the image frame and initialize the scale grid, which includes coarse and fine grids;

[0202] S4. Construct a sliding window whose parameters include the latest data. Feature points of each image frame, feature point depth values, relative feature point depth values, relative depth maps, pose, and scale grid; construct optimization variables, which are derived from... The feature points of an image frame are composed of depth values, pose, and scale grids.

[0203] S5. Calculate the reprojection residual function and scaling residual function of the sliding window and sum them by weight to obtain the total residual function of the sliding window;

[0204] S6. Based on the total residual function, calculate the Hessian matrix of the optimization variables, which includes the metric depth Hessian matrix, the pose Hessian matrix, and the scale grid Hessian matrix; apply Laplacian smoothing regularization to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix.

[0205] S7. Assemble the depth Hessian matrix, pose Hessian matrix, and regularized scaled grid Hessian matrix after applying additional damping into the final Hessian matrix.

[0206] S8. Iteratively optimize the optimization variables in two stages to obtain the optimized coarse mesh, the pose of the image frame, and the fine mesh, respectively.

[0207] S9. After all image frames are optimized, the scale map of all image frames is obtained based on the optimized fine mesh; the metric depth map of all image frames is obtained based on the scale map and relative depth map of all image frames; and a 3D visual map is constructed by fusing the metric depth map of all image frames with the optimized pose.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A visual mapping method for mobile robots, characterized in that, The mapping method includes the following steps: S1. Image frames are acquired by a camera mounted on a mobile robot, and feature extraction and feature matching are performed to obtain the feature points, feature point depth values ​​and pose of the image frames. S2. Perform monocular depth prediction on the image frame to obtain the relative depth map of the image frame and the relative depth values ​​of the feature points; S3. Construct the scale grid of the image frame and initialize the scale grid, which includes coarse and fine grids; S4. Construct a sliding window whose parameters include the latest data. Feature points of each image frame, feature point depth values, relative feature point depth values, relative depth maps, pose, and scale grid; construct optimization variables, which are derived from... The feature points of an image frame are composed of depth values, pose, and scale grids. S5. Calculate the reprojection residual function and scaling residual function of the sliding window and sum them by weight to obtain the total residual function of the sliding window; S6. Based on the total residual function, calculate the Hessian matrix of the optimization variables, which includes the metric depth Hessian matrix, the pose Hessian matrix, and the scale grid Hessian matrix; apply Laplacian smoothing regularization to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix. S7. Assemble the depth Hessian matrix, pose Hessian matrix, and regularized scaled grid Hessian matrix after applying additional damping into the final Hessian matrix. S8. The optimization variables are iteratively optimized in two stages to obtain the optimized coarse mesh, the pose of the image frame, and the fine mesh, as follows: S81. Enter the first stage of iterative optimization, construct the first stage optimization variables consisting of the depth measurement value of the image frame within the sliding window, the camera pose, and the coarse mesh, extract the first stage Hessian matrix from the final Hessian matrix, construct the first stage incremental equation based on the first stage optimization variables and the first stage Hessian matrix, iteratively solve the first stage incremental equation, and obtain the optimized coarse mesh. S82. Enter the second stage of iterative optimization, construct the second stage optimization variables consisting of the depth measurement value of the image frame within the sliding window, the camera pose, and the fine mesh. Extract the second stage Hessian matrix from the final Hessian matrix. Construct the second stage incremental equation based on the second stage optimization variables and the second stage Hessian matrix. Iteratively solve the second stage incremental equation to obtain the optimized image frame pose and fine mesh. S9. After all image frames are optimized, the scale map of all image frames is obtained based on the optimized fine mesh; the metric depth map of all image frames is obtained based on the scale map and relative depth map of all image frames; and a 3D visual map is constructed by fusing the metric depth map of all image frames with the optimized pose.

2. The mobile robot visual mapping method according to claim 1, characterized in that, Step S3 is as follows: S31. Construct the scale grid of the image frame. The scale grid of the image frame consists of two levels. The first level is a coarse grid, containing... One parameter; the second level is a fine mesh, containing There are parameters, and the number of parameters satisfies... ; S32. Initialize the coarse grid to the mean of the ratio of the feature point depth measurement value to the relative depth value of the feature point in the image frame; The fine mesh is initialized to 1.

0.

3. The mobile robot visual mapping method according to claim 1, characterized in that, Step S5 is as follows: S51. Calculate the reprojection residual function based on the feature points, feature point depth values, and pose of the image frames within the sliding window. S52. Calculate the scale residual function based on the scale grid, relative depth values ​​of feature points, and depth measurement values ​​of feature points of the image frames within the sliding window; S53. The total residual function is obtained by weighted summation of the reprojection residual function and the scaling residual function.

4. The mobile robot visual mapping method according to claim 1, characterized in that, Step S6 is as follows: S61. Calculate the gradient of the total residual function with respect to the feature point metric depth value, pose, and scale grid; calculate the second-order gradient and cross terms of the metric depth value, pose, and scale grid based on the gradient, and obtain the metric depth Hessian matrix, pose Hessian matrix, scale grid Hessian matrix, metric depth value-pose cross term matrix, pose-scale grid cross term matrix, and metric depth value-scale grid cross term matrix. S62. Apply Laplacian smoothing regularization to the scaled grid Hessian matrix; wherein, a first Laplacian smoothing regularization is applied to the coarse grid and a second Laplacian smoothing regularization is applied to the fine grid to obtain the regularized scaled grid Hessian matrix.

5. The mobile robot visual mapping method according to claim 4, characterized in that, Step S7 is as follows: S71. Calculate the diagonal dominance ratio based on the diagonal and off-diagonal elements of the Hessian matrix of the regularized scale grid. S72. Calculate the degree of weak constraint of the scale grid based on the diagonal dominance ratio, and calculate the additional damping based on the degree of weak constraint; S73. Apply additional damping to the Hessian matrix of the regularized scaled grid; The Hessian matrix for metric depth, pose, scale grid after applying additional damping, metric depth-pose cross term matrix, pose-scale grid cross term matrix, and metric depth-scale grid cross term matrix are assembled into the final Hessian matrix.

6. The mobile robot visual mapping method according to claim 1, characterized in that, Step S9 is as follows: S91. When a new image frame is acquired, repeat steps S1 to S8 to obtain the optimized pose and fine mesh of all image frames and the relative depth map of all image frames. S92. Based on the optimized fine grid, perform bilinear interpolation on all pixel positions of the image frame to obtain the scale map of all image frames. S93. Based on the scale map and relative depth map of all image frames, multiply them pixel by pixel to obtain the metric depth map of all image frames. S94. By fusing the metric depth map of all image frames and the optimized pose, a three-dimensional visual map is constructed.

7. A mobile robot visual mapping system, used to execute the mobile robot visual mapping method according to any one of claims 1 to 6, characterized in that, The mapping system includes: The image frame acquisition module acquires image frames through a camera mounted on a mobile robot, and performs feature extraction and feature matching to obtain the feature points, feature point depth values, and pose of the image frame. The monocular depth estimation module performs monocular depth prediction on the image frame to obtain the relative depth map of the image frame and the relative depth values ​​of the feature points; The scale grid module is used to construct the scale grid of the image frame and initialize the scale grid, which includes coarse and fine grids; The sliding window module is used to construct a sliding window whose parameters include the most recently acquired data. The image frame contains feature points, feature point depth measurements, feature point relative depth values, relative depth maps, pose, and scale grids. W is the maximum number of image frames in the sliding window; construct the optimization variable, which is composed of... The feature points of an image frame are composed of depth values, pose, and scale grids. The total residual function calculation module is used to calculate the reprojection residual function and the scaling residual function of the sliding window and then sum them by weight to obtain the total residual function of the sliding window. The module for calculating the regularized scale grid Hessian matrix is ​​used to calculate the Hessian matrix of the optimization variables based on the total residual function. The Hessian matrix of the optimization variables includes the metric depth Hessian matrix, the pose Hessian matrix, and the scale grid Hessian matrix. The module applies Laplacian smoothing regularization to the scale grid Hessian matrix to obtain the regularized scale grid Hessian matrix. The final Hessian matrix calculation module assembles the depth Hessian matrix, the pose Hessian matrix, and the regularized scaled grid Hessian matrix after applying additional damping into the final Hessian matrix; The iterative optimization module is used to iteratively optimize the optimization variables in two stages to obtain the optimized coarse mesh, the pose of the image frame, and the fine mesh, respectively. The visual mapping module is used to obtain the scale map of all image frames after optimization, based on the optimized fine mesh; to obtain the metric depth map of all image frames based on the scale map and relative depth map of all image frames; and to construct a 3D visual map by fusing the metric depth map of all image frames with the optimized pose.

8. An electronic device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the mobile robot visual mapping method according to any one of claims 1 to 6.

9. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the mobile robot visual mapping method according to any one of claims 1 to 6.

Images