Slam method for indoor mobile robots

Abstract

This invention relates to the field of autonomous navigation technology for indoor mobile robots, specifically disclosing a SLAM method for indoor mobile robots, comprising: acquiring environmental image information of the indoor mobile robot; performing ray sampling and attention-based encoding fusion on an RGB-D image sequence to obtain first optimized parameter information; performing pose optimization processing on the initial pose information according to a target total loss function to obtain tracked frames with pose information; determining keyframes for mapping based on the tracked frames; performing ray sampling and attention-based encoding fusion on the keyframes to obtain second optimized parameter information; and performing joint pose and mapping optimization processing on the second optimized parameter information according to the target total loss function to obtain pose and map construction results. The SLAM method for indoor mobile robots provided by this invention can improve positioning accuracy in complex indoor environments.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation technology for indoor mobile robots, and more particularly to a SLAM method for indoor mobile robots. Background Technology

[0002] For current indoor mobile robots, such as service robots used in hotel delivery and hospital disinfection, the real-time localization and high-precision map construction in complex indoor environments adopt the traditional indoor SLAM method. However, the traditional indoor SLAM method is prone to tracking loss in environments with dynamic lighting or low texture areas (such as solid color corridors), which leads to localization failure.

[0003] While existing dense SLAM methods can achieve indoor localization and tracking, current dense SLAM techniques, such as Co-SLAM, tend to overly smooth out indoor elements like plants and water bottles, which should be modeled with coarse yet fine detail. Furthermore, Co-SLAM selects one keyframe every five frames. However, as the number of keyframes increases, the computational cost of traversing the keyframe database becomes unacceptable. Therefore, while existing dense SLAM methods can achieve indoor localization, they still suffer from low localization accuracy.

[0004] Therefore, how to provide a SLAM method suitable for complex indoor environments to improve positioning accuracy in such environments has become an urgent problem for those skilled in the art. Summary of the Invention

[0005] This invention provides a SLAM method for indoor mobile robots, which solves the problem of low positioning accuracy of current SLAM methods in complex indoor environments.

[0006] As a first aspect of the present invention, a SLAM method for an indoor mobile robot is provided, comprising:

[0007] Acquire environmental image information of an indoor mobile robot, wherein the environmental image information includes an RGB-D image sequence formed by RGB image information and depth image information;

[0008] Ray sampling and attention-based encoding are fused to the RGB-D image sequence to obtain first optimization parameter information, which includes at least initial pose information and map building performance parameter information.

[0009] The initial pose information is optimized according to the target total loss function to obtain the tracked frame with pose information.

[0010] Based on the tracked frames, key frames for mapping are determined;

[0011] Ray sampling and attention-based encoding are fused based on the keyframes to obtain second optimization parameter information, which includes at least updated pose information and map building performance parameter information.

[0012] Based on the target total loss function, the second optimization parameter information is subjected to joint optimization processing of pose and mapping to obtain the pose and map construction results.

[0013] Further, the RGB-D image sequence is ray-sampled and fused with an attention-based encoding mechanism to obtain first optimized parameter information, including:

[0014] The RGB-D image sequence is preprocessed and then stored in a multi-resolution hash feature grid.

[0015] Multiple 3D environmental information sampling points are obtained by performing ray sampling in a multi-resolution hash feature grid based on NeRF rays;

[0016] The three-dimensional sampling points of the environmental information are encoded and fused based on an attention mechanism to obtain the first optimized parameter information.

[0017] Furthermore, the three-dimensional sampling points of the environmental information are encoded and fused based on an attention mechanism, including:

[0018] The three-dimensional sampling points of the environmental information are encoded according to parameter encoding and coordinate encoding respectively to obtain parameter encoding results and coordinate encoding results;

[0019] The parameter encoding result and the coordinate encoding result are respectively input into the first geometric decoder to obtain the first geometric decoding feature, and the parameter encoding result and the coordinate encoding result are respectively input into the second geometric decoder to obtain the second geometric decoding feature;

[0020] The first geometric decoding feature and the coordinate encoding result are both input to the first additional decoder to obtain the first additional decoding feature, and the first geometric decoding feature is input to the second additional decoder to obtain the second additional decoding feature;

[0021] SDF features and color features are determined based on attention weights;

[0022] The first optimization parameter information is determined based on the SDF features and color features.

[0023] Furthermore, SDF features and color features are determined based on attention weights, including:

[0024] Attention weights are determined based on a multilayer perceptron, wherein the expression for the multilayer perceptron is:

[0025] ,

[0026] in, Represents the learning parameters, and This represents the attention weights between the two inputs. This is achieved using the Softmax activation function.

[0027] Calculate the SDF feature value s and the color feature value c based on the attention weights:

[0028] .

[0029] Further, the initial pose information is subjected to pose optimization processing based on the target total loss function, including:

[0030] The target total loss function is determined based on the SDF features and color features;

[0031] The initial pose information is input into the pose optimizer, and the target total loss function is minimized to obtain the tracked frame with pose information.

[0032] Further, the target total loss function is determined based on the SDF features and color features, including:

[0033] Determine the color and depth loss functions based on the SDF features and the color features;

[0034] Determine the SDF loss function and free space loss function based on the SDF features;

[0035] Determine the photometric warpage loss function based on pose information;

[0036] The target total loss function is determined based on the color and depth loss function, SDF loss function, free space loss function, and photometric warping loss function.

[0037] Further, based on the tracked frames, keyframes for mapping are determined, including:

[0038] Iterate through the database of the tracked frames;

[0039] Divide the index of the currently tracked frame by 5. If the index of the currently tracked frame is divisible by 5, then the currently tracked frame is determined as a key frame.

[0040] Store the selected keyframes into the keyframe database.

[0041] Further, based on the keyframes, ray sampling and attention-based encoding are fused to obtain second optimized parameter information, including:

[0042] Keyframes are selected from the keyframe database according to the segmented keyframe selection method.

[0043] The selected keyframes are ray-sampled to obtain multiple 3D sampling points for the keyframes.

[0044] The keyframe 3D sampling points are encoded and fused based on an attention mechanism to obtain the second optimized parameter information.

[0045] Furthermore, keyframes are selected from the keyframe database according to the segmented keyframe selection method, including:

[0046] Determine if the number of keyframes in the keyframe database is less than [number missing]. indivual;

[0047] If less If there are 1, then randomly select 1 from all keyframes in the keyframe database. BA optimization is performed on the ray;

[0048] If not less than If there are 1, then calculate the pose difference of all keyframes in the keyframe database;

[0049] After sorting the pose differences of all keyframes in descending order, select the first one. Individual keyframes.

[0050] Further, based on the target total loss function, the second optimization parameter information is subjected to joint optimization processing for pose and mapping to obtain pose and map construction results, including:

[0051] The updated pose information and map building performance parameters are input into the pose and map joint optimizer.

[0052] The pose and map in the pose and map joint optimizer are jointly optimized according to the target total loss function to obtain the pose and map construction results.

[0053] The SLAM method for indoor mobile robots provided by this invention acquires RGB-D image sequences and performs ray sampling and encoding fusion on these sequences to optimize pose. Based on the optimized pose, keyframes are determined, and ray sampling and encoding fusion are then performed on these keyframes to achieve joint optimization of pose and map. This SLAM method for indoor mobile robots effectively utilizes the advantages of each specific encoding method through attention-based encoding fusion, thus significantly improving localization accuracy in indoor scenes. Furthermore, by first optimizing the pose and then jointly optimizing the pose and map, tracking accuracy is effectively improved, thereby enhancing indoor localization accuracy. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0055] Figure 1 A flowchart of the SLAM method for indoor mobile robots provided by the present invention.

[0056] Figure 2 This is a schematic diagram illustrating the specific implementation process of the SLAM method for indoor mobile robots provided by the present invention.

[0057] Figure 3 The flowchart for obtaining the first optimized parameter information provided by the present invention.

[0058] Figure 4 The flowchart of the method for encoding and fusing three-dimensional environmental information sampling points based on an attention mechanism provided by the present invention is shown.

[0059] Figure 5 This is a flowchart of pose optimization processing based on the target total loss function provided by the present invention.

[0060] Figure 6 A flowchart for determining the target total loss function provided by the present invention. Detailed Implementation

[0061] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0062] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0063] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0064] This embodiment provides a SLAM method for indoor mobile robots. Figure 1 This is a flowchart of a SLAM method for indoor mobile robots provided according to an embodiment of the present invention, such as... Figure 1 As shown, it includes:

[0065] S100: Acquire environmental image information of the indoor mobile robot, wherein the environmental image information includes an RGB-D image sequence formed by RGB image information and depth image information;

[0066] In this embodiment of the invention, RGB images and depth maps are acquired by an RGB-D camera at a frame rate of 30Hz. The sequence of these images is called an RGB-D image sequence. Specifically, the RGB-D image sequence... That is, color images and depth maps.

[0067] S200. Ray sampling and attention-based encoding are performed on the RGB-D image sequence to obtain first optimization parameter information, wherein the first optimization parameter information includes at least initial pose information and map building performance parameter information.

[0068] In this embodiment of the invention, three-dimensional environmental information sampling points are obtained by ray sampling of the RGB-D image sequence, and the environmental information three-dimensional sampling points are encoded and fused based on an attention mechanism to obtain first optimized parameter information. The first optimized parameter information specifically includes initial pose information and map construction performance parameter information.

[0069] During the tracking process, the initial pose information is first optimized based on the target loss function to obtain the tracked frame with pose information.

[0070] S300. Perform pose optimization processing on the initial pose information according to the target total loss function to obtain a tracked frame with pose information.

[0071] Specifically, the target total loss function is determined based on the first optimization parameter information, and then the pose optimization processing of the initial pose information is realized based on the target total loss function.

[0072] S400. Determine the key frames for mapping based on the tracked frames;

[0073] Specifically, keyframes are selected from the tracked frames obtained above to obtain keyframes for mapping.

[0074] S500. Based on the keyframe, perform ray sampling and attention-based encoding fusion to obtain second optimization parameter information, which includes at least updated pose information and map building performance parameter information.

[0075] Specifically, the same ray sampling and attention-based encoding fusion are performed based on the selected keyframes as described above. The pose information included in the second optimized parameter information obtained here is the updated pose information, that is, the updated pose information after the optimization process described above.

[0076] S600. Perform pose and mapping joint optimization processing on the second optimization parameter information according to the target total loss function to obtain pose and map construction results.

[0077] Specifically, in this embodiment of the invention, the pose and mapping information of the second optimization parameter are still jointly optimized based on the target total loss function to obtain the pose and map construction results.

[0078] In summary, the SLAM method for indoor mobile robots provided by this invention acquires RGB-D image sequences and performs ray sampling and encoding fusion on these sequences to optimize pose. Based on the optimized pose, keyframes are determined, and ray sampling and encoding fusion are then performed on these keyframes to achieve joint optimization of pose and map. This SLAM method for indoor mobile robots effectively utilizes the advantages of each specific encoding method through attention-based encoding fusion, thus significantly improving localization accuracy in indoor scenes. Furthermore, by first optimizing the pose and then jointly optimizing the pose and map, tracking accuracy is effectively improved, thereby enhancing indoor localization accuracy.

[0079] It should be noted that the SLAM method for indoor mobile robots in this embodiment of the invention is mainly applicable to the real-time localization and high-precision map construction of service robots (such as hotel delivery and hospital disinfection robots) in complex indoor environments. Compared with traditional SLAM, which is prone to losing tracking in dynamic lighting or low-texture areas (such as solid color corridors), the attention encoding mechanism of this invention significantly improves the robustness of such scenarios by fusing multi-resolution geometric features and photometric consistency.

[0080] The following is combined Figure 2 The specific implementation process of the SLAM method for indoor mobile robots of the present invention is described in detail.

[0081] As a specific implementation method, ray sampling and attention-based encoding are fused into the RGB-D image sequence to obtain first optimized parameter information, such as... Figure 3 As shown, it includes:

[0082] S210. The RGB-D image sequence is preprocessed and then stored in a multi-resolution hash feature grid;

[0083] Specifically, the RGB-D image sequence is transformed into 3D coordinates by combining depth information with the relative coordinates of RGB image pixels, and then stored in a multi-resolution hash feature grid.

[0084] S220. Based on NeRF rays, ray sampling is performed in a multi-resolution hash feature grid to obtain multiple three-dimensional sampling points for environmental information;

[0085] Specifically, NeRF rays are used for sampling on a resolution hash feature grid, i.e., around the center of the camera. With the origin as the point, facing the direction the camera is pointing. Emit a ray. (Point) At fixed intervals Sampling is performed at the location, such as in This indicates the number of points sampled by the ray. Assume the ray samples points within the grid. The code is obtained through coordinate encoding method. Simultaneously, the sampling features of the resolution hash feature grid are obtained through parameter encoding. ,in This indicates the resolution of the grid.

[0086] S230. The three-dimensional sampling points of the environmental information are encoded and fused based on an attention mechanism to obtain the first optimized parameter information.

[0087] In this embodiment of the invention, attention-based encoding fusion is performed on the three-dimensional sampling points of environmental information to integrate multi-resolution geometric features and photometric consistency, thereby improving the robustness of indoor scenes.

[0088] Specifically, the three-dimensional sampling points of the environmental information are encoded and fused based on an attention mechanism, such as... Figure 4 As shown, it includes:

[0089] S231. Encode the three-dimensional sampling points of the environmental information according to the parameter encoding and coordinate encoding respectively to obtain the parameter encoding result and the coordinate encoding result;

[0090] S232. Input the parameter encoding result and coordinate encoding result into the first geometric decoder to obtain the first geometric decoding feature, and input the parameter encoding result and coordinate encoding result into the second geometric decoder to obtain the second geometric decoding feature;

[0091] In this embodiment of the invention, the parameterized encoding result at sampling point x is... And coordinate encoding results Input to geometry encoder Obtain the first geometric decoding feature, which may specifically include the first signed distance function (SDF) value s1 and the first feature vector h1, and then perform the parameterized encoding of the sampling point x. And coordinate encoding results Input to geometry encoder Output the second geometric decoding feature, which may specifically include the second signed distance function (SDF) value s2 and the second feature vector h2:

[0092] .

[0093] S233. Input the first geometric decoding feature and the coordinate encoding result into the first additional decoder to obtain the first additional decoding feature, and input the first geometric decoding feature into the second additional decoder to obtain the second additional decoding feature;

[0094] In this embodiment of the invention, color features are obtained through two additional decoders. Specifically, the first feature vector and coordinate encoding result from the first geometric decoding feature are input to the first additional decoder to obtain the first color feature c1, and the first feature vector from the first geometric decoding feature is input to the second additional decoder to obtain the second color feature c2, as specifically expressed as follows:

[0095] .

[0096] S234. Determine SDF features and color features based on attention weights;

[0097] In this embodiment of the invention, determining SDF features and color features based on attention weights includes:

[0098] 1) Determine attention weights based on a multilayer perceptron, wherein the expression for the multilayer perceptron is:

[0099] ,

[0100] in, Represents the learning parameters, and This represents the attention weights between the two inputs. This is achieved using the Softmax activation function.

[0101] 2) Calculate the SDF feature value s and color feature value c based on the attention weights:

[0102] .

[0103] It should be understood that this is specifically achieved through a six-layer multilayer perceptron (MLP). To implement the attention mechanism:

[0104] ,

[0105] in, Represents the learnable parameters. and This represents the attention weights of the two inputs. Here, This is achieved using the Softmax activation function. If the weights... A larger attention weight will result in more focused attention on coordinate encoding, leading to a smoother final reconstruction. Finally, the attention weights are used to calculate the SDF value. and color :

[0106] .

[0107] Furthermore, to ensure faster training speeds in this embodiment of the invention, the number of layers in the MLP can be reduced, and a simpler network architecture can be used. Regarding the specific structure of the MLP, the input of the first layer and the output of the fifth layer have a size of 2, followed by the input of the second layer and the output of the fourth layer with a size of 64, with a linear layer of size 128 in between. The output layer uses Softmax. Except for the last layer, all layers are a combination of linear layers and Rectified Linear Units (ReLUs).

[0108] S235. Determine the first optimization parameter information based on the SDF features and color features.

[0109] In this embodiment of the invention, the initial pose information is subjected to pose optimization processing according to the target total loss function, such as... Figure 5 As shown, it includes:

[0110] S310. Determine the target total loss function based on the SDF features and color features;

[0111] It should be noted that depth and color are rendered by sampling along specific rays and feeding the samples into the decoder. Specifically, with the center of the camera... With the origin as the point, facing the direction the camera is pointing. Emit a ray. (Point) At fixed intervals Sampling is performed at the location, such as ,in This refers to the number of points sampled by the ray. Then, after voxel rendering, the predicted SDF value and color for each point are calculated. Specifically, the obtained depth value... With color value :

[0112] ,

[0113] in, It can be obtained through SDF values Calculated:

[0114] ,

[0115] in, This represents the Sigmoid function. Indicates the cutoff distance.

[0116] It should be noted that, The purpose of the calculation formula is to weight the points in the 3D reconstruction. When the SDF value approaches zero, the weights... Reaching its maximum value, therefore, The calculation formula uses the Sigmoid function to balance the importance of points, allowing for significant changes in weights as points converge to the surface. This mechanism ensures that the effect of the cutoff distance is more accurately considered in 3D reconstruction.

[0117] In this embodiment of the invention, the target total loss function is determined based on the SDF features and color features, such as... Figure 6 As shown, it includes:

[0118] S311. Determine the color and depth loss functions based on the SDF features and the color features;

[0119] In this embodiment of the invention, sampling is first performed. The 3D map was optimized by minimizing the color and depth loss functions, which are expressed as follows:

[0120] ,

[0121] in, and These represent the estimated values ​​for color and depth, respectively. and These represent the actual values ​​for color and depth, respectively.

[0122] S312. Determine the SDF loss function and free space loss function based on the SDF features;

[0123] In this embodiment of the invention, considering the case where the sampling point is within the cutoff distance, an SDF loss is defined to fully utilize the predicted SDF value:

[0124] ,

[0125] in, Indicates the number of rays, Indicates direction The estimated values ​​of sampling points within the cutoff distance, in addition... and This represents the relative coordinates of the sampled pixels on the image. For sampling points beyond the truncation distance, free space loss is used to limit the sampling points from moving too far from the surface.

[0126] .

[0127] S313. Determine the photometric warpage loss function based on the pose information;

[0128] In this embodiment of the invention, a new photometric warp loss is introduced, specifically by calculating the loss of sampled image patches to make the 3D map reconstruction of indoor scenes smoother. Specifically, this involves calculating the loss of each pixel on the image... Centered on, in Sample a square image patch within the range Then set the depth value of the image patch to the pixel value. Same. First, the pixels Projected onto another frame Above, below:

[0129] ,

[0130] in, Represents the camera intrinsic parameter matrix. Indicates camera pose. Represents homogeneous coordinates.

[0131] Then, the sampled image patches Projected onto another frame Above, and employing structural similarity loss (SSIM):

[0132] ,

[0133] in, Represents the current set of keyframes One of the keyframes, Indicates the current frame. This indicates the size of the image patch. Embodiments of the present invention remove the visibility mask, which improves 3D smoothing performance. Furthermore, by reducing the size of the sampled image patch, it accommodates partially incomplete pixels preserved by keyframes.

[0134] S314. Determine the target total loss function based on the color and depth loss function, SDF loss function, free space loss function, and photometric warping loss function.

[0135] In summary, the target total loss function is obtained as follows:

[0136] .

[0137] S320. Input the initial pose information into the pose optimizer and perform minimization optimization processing according to the target total loss function to obtain the tracked frame with pose information.

[0138] In this embodiment of the invention, the ultimate goal of camera tracking is to optimize camera pose. ,in and Corresponding to frames respectively The camera's rotation and translation are considered. Specifically, a constant velocity model is first applied to initialize the camera pose, as shown below:

[0139] ,

[0140] Then, the initialized camera pose is fed into the AdamW optimizer. Afterwards, the loss function of the selected keyframes, i.e., the aforementioned total loss function, is minimized to complete the joint optimization of reconstruction and camera motion, and obtain the corresponding pose.

[0141] In this embodiment of the invention, determining key frames for mapping based on the tracked frames includes:

[0142] 1) Traverse the database of the tracked frames;

[0143] 2) Divide the index of the currently tracked frame by 5. If the index of the currently tracked frame is divisible by 5, then the currently tracked frame is determined as a key frame.

[0144] 3) Store the selected keyframes into the keyframe database.

[0145] In this embodiment of the invention, a keyframe is selected every five frames, and a subset of the keyframe's pixels (approximately 5%) and its relative coordinates are... and Add to the keyframe database.

[0146] Specifically, based on the keyframes, ray sampling and attention-based encoding are fused to obtain second optimized parameter information, including:

[0147] 1) Select keyframes from the keyframe database according to the segmented keyframe selection method;

[0148] To reduce the time cost of traversing the keyframe database, this embodiment of the invention selects keyframes from the keyframe database according to a segmented keyframe selection method, including:

[0149] 11) Determine if the number of keyframes in the keyframe database is less than [a certain number]. indivual;

[0150] 12) If less than If there are 1, then randomly select 1 from all keyframes in the keyframe database. BA optimization is performed on the ray;

[0151] 13) If not less than If there are 1, then calculate the pose difference of all keyframes in the keyframe database;

[0152] 14) Sort all keyframe pose differences in descending order and then select the previous one. Individual keyframes.

[0153] It should be understood that if the number of keyframes in the keyframe database is less than [a certain number], [the following applies]. One, randomly selected from all keyframes. Optimize the ray using BA; otherwise, select the previous one. The keyframes with the largest pose differences are selected and maintained in a sorted list. This segmented keyframe selection method saves time traversing the keyframe database, avoids catastrophic omissions, and ensures the accuracy and efficiency of the optimization.

[0154] 2) Perform ray sampling on the selected keyframes to obtain multiple 3D sampling points for the keyframes;

[0155] 3) Perform attention-based encoding fusion on the three-dimensional sampling points of the keyframe to obtain the second optimized parameter information.

[0156] After segmented keyframe selection, NeRF rays are applied to the selected keyframes to sample points in the same manner as before, generating SDF, color, and depth. By minimizing the total target loss function described earlier, the final joint optimization of scene representation and camera pose is achieved, resulting in the final 3D scene map and camera pose, thus completing the SLAM process.

[0157] In this embodiment of the invention, pose and mapping joint optimization processing is performed on the second optimization parameter information according to the target total loss function to obtain pose and map construction results, including:

[0158] 1) Input the updated pose information and map building performance parameters into the pose and map joint optimizer;

[0159] 2) Perform joint optimization of pose and map in the pose and map joint optimizer according to the target total loss function to obtain pose and map construction results.

[0160] It should be noted that after SLAM is completed, the moving cube algorithm is used to extract isosurfaces with a threshold of 0.003m to obtain the final 3D reconstructed map scene.

[0161] The specific effects of the SLAM method for indoor mobile robots according to embodiments of the present invention will be explained below with reference to experiments.

[0162] Specifically, the SLAM method of this embodiment, namely ATE-SLAM, was evaluated on four different indoor scene datasets. Regarding reconstruction performance, the Replica dataset [Straub J, Whelan T, Ma L, et al.The Replica dataset: A digital replica of indoor spaces[J]. arXiv preprintarXiv:1906.05797, 2019.], containing eight indoor scenes, and the NeuralRGBD synthetic dataset [Azinović D, Martin-Brualla R, Goldman DB, et al. Neural rgb-d surface reconstruction[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 6290-6301.], containing seven scenes, were used. The Replica indoor dataset is specifically a large-scale multi-sensory 3D reconstruction database containing various scenes from the real world, such as offices, apartments, and libraries. The NeuralRGBD synthetic RGBD dataset was captured using a Structure IO camera. Regarding camera tracking performance, the ScanNet dataset [Dai A, Chang AX, Savva M, et al. Scannet: Richly-annotated 3d reconstructions of indoor scenes[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 5828-5839.] was used. This dataset contains RGB-D video data from 2.5 million viewpoints collected from various indoor environments, used for annotation of 3D camera pose detection, surface reconstruction, and instance-level semantic segmentation. Furthermore, ScanNet includes over 1500 scanned indoor environments, covering various scenes such as homes, offices, and public spaces.TUM [Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C] / / 2012 IEEE / RSJinternational conference on intelligent robots and systems. IEEE, 2012: 573-580.] is a dataset for evaluating visual odometry and visual SLAM systems. Real-world trajectories were obtained from the TUM dataset using a high-precision motion capture system and a high-speed tracking camera (100 Hz).

[0163] To eliminate noise clouds outside the scene, a mesh culling operation was performed, and a virtual camera view covering the occluded areas was also predicted. After mesh culling, the performance of 3D reconstruction was evaluated by accuracy error, completion error, and completion rate [< 5cm%]. For camera tracking evaluation, the root mean square error of the absolute trajectory error (RMSE) was used. First, two point clouds, P and Q, were uniformly sampled, one from a grid on the real map and the other from a grid on the reconstructed map. These point clouds had the same dimension, |P| = |Q| = 200000. The accuracy error was defined as the average distance between the nearest point on the real map grid and the nearest point on the reconstructed map grid. The mapping metrics are defined as shown in Table 1.

[0164] Table 1 Mapping Evaluation Indicators

[0165] This invention, ATE-SLAM, was implemented on a server equipped with a 2.60GHz Intel Xeon® Platinum 8358 CPU and an NVIDIA RTX 3090 GPU. Regarding segmented keyframe selection, if the keyframe database contains fewer than 300 keyframes for 20 iterations of bundled adjustments, random sampling is used. A ray of light. Regarding camera tracking, it samples 1024 pixels 10 times in the default settings. Additionally, the truncation distance is... Set to 10 cm. For camera pose and dense mapping, use the AdamW optimizer for optimization.

[0166] Compared to Co-SLAM, the ATE-SLAM implementation of this invention shows an overall improvement in mapping performance on the Replica dataset. Specifically, Co-SLAM achieved an average completion rate of 93.44%. As shown in Table 2, the ATE-SLAM implementation of this invention slightly improved the completion error, reaching 93.66%. This improvement is also reflected in the accuracy error and completion error, reaching 2.00 cm and 2.07 cm, respectively. The ATE-SLAM implementation of this invention also significantly outperforms NICE-SLAM in high-fidelity reconstruction performance, improving the completion rate by 2.53%. ATE-SLAM also far surpasses iMAP in completion rate, improving it by approximately 13%. The 3D reconstruction performance evaluation on the Replica dataset is shown in Table 2. In Table 2, bold text indicates that the current method achieved the best results in the current dataset scenario. When there are local lighting changes in the scene (such as the movement of sunlight patches caused by swaying curtains), existing neural implicit SLAM (such as NICE-SLAM) tends to misjudge appearance changes as geometric changes. This invention dynamically allocates the contribution of hash features and One-blob encoding by using attention weights. Experiments show that the reconstruction completion rate is as high as 91.22% in the illumination mutation scenario of the Replica dataset.

[0167] Table 2 Evaluation of 3D Reconstruction Performance on the Replica Dataset

[0168] Unlike the Replica dataset, the NeuralRGBD synthetic dataset contains seven scenes and includes noise from the depth sensor. In Table 3, ATE-SLAM slightly outperforms Co-SLAM across all metrics and significantly surpasses NICE-SLAM. Specifically, ATE-SLAM achieves a completion rate of 90.22%, while Co-SLAM only reaches approximately 89%. Furthermore, it shows additional improvements in accuracy error and completion error, reaching 2.64 cm and 2.59 cm respectively.

[0169] Table 3 Evaluation of 3D Reconstruction Results on NeuralRGBD Synthetic Dataset

[0170] As one of the most challenging indoor scene datasets, the ScanNet dataset was used for camera tracking evaluation. Using absolute trajectory error (ATE RMSE) as a measure of tracking accuracy, this invention still demonstrates competitiveness compared to benchmark methods. Specifically, as shown in Table 4, the average absolute trajectory error of this invention reaches 8.70 cm, which is 0.67 cm lower than Co-SLAM. Experimental results show that the method of this invention performs well in many scenes, such as scene0000, scene0059, and scene0181. It should be noted that this invention uses the average of 10 runs for fair comparison. Traditional voxel meshes face memory explosion or loss of detail in scenes such as large shopping malls. The multi-resolution hash feature mesh, combined with an 8-layer MLP decoder, achieves 0.8 mm geometric accuracy on the ScanNet dataset while using only 1.2 GB of GPU memory, saving 73% of memory compared to NICE-SLAM. The absolute trajectory error evaluation results on the ScanNet dataset are shown in Table 4.

[0171] Table 4. Absolute trajectory error evaluation on the ScanNet dataset.

[0172] To further demonstrate the robustness of this invention, ATE-SLAM, camera tracking performance was evaluated using the TUM dataset in the same manner as the ScanNet dataset. As shown in Table 5, by averaging 10 runs, it was found that ATE-SLAM maintains the same performance as Co-SLAM, while being more accurate than NICE-SLAM in tracking trajectories (see Table 5). Specifically, the absolute trajectory error of ATE-SLAM reached 2.3 cm, significantly lower than the 2.4 cm of Co-SLAM. Meanwhile, the SLAM algorithm suffers computational delay due to traversing all keyframes during BA optimization. The segmented selection strategy dynamically filters keyframes using a pose difference threshold, reducing the average optimization time from 210 ms to 89 ms on a 30m trajectory in the TUM dataset, while maintaining the ATE error below 0.11m. The absolute trajectory error evaluation on the TUM dataset is shown in Table 5.

[0173] Table 5. Absolute trajectory error evaluation under the TUM dataset.

[0174] In this embodiment of the invention, time performance is also a key metric for the SLAM method. A comprehensive evaluation of the SLAM system includes tracking time and mapping time. The average time for each tracking iteration or mapping iteration was calculated on the ScanNet and TUM datasets. In this ablation experiment, the addition of... The impact of the segmented keyframe selection method on overall time is shown in Table 6. As shown, the segmented keyframe selection method can improve tracking and mapping speed to some extent. In the ScanNet dataset, ATE-SLAM achieved a single-frame execution speed of 79.88 ms, an improvement of 6.24 ms. Furthermore, the photometric warpage loss had no significant impact on reconstruction speed. The entire SLAM system ran at a frequency of 5–8 Hz during the test; detailed time performance analysis is shown in Table 6.

[0175] Table 6 Time Performance Analysis

[0176] In summary, the experimental results demonstrate that the attention mechanism of this invention can more effectively leverage the advantages of both encoding methods to improve mapping performance. Furthermore, by introducing photometric warp loss and performing volume rendering through sampled image patches, smoother reconstruction is encouraged. The segmented keyframe selection method included in this invention only traverses a fixed number of keyframes per sample, instead of traversing all keyframes in the database. This keyframe selection method reduces time overhead and slightly improves camera tracking accuracy. Experimental data show that, compared to Co-SLAM, the photometric warp loss function can effectively improve reconstruction quality.

[0177] Secondly, when there are local lighting changes in the scene (such as sunlight patches moving due to curtains swaying), existing neural implicit SLAM tends to misjudge appearance changes as geometric changes. This invention addresses the impact of factors such as lighting on SLAM performance by dynamically allocating the contribution of hash features and One-blob encoding through attention weights.

[0178] Meanwhile, traditional voxel meshes face memory explosion or loss of detail in scenarios such as large shopping malls. The multi-resolution hash feature mesh of this invention, combined with an 8-layer MLP decoder, greatly saves video memory overhead compared with traditional Co-SLAM.

[0179] This invention was evaluated on four different datasets to demonstrate its superiority over state-of-the-art methods such as Co-SLAM and NICE-SLAM. Ablation studies and additional visualizations show that the method of this invention improves 3D reconstruction performance while maintaining tracking accuracy.

[0180] Therefore, the SLAM method for indoor mobile robots provided by this invention has the following advantages: First, it effectively utilizes the advantages of each specific encoding, enabling efficient volume rendering of neural implicit RGBD SLAM in bright indoor environments; second, it uses a novel photometric warp loss, which can produce smoother, dense reconstructions of large indoor scenes; third, it uses a segmented keyframe selection method, which can save time as much as possible and improve tracking and dense mapping performance. Finally, in evaluations on four different indoor scene datasets (Replica, TUM, ScanNet, Synthetic RGBD), the SLAM method for indoor mobile robots of this invention can effectively improve high-fidelity reconstruction performance while maintaining tracking accuracy.

[0181] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A SLAM method for indoor mobile robots, characterized in that, include: Acquire environmental image information of an indoor mobile robot, wherein the environmental image information includes an RGB-D image sequence formed by RGB image information and depth image information; Ray sampling and attention-based encoding are fused to the RGB-D image sequence to obtain first optimization parameter information, which includes at least initial pose information and map building performance parameter information. The initial pose information is optimized according to the target total loss function to obtain the tracked frame with pose information. Based on the tracked frames, key frames for mapping are determined; Ray sampling and attention-based encoding are fused based on the keyframes to obtain second optimization parameter information, which includes at least updated pose information and map building performance parameter information. Based on the target total loss function, the second optimization parameter information is subjected to joint optimization processing of pose and mapping to obtain pose and map construction results; Specifically, the RGB-D image sequence is fused with ray sampling and attention-based encoding to obtain first optimized parameter information, including: The RGB-D image sequence is preprocessed and then stored in a multi-resolution hash feature grid. Multiple 3D environmental information sampling points are obtained by performing ray sampling in a multi-resolution hash feature grid based on NeRF rays; The three-dimensional sampling points of the environmental information are encoded and fused based on an attention mechanism to obtain the first optimized parameter information; The process of encoding and fusing the three-dimensional sampling points of the environmental information based on an attention mechanism includes: The three-dimensional sampling points of the environmental information are encoded according to parameter encoding and coordinate encoding respectively to obtain parameter encoding results and coordinate encoding results; The parameter encoding result and the coordinate encoding result are respectively input into the first geometric decoder to obtain the first geometric decoding feature, and the parameter encoding result and the coordinate encoding result are respectively input into the second geometric decoder to obtain the second geometric decoding feature; The first geometric decoding feature and the coordinate encoding result are both input to the first additional decoder to obtain the first additional decoding feature, and the first geometric decoding feature is input to the second additional decoder to obtain the second additional decoding feature; SDF features and color features are determined based on attention weights; The first optimization parameter information is determined based on the SDF features and color features; Wherein, the first additional decoding feature represents a first color feature, and the second additional decoding feature represents a second color feature; the first color feature and the second color feature are combined with the attention weight to form the color feature; The target total loss function is determined based on the SDF features and color features.

2. The SLAM method for indoor mobile robots according to claim 1, characterized in that, SDF features and color features are determined based on attention weights, including: Attention weights are determined based on a multilayer perceptron, wherein the expression for the multilayer perceptron is: ， in, Represents the learning parameters, and This represents the attention weights between the two inputs. This is achieved using the Softmax activation function. Calculate the SDF feature value s and the color feature value c based on the attention weights: 。 3. The SLAM method for indoor mobile robots according to claim 1, characterized in that, The initial pose information is subjected to pose optimization processing based on the target total loss function, including: The target total loss function is determined based on the SDF features and color features; The initial pose information is input into the pose optimizer, and the target total loss function is minimized to obtain the tracked frame with pose information.

4. The SLAM method for indoor mobile robots according to claim 3, characterized in that, The target total loss function is determined based on the SDF features and color features, including: Determine the color and depth loss functions based on the SDF features and the color features; Determine the SDF loss function and free space loss function based on the SDF features; Determine the photometric warpage loss function based on pose information; The target total loss function is determined based on the color and depth loss function, SDF loss function, free space loss function, and photometric warping loss function.

5. The SLAM method for indoor mobile robots according to claim 1, characterized in that, Based on the tracked frames, key frames for mapping are determined, including: Iterate through the database of the tracked frames; Divide the index of the currently tracked frame by 5. If the index of the currently tracked frame is divisible by 5, then the currently tracked frame is determined as a key frame. Store the selected keyframes into the keyframe database.

6. The SLAM method for indoor mobile robots according to claim 1, characterized in that, Based on the keyframes, ray sampling and attention-based encoding are fused to obtain second optimized parameter information, including: Keyframes are selected from the keyframe database according to the segmented keyframe selection method. The selected keyframes are ray-sampled to obtain multiple 3D sampling points for the keyframes. The keyframe 3D sampling points are encoded and fused based on an attention mechanism to obtain the second optimized parameter information.

7. The SLAM method for indoor mobile robots according to claim 6, characterized in that, Keyframes are selected from the keyframe database according to a segmented keyframe selection method, including: Determine if the number of keyframes in the keyframe database is less than [number missing]. indivual; If less If there are 1, then randomly select 1 from all keyframes in the keyframe database. BA optimization is performed on the ray; If not less than If there are 1, then calculate the pose difference of all keyframes in the keyframe database; After sorting the pose differences of all keyframes in descending order, select the first one. Individual keyframes.

8. The SLAM method for indoor mobile robots according to claim 1, characterized in that, Based on the target total loss function, the second optimization parameter information is subjected to joint optimization processing for pose and mapping to obtain pose and map construction results, including: The updated pose information and map building performance parameters are input into the pose and map joint optimizer. The pose and map in the pose and map joint optimizer are jointly optimized according to the target total loss function to obtain the pose and map construction results.

Images