Real-time simultaneous localization and mapping system based on implicit representation

Abstract

A real-time simultaneous localization and mapping system based on implicit representation, the system comprising a multi-thread localization and mapping module, wherein the multi-thread localization and mapping module comprises a camera tracking thread, a local mapping thread and a global mapping thread that are in parallel; the camera tracking thread is used for tracking, on the basis of a color-depth video frame that is collected in real time, a camera pose in real time by means of feature point extraction and matching; the local mapping thread is used for constructing, on the basis of the color-depth video frame and the camera pose, a real-time local map by means of implicit representation; and the global mapping thread is used for stitching and updating all local maps in real time, so as to obtain a complete global map. The real-time simultaneous localization and mapping system exerts both the characteristic of accurate localization of conventional simultaneous localization and mapping methods and the characteristic of high-precision map acquisition of an implicit representation method, thereby obtaining corresponding high-precision maps while implementing accurate localization.

Description

A real-time simultaneous localization and mapping system based on implicit representation Technical Field

[0001] The present invention belongs to the field of visual simultaneous localization and mapping technology, and specifically relates to a real-time simultaneous localization and mapping system based on implicit representation. Background Art

[0002] Visual Simultaneous Localization and Mapping (vSLAM) is a fundamental yet challenging computer vision task. It enables navigation and path planning by simultaneously mapping an unknown environment and locating its own position. In research both domestically and internationally, vSLAM systems have been widely applied in fields such as autonomous driving and robotics.

[0003] In recent years, many simultaneous localization and mapping systems have emerged. Based on their technical solutions, these systems can be categorized into traditional simultaneous localization and mapping systems and deep learning-based simultaneous localization and mapping systems. Traditional simultaneous localization and mapping systems primarily rely on geometric information from sensor data for motion estimation and map construction, using traditional computer vision algorithms such as feature point extraction and matching, optical flow, and stereo matching. There are many representative works on traditional simultaneous localization and mapping systems. In 2011, Richard Newcombe et al. proposed DTAM. In this paper, the authors proposed a vision-based simultaneous localization and mapping system that generates a dense 3D map in real time from a sequence of continuous camera images and simultaneously tracks the camera's position and pose, achieving real-time localization and mapping. This method employs an optimization-based framework to convert sparse feature point matches into dense pixel matches, thereby improving the accuracy and efficiency of tracking and mapping. In 2015, Raúl Mur-Artal proposed ORB-SLAM. In this paper, the authors propose a simultaneous localization and mapping system based on ORB feature points. By extracting and matching ORB feature points, they achieve real-time camera localization and 3D map construction. This method uses a sliding window-based optimization framework to effectively handle errors in camera pose and map estimation, and achieves high accuracy and robustness in a variety of scenarios.

[0004] With the development of deep learning, many simultaneous localization and mapping systems based on deep learning have emerged. These systems learn the processes of motion estimation and map construction directly from images or lidar data by using deep learning models. In 2020, R. Li et al. proposed the DeepSLAM system. The DeepSLAM system uses an unsupervised training method that does not require the use of labeled data and can be trained on a large amount of unlabeled data, thereby achieving more efficient and flexible deep learning feature extraction and depth estimation. Traditional simultaneous localization and mapping systems can provide relatively accurate positioning accuracy, but can only output point cloud maps that lack geometric connections and cannot meet the interaction needs of robots. Simultaneous localization and mapping systems based on deep learning can output dense maps, but they also have problems with slow training speed and poor generalization.

[0005] With the development of implicit representation, many simultaneous localization and mapping systems have emerged that use implicit representation as maps. Implicit representation maps are lightweight, can render high-precision images, and extract high-precision surfaces. They are a map representation method with geometric continuity and are very suitable for mapping tasks. Z. Zhu et al. proposed NICE-SLAM. In this paper, the authors proposed a method for simultaneous localization and reconstruction using implicit representation maps. This system can extract high-quality, interactive, dense meshes for downstream tasks. However, the implicit representation localization method lacks the global optimization and closed-loop detection of traditional simultaneous localization and navigation systems, resulting in low positioning accuracy.

[0006] Summary of the Invention

[0007] In order to solve the technical problems that traditional simultaneous positioning and mapping methods cannot obtain high-precision maps and implicit representation methods cannot obtain accurate positioning, the embodiments provide a real-time simultaneous positioning and mapping system based on implicit representation. The system simultaneously takes advantage of the accurate positioning characteristics of traditional simultaneous positioning and mapping methods and the characteristics of implicit representation methods in obtaining high-precision maps, so as to achieve accurate positioning and obtain corresponding high-precision maps.

[0008] To achieve the above-mentioned object of the invention, an embodiment provides a real-time simultaneous localization and mapping system based on implicit representation, including a multi-threaded localization and mapping module, wherein the multi-threaded localization and mapping module includes a parallel camera tracking thread, a local mapping thread, and a global mapping thread;

[0009] The camera tracking thread is used to track the camera pose in real time by extracting and matching feature points based on the real-time acquired color-depth video frames;

[0010] The local mapping thread is used to construct a real-time local map based on color-depth video frames and camera pose using an implicit representation method;

[0011] The global mapping thread is used to stitch and update all local maps in real time to obtain a complete global map;

[0012] The global map and local map obtained by the system can realize the rendering of high-resolution color-depth images and the extraction of high-precision surfaces.

[0013] Preferably, the camera tracking thread includes a key frame selection unit, a local optimization unit, a closed loop detection unit, and a global optimization unit;

[0014] The key frame selection unit is used to extract ORB features from the color-depth video frame, select key frames based on the number of ORB features, and construct an eigengraph based on the key frames. In the eigengraph, the key frames are used as graph nodes, and the edges between the graph nodes are established by the degree of common view, wherein the degree of common view is the degree of matching between the ORB features of two key frames;

[0015] The local optimization unit is used to locally optimize the relative posture of the local key frame corresponding to the current moment according to the eigengraph;

[0016] The closed loop detection unit is used to monitor the ORB features of the color-depth video frame in real time, and detect whether a closed loop occurs in the color-depth video frame according to the number of ORB features;

[0017] The global optimization unit is used to globally optimize the relative positions of the key frames in the current eigengraph when a closed loop occurs.

[0018] Preferably, in the local optimization unit, local optimization is performed on the relative pose of the local key frame corresponding to the current moment according to the eigengraph, including:

[0019] At the current moment, based on the eigengraph at the current moment, the keyframes corresponding to the graph nodes whose co-viewing degree is greater than the first co-viewing degree threshold are selected as local keyframes. For each local keyframe, the relative position of the local keyframe is locally optimized using the keyframes connected to each local keyframe and whose co-viewing degree is greater than the first co-viewing degree threshold.

[0020] The process of optimizing the relative pose is as follows: determining the relative pose of the local keyframe based on the relative pose of the matching ORB features of the two keyframes.

[0021] Preferably, in the closed loop detection unit, detecting whether a closed loop occurs in the color-depth video frame according to the number of ORB features includes:

[0022] A frame number threshold is introduced. When the number of historical color-depth video frames exceeds the frame number threshold and the number of ORB feature matches between the current color-depth video frame and the previous color-depth video frame reaches the feature number threshold, the color-depth video frame is considered to have a closed loop.

[0023] Preferably, in the global optimization unit, the pose of the key frame in the current eigengraph is globally optimized, including:

[0024] For each keyframe, the relative pose of the keyframe is determined based on the relative poses between the keyframe and the matching ORB features of all its adjacent keyframes to achieve global optimization of the keyframe.

[0025] Preferably, the local mapping thread includes a local map initialization unit and a local map training unit;

[0026] The local map initialization unit is used to initialize a new local map when it is determined based on the eigenmap that the degree of common view between the first key frame of the current local map and the current key frame is less than a second common view threshold, wherein the local map is an incremental implicit representation network;

[0027] The local map training unit is used to use the local key frames and corresponding relative poses output by the local optimization unit as inputs to the incremental implicit representation network, and to achieve training optimization of the local map by training the incremental implicit representation network.

[0028] Preferably, the incremental implicit representation network includes a feature grid, trilinear interpolation, and a feature decoder, wherein the feature network adopts an octree structure and stores features at corner points of different levels of the octree. The specific construction process of the feature network is as follows: converting the depth image corresponding to the local key frame into a point cloud, generating an octree structure network based on the point cloud, determining the light of the color image corresponding to the local key frame at the relative pose based on the relative pose, the light is directed to the grid and intersects with the grid, and features are stored at the corner points of the grid, and the features are optimized during the training process;

[0029] The trilinear interpolation is used to perform trilinear interpolation on the corner point features of the grid to which the sampling point belongs based on the feature grid when querying the sampling point, and to fuse the interpolation features of multiple levels to obtain the feature vector of the sampling point;

[0030] The feature decoder includes a color decoder and a geometry decoder, which are used to perform color decoding and geometry decoding on the feature vectors of the sampling points respectively to obtain color information and geometry information.

[0031] Preferably, in the local map training unit, the process of training the incremental implicit representation network is:

[0032] Volume rendering is used to render the color information and geometric information to obtain a rendered color image and a rendered depth image. The photometric loss is constructed based on the difference between the rendered color image and the input color image, and the depth loss is constructed based on the difference between the rendered depth image and the input depth image. The features and feature decoder parameters of the feature grid in the incremental implicit representation network are optimized based on the photometric loss and depth loss.

[0033] Preferably, the global mapping thread includes a multi-map stitching unit and a global map updating unit.

[0034] The multi-map stitching unit is used to stitch the local maps generated by the local mapping threads in real time to obtain a global map;

[0035] The global map updating unit is used to update all local maps constituting the global map according to the relative posture of the global optimization after the global optimization by the global optimization unit, thereby updating the global map.

[0036] Preferably, it also includes an uncertainty-based image rendering module; the uncertainty-based image rendering module is used to render according to the geometric information output by multiple local maps. Specifically, in each local map, occupancy is used to characterize the geometric information. After determining the occupancy p of each pixel point, the occupancy variance var of each pixel point is calculated by the variance formula var=p(1-p) of the Bernoulli distribution. The volume rendering method is used to calculate the uncertainty of each pixel point in the rendered image based on the occupancy variance var of each point, and then the overall uncertainty of each rendered image is obtained. The rendered image with the lowest uncertainty is screened as the final output based on the overall uncertainty of the rendered image corresponding to each local map.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] Through the parallel camera tracking thread, local mapping thread and global mapping thread, while tracking and locating the camera pose in real time, local mapping and global mapping are performed to achieve accurate positioning and obtain corresponding high-precision maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] FIG1 is a schematic structural diagram of a real-time simultaneous positioning and mapping system based on implicit representation provided by an embodiment;

[0041] FIG2 is a schematic diagram of the structure of a multi-threaded positioning and mapping module provided in an embodiment;

[0042] FIG3 is a flow chart of training an incremental implicit representation network corresponding to a local map provided in an embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0044] In order to give full play to the characteristics of accurate positioning of traditional simultaneous positioning and mapping systems and the ability of implicit representation methods to obtain high-precision maps, and to solve the technical problems that traditional methods cannot obtain high-precision maps and implicit representation methods cannot obtain accurate positioning, an embodiment of the present invention provides a real-time simultaneous positioning and mapping system based on implicit representation to achieve accurate positioning and obtain corresponding high-precision maps.

[0045] As shown in Figure 1, the real-time simultaneous positioning and mapping system based on implicit representation provided by the embodiment includes two modules: multi-threaded positioning and mapping and uncertainty-based image rendering. To enhance real-time performance, the multi-threaded positioning (camera pose tracking) and mapping modules are divided into three threads and performed simultaneously. Among them, positioning is the camera tracking thread, and mapping is divided into two threads: local mapping thread and global mapping thread. In the camera tracking thread, feature point extraction and matching methods are used to track the relative pose of the camera. During the tracking process, the system has multiple optimization strategies of local optimization, global optimization and closed-loop detection to ensure the accuracy of tracking. Its input is a color-depth image, and the output is the relative pose of each frame image. In the local mapping thread, the camera pose and color-depth image obtained by the tracking thread are used to construct a local map of the current camera using an implicit representation method to obtain a local implicit map of the scene. The local mapping process corresponds to the local optimization in the pose tracking thread. In the global mapping thread, the local maps obtained by the local mapping thread are spliced ​​and updated to obtain a complete global map. The following is a detailed description of each thread.

[0046] In this embodiment, the camera tracking thread is used to track the camera pose in real time based on the input color-depth video frames using feature point extraction and matching. As shown in Figure 2, the camera tracking thread includes a keyframe selection unit, a local optimization unit, a loop closure detection unit, and a global optimization unit. Specifically, it implements real-time pose tracking of the input color-depth video stream, tracking each frame by extracting and matching ORB features.

[0047] In order to reduce the amount of calculation while maintaining the consistency of tracking, a key frame selection unit is introduced. The key frame selection unit is used to extract ORB features from the color-depth video frame, and then select the key frame by counting the number of ORB features. Specifically, when the number of ORB features is greater than the set threshold, the corresponding color-depth video frame is used as the key frame. After obtaining the key frame, the eigengraph is also constructed and updated in real time based on the key frame. Specifically, in the eigengraph, the key frame is used as a graph node, and the edges between the graph nodes are established through the degree of common view, wherein the degree of common view is the degree of matching between the ORB features of the two key frames, and the degree of matching is represented by the matching value between the quantitative features. The matching value is used as the degree of common view, that is, the weight value of the edge. The larger the matching value, the larger the weight value of the corresponding edge.

[0048] The local optimization unit is used to locally optimize the relative pose of the local keyframe corresponding to the current moment based on the eigengraph. Specifically, for the current moment, based on the eigengraph at the current moment, the keyframes corresponding to the graph nodes whose co-viewing degree is greater than the first co-viewing degree threshold are selected as local keyframes. For each local keyframe, the relative pose of the local keyframe is locally optimized using the keyframes connected to each local keyframe and whose co-viewing degree is greater than the first co-viewing degree threshold. The process of optimizing the relative pose is as follows: the relative pose of the local keyframe is determined based on the relative pose of the matching ORB features of the two keyframes, so that the pose of the local keyframe can be continuously optimized through graph optimization.

[0049] Long-distance camera tracking often causes cumulative errors. To eliminate these cumulative errors, the camera tracking thread of this system has added a closed-loop detection unit and a global optimization unit. The closed-loop detection unit is used to monitor the ORB features of the color-depth video frame in real time, and detect whether the color-depth video frame has a closed loop based on the number of ORB features. Specifically, a frame number threshold is introduced. If the number of historical color-depth video frames exceeds the frame number threshold, and the number of ORB feature matches between the current color-depth video frame and the previous color-depth video frame reaches the feature number threshold, the color-depth video frame is considered to have a closed loop. The frame number threshold and the feature number threshold are set according to the situation.

[0050] The global optimization unit is used to globally optimize the relative poses of keyframes in the current eigengraph when a closed loop occurs. Specifically, for each keyframe, the relative pose of the keyframe is determined based on the relative poses of the keyframe and all its adjacent keyframes' matching ORB features, achieving global optimization of the keyframes and eliminating accumulated errors.

[0051] In this embodiment, the local mapping thread is used to construct a real-time local map using an implicit representation method based on color-depth video frames and camera pose. As shown in Figure 2, the local mapping thread includes a local map initialization unit and a local map training unit, where the local map is an incremental implicit representation network.

[0052] The local map initialization unit is responsible for determining whether a new local map needs to be established for the current map. The determination is based on the local optimization unit of the camera tracking thread. Specifically, when the degree of common visibility between the first key frame of the current local map and the current key frame is determined to be less than a second common visibility threshold based on the intrinsic map, a new local map is initialized.

[0053] The local map training unit is used to use the local key frames and corresponding relative poses output by the local optimization unit as the input of the incremental implicit representation network, and output the geometric information and color information of the target sampling points. The training optimization of the local map is achieved by training the incremental implicit representation network.

[0054] Specifically, as shown in Figure 3, the incremental implicit representation network includes a feature grid, trilinear interpolation, and a feature decoder, wherein the feature network adopts an octree structure and stores features at the corner points of different levels of the octree. The specific construction process of the feature network is: the depth image corresponding to the local key frame is converted into a point cloud, and an octree structure network is generated according to the point cloud. The light of the color image corresponding to the local key frame at the relative pose is determined according to the relative pose. The light is directed to the grid and intersects with the grid, and features are stored at the corner points of the grid. The features are optimized during the training process; trilinear interpolation is used to perform trilinear interpolation on the corner point features of the grid to which the sampling point belongs based on the feature grid when querying the sampling point, and the interpolation features of multiple levels are fused to obtain the feature vector of the sampling point; the feature decoder includes a color decoder and a geometry decoder, which are used to perform color decoding and geometry decoding on the feature vector of the sampling point respectively to obtain color information and geometry information.

[0055] The incremental implicit representation network can render images using volume rendering. For a given viewpoint, the depth and color of each pixel are rendered by sampling the geometric and color information of the light rays emitted by each pixel. The training process of the incremental implicit representation network is as follows: the color information and geometric information are rendered using volume rendering to obtain a rendered color image and a rendered depth image. A photometric loss is constructed based on the difference between the rendered color image and the input color image, and a depth loss is constructed based on the difference between the rendered depth image and the input depth image. The photometric and depth losses are used to optimize the features of the feature grid in the incremental implicit representation network and the parameters of the feature decoder.

[0056] In an embodiment, the global mapping thread is used to stitch and update all local maps in real time to obtain a complete global map. As shown in Figure 2, the global mapping thread includes a multi-map stitching unit and a global map update unit. Since the local mapping thread has trained multiple local maps, the multi-map stitching unit is used to stitch these local maps generated by the local mapping thread in real time to obtain a global map. After the global optimization occurs, the camera pose of each local map will change. The global map update unit updates all local maps that make up the global map according to the relative pose of the global optimization, and then updates the global map. Specifically, the obtained global map and local map can achieve high-resolution color-depth image rendering and high-precision surface extraction.

[0057] In the embodiment, the uncertainty-based image rendering module is responsible for rendering based on the geometric information output by multiple local maps, and selecting the highest quality map in the pictures rendered by multiple local maps. In the local map implicit representation network, the present invention uses occupancy to characterize geometric information. Among them, the occupancy p is a real number belonging to the interval [0,1], which represents the probability that the point is occupied. An occupancy of 0 means that the point is not occupied, and an occupancy of 1 means that the point is occupied. The distribution of occupancy obeys the Bernoulli distribution. Therefore, the occupancy variance of each pixel point can be calculated by the variance formula var=p(1-p) of the Bernoulli distribution. Then, through the volume rendering method, the uncertainty of each pixel point in the rendered image can be calculated based on the occupancy variance var of each point, and then the overall uncertainty of each rendered image can be obtained. Finally, based on the overall uncertainty of the rendered image corresponding to each local map, the image with the lowest overall uncertainty is selected as the rendered image output of the final model.

[0058] The real-time simultaneous positioning and mapping system based on implicit representation provided in the above embodiment can quickly and accurately complete positioning and mapping tasks simultaneously, and has the following effects:

[0059] 1. High-precision tracking. This system tracks the input color-depth video stream, estimates the camera pose with high precision for each frame, and selects keyframes. During the tracking process, the system simultaneously optimizes the camera pose using local optimization, global optimization, and closed-loop detection, ensuring accurate tracking and positioning even in long-distance tracking tasks.

[0060] 2. High-precision mapping. The mapping process of this system is divided into two parts: local mapping and global mapping. Local mapping corresponds to the local optimization part of high-precision tracking. Local mapping maintains multiple local maps and optimizes the local map currently being optimized to ensure the accuracy of the local map. Global mapping stitches multiple local maps together. After a closed loop or global optimization occurs during the tracking process, the relative positions of the local maps are quickly adjusted to maintain the accuracy of the global map.

[0061] 3. High-precision image rendering and mesh extraction. Leveraging the advantages of implicit representation methods, the implicit map constructed by this system can render images from any perspective and use uncertainty to select images, resulting in high-quality color and depth images. Furthermore, this implicit map can also extract a dense mesh that can be used for interaction.

[0062] 4. Real-time performance. This system can simultaneously perform positioning and mapping at a frequency of 10 Hz, which can be used in most practical mission scenarios and meets the real-time requirements of these scenarios.

[0063] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time simultaneous localization and mapping system based on implicit representation, characterized in that: It includes a multi-threaded positioning and mapping module, which includes a parallel camera tracking thread, a local mapping thread, and a global mapping thread; The camera tracking thread is used to track the camera posture in real time by extracting and matching feature points based on the color-depth video frames collected in real time; The local mapping thread is used to construct a local map in real time using an implicit representation method based on color-depth video frames and camera poses; The global mapping thread is used to stitch and update all local maps in real time to obtain a complete global map; The global map and local map obtained by the system can realize the rendering of high-resolution color-depth images and the extraction of high-precision surfaces.

2. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: The camera tracking thread includes a key frame selection unit, a local optimization unit, a closed loop detection unit and a global optimization unit; The key frame selection unit is used to extract ORB features from the color-depth video frame, select key frames by counting the number of ORB features, and construct an eigengraph based on the key frames. In the eigengraph, the key frames are used as graph nodes, and the edges between the graph nodes are established by the degree of common view, wherein the degree of common view is the degree of matching between the ORB features of two key frames; The local optimization unit is used to locally optimize the relative position and posture of the local key frame corresponding to the current moment according to the eigengraph; The closed loop detection unit is used to monitor the ORB features of the color-depth video frame in real time, and detect whether a closed loop occurs in the color-depth video frame according to the number of ORB features; The global optimization unit is used to globally optimize the relative positions and postures of the key frames in the current eigengram when a closed loop occurs.

3. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: In the local optimization unit, the relative position of the local key frame corresponding to the current moment is locally optimized according to the eigengraph, including: At the current moment, according to the eigengraph at the current moment, the key frames corresponding to the graph nodes whose co-viewing degree is greater than the first co-viewing degree threshold are selected as local key frames. For each local key frame, the relative position and posture of the local key frame are locally optimized by using the key frames connected to each local key frame and whose co-viewing degree is greater than the first co-viewing degree threshold. The process of optimizing the relative pose is as follows: determining the relative pose of the local keyframe based on the relative pose of the matching ORB features of the two keyframes.

4. The real-time simultaneous localization and mapping system based on implicit representation according to claim 2, characterized in that: In the closed loop detection unit, whether a closed loop occurs in the color-depth video frame is detected based on the number of ORB features, including: A frame number threshold is introduced. When the number of historical color-depth video frames exceeds the frame number threshold, and the number of ORB feature matches between the current color-depth video frame and the previous color-depth video frame reaches the feature number threshold, the color-depth video frame is considered to have a closed loop.

5. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: In the global optimization unit, the pose of the key frame in the current eigenmap is globally optimized, including: For each keyframe, the relative pose of the keyframe is determined based on the relative poses between the keyframe and the matching ORB features of all its adjacent keyframes to achieve global optimization of the keyframe.

6. The real-time simultaneous localization and mapping system based on implicit representation according to claim 2, characterized in that: The local mapping thread includes a local map initialization unit and a local map training unit; The local map initialization unit is used to initialize a new local map when it is determined based on the intrinsic map that the co-viewing degree between the first key frame of the current local map and the current key frame is less than a second co-viewing degree threshold, wherein the local map is an incremental implicit representation network; The local map training unit is used to use the local key frames and the corresponding relative poses output by the local optimization unit as inputs of the incremental implicit representation network, and to achieve training optimization of the local map by training the incremental implicit representation network.

7. The real-time simultaneous localization and mapping system based on implicit representation according to claim 6, characterized in that: The incremental implicit representation network includes a feature grid, trilinear interpolation, and a feature decoder, wherein the feature network adopts an octree structure, and stores features at corner points of different levels of the octree. The specific construction process of the feature network is: converting a depth image corresponding to a local key frame into a point cloud, generating an octree structure network according to the point cloud, determining the light of the color image corresponding to the local key frame at the relative posture according to the relative posture, the light is directed to the grid and intersects with the grid, and features are stored at the corner points of the grid, and the features are optimized during the training process; The trilinear interpolation is used to perform trilinear interpolation on the corner point features of the grid to which the sampling point belongs based on the feature grid when querying the sampling point, and to fuse the interpolation features of multiple levels to obtain the feature vector of the sampling point; The feature decoder includes a color decoder and a geometry decoder, which are used to perform color decoding and geometry decoding on the feature vector of the sampling point to obtain color information and geometry information.

8. The real-time simultaneous localization and mapping system based on implicit representation according to claim 7, characterized in that: In the local map training unit, the process of training the incremental implicit representation network is: The color information and geometric information are rendered using volume rendering to obtain a rendered color image and a rendered depth image, and a luminosity image is constructed based on the difference between the rendered color image and the input color image. The depth loss is constructed by the difference between the rendered depth image and the input depth image, and the features and feature decoder parameters of the feature grid in the incremental implicit representation network are optimized based on the photometric loss and the depth loss.

9. The real-time simultaneous localization and mapping system based on implicit representation according to claim 2, characterized in that: The global map building thread includes a multi-map stitching unit and a global map update unit. The multi-map stitching unit is used to stitch the local maps generated by the local mapping threads in real time to obtain a global map; The global map updating unit is used to update all local maps constituting the global map according to the relative positions of the global optimization after the global optimization by the global optimization unit, thereby updating the global map.

10. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: It also includes an uncertainty-based image rendering module; The uncertainty-based image rendering module is used to render according to the geometric information output by multiple local maps. Specifically, in each local map, occupancy is used to characterize the geometric information. After the occupancy p of each pixel is determined, the occupancy variance var of each pixel is calculated by the variance formula var=p(1-p) of the Bernoulli distribution. The uncertainty of each pixel in the rendered image is calculated according to the occupancy variance var of each point using a volume rendering method, and then the overall uncertainty of each rendered image is obtained. The rendered image with the lowest uncertainty is selected as the final output according to the overall uncertainty of the rendered image corresponding to each local map.

Images