A real-time simultaneous localization and mapping system based on implicit representation

Abstract

The present invention discloses a real-time simultaneous positioning and mapping system based on implicit representation, including a multi-threaded positioning and mapping module, wherein the multi-threaded positioning and mapping module includes a parallel camera tracking thread, a local mapping thread and a global mapping thread; the camera tracking thread is used to track the camera posture in real time based on the real-time collected color-depth video frame by extracting and matching feature points; the local mapping thread is used to construct a real-time local map based on the color-depth video frame and the camera posture by implicit representation; the global mapping thread is used to splice and update all local maps in real time to obtain a complete global map. The system simultaneously gives play to the accurate positioning characteristics of the traditional simultaneous positioning and mapping method and the characteristics of the implicit representation method to obtain high-precision maps, so as to achieve accurate positioning and obtain the corresponding high-precision map.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual simultaneous localization and mapping, and in particular relates to a real-time simultaneous localization and mapping system based on implicit representation. Background Art

[0002] Visual Simultaneous Localization and Mapping (vSLAM) is a very basic but also challenging computer vision task. It can achieve device navigation and path planning by building a map and locating its own position in an unknown environment. In domestic and foreign research, the simultaneous localization and mapping system has been widely used in many fields such as autonomous driving and robot operation.

[0003] In recent years, many simultaneous localization and mapping systems have emerged. According to different technical solutions, these systems can be divided into traditional simultaneous localization and mapping systems and deep learning-based simultaneous localization and mapping systems. Among them, traditional simultaneous localization and mapping systems mainly rely on the geometric information of sensor data for motion estimation and map construction, and use traditional computer vision algorithms such as feature point extraction and matching, optical flow method, stereo matching, etc. to achieve motion estimation and map construction. There are many representative works on traditional simultaneous localization and mapping systems. In 2011, Richard Newcombe et al. proposed DTAM. In this paper, the authors proposed a vision-based simultaneous localization and mapping system, which can generate dense three-dimensional maps in real time from continuous image sequences captured by the camera, and simultaneously track the position and posture of the camera, realizing real-time localization and mapping. This method adopts an optimization-based framework to convert sparse feature point matching into dense pixel matching, thereby improving the accuracy and efficiency of tracking and mapping. In 2015, Raúl Mur-Artal proposed ORB-SLAM. In this paper, the author proposed a simultaneous positioning and mapping system based on ORB feature points. Through the extraction and matching of ORB feature points, real-time camera positioning and three-dimensional map construction are realized. This method adopts a sliding window-based optimization framework, which can effectively handle the estimation errors of camera posture and map, and has high accuracy and robustness in various scenarios.

[0004] With the development of deep learning, many simultaneous localization and mapping systems based on deep learning have emerged. These systems learn the process of motion estimation and map construction directly from images or lidar data by using deep learning models to learn the process of motion estimation and map construction. In 2020, R. Li et al. proposed the DeepSLAM system. The DeepSLAM system uses an unsupervised training method that does not require the use of labeled data and can be trained on a large amount of unlabeled data, thereby achieving more efficient and flexible deep learning feature extraction and depth estimation. Traditional simultaneous localization and mapping systems can provide relatively accurate positioning accuracy, but can only output point cloud maps that lack geometric connections and cannot meet the interaction needs of robots. The simultaneous localization and mapping system based on deep learning can output dense maps, but it also has the problems of slow training speed and poor generalization.

[0005] With the development of implicit representation, many simultaneous localization and mapping systems have emerged that use implicit representation as maps. Implicit representation maps are lightweight, can render high-precision images, and extract high-precision surfaces. They are a map representation method with geometric continuity and are very suitable for mapping tasks. Z. Zhu et al. proposed NICE-SLAM. In this paper, the authors proposed a method for simultaneous localization and reconstruction using implicit representation maps. The system can extract high-quality, interactive, dense grids for downstream tasks. However, the method of using implicit representation for localization lacks the global optimization and closed-loop detection of traditional simultaneous localization and navigation systems, resulting in low positioning accuracy of this method. Summary of the invention

[0006] In order to solve the technical problem that the traditional simultaneous positioning and mapping method cannot obtain high-precision maps and the implicit characterization method cannot obtain accurate positioning, the embodiment provides a real-time simultaneous positioning and mapping system based on implicit characterization. The system simultaneously gives play to the accurate positioning characteristics of the traditional simultaneous positioning and mapping method and the characteristics of the implicit characterization method for obtaining high-precision maps, so as to achieve accurate positioning and obtain the corresponding high-precision map.

[0007] To achieve the above-mentioned object of the invention, an embodiment provides a real-time simultaneous positioning and mapping system based on implicit representation, including a multi-threaded positioning and mapping module, wherein the multi-threaded positioning and mapping module includes a parallel camera tracking thread, a local mapping thread and a global mapping thread;

[0008] The camera tracking thread is used to track the camera posture in real time by extracting and matching feature points based on the color-depth video frames collected in real time;

[0009] The local mapping thread is used to construct a real-time local map based on the color-depth video frame and the camera pose using an implicit representation method;

[0010] The global mapping thread is used to stitch and update all local maps in real time to obtain a complete global map;

[0011] The global map and local map obtained by the system can realize the rendering of high-resolution color-depth images and the extraction of high-precision surfaces.

[0012] Preferably, the camera tracking thread includes a key frame selection unit, a local optimization unit, a closed loop detection unit and a global optimization unit;

[0013] The key frame selection unit is used to extract ORB features from the color-depth video frame, select key frames by counting the number of ORB features, and construct an eigengraph based on the key frames. In the eigengraph, the key frames are used as graph nodes, and the edges between the graph nodes are established by the degree of common view, wherein the degree of common view is the degree of matching between the ORB features of two key frames;

[0014] The local optimization unit is used to locally optimize the relative position and posture of the local key frame corresponding to the current moment according to the eigengraph;

[0015] The closed loop detection unit is used to monitor the ORB features of the color-depth video frame in real time, and detect whether a closed loop occurs in the color-depth video frame according to the number of ORB features;

[0016] The global optimization unit is used to globally optimize the relative positions and postures of the key frames in the current eigengram when a closed loop occurs.

[0017] Preferably, in the local optimization unit, the relative position and posture of the local key frame corresponding to the current moment is locally optimized according to the eigengraph, including:

[0018] At the current moment, according to the eigengraph at the current moment, the key frames corresponding to the graph nodes whose co-viewing degree is greater than the first co-viewing degree threshold are selected as local key frames. For each local key frame, the relative position and posture of the local key frame are locally optimized by using the key frames connected to each local key frame and whose co-viewing degree is greater than the first co-viewing degree threshold.

[0019] The process of optimizing the relative pose is as follows: determining the relative pose of the local keyframe based on the relative pose of the matching ORB features of the two keyframes.

[0020] Preferably, in the closed loop detection unit, detecting whether a closed loop occurs in the color-depth video frame according to the number of ORB features includes:

[0021] A frame number threshold is introduced. When the number of historical color-depth video frames exceeds the frame number threshold, and the number of ORB feature matches between the current color-depth video frame and the previous color-depth video frame reaches the feature number threshold, the color-depth video frame is considered to have a closed loop.

[0022] Preferably, in the global optimization unit, the pose of the key frame in the current eigenimage is globally optimized, including:

[0023] For each keyframe, the relative pose of the keyframe is determined based on the relative poses between the keyframe and the matching ORB features of all its adjacent keyframes to achieve global optimization of the keyframe.

[0024] Preferably, the local mapping thread includes a local map initialization unit and a local map training unit;

[0025] The local map initialization unit is used to initialize a new local map when it is determined based on the intrinsic map that the co-viewing degree between the first key frame of the current local map and the current key frame is less than a second co-viewing degree threshold, wherein the local map is an incremental implicit representation network;

[0026] The local map training unit is used to use the local key frames and the corresponding relative poses output by the local optimization unit as inputs of the incremental implicit representation network, and to achieve training optimization of the local map by training the incremental implicit representation network.

[0027] Preferably, the incremental implicit representation network includes a feature grid, trilinear interpolation, and a feature decoder, wherein the feature network adopts an octree structure, and features are stored at corner points of different levels of the octree. The specific construction process of the feature network is: converting the depth image corresponding to the local key frame into a point cloud, generating an octree structure network according to the point cloud, determining the light of the color image corresponding to the local key frame at the relative posture according to the relative posture, the light is directed to the grid and intersects with the grid, and features are stored at the corner points of the grid, and the features are optimized during the training process;

[0028] The trilinear interpolation is used to perform trilinear interpolation on the corner point features of the grid to which the sampling point belongs based on the feature grid when querying the sampling point, and to fuse the interpolation features of multiple levels to obtain the feature vector of the sampling point;

[0029] The feature decoder includes a color decoder and a geometry decoder, which are used to perform color decoding and geometry decoding on the feature vector of the sampling point to obtain color information and geometry information.

[0030] Preferably, in the local map training unit, the process of training the incremental implicit representation network is:

[0031] Volume rendering is used to render the color information and geometric information to obtain a rendered color image and a rendered depth image. The photometric loss is constructed based on the difference between the rendered color image and the input color image, and the depth loss is constructed based on the difference between the rendered depth image and the input depth image. The features and feature decoder parameters of the feature grid in the incremental implicit representation network are optimized based on the photometric loss and the depth loss.

[0032] Preferably, the global mapping thread includes a multi-map stitching unit and a global map updating unit.

[0033] The multi-map stitching unit is used to stitch the local maps generated by the local mapping threads in real time to obtain a global map;

[0034] The global map updating unit is used to update all local maps constituting the global map according to the relative positions of the global optimization after the global optimization by the global optimization unit, thereby updating the global map.

[0035] Preferably, it also includes an uncertainty-based image rendering module; the uncertainty-based image rendering module is used to render according to the geometric information output by multiple local area maps. Specifically, in each local area map, occupancy is used to characterize the geometric information. After determining the occupancy p of each pixel point, the occupancy variance var of each pixel point is calculated by the variance formula of Bernoulli distribution var=p(1-p). The volume rendering method is used to calculate the uncertainty of each pixel point in the rendered image according to the occupancy variance var of each point, and then the overall uncertainty of each rendered image is obtained. The rendered image with the lowest uncertainty is selected as the final output according to the overall uncertainty of the rendered image corresponding to each local area map.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] Through the three parallel threads of camera tracking, local mapping and global mapping, local and global mapping are performed while tracking and locating the camera pose in real time to achieve accurate positioning and obtain the corresponding high-precision map. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a structural schematic diagram of a real-time simultaneous positioning and mapping system based on implicit representation provided in an embodiment;

[0040] Figure 2 is a schematic diagram of the structure of a multi-threaded positioning and mapping module provided in an embodiment;

[0041] Figure 3 It is a training flow chart of an incremental implicit representation network corresponding to a local map provided in an embodiment. DETAILED DESCRIPTION

[0042] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0043] In order to give full play to the characteristics of accurate positioning of traditional simultaneous positioning and mapping systems and the ability of implicit characterization methods to obtain high-precision maps, and to solve the technical problem that traditional methods cannot obtain high-precision maps and implicit characterization methods cannot obtain accurate positioning, an embodiment of the present invention provides a real-time simultaneous positioning and mapping system based on implicit characterization to achieve accurate positioning and simultaneously obtain corresponding high-precision maps.

[0044] like Figure 1 As shown, the real-time simultaneous positioning and mapping system based on implicit representation provided by the embodiment includes two modules: multi-threaded positioning and mapping and image rendering based on uncertainty. To enhance real-time performance, the multi-threaded positioning (camera pose tracking) and mapping modules are divided into three threads and performed simultaneously, wherein positioning is a camera tracking thread, and mapping is divided into two threads: a local mapping thread and a global mapping thread. In the camera tracking thread, the relative pose of the camera is tracked by feature point extraction and matching methods, and in the tracking process, the system has multiple optimization strategies of local optimization, global optimization and closed-loop detection to ensure the accuracy of tracking. Its input is a color-depth image, and the output is the relative pose of each frame of the image. In the local mapping thread, the camera pose and color-depth image obtained by the tracking thread are used in an implicit representation method to construct the local map where the current camera is located to obtain a local scene implicit map, and the local mapping process corresponds to the local optimization in the pose tracking thread. In the global mapping thread, the local maps obtained by the local mapping thread are spliced ​​and updated to obtain a complete global map. The following is a detailed description of each thread.

[0045] In the embodiment, the camera tracking thread is used to track the camera posture in real time by extracting and matching feature points based on the real-time input color-depth video frame. Figure 2As shown, the camera tracking thread includes a key frame selection unit, a local optimization unit, a closed loop detection unit, and a global optimization unit. Specifically, the real-time pose tracking of the input color-depth video stream is realized, and each frame image is tracked by extracting ORB features and matching the features.

[0046] In order to reduce the amount of calculation while maintaining the consistency of tracking, a key frame selection unit is introduced. The key frame selection unit is used to extract the ORB features of the color-depth video frame, and then select the key frame by counting the number of ORB features. Specifically, when the number of ORB features is greater than the set threshold, the corresponding color-depth video frame is used as the key frame. After obtaining the key frame, the eigengraph is also constructed and updated in real time according to the key frame. Specifically, in the eigengraph, the key frame is used as a graph node, and the edges between the graph nodes are established by the degree of common view, wherein the degree of common view is the degree of matching between the ORB features of the two key frames, and the degree of matching is characterized by the matching value between the quantitative features, and the matching value is used as the degree of common view, that is, the weight value of the edge. The larger the matching value, the larger the weight value of the corresponding edge.

[0047] The local optimization unit is used to locally optimize the relative pose of the local keyframe corresponding to the current moment according to the eigengraph. Specifically, for the current moment, according to the eigengraph at the current moment, the keyframes corresponding to the graph nodes whose co-viewing degree is greater than the first co-viewing degree threshold are selected as local keyframes, and for each local keyframe, the relative pose of the local keyframe is locally optimized by using the keyframes connected to each local keyframe and whose co-viewing degree is greater than the first co-viewing degree threshold; wherein, the process of optimizing the relative pose is: determining the relative pose of the local keyframe according to the relative pose of the matching ORB features of the two keyframes, so that the pose of the local keyframe can be continuously optimized by graph optimization.

[0048] Long-distance camera tracking often causes cumulative errors. In order to eliminate the cumulative errors, the camera tracking thread of this system has added a closed-loop detection unit and a global optimization unit. Among them, the closed-loop detection unit is used to monitor the ORB features of the color-depth video frame in real time, and detect whether the color-depth video frame has a closed loop based on the number of ORB features. Specifically, a frame number threshold is introduced. When it is judged that the number of historical color-depth video frames exceeds the frame number threshold, and it is judged that the number of ORB feature matches between the current color-depth video frame and the previous color-depth video frame reaches the feature number threshold, it is considered that the color-depth video frame has a closed loop. The frame number threshold and the feature number threshold are set according to the situation.

[0049] The global optimization unit is used to globally optimize the relative poses of the keyframes in the current eigenmap when a closed loop occurs. Specifically, for each keyframe, the relative pose of the keyframe is determined based on the relative poses between the keyframe and the matching ORB features of all adjacent keyframes to achieve global optimization of the keyframe, which can eliminate the cumulative error.

[0050] In the embodiment, the local mapping thread is used to construct a real-time local map based on the color-depth video frame and the camera pose using an implicit representation method. Figure 2 As shown, the local mapping thread includes a local map initialization unit and a local map training unit, wherein the local map is an incremental implicit representation network.

[0051] The local map initialization unit is responsible for determining whether a new local map needs to be established for the current map. The determination is based on the local optimization unit of the camera tracking thread. Specifically, when it is determined based on the intrinsic map that the degree of co-visibility between the first key frame of the current local map and the current key frame is less than a second co-visibility threshold, the new local map is initialized.

[0052] The local map training unit is used to use the local key frames and corresponding relative poses output by the local optimization unit as the input of the incremental implicit representation network, and output the geometric information and color information of the target sampling points. The training optimization of the local map is achieved by training the incremental implicit representation network.

[0053] Specifically, Figure 3 As shown, the incremental implicit representation network includes a feature grid, trilinear interpolation, and a feature decoder, wherein the feature network adopts an octree structure and stores features at corner points of different levels of the octree. The specific construction process of the feature network is: the depth image corresponding to the local key frame is converted into a point cloud, an octree structure network is generated according to the point cloud, and the light of the color image corresponding to the local key frame at the relative pose is determined according to the relative pose. The light is directed to the grid and intersects with the grid, and features are stored at the corner points of the grid, which are optimized during the training process; trilinear interpolation is used to perform trilinear interpolation on the corner point features of the grid to which the sampling point belongs based on the feature grid when querying the sampling point, and the interpolation features of multiple levels are fused to obtain the feature vector of the sampling point; the feature decoder includes a color decoder and a geometric decoder, which are used to perform color decoding and geometric decoding on the feature vector of the sampling point to obtain color information and geometric information respectively.

[0054] The incremental implicit representation network can render images by volume rendering. For a given viewing angle, the depth and color of the pixel are rendered by sampling the geometric information and color information of the light line emitted by each pixel. The process of training the incremental implicit representation network is as follows: the color information and geometric information are rendered by volume rendering to obtain a rendered color image and a rendered depth image, and a photometric loss is constructed based on the difference between the rendered color image and the input color image, and a depth loss is constructed based on the difference between the rendered depth image and the input depth image. The features of the feature grid in the incremental implicit representation network and the parameters of the feature decoder are optimized based on the photometric loss and the depth loss.

[0055] In the embodiment, the global mapping thread is used to stitch and update all local maps in real time to obtain a complete global map. Figure 2 As shown, the global mapping thread includes a multi-map stitching unit and a global map updating unit. Since the local mapping thread has trained multiple local maps, the multi-map stitching unit is used to stitch these local maps generated by the local mapping thread in real time to obtain a global map. After the global optimization occurs, the camera pose of each local map will change. The global map updating unit updates all local maps that make up the global map according to the relative pose of the global optimization, and then updates the global map. Specifically, the obtained global map and local map can realize the rendering of high-resolution color-depth images and the extraction of high-precision surfaces.

[0056] In the embodiment, the uncertainty-based image rendering module is responsible for rendering according to the geometric information output by multiple local maps, and selecting the highest quality map in the pictures rendered by multiple local maps. In the local map implicit representation network, the present invention uses occupancy to characterize geometric information. Among them, the occupancy p is a real number belonging to the interval [0,1], which represents the probability that the point is occupied. An occupancy rate of 0 means that the point is not occupied, and an occupancy rate of 1 means that the point is occupied. The distribution of occupancy obeys the Bernoulli distribution. Therefore, the occupancy variance of each pixel point can be calculated by the variance formula var=p(1-p) of the Bernoulli distribution. Then, through the volume rendering method, the uncertainty of each pixel point in the rendered image can be calculated based on the occupancy variance var of each point, and then the overall uncertainty of each rendered image is obtained. Finally, based on the overall uncertainty of the rendered image corresponding to each local map, the image with the lowest overall uncertainty is selected as the rendered image output of the final model.

[0057] The real-time simultaneous positioning and mapping system based on implicit representation provided in the above embodiment can quickly and accurately complete the positioning and mapping tasks simultaneously, and has the following effects:

[0058] 1. High-precision tracking. The system tracks the input color-depth video stream, estimates the camera pose with high precision for each frame, and selects key frames. During the tracking process, the system uses local optimization, global optimization, and closed-loop detection to optimize the camera pose, ensuring accurate tracking and positioning even in long-distance tracking tasks.

[0059] 2. High-precision mapping. The mapping process of this system is divided into two parts: local mapping and global mapping. Among them, local mapping corresponds to the local optimization part of high-precision tracking. Local mapping maintains multiple local maps and optimizes the local map currently being optimized to ensure the accuracy of the local map. Global mapping splices multiple local maps. After a closed loop or global optimization occurs in the tracking process, the relative position of the local map is quickly adjusted to maintain the accuracy of the global map.

[0060] 3. High-precision image rendering and mesh extraction. Thanks to the advantages of implicit representation methods, the implicit map constructed by this system can render images of any perspective and select images using uncertainty to obtain high-quality color images and depth images. At the same time, the implicit map of this system can also extract dense meshes that can be used for interaction.

[0061] 4. Real-time performance. The system can simultaneously locate and build maps at a frequency of 10 Hz, which can be used in most practical mission scenarios and meet the real-time requirements of these scenarios.

[0062] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time simultaneous localization and mapping system based on implicit representation, characterized in that: It includes a multi-threaded positioning and mapping module, which includes a parallel camera tracking thread, a local mapping thread, and a global mapping thread; The camera tracking thread is used to track the camera pose in real time by extracting and matching feature points based on the color-depth video frames collected in real time: wherein the camera tracking thread includes a key frame selection unit, a local optimization unit, a closed loop detection unit and a global optimization unit; the key frame selection unit is used to extract ORB features from the color-depth video frames, select key frames by counting the number of ORB features, and construct an eigengraph based on the key frames. In the eigengraph, the key frames are used as graph nodes, and the edges between the graph nodes are established by the degree of common view, wherein the degree of common view is the degree of matching between the ORB features of two key frames; the local optimization unit is used to locally optimize the relative pose of the local key frames corresponding to the current moment based on the eigengraph; the closed loop detection unit is used to monitor the ORB features of the color-depth video frames in real time, and detect whether a closed loop occurs in the color-depth video frames based on the number of ORB features; the global optimization unit is used to globally optimize the relative pose of the key frames in the current eigengraph when a closed loop occurs; The local mapping thread is used to construct a local map in real time based on the color-depth video frame and the camera pose by an implicit representation method; wherein the local mapping thread includes a local map initialization unit and a local map training unit; the local map initialization unit is used to determine, based on the intrinsic map, that the co-viewing degree between the first key frame of the current local map and the current key frame is less than a second co-viewing degree threshold, and then initialize a new local map, wherein the local map is an incremental implicit representation network; the local map training unit is used to use the local key frame and the corresponding relative pose output by the local optimization unit as the input of the incremental implicit representation network, and to train the incremental implicit representation network to achieve training optimization of the local map; The global mapping thread is used to stitch and update all local maps in real time to obtain a complete global map; The global map and local map obtained by the system can realize the rendering of high-resolution color-depth images and the extraction of high-precision surfaces.

2. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: In the local optimization unit, the relative position of the local key frame corresponding to the current moment is locally optimized according to the eigengraph, including: At the current moment, according to the eigengraph at the current moment, the key frames corresponding to the graph nodes whose co-viewing degree is greater than the first co-viewing degree threshold are selected as local key frames. For each local key frame, the relative position and posture of the local key frame are locally optimized by using the key frames connected to each local key frame and whose co-viewing degree is greater than the first co-viewing degree threshold. The process of optimizing the relative pose is as follows: determining the relative pose of the local keyframe based on the relative pose of the matching ORB features of the two keyframes.

3. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: In the closed loop detection unit, whether a closed loop occurs in the color-depth video frame is detected based on the number of ORB features, including: A frame number threshold is introduced. When the number of historical color-depth video frames exceeds the frame number threshold, and the number of ORB feature matches between the current color-depth video frame and the previous color-depth video frame reaches the feature number threshold, the color-depth video frame is considered to have a closed loop.

4. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: In the global optimization unit, the pose of the key frame in the current eigenmap is globally optimized, including: For each keyframe, the relative pose of the keyframe is determined based on the relative poses between the keyframe and the matching ORB features of all its adjacent keyframes to achieve global optimization of the keyframe.

5. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: The incremental implicit representation network includes a feature grid, trilinear interpolation, and a feature decoder, wherein the feature grid adopts an octree structure, and features are stored at corner points of different levels of the octree. The specific construction process of the feature grid is: converting a depth image corresponding to a local key frame into a point cloud, generating an octree structured grid according to the point cloud, determining the light of the color image corresponding to the local key frame at the relative posture according to the relative posture, the light is directed to the grid and intersects with the grid, and features are stored at the corner points of the grid, and the features are optimized during the training process; The trilinear interpolation is used to perform trilinear interpolation on the corner point features of the grid to which the sampling point belongs based on the feature grid when querying the sampling point, and to fuse the interpolation features of multiple levels to obtain the feature vector of the sampling point; The feature decoder includes a color decoder and a geometry decoder, which are used to perform color decoding and geometry decoding on the feature vector of the sampling point to obtain color information and geometry information.

6. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: In the local map training unit, the process of training the incremental implicit representation network is: Volume rendering is used to render the color information and geometric information to obtain a rendered color image and a rendered depth image. The photometric loss is constructed based on the difference between the rendered color image and the input color image, and the depth loss is constructed based on the difference between the rendered depth image and the input depth image. The features and feature decoder parameters of the feature grid in the incremental implicit representation network are optimized based on the photometric loss and the depth loss.

7. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: The global map building thread includes a multi-map stitching unit and a global map update unit. The multi-map stitching unit is used to stitch the local maps generated by the local mapping threads in real time to obtain a global map; The global map updating unit is used to update all local maps constituting the global map according to the relative positions of the global optimization after the global optimization unit has globally optimized, thereby updating the global map.

8. The real-time simultaneous localization and mapping system based on implicit representation according to claim 1, characterized in that: It also includes an uncertainty-based image rendering module; The uncertainty-based image rendering module is used to render according to the geometric information output by multiple local maps. Specifically, in each local map, occupancy is used to characterize the geometric information. After the occupancy p of each pixel is determined, the occupancy variance var of each pixel is calculated by the variance formula var=p(1-p) of the Bernoulli distribution. The uncertainty of each pixel in the rendered image is calculated according to the occupancy variance var of each point using a volume rendering method, and then the overall uncertainty of each rendered image is obtained. The rendered image with the lowest uncertainty is selected as the final output according to the overall uncertainty of the rendered image corresponding to each local map.

Images