Relocation method of lightweight implicit neural map
By combining neural network training with an inertial navigation system, the problem of low storage and processing efficiency of traditional map methods in large-scale scenarios is solved, achieving high-precision, lightweight implicit neural map relocalization and improving the accuracy of localization and attitude estimation.
Patent Information
- Application Number
- CN202411156421.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-22
AI Technical Summary
Traditional explicit mapping methods suffer from high memory consumption, low resolution, and difficulty in achieving global consistency in large-scale scenarios. Visual locators are greatly affected by the environment, and LiDAR data processing is complex and requires efficient storage and processing within a lightweight implicit neural mapping framework.
By using neural network training map data, combined with inertial navigation system and point-to-model matching technology, and through motion compensation correction, loop closure detection and correction, pre-integration estimation and factor graph framework fusion, a lightweight implicit neural map relocalization is achieved.
It achieves efficient construction of large-scale field maps, improves relocation accuracy and robustness, reduces the storage requirements of traditional methods, and enhances the accuracy of position and attitude estimation.
Smart Images

Figure CN118913251A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of lightweight map repositioning, relates to deep learning technology of laser radar, and specifically to a method for applying an optimization model and lightweight map training construction. Background Art
[0002] Map matching and positioning is a necessary means for robot navigation and positioning, but a priori map matching and positioning in large-scale scenes are still challenging. Traditional explicit map expression methods have problems such as large memory usage, low resolution, and difficulty in achieving global consistency. Neural implicit expression fits the observed data in the scene through a neural network, and can implicitly query attributes such as distance, color, and semantic information at any location. This method not only has advantages in storage efficiency and continuity, but also can achieve high-fidelity map reconstruction by optimizing local latent features.
[0003] In the field of visual positioning, implicit radiation fields have made significant progress in recent years and can achieve real-time and high-precision implicit 3D reconstruction. The core advantage of this technology is that it can effectively model complex 3D scenes through deep learning methods. However, despite the excellent performance of implicit radiation fields in terms of accuracy and real-time performance, visual sensors still face some challenges compared to lidar. It is easily affected by the environment and is highly sensitive to factors such as lighting changes, weather conditions, and changes in viewing angles. Lidar provides rich spatial information, and its characteristics are suitable for large-scale 3D reconstruction.
[0004] Based on the above principles, LiDAR sensors are used in large-scale 3D field map construction and relocalization. LiDAR data is large and complex, and it is necessary to efficiently process and store LiDAR point cloud data in a lightweight implicit neural map framework. The accuracy of point-to-model matching technology when processing LiDAR data needs to be guaranteed, especially in the presence of noise, occlusion or environmental changes. In combination with neural networks and inertial navigation systems, LiDAR data needs to be effectively fused with other sensor data to improve the accuracy and robustness of relocalization. Summary of the invention
[0005] To solve the above problems, the present invention discloses a lightweight implicit neural map repositioning method. According to the current map storage requirements, a neural network is used to train map data to achieve large-scale field map construction, and an inertial navigation system and point-to-model matching technology are used to realize the repositioning function.
[0006] To achieve the above object, the technical solution of the present invention is as follows:
[0007] A lightweight implicit neural map relocalization method, the specific steps are as follows:
[0008] Step 1: Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map;
[0009] Step 2: Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map.
[0010] Step 3: Introduce the pre-integrated estimation of the inertial navigation system to provide a priori initial values for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map.
[0011] Step 4: The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination.
[0012] Furthermore, the point cloud motion compensation correction in step 1 is as follows:
[0013]
[0014] The loss function of the training process is designed as:
[0015] loss = L bce +λ e L e (2)
[0016] In formula (1), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (2), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.
[0017] Furthermore, the neural point features described in step 2 are subjected to closed-loop detection and correction, and the closed-loop detection judgment formula is as follows:
[0018] p c -p h <d th (3)
[0019] pc and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th , it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative position calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows:
[0020] x i ←δ T x i ,q i ←δ q q i (4)
[0021] x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, while retaining the neural points with high stability in each voxel of the map.
[0022] Furthermore, the pre-integrated estimate of the inertial navigation system introduced in step 3 is used to provide a priori initial values for implicit registration, and the point-to-implicit neural model registration method is used to realize pose estimation; wherein the measurement equation of the inertial unit is as follows:
[0023]
[0024] in and Respectively represent the measurement values of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the bias of the gyroscope and accelerometer respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows:
[0025]
[0026] and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows:
[0027]
[0028] Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .
[0029] Furthermore, the laser relocation factor and the pre-integration factor are fused based on the factor graph framework described in step 4. According to the difference between the observed value and the model predicted value, these error functions are combined into an overall optimization objective, and the minimization equation is as follows:
[0030]
[0031] Where M(x) represents the marginalized prior residual of the optimization process. Marginalization can remove some redundant information and retain only the most important information for the current optimization problem, thereby reducing data redundancy and improving optimization efficiency. There are a total of n states that need to be optimized, among which the relocation and pre-integration residuals of two adjacent frames i and j are respectively and
[0032] The beneficial effects of the present invention are:
[0033] By adopting neural network to train map data, the present invention can efficiently realize large-scale field map construction; neural implicit map can effectively represent complex environmental structure, reducing the demand for large-scale explicit map storage in traditional methods; combining inertial navigation system and point-to-model matching technology, the present invention realizes high-precision repositioning function; inertial navigation system provides continuous estimation of motion state, while point-to-model matching technology improves the accuracy of position and attitude estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the lightweight implicit neural map relocalization based on the present invention;
[0035] Figure 2 It is a diagram of the experimental equipment;
[0036] Figure 3 It is a real-time test reconstruction image of the system in a large indoor scene. DETAILED DESCRIPTION
[0037] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.
[0038] As shown in the figure, the relocalization algorithm of a lightweight implicit neural map described in the present invention has the following specific steps:
[0039] Step 1. Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map. The point cloud motion compensation correction is as follows:
[0040]
[0041] The loss function of the training process is designed as:
[0042] loss = L bce +λ e L e (2)
[0043] In formula (1), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (2), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.
[0044] Step 2. Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map and extract the environmental point cloud features. The closed-loop detection judgment formula is as follows:
[0045] p c -p h <d th (3)
[0046] p c and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th, it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative position calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows:
[0047] x i ←δ T x i ,q i ←δ q q i (4)
[0048] x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, while retaining the neural points with high stability in each voxel of the map.
[0049] Step 3. Introduce the pre-integrated estimate of the inertial navigation system to provide a priori initial values for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map, where the measurement equation of the inertial unit is as follows:
[0050]
[0051] in and Respectively represent the measurement values of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the bias of the gyroscope and accelerometer respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows:
[0052]
[0053] and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows:
[0054]
[0055] Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .
[0056] Step 4. The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination. Specifically, these error functions are combined into an overall optimization objective based on the difference between the observed value and the model prediction value. The minimization equation is as follows:
[0057]
[0058] Where M(x) represents the marginalized prior residual of the optimization process. Marginalization can remove some redundant information and retain only the most important information for the current optimization problem, thereby reducing data redundancy and improving optimization efficiency. There are a total of n states that need to be optimized, among which the relocation residual and pre-integration residual of two adjacent frames i and j are respectively and
[0059] Attached Figure 2 It is an experimental device, using a Suteng 32-line laser radar and an EPSON MG-370 inertial navigation unit. The data information between sensors and the interaction information between modules are transmitted through the topic of the Robot Operating System (ROS). The external parameters of the laser radar and inertial navigation have been accurately calibrated. Figure 3 It is the re-localization experiment and mapping result of the experimental vehicle-mounted platform in a large scene. The experiment shows that the memory usage of the map can be reduced from 13.3GB of the original point cloud to 138MB. The accuracy of the pure lidar implicit registration algorithm is better than the traditional voxel matching positioning algorithm, and the positioning accuracy is improved by 61.2%. After integrating inertial navigation, the positioning accuracy is further improved by 51.2% compared with the traditional fusion algorithm.
[0060] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.
Claims
1. A lightweight implicit neural map relocalization method, characterized in that: The specific method is as follows: Step 1: Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map; Step 2: Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map. Step 3: Introduce the pre-integrated estimation of the inertial navigation system to provide a priori initial values for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map. Step 4: The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination.
2. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The point cloud motion compensation correction in step 1 is as follows: The loss function of the training process is designed as: loss=L bce +λ e L e (2) In formula (1), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (2), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.
3. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The neural point features described in step 2 are closed-loop detected and corrected, and the closed-loop detection judgment formula is as follows: p c -p h <d th (3) p c and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th , it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative posture calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows: x i ←δ T x i ,q i ←δ q q i (4) x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, while retaining the neural points with high stability in each voxel of the map.
4. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The pre-integrated estimate of the inertial navigation system introduced in step 3 is used to provide a priori initial values for implicit registration, and the point-to-implicit neural model registration method is used to realize pose estimation; the measurement equation of the inertial unit is as follows: in and Respectively represent the measurement values of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the biases of the gyroscope and accelerometer, respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows: and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows: Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .
5. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The laser relocation factor and the pre-integration factor are fused based on the factor graph framework described in step 4. Specifically, these error functions are combined into an overall optimization objective based on the difference between the observed value and the model predicted value. The minimization equation is as follows: Where M(x) represents the marginalized prior residual of the optimization process. There are n states that need to be optimized at present, among which the lidar relocalization residual and pre-integration residual of two adjacent frames i and j are respectively and
Citation Information
Patent Citations
Simultaneous localization and mapping method based on vision and laser radar
CN112258600A
Positioning method and device based on multi-sensor fusion and storage medium
CN112304307A
Continuous laser scanning registration assisted inertial positioning and attitude determination method
CN113639722A
Model training method, repositioning method, equipment and medium
CN116935159A
Missile-borne strapdown inertial navigation system attitude keeping method based on attitude pre-integration
CN118111427A
Cited By
Mobile robot positioning and navigation method based on laser SLAM
CN119828161A
Control method and device for cooperative movement of multiple mechanical arms
CN120056135A
