Relocation method of lightweight implicit neural map

By combining neural network training with an inertial navigation system, the problem of low storage and processing efficiency of traditional map methods in large-scale scenarios is solved, achieving high-precision, lightweight implicit neural map relocalization and improving the accuracy of localization and attitude estimation.

CN118913251AActive Publication Date: 2024-11-08SOUTHEAST UNIV

Patent Information

Application Number
CN202411156421.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-08
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

Traditional explicit mapping methods suffer from high memory consumption, low resolution, and difficulty in achieving global consistency in large-scale scenarios. Visual locators are greatly affected by the environment, and LiDAR data processing is complex and requires efficient storage and processing within a lightweight implicit neural mapping framework.

Method used

By using neural network training map data, combined with inertial navigation system and point-to-model matching technology, and through motion compensation correction, loop closure detection and correction, pre-integration estimation and factor graph framework fusion, a lightweight implicit neural map relocalization is achieved.

Benefits of technology

It achieves efficient construction of large-scale field maps, improves relocation accuracy and robustness, reduces the storage requirements of traditional methods, and enhances the accuracy of position and attitude estimation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118913251A_ABST
    Figure CN118913251A_ABST
Patent Text Reader

Abstract

The invention discloses a relocation method for a lightweight implicit neural map, which comprises the following steps: firstly, performing motion compensation correction on a current point cloud, performing voxel downsampling, and training to construct a lightweight and high-resolution implicit neural map; closed-loop detection and correction are carried out by using neural point features, so that the consistency and accuracy of the map are ensured; pre-integral estimation of an inertial navigation system is introduced to provide a priori initial value for implicit registration, and state estimation based on a lightweight implicit neural map is realized by using a registration method from a point to an implicit neural model; environmental constraints provided by laser repositioning and dynamic estimation of an inertial navigation system are effectively combined, and real-time robust positioning and attitude determination are realized by fusing a repositioning factor and a pre-integration factor based on a factor graph framework. Through the novel lightweight implicit neural map storage model, the problems that the traditional point cloud data storage amount is large and the relocation performance is influenced by the map resolution can be solved, and meanwhile, the improvement of the location precision can be realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of lightweight map repositioning, relates to deep learning technology of laser radar, and specifically to a method for applying an optimization model and lightweight map training construction. Background Art

[0002] Map matching and positioning is a necessary means for robot navigation and positioning, but a priori map matching and positioning in large-scale scenes are still challenging. Traditional explicit map expression methods have problems such as large memory usage, low resolution, and difficulty in achieving global consistency. Neural implicit expression fits the observed data in the scene through a neural network, and can implicitly query attributes such as distance, color, and semantic information at any location. This method not only has advantages in storage efficiency and continuity, but also can achieve high-fidelity map reconstruction by optimizing local latent features.

[0003] In the field of visual positioning, implicit radiation fields have made significant progress in recent years and can achieve real-time and high-precision implicit 3D reconstruction. The core advantage of this technology is that it can effectively model complex 3D scenes through deep learning methods. However, despite the excellent performance of implicit radiation fields in terms of accuracy and real-time performance, visual sensors still face some challenges compared to lidar. It is easily affected by the environment and is highly sensitive to factors such as lighting changes, weather conditions, and changes in viewing angles. Lidar provides rich spatial information, and its characteristics are suitable for large-scale 3D reconstruction.

[0004] Based on the above principles, LiDAR sensors are used in large-scale 3D field map construction and relocalization. LiDAR data is large and complex, and it is necessary to efficiently process and store LiDAR point cloud data in a lightweight implicit neural map framework. The accuracy of point-to-model matching technology when processing LiDAR data needs to be guaranteed, especially in the presence of noise, occlusion or environmental changes. In combination with neural networks and inertial navigation systems, LiDAR data needs to be effectively fused with other sensor data to improve the accuracy and robustness of relocalization. Summary of the invention

[0005] To solve the above problems, the present invention discloses a lightweight implicit neural map repositioning method. According to the current map storage requirements, a neural network is used to train map data to achieve large-scale field map construction, and an inertial navigation system and point-to-model matching technology are used to realize the repositioning function.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A lightweight implicit neural map relocalization method, the specific steps are as follows:

[0008] Step 1: Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map;

[0009] Step 2: Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map.

[0010] Step 3: Introduce the pre-integrated estimation of the inertial navigation system to provide a priori initial values ​​for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map.

[0011] Step 4: The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination.

[0012] Furthermore, the point cloud motion compensation correction in step 1 is as follows:

[0013]

[0014] The loss function of the training process is designed as:

[0015] loss = L bce +λ e L e (2)

[0016] In formula (1), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (2), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.

[0017] Furthermore, the neural point features described in step 2 are subjected to closed-loop detection and correction, and the closed-loop detection judgment formula is as follows:

[0018] p c -p h <d th (3)

[0019] pc and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th , it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative position calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows:

[0020] x i ←δ T x i ,q i ←δ q q i (4)

[0021] x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, while retaining the neural points with high stability in each voxel of the map.

[0022] Furthermore, the pre-integrated estimate of the inertial navigation system introduced in step 3 is used to provide a priori initial values ​​for implicit registration, and the point-to-implicit neural model registration method is used to realize pose estimation; wherein the measurement equation of the inertial unit is as follows:

[0023]

[0024] in and Respectively represent the measurement values ​​of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the bias of the gyroscope and accelerometer respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows:

[0025]

[0026] and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows:

[0027]

[0028] Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .

[0029] Furthermore, the laser relocation factor and the pre-integration factor are fused based on the factor graph framework described in step 4. According to the difference between the observed value and the model predicted value, these error functions are combined into an overall optimization objective, and the minimization equation is as follows:

[0030]

[0031] Where M(x) represents the marginalized prior residual of the optimization process. Marginalization can remove some redundant information and retain only the most important information for the current optimization problem, thereby reducing data redundancy and improving optimization efficiency. There are a total of n states that need to be optimized, among which the relocation and pre-integration residuals of two adjacent frames i and j are respectively and

[0032] The beneficial effects of the present invention are:

[0033] By adopting neural network to train map data, the present invention can efficiently realize large-scale field map construction; neural implicit map can effectively represent complex environmental structure, reducing the demand for large-scale explicit map storage in traditional methods; combining inertial navigation system and point-to-model matching technology, the present invention realizes high-precision repositioning function; inertial navigation system provides continuous estimation of motion state, while point-to-model matching technology improves the accuracy of position and attitude estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of the lightweight implicit neural map relocalization based on the present invention;

[0035] Figure 2 It is a diagram of the experimental equipment;

[0036] Figure 3 It is a real-time test reconstruction image of the system in a large indoor scene. DETAILED DESCRIPTION

[0037] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0038] As shown in the figure, the relocalization algorithm of a lightweight implicit neural map described in the present invention has the following specific steps:

[0039] Step 1. Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map. The point cloud motion compensation correction is as follows:

[0040]

[0041] The loss function of the training process is designed as:

[0042] loss = L bce +λ e L e (2)

[0043] In formula (1), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (2), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.

[0044] Step 2. Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map and extract the environmental point cloud features. The closed-loop detection judgment formula is as follows:

[0045] p c -p h <d th (3)

[0046] p c and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th, it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative position calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows:

[0047] x i ←δ T x i ,q i ←δ q q i (4)

[0048] x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, while retaining the neural points with high stability in each voxel of the map.

[0049] Step 3. Introduce the pre-integrated estimate of the inertial navigation system to provide a priori initial values ​​for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map, where the measurement equation of the inertial unit is as follows:

[0050]

[0051] in and Respectively represent the measurement values ​​of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the bias of the gyroscope and accelerometer respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows:

[0052]

[0053] and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows:

[0054]

[0055] Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .

[0056] Step 4. The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination. Specifically, these error functions are combined into an overall optimization objective based on the difference between the observed value and the model prediction value. The minimization equation is as follows:

[0057]

[0058] Where M(x) represents the marginalized prior residual of the optimization process. Marginalization can remove some redundant information and retain only the most important information for the current optimization problem, thereby reducing data redundancy and improving optimization efficiency. There are a total of n states that need to be optimized, among which the relocation residual and pre-integration residual of two adjacent frames i and j are respectively and

[0059] Attached Figure 2 It is an experimental device, using a Suteng 32-line laser radar and an EPSON MG-370 inertial navigation unit. The data information between sensors and the interaction information between modules are transmitted through the topic of the Robot Operating System (ROS). The external parameters of the laser radar and inertial navigation have been accurately calibrated. Figure 3 It is the re-localization experiment and mapping result of the experimental vehicle-mounted platform in a large scene. The experiment shows that the memory usage of the map can be reduced from 13.3GB of the original point cloud to 138MB. The accuracy of the pure lidar implicit registration algorithm is better than the traditional voxel matching positioning algorithm, and the positioning accuracy is improved by 61.2%. After integrating inertial navigation, the positioning accuracy is further improved by 51.2% compared with the traditional fusion algorithm.

[0060] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A lightweight implicit neural map relocalization method, characterized in that: The specific method is as follows: Step 1: Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map; Step 2: Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map. Step 3: Introduce the pre-integrated estimation of the inertial navigation system to provide a priori initial values ​​for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map. Step 4: The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination.

2. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The point cloud motion compensation correction in step 1 is as follows: The loss function of the training process is designed as: loss=L bce +λ e L e (2) In formula (1), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (2), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.

3. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The neural point features described in step 2 are closed-loop detected and corrected, and the closed-loop detection judgment formula is as follows: p c -p h <d th (3) p c and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th , it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative posture calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows: x i ←δ T x i ,q i ←δ q q i (4) x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, while retaining the neural points with high stability in each voxel of the map.

4. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The pre-integrated estimate of the inertial navigation system introduced in step 3 is used to provide a priori initial values ​​for implicit registration, and the point-to-implicit neural model registration method is used to realize pose estimation; the measurement equation of the inertial unit is as follows: in and Respectively represent the measurement values ​​of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the biases of the gyroscope and accelerometer, respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows: and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows: Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .

5. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The laser relocation factor and the pre-integration factor are fused based on the factor graph framework described in step 4. Specifically, these error functions are combined into an overall optimization objective based on the difference between the observed value and the model predicted value. The minimization equation is as follows: Where M(x) represents the marginalized prior residual of the optimization process. There are n states that need to be optimized at present, among which the lidar relocalization residual and pre-integration residual of two adjacent frames i and j are respectively and

Citation Information

Patent Citations

  • Simultaneous localization and mapping method based on vision and laser radar

    CN112258600A

  • Positioning method and device based on multi-sensor fusion and storage medium

    CN112304307A

  • Continuous laser scanning registration assisted inertial positioning and attitude determination method

    CN113639722A

  • Model training method, repositioning method, equipment and medium

    CN116935159A

  • Missile-borne strapdown inertial navigation system attitude keeping method based on attitude pre-integration

    CN118111427A

Cited By

  • Mobile robot positioning and navigation method based on laser SLAM

    CN119828161A

  • Control method and device for cooperative movement of multiple mechanical arms

    CN120056135A