A Relocalization Method for Lightweight Implicit Neural Maps

Abstract

The present invention discloses a relocalization method for a lightweight implicit neural map, including: First, perform motion compensation and correction on the current point cloud, and downsample it by voxel to train and construct a lightweight and high-resolution implicit neural map; use neural point features for loop detection and correction to ensure the consistency and accuracy of the map; introduce the pre-integration estimation of the inertial navigation system to provide a prior initial value for implicit registration, and at the same time use the registration method from points to the implicit neural model to achieve state estimation based on the lightweight implicit neural map; effectively combine the environmental constraints provided by laser relocalization and the dynamic estimation of the inertial navigation system, and use the factor graph framework to fuse the relocalization factor and the pre-integration factor to achieve real-time and robust positioning and pose determination. The present invention can solve the problems of large storage capacity of traditional point cloud data and the influence of map resolution on relocalization performance through a novel lightweight implicit neural map storage model, and at the same time can improve the positioning accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of lightweight map relocalization, involves the deep learning technology of lidar, and specifically relates to a method for constructing an application optimization model and lightweight map training. Background Art

[0002] Map matching localization is a necessary means in robot navigation and localization. However, prior map matching localization in large-scale scenarios still poses challenges. Traditional explicit map representation methods have problems such as large memory occupancy, low resolution, and difficulty in achieving global consistency. Neural implicit representation fits the observed data in the scene through a neural network and can implicitly query attributes such as distance, color, and semantic information at any position. This method not only has advantages in storage efficiency and continuity but also can achieve high-fidelity map reconstruction by optimizing local latent features.

[0003] In the field of visual localization, implicit radiance fields have made significant progress in recent years, enabling real-time high-precision implicit three-dimensional reconstruction. The core advantage of this technology lies in its ability to effectively model complex three-dimensional scenes through deep learning methods. However, although implicit radiance fields perform well in terms of accuracy and real-time performance, visual sensors still face some challenges compared to lidar. They are easily affected by the environment and are highly sensitive to factors such as lighting changes, weather conditions, and perspective changes. Lidar provides rich spatial information, and its characteristics are suitable for realizing large-scale three-dimensional reconstruction.

[0004] According to the above principle, lidar sensors are applied in large-scale three-dimensional field map construction and relocalization. Lidar data is large and complex, and it is necessary to efficiently process and store lidar point cloud data in a lightweight implicit neural map framework. The accuracy of point-to-model matching technology needs to be ensured when processing lidar data, especially in the presence of noise, occlusion, or environmental changes. When combining a neural network and an inertial navigation system, it is necessary to effectively fuse lidar data with other sensor data to improve the accuracy and robustness of relocalization. Summary of the Invention

[0005] To solve the above problems, the present invention discloses a relocalization method for a lightweight implicit neural map. According to the current requirements for map storage, neural networks are used to train map data to achieve large-scale field map construction, and an inertial navigation system and point-to-model matching technology are used to achieve the relocalization function.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A relocalization method for a lightweight implicit neural map, the specific steps are as follows:

[0008] Step 1: Perform motion compensation and correction on the current point cloud, and perform voxel downsampling to train and construct a lightweight and high-resolution implicit neural map;

[0009] Step 2: Use neural point features for loop closure detection and correction. The correction result of the loop closure is used to correct the pose state of the neural points to ensure the consistency and accuracy of the map;

[0010] Step 3: Introduce the pre-integration estimation of the inertial navigation system to provide a prior initial value for implicit registration. At the same time, use the registration method from points to the implicit neural model to achieve state estimation based on the lightweight implicit neural map;

[0011] Step 4: Effectively combine the environmental constraints provided by laser relocalization and the dynamic estimation of the inertial navigation system. Use the factor graph framework to fuse the relocalization factor and the pre-integration factor to achieve real-time and robust pose estimation.

[0012] Further, the point cloud motion compensation and correction in Step 1 is as follows:

[0013]

[0014] The loss function loss in the training process is designed as:

[0015] loss = L bce + λ e L e (2)

[0016] In Equation (1), k and j respectively represent the sampling time of the last point in this frame and the time when the current point is located; and respectively represent the values of the points corresponding to the two times in the lidar coordinate system; is the external parameter matrix of the inertial navigation and the lidar; is the pose of the current point predicted by the inertial navigation recursion in the global coordinate system; is the pose change amount between the two times, where (·) -1 represents the inverse operation; in Equation (2), L bce and L e respectively represent the cross-entropy loss and the regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.

[0017] Further, for the loop closure detection and correction using the neural point features in Step 2, the loop closure detection judgment formula is as follows:

[0018] p c - p h <d th (3)

[0019] pc and p h represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th , it is determined as a closed loop. The adopted scan context and its search algorithm make the closed loop unaffected by the viewpoint change, with translational and rotational invariance, and can achieve closed-loop detection of reverse equal-angle transformation. At the same time, the relative pose calculated by the closed-loop module is used to update the neural point state of the implicit neural field map, enabling the lightweight map to maintain global consistency. The neural point position update correction formula is as follows:

[0020] x i ←δ T x i , q i ←δ q q i (4)

[0021] x i and q i are the position and attitude of the neural point. The position correction amount δ T and the rotation correction amount δ q are used to update the neural points in sequence. After global optimization, the results in the neural point storage structure are updated, and at the same time, the neural points with high stability are retained in each voxel of the map.

[0022] Furthermore, the pre-integration estimation of the inertial navigation system introduced in step 3 is used to provide a prior initial value for implicit registration, and at the same time, the pose estimation is realized by using the registration method from points to the implicit neural model. The measurement equation of the inertial unit is as follows:

[0023]

[0024] where and represent the measurement values of the gyroscope and the accelerometer at time t respectively, ω B (t) and a W (t) represent the true value of the gyroscope in the body frame and the true value of the acceleration in the world frame; g w is the gravity vector in the world frame; is the attitude matrix of the current vehicle in the world frame; b ω (t) and b a (t) are the biases of the gyroscope and the accelerometer respectively; η ω (t) and η a (t) are the process noises of the gyroscope and the accelerometer respectively. The recurrence formula of the inertial navigation system is as follows:

[0025]

[0026] and are the state variables of inertial navigation at the current and next moments, is the input of the inertial measurement unit, is the process noise, and Δt is the time variation; (is the generalized addition. After obtaining the initial estimate by recursive calculation in the inertial navigation system, the directed distance value is predicted for each point in the current frame in the map model, and the optimal solution is obtained by optimizing to minimize the value of the directed distance field. The solution process is as follows:

[0027]

[0028] In the formula, SDF represents the effective distance field; for all points in the current frame calculate the directed distance for each point p to obtain the optimal pose T * .

[0029] Furthermore, the laser relocalization factor and the pre-integration factor are fused based on the factor graph framework in step 4. According to the difference between the observed value and the model prediction value, these error functions are combined into an overall optimization objective, and the minimization equation is as follows:

[0030]

[0031] In the formula, M(x) represents the marginalized prior residual of the optimization process. Marginalization can remove some redundant information and only retain the most important information for the current optimization problem, thereby reducing data redundancy and improving optimization efficiency; there are a total of n state variables to be optimized currently, and the relocalization and pre-integration residuals between two adjacent frames i and j are respectively and

[0032] The beneficial effects of the present invention are:

[0033] By using neural network to train map data, the present invention can efficiently realize large-scale field map construction; the neural implicit map can effectively represent complex environmental structures, reducing the need for storing large-scale explicit maps in traditional methods; combining the inertial navigation system and the point-to-model matching technology, the present invention realizes high-precision relocalization function; the inertial navigation system provides continuous estimation of the motion state, while the point-to-model matching technology improves the accuracy of position and attitude estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is the flowchart of relocalization based on the lightweight implicit neural map of the present invention;

[0035] Figure 2 is the experimental equipment diagram;

[0036] Figure 3 is the real-time test reconstruction diagram of the system in a large indoor scene. Detailed implementation manners

[0037] The present invention will be further illustrated below in conjunction with the accompanying drawings and specific implementation manners. It should be understood that the following specific implementation manners are only used to illustrate the present invention and not to limit the scope of the present invention.

[0038] As shown in the figure, a relocalization algorithm for a lightweight implicit neural map according to the present invention specifically includes the following steps:

[0039] Step 1. Perform motion compensation correction on the current point cloud and voxel downsampling, and train and construct a lightweight and high-resolution implicit neural map. The motion compensation correction of the point cloud is as follows:

[0040]

[0041] The loss function loss in the training process is designed as:

[0042] loss = L bce + λ e L e (2)

[0043] In formula (1), k and j respectively represent the sampling time of the last point in this frame and the time when the current point is located; and respectively represent the values of the points corresponding to the two times in the lidar coordinate system; is the external parameter matrix of the inertial navigation and the lidar; is the pose of the current point predicted by the inertial navigation recursion in the global coordinate system; is the pose change amount between the two times, where (·) -1 represents the inverse operation; in formula (2), L bce and L e respectively represent the cross-entropy loss and the regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.

[0044] Step 2. Use the neural point features for loop closure detection and correction. The correction result of the loop closure is used to correct the pose state of the neural points to ensure the consistency and accuracy of the map. Extract the environmental point cloud features. The loop closure detection judgment formula is as follows:

[0045] p c - p h <d th (3)

[0046] p c and p h respectively represent the positions of the current frame and the historical frame. When the position distance is less than a certain threshold d th, it is determined as a closed loop; the adopted scan context and its search algorithm make the closed loop unaffected by the viewpoint change, with translational and rotational invariance, and can achieve closed-loop detection of reverse equal-angle transformation; at the same time, the relative pose calculated by the closed-loop module is used to update the neural point state of the implicit neural field map, enabling the lightweight map to maintain global consistency. The neural point position update correction formula is as follows:

[0047] x i ←δ T x i ,q i ←δ q q i (4)

[0048] x i and q i are the position and attitude of the neural point, and the position correction amount δ T and the rotation correction amount δ q are used to update the neural points in sequence; after global optimization, the results in the neural point storage structure are updated, and at the same time, the neural points with high stability are retained in each voxel of the map.

[0049] Step 3. Introduce the pre-integration estimation of the inertial navigation system to provide a prior initial value for implicit registration. At the same time, use the registration method from points to the implicit neural model to achieve state estimation based on the lightweight implicit neural map. The measurement equation of the inertial unit is as follows:

[0050]

[0051] where and represent the measurement values of the gyroscope and accelerometer at time t respectively, ω B (t) and a W (t) represent the true value of the gyroscope in the vehicle coordinate system and the true value of the acceleration in the world coordinate system; g w is the gravity vector in the world coordinate system; is the attitude matrix of the current vehicle in the world coordinate system; b ω (t) and b a (t) are the biases of the gyroscope and accelerometer respectively; η ω (t) and η a (t) are the process noises of the gyroscope and accelerometer respectively. The recurrence formula of the inertial navigation system is as follows:

[0052]

[0053] and are the state quantities of the inertial navigation at the current time and the next time, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; ( is the generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is to minimize the value of the directed distance field. The solution process is as follows:

[0054]

[0055] Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * .

[0056] Step 4. The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination. Specifically, these error functions are combined into an overall optimization objective based on the difference between the observed value and the model prediction value. The minimization equation is as follows:

[0057]

[0058] Where M(x) represents the marginalized prior residual of the optimization process. Marginalization can remove some redundant information and retain only the most important information for the current optimization problem, thereby reducing data redundancy and improving optimization efficiency. There are a total of n states that need to be optimized, among which the relocation residual and pre-integration residual of two adjacent frames i and j are respectively and

[0059] Attached Figure 2 It is an experimental device, using a Suteng 32-line laser radar and an EPSON MG-370 inertial navigation unit. The data information between sensors and the interaction information between modules are transmitted through the topic of the Robot Operating System (ROS). The external parameters of the laser radar and inertial navigation have been accurately calibrated. Figure 3 It is the re-localization experiment and mapping result of the experimental vehicle-mounted platform in a large scene. The experiment shows that the memory usage of the map can be reduced from 13.3GB of the original point cloud to 138MB. The accuracy of the pure lidar implicit registration algorithm is better than the traditional voxel matching positioning algorithm, and the positioning accuracy is improved by 61.2%. After integrating inertial navigation, the positioning accuracy is further improved by 51.2% compared with the traditional fusion algorithm.

[0060] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A lightweight implicit neural map relocalization method, characterized in that: The specific method is as follows: Step 1: Perform motion compensation correction on the current point cloud, downsample the voxels, and train to build a lightweight and high-resolution implicit neural map; Step 2: Use the neural point features to perform closed-loop detection and correction. The closed-loop correction results are used to correct the position and posture of the neural points to ensure the consistency and accuracy of the map. The closed-loop detection judgment formula is as follows: p c -p h <d th (1) p c and p h Represent the positions of the current frame and the historical frame respectively. When the position distance is less than a certain threshold d th , it is determined to be a closed loop; the scanning context and its search algorithm adopted make the closed loop unaffected by the change of viewpoint, with translation and rotation invariance, and can realize closed loop detection of reverse and large angle transformation; at the same time, the relative posture calculated by the closed loop module is used to update the neural point state of the implicit neural field map, so that the lightweight map can maintain global consistency. The neural point position update correction formula is as follows: x i ←δ T x i ,q i ←δ q q i (2) x i and q i is the position and posture of the neural point, and the closed-loop position correction δ T and the rotation correction δ q Used to update the neural points sequentially; after global optimization, update the results in the neural point storage structure, and retain the neural points with high stability in each voxel of the map; Step 3: Introduce the pre-integrated estimation of the inertial navigation system to provide a priori initial values ​​for implicit registration. At the same time, use the point-to-implicit neural model registration method to achieve state estimation based on a lightweight implicit neural map. The measurement equation of the inertial unit is as follows: in and Respectively represent the measurement values ​​of the gyroscope and accelerometer at time t, ω B (t) and a W (t) represents the true value of the gyro in the carrier system and the true value of the acceleration in the world system; g w is the gravity vector in the world system; is the posture matrix of the current carrier in the world system; b ω (t) and b a (t) are the biases of the gyroscope and accelerometer, respectively; η ω (t) and η a (t) are the process noise of the gyroscope and accelerometer respectively; the recursive formula of the inertial navigation system is as follows: and is the state quantity of the inertial navigation at the current moment and the next moment, is the input of the inertial measurement unit, is the process noise, Δt is the time variation; It is a generalized addition. After the inertial navigation system recursively obtains the initial estimate, each point in the current frame is used to predict the directed distance value in the map model. The optimization solution is made to minimize the value of the directed distance field. The solution process is as follows: Where SDF represents the effective distance field; for all points in the current frame Calculate the directed distance point by point p to get the optimal pose T * ; Step 4: The environmental constraints provided by laser relocation and the dynamic estimation of the inertial navigation system are effectively combined, and the relocation factor and the pre-integration factor are fused based on the factor graph framework to achieve real-time robust positioning and attitude determination.

2. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The point cloud motion compensation correction in step 1 is as follows: The loss function of the training process is designed as: loss=L bce +λ e L e (8) In formula (7), k and j represent the sampling time of the last point of this frame and the time of the current point respectively; and It represents the value of the corresponding point at two moments in the laser radar coordinate system; is the external parameter matrix of inertial navigation and lidar; is the position and posture of the current point in the global coordinate system predicted by inertial navigation recursion; is the change in posture between two moments, where (·) -1 represents the inverse operation; in formula (8), L bce and L e Respectively represent the cross entropy loss and regularization loss function, and λ e is the hyperparameter adjustment term corresponding to the regularization loss.

3. The method for relocating a lightweight implicit neural map according to claim 1, characterized in that: The laser relocation factor and the pre-integration factor are fused based on the factor graph framework described in step 4. Specifically, these error functions are combined into an overall optimization objective based on the difference between the observed value and the model predicted value. The minimization equation is as follows: Where M(x) represents the marginalized prior residual of the optimization process. There are n states that need to be optimized at present, among which the lidar relocalization residual and pre-integration residual of two adjacent frames i and j are respectively and

Images