Odometry method and apparatus based on neural distance field, and electronic device and storage medium

Abstract

An odometry method and apparatus based on a neural distance field, and an electronic device and a storage medium. In the method, collected point cloud data is divided into two parts. One part serves as positive samples for performing real-time training on a machine learning model which uses a neural radiance field (NeRF). Then, the remaining point cloud data is inputted into the trained model to generate signed distance function (SDF) values, which serve as neural distance field values. Moreover, under ideal conditions, the signed distance function (SDF) values of points that are reflected from an object surface should all be zero. Therefore, the neural distance field values generated in this way can be used for iterative adjustment toward zero, so as to reduce or eliminate errors in the point cloud data, such that movement directions of points in an odometry map and corresponding movement distances can be obtained after the iteration is completed, thereby obtaining an updated odometry map that can more accurately represent a scanned environment, and significantly improving the measurement accuracy of SLAM.

Description

Odometry methods and devices, electronic devices and storage media based on neural distance fields

[0001] This patent application claims priority to Chinese patent application No. 2025103535660, filed on March 24, 2025, which is incorporated herein by reference in its entirety. Technical Field

[0002] This application relates to the technical field, and more particularly to an odometry method and apparatus, electronic device and storage medium based on neural distance field. Background Technology

[0003] Simultaneous Localization and Mapping (SLAM) is a key technology for autonomous mobile robots in large-scale map building. Especially in outdoor environments, mobile robots typically rely on LiDAR (LiDAR) sensors to perform SLAM tasks, achieving high-precision localization and navigation. This high-precision localization capability not only improves the robot's efficiency in downstream tasks such as navigation, detection, and perception, but also significantly enhances its environmental adaptability. However, due to the characteristics of large-scale mapping, SLAM systems often face the challenge of accumulated errors in outdoor environments, which can severely affect the robot's localization accuracy.

[0004] The accumulated error in SLAM primarily stems from inaccurate attitude estimation during odometry, and this error intensifies as alignment errors between adjacent frames gradually accumulate. To address this issue, various solutions have been proposed in existing technologies. For example, pioneering methods such as LOAM (LiDAR Odometry and Mapping) and KISS-ICP significantly improve alignment efficiency and accuracy by directly aligning point cloud data, rather than relying on feature extraction. However, these methods also have limitations: LOAM performs poorly in environments with few curvature features, while KISS-ICP is prone to drift in the absence of loop closure detection and backend optimization. Furthermore, these methods still fall short in meeting the requirements for high-precision attitude estimation.

[0005] Loop closure detection is considered an effective method to reduce accumulated errors. In SLAM, detecting and correcting loop closures can reduce the accumulated error from the first frame to the Nth frame, thereby improving overall system performance. For example, point cloud descriptors based on Scan Context perform well in loop closure detection, leading to a series of algorithms (such as SC-LEGO-LOAM and SC-LIO-SAM), which significantly reduce accumulated errors through loop closure detection. However, Scan Context methods are prone to producing erroneous loop closure detection results, thus weakening their reliability.

[0006] Therefore, a technical solution is needed to improve the accuracy of odometry in the SLAM process. Summary of the Invention

[0007] This application provides an odometry method, apparatus, electronic device, and storage medium based on neural distance fields to address the shortcomings of existing technologies where large accumulated errors during point cloud SLAM lead to low odometry accuracy.

[0008] To achieve the above objectives, embodiments of this application provide an odometry method based on a neural distance field, comprising:

[0009] Acquire the first frame of point cloud data;

[0010] The first frame point cloud data is divided into a first point cloud sub-data and a second point cloud sub-data, wherein the first point cloud sub-data is multiple beam point cloud data obtained from at least one laser beam reflected from the surface of the scanned object;

[0011] Positive sample data for the first machine learning model is generated based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on neural radiation field, and the positive sample data includes the first point cloud sub-data and the first neural distance field value of each point in the first point cloud sub-data, and the first neural distance field value of each point in the first point cloud sub-data is equal to zero.

[0012] The first machine learning model is trained using the positive sample data to generate a trained first machine learning model.

[0013] The second point cloud sub-data is input into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data;

[0014] An initial odometry map is generated using the first frame of point cloud data, wherein the initial odometry map uses a first coordinate system to represent the coordinates of each point;

[0015] For the second neural distance field value of each point in the second point cloud data, determine the gradient of the second neural distance field value in the three axial directions of the first coordinate system;

[0016] For each point in the second point cloud data, the direction of movement of the point is determined by the gradients in the three axial directions, and the distance of movement of the point is determined by the corresponding second neural distance field value.

[0017] The positions of each point in the initial odometer map are updated based on the determined direction and distance of movement of each point to obtain an updated odometer map.

[0018] According to an embodiment of this application, the odometer method further includes:

[0019] Select a second frame of point cloud data from the historical frame point cloud data of the first frame point cloud data, whose distance from the first frame point cloud data is less than a preset frame distance threshold;

[0020] The second frame point cloud data is input into the trained first machine learning model to generate a third neural distance field value for each point in the second frame point cloud data;

[0021] When the first difference between the third neural distance field value of each point in the second frame point cloud data and the first neural distance field value and the second neural distance field value of the first frame point cloud data is less than a preset neural distance threshold, the position adjustment information of each point in the first frame point cloud data is generated using the pose data of each point in the second frame point cloud data and the first difference.

[0022] The location adjustment information is used to update the position of each point in the initial odometer map to obtain an updated odometer map.

[0023] According to an embodiment of this application, the first point cloud sub-data is determined in the following manner:

[0024] The distal endpoint of at least one laser beam reflected from the surface of the object being scanned is selected as the target endpoint, wherein the distal endpoint represents the endpoint that is far from the end of the sensor receiving the laser beam, and the first neural distance field value of the target endpoint is equal to zero.

[0025] The line from the endpoint to the sensor is formed as r = p / ||P||2, and the i-th sampling point on this line is denoted as N. i =dr, where p represents the coordinate vector of the endpoint and d represents the depth on the line relative to the endpoint;

[0026] At the endpoint, a Gaussian distribution is applied at a first interval. Sampling is performed to obtain the endpoint N i Adjacent surface points N s Each point corresponds to the first point of cloud sub-data.

[0027] According to an embodiment of this application, training the first machine learning model using the positive sample data includes: training the first machine learning model based on the positive sample data using the following loss function:

[0028] in,

[0029] λ1, λ2, and λ3 are respectively related to the loss function. and The corresponding preset coefficients, where N represents the number of points in the first frame of point cloud data, and s i This is the second neural distance field value. For the target value, ρ(s) i )and They are respectively with s i and Related reflectivity,

[0030] R is the measured distance between each point in the first frame of point cloud data. The intensity I after angle correction was combined with a preset replacement value r. I The obtained compensated strength, where the replacement value r I This is the preset strength compensation value, n, for points where the strength is zero. all It is a preset constant.

[0031] According to an embodiment of this application, in the odometer method, updating the position of each point in the initial odometer map based on the determined direction and distance of movement of each point includes:

[0032] The translation and transformation matrices of each point are determined based on the gradient and the second neural distance field value;

[0033] The translation matrix and the transformation matrix are used to determine the updated positions of each point in the initial odometer map, so as to update the positions of each point in the initial odometer map.

[0034] According to an embodiment of this application, in the odometry method, determining the translation matrix and transformation matrix of each point based on the gradient and the second neural distance field value includes:

[0035] Construct the Hessian matrix H and the gradient vector g* of the objective function, where,

[0036] θ = log(R) and represents the axial angle of the rotation matrix R, where ω is the weight matrix.

[0037] And this represents the distance gradient of point P, where P represents the coordinates of point P.

[0038] g * =J T ωb, where b is the residual of the neural distance field between point P and the objective function;

[0039] The translation and transformation matrices of each point are calculated iteratively in each iteration, and the increment δ of the current iteration is determined based on the translation and transformation matrices calculated in the current iteration. ∈ =-(H+μ)d diag(H)) -1 g * And the gain ratio ρ is calculated to adjust the damping factor μ. d ,

[0040] When the loop increment is less than a preset threshold, output the translation matrix and transformation matrix of the current iteration round.

[0041] This application also provides an odometry device based on a neural distance field, comprising:

[0042] The acquisition module is used to acquire the first frame of point cloud data;

[0043] The segmentation module is used to divide the first frame point cloud data into a first point cloud sub-data and a second point cloud sub-data, wherein the first point cloud sub-data is multiple beam point cloud data obtained from at least one laser beam reflected from the surface of the scanned object.

[0044] The sample generation module is used to generate positive sample data for the first machine learning model based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on neural radiation field, and the positive sample data includes the first point cloud sub-data and the first neural distance field value of each point in the first point cloud sub-data, and the first neural distance field value of each point in the first point cloud sub-data is equal to zero.

[0045] A training module is used to train the first machine learning model using the positive sample data to generate a trained first machine learning model.

[0046] The first generation module is used to input the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data.

[0047] The odometer map generation module is used to generate an initial odometer map using the first frame point cloud data, wherein the initial odometer map uses a first coordinate system to represent the coordinates of each point;

[0048] The first determining module is used to determine the gradient of the second neural distance field value of each point in the second point cloud data in the three axial directions of the first coordinate system; and to determine the movement direction of each point in the second point cloud data using the gradients in the three axial directions and to determine the movement distance of the point using the corresponding second neural distance field value.

[0049] An update module is used to update the position of each point in the initial odometer map according to the determined direction and distance of movement of each point, so as to obtain an updated odometer map.

[0050] This application also provides an electronic device, including:

[0051] Memory, used to store programs;

[0052] A processor is configured to run the program stored in the memory, wherein the program executes the odometry method based on neural distance field provided in the embodiments of this application.

[0053] This application also provides a computer-readable storage medium storing a computer program executable by a processor, wherein the program, when executed by the processor, implements the odometry method based on neural distance field provided in this application.

[0054] The odometry method, apparatus, electronic device, and storage medium based on neural distance field provided in this application divide the acquired point cloud data into two parts. One part is used as positive samples to train a machine learning model using neural radiation field (NeRF) in real time. The remaining point cloud data is then input into the trained model to generate signed distance function (SDF) values, which serve as neural distance field values. Since, ideally, the signed distance function (SDF) values ​​of points reflected from the object surface should all be zero, the generated neural distance field values ​​can be iteratively adjusted towards zero to reduce or eliminate errors in the point cloud data. After iteration, the movement direction and corresponding movement distance of each point in the odometry map can be obtained, resulting in an updated odometry map that more accurately represents the scanned environment, greatly improving the measurement accuracy of SLAM.

[0055] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0056] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0057] Figure 1 is a flowchart illustrating an embodiment of the odometry method based on neural distance field according to this application.

[0058] Figure 2 is a schematic flowchart of loop closure detection in the odometry method based on neural distance field according to an embodiment of this application;

[0059] Figure 3 is a schematic diagram of an embodiment of the odometer device based on the neural distance field according to this application;

[0060] Figure 4 is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0061] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0062] Simultaneous Localization and Mapping (SLAM) is a key technology for autonomous mobile robots in large-scale map building. Especially in outdoor environments, mobile robots typically rely on LiDAR (LiDAR) sensors to perform SLAM tasks, achieving high-precision localization and navigation. This high-precision localization capability not only improves the robot's efficiency in downstream tasks such as navigation, detection, and perception, but also significantly enhances its environmental adaptability. However, due to the characteristics of large-scale mapping, SLAM systems often face the challenge of accumulated errors in outdoor environments, which can severely affect the robot's localization accuracy.

[0063] The accumulated error in SLAM primarily stems from inaccurate attitude estimation during odometry, and this error intensifies as alignment errors between adjacent frames gradually accumulate. To address this issue, various solutions have been proposed in existing technologies. For example, pioneering methods such as LOAM (LiDAR Odometry and Mapping) and KISS-ICP significantly improve alignment efficiency and accuracy by directly aligning point cloud data, rather than relying on feature extraction. However, these methods also have limitations: LOAM performs poorly in environments with few curvature features, while KISS-ICP is prone to drift in the absence of loop closure detection and backend optimization. Furthermore, these methods still fall short in meeting the requirements for high-precision attitude estimation.

[0064] Loop closure detection is considered an effective method to reduce accumulated errors. In SLAM, detecting and correcting loop closures can reduce the accumulated error from the first frame to the Nth frame, thereby improving overall system performance. For example, point cloud descriptors based on Scan Context perform well in loop closure detection, leading to a series of algorithms (such as SC-LEGO-LOAM and SC-LIO-SAM), which significantly reduce accumulated errors through loop closure detection. However, Scan Context methods are prone to producing erroneous loop closure detection results, thus weakening their reliability.

[0065] In recent years, LiDAR SLAM techniques based on Neural Radiation Fields (NeRF) have attracted widespread attention. These methods utilize NeRF to generate signed distance functions (SDFs) to characterize the distance from a point to the nearest surface, thus exhibiting excellent feature learning capabilities. For example, SHINE-Mapping and NeRF-LOAM have achieved remarkable mapping effects in 3D space. These methods supervise the generation of SDF values ​​through depth information; for instance, TNDF-Fusion enhances ground point segmentation, while PIN-SLAM directly calculates the target SDF value from depth data. However, these NeRF-based SLAM methods still face many challenges in pose estimation, including improvements in SDF representation, real-time alignment rates, the effectiveness of loop closure detection, and the accuracy of loop closure detection in large-scale scenes.

[0066] To address this, this application provides an odometry method based on a neural distance field. This method divides a frame of point cloud data into two parts. For example, point cloud data corresponding to certain laser scan lines can be used as positive samples to train a NeRF-based machine learning model according to this application. After training, the remaining point cloud data is input into the trained machine learning model to generate the corresponding signed distance function (SDF) value. In this application, since a neural radiation field (NeRF) is introduced into the machine learning model, the SDF value generated by such a model can be called the neural distance field value. Therefore, the signed distance function value (i.e., the neural distance field value) corresponding to the selected positive sample point cloud data can be set to zero, thus obtaining training data with input samples and sample values. After training the machine learning model using this training data, the remaining point cloud data can be input into the trained machine learning model to generate the corresponding neural distance field value. It is worth noting that since the SDF value is defined as the distance from a point to the nearest surface, ideally, the SDF value of the point cloud data returned from the surface of an object after laser irradiation should be zero. However, in reality, due to environmental complexity and data acquisition errors, the acquired point cloud data cannot accurately reflect the surface of the scanned object. Therefore, the calculated SDF value is usually not zero. In this embodiment, the positions of the points projected onto the odometry map can be adjusted by optimizing these non-zero SDF values ​​towards zero, thereby correcting the acquired point cloud data and achieving a more accurate reflection of the scanned environment.

[0067] Furthermore, in this embodiment, a damping factor based on dynamically adjusted intensity values ​​is introduced as a further constraint during the optimization process. This avoids premature entrapment in local optima during iteration, thereby further improving registration accuracy. In addition to adjusting the position of points in the odometry map based on the SDF optimization results, this embodiment also introduces a loop closure detection step. This involves comparing the neural distance field values ​​of the current frame point cloud data with those of historical frame point cloud data. When the two values ​​are sufficiently close, a loop closure is considered to have occurred, thus confirming that the optimization result of the current frame is usable.

[0068] The odometry scheme based on neural distance field of this application will be described in detail below with reference to specific embodiments.

[0069] Example 1

[0070] Figure 1 is a flowchart illustrating an embodiment of the odometry method based on a neural distance field according to this application. As shown in Figure 1, the odometry method based on a neural distance field according to an embodiment of this application may include:

[0071] S101, acquire the first frame of point cloud data.

[0072] In this embodiment, point cloud data can be extracted from the reflected signals received by the LiDAR scanning of the surrounding environment. After acquiring the point cloud data, it can be preprocessed, for example, by performing distortion correction and / or filtering and downsampling to remove noise.

[0073] S102, divide the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data.

[0074] In step S102, the point cloud data obtained in step S101, or the point cloud data obtained after preprocessing in step S101, can be divided into two parts: a first point cloud sub-data and a second point cloud sub-data. In this embodiment, since the point cloud data is generated based on the laser beam radiated by the lidar and the reflected signals from the laser beam on surrounding objects, one or more laser beams can be randomly or specifically selected in step S102. The point cloud data corresponding to the selected laser beam can be used as the first point cloud sub-data, and the remaining point cloud data can be used as the second point cloud sub-data. Of course, in this embodiment, the method of dividing the point cloud data is not limited to this; other methods can also be used.

[0075] For example, in this embodiment of the application, the first point cloud sub-data can be determined in the following manner:

[0076] The distal endpoint of at least one laser beam reflected from the surface of the object being scanned is selected as the target endpoint, where the distal endpoint represents the endpoint furthest from the sensor receiving the laser beam, and the first neural distance field value of the target endpoint is equal to zero. A line r = p / ||P||² is formed from the endpoint to the sensor, and the i-th sampling point on this line is denoted as N. i =dr, where p represents the coordinate vector of the endpoint, and d represents the depth on the line relative to the endpoint, distributed along a Gaussian pattern at the endpoint at the first interval. Sampling is performed to obtain the endpoint N i Adjacent surface points N s Each point corresponds to the first point of cloud sub-data.

[0077] S103, Generate positive sample data for the first machine learning model based on the first point cloud sub-data.

[0078] In step S103, the first point cloud data obtained after segmenting the first frame of point cloud data in step S102 can be used to generate positive sample data for the machine learning model. For example, in this embodiment, a machine learning model based on Neural Radiation Field (NeRF) can be used to generate the signed distance function (SDF) value of the point cloud data. In this embodiment, the SDF value generated using such a NeRF-based machine learning model can also be referred to as the neural distance field (NDF) value.

[0079] When generating positive sample data, the neural distance field value of each point in the first point cloud sub-data can be set to zero. That is, it is assumed that each point in the first point cloud sub-data is point cloud data generated by reflection from the surface of an object in the environment, and therefore its NDF value generated by the model should be zero. Thus, the first point cloud sub-data with such a preset zero NDF value can be used as positive sample data.

[0080] S104, Use positive sample data to train the first machine learning model to generate the trained first machine learning model.

[0081] In step S104, the positive sample data generated in step S103 can be used to train the machine learning model based on NeRF (Neural Radiation Field). That is, the first point cloud sub-data obtained after division in step S102 can be used as input, and the NDF value preset for each point in the first point cloud sub-data in step S103 can be used as the value of the positive sample for training, thereby obtaining the trained first machine learning model.

[0082] For example, in this embodiment of the application, the first machine learning model can be trained using the loss function shown in equation (1) based on the positive sample data generated in step S103:

[0083] in,

[0084] λ1, λ2, and λ3 are respectively related to the loss function and The corresponding preset coefficients, where N represents the number of points in the first frame of point cloud data, and s i This is the distance field value of the second nerve. For the target value, ρ(s) i )and They are respectively with s i and Related reflectivity,

[0085] R is the measured distance between each point in the first frame of point cloud data. The intensity I after angle correction was combined with a preset replacement value r. I The obtained compensated strength, where the replacement value r I This is the preset strength compensation value, n, for points where the strength is zero. all It is a preset constant; in this embodiment of the application, n all It can be a value of 20 or greater.

[0086] In this embodiment, the laser intensity I at each point actually depends on the incident angle and the measurement distance R. In the prior art, the measurement distance is typically used to calibrate the intensity. In this embodiment, the incident angle is calibrated to adjust the laser intensity of points in the point cloud data. Furthermore, during the scanning process, due to the low reflectivity of the target object, various interferences including strong light interference, or occlusions, the intensity of points captured by LiDAR may be zero. In this embodiment, a preset replacement value r is used. I To recalculate the zero strength value, thus obtaining the adjusted compensation strength.

[0087] S105, the second point cloud sub-data is input into the trained first machine learning model to generate the second neural distance field value for each point in the second point cloud sub-data.

[0088] S106, use the first frame of point cloud data to generate the initial odometry map.

[0089] In step S105, the second point cloud sub-data can be input into the first machine learning model trained in step S104 to calculate the NDF value of each point, and at the same time or afterward, the first frame of point cloud data can be used to generate an initial odometry map.

[0090] Due to environmental factors, point cloud data generated based on the reflected signal of a laser beam cannot accurately reflect the shape of objects in the environment. That is, at least some points in the returned point cloud data are not actually on the object's surface. Therefore, the NDF value obtained after calculation by the machine learning model for such points will inevitably be non-zero. For example, in this embodiment, the NDF value of the first point cloud sub-data in the first frame of point cloud data has been set to zero. The second point cloud sub-data will inevitably contain points with non-zero NDF values, and these points can become the targets for optimization. That is, optimization can be performed in the direction of zero, and the corresponding adjusted pose of the point in the odometry map can be obtained accordingly, thereby improving the accuracy of the odometry map generated based on the point cloud data.

[0091] S107, for the second neural distance field value of each point in the second point cloud data, determine the gradient of the second neural distance field value in the three axial directions of the first coordinate system.

[0092] S108, for each point in the second point cloud sub-data, the direction of movement of the point is determined by the gradients in the three axial directions, and the distance of movement of the point is determined by the corresponding second neural distance field value.

[0093] In step S107, the gradient can be calculated in three axial directions, such as x, y, and z, using the second point cloud sub-data and the second NDF value generated in step S105. In step S108, the gradient and the second NDF value can be used to determine the direction and distance of movement of the corresponding point in the initial odometry map.

[0094] For example, in step S108, the translation matrix and transformation matrix of each point can be determined based on the gradient of each point and the second neural distance field value. Then, the translation matrix and transformation matrix are used to determine the updated position of each point in the initial odometry map, so as to update the position of each point in the initial odometry map.

[0095] Specifically, in step S108, the translation matrix and transformation matrix of each point can be determined iteratively.

[0096] For example, we can first construct the Hessian matrix H and the gradient vector g* of the objective function, where H = J T ωJ, θ = log(R), where θ is the axial angle representation of the rotation matrix R, and ω is the weight matrix. And g represents the distance gradient of point P, P represents the coordinates of point P, g * =J T ωb, where b is the residual of the neural distance field between point P and the objective function. Then, in each iteration, the translation and transformation matrices of each point can be calculated, and the increment δ for the current iteration is determined based on the translation and transformation matrices calculated in the current iteration. ε =-(H+μ) d diag(H)) -1 g * And the gain ratio ρ is calculated to adjust the damping factor μ. d ,

[0097] v is used to adjust μ d The factor is then determined. Finally, when the loop increment is less than a preset threshold, the translation and transformation matrices of the current iteration are output as the final transformation matrices used to adjust the positions of each point in the odometer map.

[0098] S109, update the position of each point in the initial odometer map according to the determined direction and distance of movement of each point, so as to obtain the updated odometer map.

[0099] In step S109, the transformation matrix, namely the translation matrix and the conversion matrix, calculated and determined in step S108 can be used to adjust the position of each point in the odometer map.

[0100] Furthermore, in this embodiment of the application, since cumulative errors may occur in large-scale point cloud mapping, after the transformation matrix is ​​generated in step S108, at the same time or after the update process in step S109, the distance between the adjusted pose determined by the current frame point cloud data and the pose of the historical frame can be used to determine the historical frames that may have a closed loop with the current frame. When it is determined that such a historical frame exists, the point cloud data of such a historical frame can be further input into the first machine learning model to calculate the NDF value, and the difference between the NDF value of the historical frame and the NDF value of the current frame can be calculated to confirm whether it is less than a preset threshold.

[0101] For example, as shown in Figure 2, which is a flowchart illustrating loop closure detection in the odometry method of this application embodiment, in loop closure detection, a second frame of point cloud data can be selected from the historical frame point cloud data of the first frame point cloud data. The distance between the second frame and the first frame point cloud data is less than a preset frame distance threshold. For example, as mentioned above, the pose of the current frame point cloud data can be used to calculate the distance between the poses of the historical frame point cloud data and the poses of the historical frame point cloud data. When the distance is less than the preset distance threshold, it can be considered that there may be a closed loop between the historical frame and the current frame. Thus, the point cloud data of the historical frame can be input into the trained first machine learning model trained in step S104 to generate the third neural distance field value of each point in the historical frame point cloud data. When the first difference between the third neural distance field value of each point in the historical frame point cloud data and the first and second neural distance field values ​​of the first frame point cloud data is less than the preset neural distance threshold, it can be confirmed that the current frame and the historical frame have formed a closed loop, thereby confirming the accuracy of the pose adjustment of the current frame.

[0102] The odometry method based on neural distance field provided in this application divides the collected point cloud data into two parts. One part is used as positive samples to train a machine learning model using neural radiation field (NeRF) in real time. The remaining point cloud data is then input into the trained model to generate signed distance function (SDF) values, which serve as neural distance field values. Since, ideally, the signed distance function (SDF) values ​​of points reflected from the object surface should all be zero, the generated neural distance field values ​​can be iteratively adjusted towards zero to reduce or eliminate errors in the point cloud data. After iteration, the movement direction and corresponding movement distance of each point in the odometry map can be obtained, resulting in an updated odometry map that more accurately represents the scanned environment, greatly improving the measurement accuracy of SLAM.

[0103] Example 2

[0104] This application also provides a sparse temporal fusion method for detecting 3D objects in LiDAR point clouds. As shown in Figure 3, Figure 3 is a structural schematic diagram of an embodiment of the odometry device based on a neural distance field provided in this application. The odometry device of this application embodiment may include an acquisition module 31, a partitioning module 32, a sample generation module 33, a training module 34, a first generation module 35, an odometry map generation module 36, a first determination module 37, and an update module 38.

[0105] The acquisition module 31 can be used to acquire the first frame of point cloud data.

[0106] In this embodiment, the acquisition module 31 can extract point cloud data from the reflected signals received by the lidar scanning the surrounding environment. After acquiring the point cloud data, the acquisition module 31 can preprocess the point cloud data, for example, by performing distortion correction processing and / or filtering and downsampling processing to remove noise.

[0107] The partitioning module 32 can be used to partition the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data.

[0108] The segmentation module 32 can segment the point cloud data acquired by the acquisition module 31, or the point cloud data obtained after preprocessing by the acquisition module 31, into two parts: a first point cloud sub-data and a second point cloud sub-data. In this embodiment, since the point cloud data is generated based on the laser beam radiated outward by the lidar and the reflected signals of the laser beam on surrounding objects, the segmentation module 32 can randomly or specifically select one or more laser beams and use the point cloud data corresponding to the selected laser beam as the first point cloud sub-data, and use the remaining point cloud data as the second point cloud sub-data. Of course, in this embodiment, the segmentation method of the point cloud data is not limited to this, and other methods can also be used for segmentation.

[0109] For example, in this embodiment of the application, the partitioning module 32 can determine the first point cloud sub-data in the following manner:

[0110] The distal endpoint of at least one laser beam reflected from the surface of the object being scanned is selected as the target endpoint, where the distal endpoint represents the endpoint furthest from the sensor receiving the laser beam, and the first neural distance field value of the target endpoint is equal to zero. A line r = p / ||P||² is formed from the endpoint to the sensor, and the i-th sampling point on this line is denoted as N. i =dr, where p represents the coordinate vector of the endpoint, and d represents the depth on the line relative to the endpoint, distributed along a Gaussian pattern at the endpoint at the first interval. Sampling is performed to obtain the endpoint N i Adjacent surface points N s Each point corresponds to the first point of cloud sub-data.

[0111] The sample generation module 33 can be used to generate positive sample data for the first machine learning model based on the first point cloud sub-data.

[0112] In the sample generation module 33, the first point cloud data obtained by dividing the first frame of point cloud data using the partitioning module 32 can be used to generate positive sample data for the machine learning model. For example, in this embodiment, a machine learning model based on Neural Radiation Field (NeRF) can be used to generate the signed distance function (SDF) value of the point cloud data. In this embodiment, the SDF value generated using such a NeRF-based machine learning model can also be referred to as the neural distance field (NDF) value.

[0113] When generating positive sample data, the neural distance field value of each point in the first point cloud sub-data can be set to zero. That is, it is assumed that each point in the first point cloud sub-data is point cloud data generated by reflection from the surface of an object in the environment, and therefore its NDF value generated by the model should be zero. Thus, the first point cloud sub-data with such a preset zero NDF value can be used as positive sample data.

[0114] Training module 34 can be used to train the first machine learning model using positive sample data to generate the trained first machine learning model.

[0115] The training module 34 can use the positive sample data generated by the sample generation module 33 to train the machine learning model based on NeRF (Neural Radiation Field). That is, it can use the first point cloud sub-data obtained after partitioning by the partitioning module 32 as input, and use the NDF value preset by the sample generation module 33 for each point in the first point cloud sub-data as the value of the positive sample for training, so as to obtain the trained first machine learning model.

[0116] For example, in this embodiment of the application, the training module 34 can train the first machine learning model based on the positive sample data generated by the sample generation module 33 using the loss function shown in equation (1):

[0117] in,

[0118] λ1, λ2, and λ3 are respectively related to the loss function and The corresponding preset coefficients, where N represents the number of points in the first frame of point cloud data, and s i This is the distance field value of the second nerve. For the target value, ρ(s) i )and They are respectively with s i and Related reflectivity,

[0119] R is the measured distance between each point in the first frame of point cloud data. The intensity I after angle correction was combined with a preset replacement value r. I The obtained compensated strength, where the replacement value r I This is the preset strength compensation value, n, for points where the strength is zero. all It is a preset constant; in this embodiment of the application, n all It can be a value of 20 or greater.

[0120] In this embodiment, the laser intensity I at each point actually depends on the incident angle and the measurement distance R. In the prior art, the measurement distance is typically used to calibrate the intensity. In this embodiment, the incident angle is calibrated to adjust the laser intensity of points in the point cloud data. Furthermore, during the scanning process, due to the low reflectivity of the target object, various interferences including strong light interference, or occlusions, the intensity of points captured by LiDAR may be zero. In this embodiment, a preset replacement value r is used. I To recalculate the zero strength value, thus obtaining the adjusted compensation strength.

[0121] The first generation module 35 can be used to input the second point cloud sub-data into a trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data.

[0122] The odometer map generation module 36 can be used to generate an initial odometer map using the first frame of point cloud data.

[0123] The first generation module 35 can input the second point cloud sub-data into the first machine learning model trained by the training module 34 to calculate the NDF value of each point, and at the same time or later, the first frame of point cloud data can be used to generate an initial odometry map.

[0124] Due to environmental factors, point cloud data generated based on the reflected signal of a laser beam cannot accurately reflect the shape of objects in the environment. That is, at least some points in the returned point cloud data are not actually on the object's surface. Therefore, the NDF value obtained after calculation by the machine learning model for such points will inevitably be non-zero. For example, in this embodiment, the NDF value of the first point cloud sub-data in the first frame of point cloud data has been set to zero. The second point cloud sub-data will inevitably contain points with non-zero NDF values, and these points can become the targets for optimization. That is, optimization can be performed in the direction of zero, and the corresponding adjusted pose of the point in the odometry map can be obtained accordingly, thereby improving the accuracy of the odometry map generated based on the point cloud data.

[0125] The first determining module 37 can be used to determine the gradient of the second neural distance field value of each point in the second point cloud data in the three axial directions of the first coordinate system, and to determine the movement direction of each point in the second point cloud data using the gradients in the three axial directions and to determine the movement distance of the point using the corresponding second neural distance field value.

[0126] In the first determining module 37, the gradient can be calculated in three axis directions, such as x, y, and z, using the second point cloud sub-data and the second NDF value generated in the first generating module 35. The gradient and the second NDF value can be used to determine the direction and distance of movement of the corresponding point in the initial odometry map.

[0127] For example, the first determining module 37 can determine the translation matrix and transformation matrix of each point based on the gradient of each point and the second neural distance field value, and then use the translation matrix and transformation matrix to determine the updated position of each point in the initial odometry map, so as to update the position of each point in the initial odometry map.

[0128] Specifically, the first determining module 37 can determine the translation matrix and transformation matrix of each point in an iterative manner.

[0129] For example, we can first construct the Hessian matrix H and the gradient vector g* of the objective function, where H = J T ωJ, θ = log(R), where θ is the axial angle representation of the rotation matrix R, and ω is the weight matrix. And g represents the distance gradient of point P, P represents the coordinates of point P, g * =J T ωb, where b is the residual of the neural distance field between point P and the objective function. Then, in each iteration, the translation and transformation matrices of each point can be calculated, and the increment δ for the current iteration is determined based on the translation and transformation matrices calculated in the current iteration. ∈ =-(H+μ) d diag(H)) -1 g * And the gain ratio ρ is calculated to adjust the damping factor μ. d ,

[0130] v is used to adjust μ d The factor is then determined. Finally, when the loop increment is less than a preset threshold, the translation and transformation matrices of the current iteration are output as the final transformation matrices used to adjust the positions of each point in the odometer map.

[0131] The update module 38 can be used to update the position of each point in the initial odometer map according to the determined direction and distance of movement of each point, so as to obtain the updated odometer map.

[0132] The update module 38 can use the first determining module 37 to calculate the determined transformation matrix, namely the translation matrix and the transformation matrix, to adjust the position of each point in the odometer map.

[0133] Furthermore, in this embodiment, since large-scale point cloud mapping may lead to cumulative errors, after the first determining module 37 generates the transformation matrix, simultaneously or after the update module 38 updates, the distance between the adjusted pose determined by the current frame point cloud data and the pose of the historical frame can be used to determine the historical frames that may have a closed loop with the current frame. When it is determined that such a historical frame exists, the point cloud data of such a historical frame can be further input into the first machine learning model to calculate the NDF value, and the difference between the NDF value of the historical frame and the NDF value of the current frame can be calculated to confirm whether it is less than a preset threshold.

[0134] For example, a second frame of point cloud data can be selected from the historical frame point cloud data of the first frame point cloud data, where the distance to the first frame point cloud data is less than a preset frame distance threshold. For example, as mentioned above, the pose of the current frame point cloud data can be used to calculate the distance between the poses of the historical frame point cloud data and the poses of the historical frame point cloud data. When the distance is less than the preset distance threshold, it can be considered that there may be a closed loop between the historical frame and the current frame. Thus, the point cloud data of the historical frame can be input into the trained first machine learning model after training in training module 34 to generate the third neural distance field value of each point in the historical frame point cloud data. When the first difference between the third neural distance field value of each point in the historical frame point cloud data and the first neural distance field value and the second neural distance field value of the first frame point cloud data is less than the preset neural distance threshold, the pose data of each point in the historical frame point cloud data and the first difference are used to generate the position adjustment information of each point in the first frame point cloud data. The position adjustment information is used to update the position of each point in the odometry map to obtain the updated odometry map.

[0135] The odometry device based on neural distance field provided in this application divides the collected point cloud data into two parts. One part is used as positive samples to train a machine learning model using neural radiation field (NeRF) in real time. The remaining point cloud data is then input into the trained model to generate signed distance function (SDF) values, which serve as neural distance field values. Since, ideally, the signed distance function (SDF) values ​​of points reflected from the object surface should all be zero, the generated neural distance field values ​​can be iteratively adjusted towards zero to reduce or eliminate errors in the point cloud data. After iteration, the movement direction and corresponding movement distance of each point in the odometry map can be obtained, thus obtaining an updated odometry map that more accurately represents the scanned environment, greatly improving the measurement accuracy of SLAM.

[0136] Example 3

[0137] The internal functions and structure of the odometry method based on neural distance fields have been described above, which can be implemented as an electronic device. Figure 4 is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. As shown in Figure 4, the electronic device includes a memory 41 and a processor 42.

[0138] Memory 41 is used to store programs. In addition to the programs described above, memory 41 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0139] The memory 41 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0140] Processor 42 is not limited to a processor (CPU), but may also be a graphics processing unit (GPU), a field-programmable gate array (FPGA), an embedded neural network processor (NPU), or an artificial intelligence (AI) chip. Processor 42 is coupled to memory 41 and executes the program stored in memory 41 to perform the odometry method based on neural distance field described in Embodiment 1 above.

[0141] Furthermore, as shown in Figure 4, the electronic device may also include other components such as a communication component 43, a power supply component 44, an audio component 45, and a display 46. Figure 4 only schematically illustrates some components and does not imply that the electronic device includes only the components shown in Figure 4.

[0142] Communication component 43 is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic devices can access wireless networks based on communication standards, such as WiFi, 3G, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 43 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 43 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0143] Power supply component 44 provides power to various components of the electronic device. Power supply component 44 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0144] Audio component 45 is configured to output and / or input audio signals. For example, audio component 45 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 41 or transmitted via communication component 43. In some embodiments, audio component 45 also includes a speaker for outputting audio signals.

[0145] Display 46 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0146] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for odometry based on neural distance fields, characterized in that, include: Acquire the first frame of point cloud data; The first frame point cloud data is divided into a first point cloud sub-data and a second point cloud sub-data, wherein the first point cloud sub-data is multiple beam point cloud data obtained from at least one laser beam reflected from the surface of the scanned object; Positive sample data for the first machine learning model is generated based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on neural radiation field, and the positive sample data includes the first point cloud sub-data and the first neural distance field value of each point in the first point cloud sub-data, and the first neural distance field value of each point in the first point cloud sub-data is equal to zero. The first machine learning model is trained using the positive sample data to generate a trained first machine learning model. The second point cloud sub-data is input into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data; An initial odometry map is generated using the first frame of point cloud data, wherein the initial odometry map uses a first coordinate system to represent the coordinates of each point; For the second neural distance field value of each point in the second point cloud data, determine the gradient of the second neural distance field value in the three axial directions of the first coordinate system; For each point in the second point cloud data, the direction of movement of the point is determined by the gradients in the three axial directions, and the distance of movement of the point is determined by the corresponding second neural distance field value. The positions of each point in the initial odometer map are updated based on the determined direction and distance of movement of each point to obtain an updated odometer map.

2. The odometry method based on neural distance field according to claim 1, characterized in that, The method further includes: Select a second frame of point cloud data from the historical frame point cloud data of the first frame point cloud data, whose distance from the first frame point cloud data is less than a preset frame distance threshold; The second frame point cloud data is input into the trained first machine learning model to generate a third neural distance field value for each point in the second frame point cloud data; When the first difference between the third neural distance field value of each point in the second frame point cloud data and the first neural distance field value and the second neural distance field value of the first frame point cloud data is less than a preset neural distance threshold, it is confirmed that the first frame point cloud data and the second frame point cloud data form a closed loop.

3. The odometry method based on neural distance fields according to claim 1, characterized in that, The first point cloud sub-data is determined in the following manner: The distal endpoint of at least one laser beam reflected from the surface of the object being scanned is selected as the target endpoint, wherein the distal endpoint represents the endpoint that is far from the end of the sensor receiving the laser beam, and the first neural distance field value of the target endpoint is equal to zero. The line from the endpoint to the sensor is formed as r = p / ||P||2, and the i-th sampling point on this line is denoted as N. i =dr, where p represents the coordinate vector of the endpoint and d represents the depth on the line relative to the endpoint; At the endpoint, along a Gaussian distribution at a first interval Sampling is performed to obtain the endpoint N i Adjacent surface points N s Each point corresponds to the first point of cloud sub-data.

4. The odometry method based on neural distance field according to claim 1, characterized in that, Training the first machine learning model using the positive sample data includes: training the first machine learning model based on the positive sample data using the following loss function: in, λ1, λ2, and λ3 are respectively related to the loss function and The corresponding preset coefficients, where N represents the number of points in the first frame of point cloud data, and s i This is the second neural distance field value. For the target value, ρ(s) i )and They are respectively with s i and Related reflectivity, R is the measured distance between each point in the first frame of point cloud data. The intensity I after angle correction was combined with a preset replacement value r. I The obtained compensated strength, where the replacement value r I This is the preset strength compensation value, n, for points where the strength is zero. all It is a preset constant.

5. The odometry method based on neural distance field according to claim 1, characterized in that, The step of updating the position of each point in the initial odometry map based on the determined direction and distance of movement of each point includes: The translation and transformation matrices of each point are determined based on the gradient and the second neural distance field value; The translation matrix and the transformation matrix are used to determine the updated positions of each point in the initial odometer map, so as to update the positions of each point in the initial odometer map.

6. The odometry method based on neural distance field according to claim 5, characterized in that, The step of determining the translation matrix and transformation matrix of each point based on the gradient and the second neural distance field value includes: Construct the Hessian matrix H and the gradient vector g* of the objective function, where, H=J T ωJ, θ = log(R) and represents the axial angle of the rotation matrix R, where ω is the weight matrix. And this represents the distance gradient of point P, where P represents the coordinates of point P. g * =J T ωb, where b is the residual of the neural distance field between point P and the objective function; The translation and transformation matrices of each point are calculated iteratively in each iteration, and the increment δ∈=-(H+μ) is determined based on the translation and transformation matrices of each point calculated in the current iteration. d diag(H)) -1 g * And the gain ratio ρ is calculated to adjust the damping factor μ. d , When the loop increment is less than a preset threshold, output the translation matrix and transformation matrix of the current iteration round.

7. An odometry device based on neural distance fields, characterized in that, The device includes: The acquisition module is used to acquire the first frame of point cloud data; The segmentation module is used to divide the first frame point cloud data into a first point cloud sub-data and a second point cloud sub-data, wherein the first point cloud sub-data is multiple beam point cloud data obtained from at least one laser beam reflected from the surface of the scanned object. The sample generation module is used to generate positive sample data for the first machine learning model based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on neural radiation field, and the positive sample data includes the first point cloud sub-data and the first neural distance field value of each point in the first point cloud sub-data, and the first neural distance field value of each point in the first point cloud sub-data is equal to zero. A training module is used to train the first machine learning model using the positive sample data to generate a trained first machine learning model. The first generation module is used to input the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data. The odometer map generation module is used to generate an initial odometer map using the first frame point cloud data, wherein the initial odometer map uses a first coordinate system to represent the coordinates of each point; The first determining module is used to determine the gradient of the second neural distance field value of each point in the second point cloud data in the three axial directions of the first coordinate system; and to determine the movement direction of each point in the second point cloud data using the gradients in the three axial directions and to determine the movement distance of the point using the corresponding second neural distance field value. An update module is used to update the position of each point in the initial odometer map according to the determined direction and distance of movement of each point, so as to obtain an updated odometer map.

8. An electronic device, characterized in that, include: Memory, used to store programs; A processor for running the program stored in the memory to perform the odometry method based on a neural distance field as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon that can be executed by a processor, characterized in that, When the program is executed by the processor, it implements the odometry method based on the neural distance field as described in any one of claims 1-6.

Images