Milemeter method and device based on nerve distance field, electronic equipment and storage medium

Abstract

The invention discloses an odometer method and device based on a nerve distance field, electronic equipment and a storage medium. Collected point cloud data is divided into two parts, one part serves as a positive sample to be used for training a machine learning model adopting a neural radiation field (NeRF) in real time, then the remaining point cloud data is input into the trained model to generate a signed distance function (SDF) value, the signed distance function (SDF) value serves as a neural distance field value, and the neural distance field value serves as a neural distance field. And under ideal conditions, the signed distance function SDF values of the points reflected back from the surface of the object should be zero, so that the generated neural distance field values can be utilized to carry out iterative adjustment towards the zero direction to reduce or eliminate errors in the point cloud data, and the accuracy of the point cloud data is improved. Therefore, the moving direction and the corresponding moving distance of each point in the speedometer diagram can be obtained after iteration is completed, the updated speedometer diagram which can more accurately represent the scanned environment can be obtained, and the measurement precision of SLAM is greatly improved.

Description

Technical Field

[0001] The present application relates to the technical field, and in particular to an odometer method and device based on a neural distance field, an electronic device, and a storage medium. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) technology is a key technology for large-scale map building for autonomous mobile robots. Especially in outdoor environments, mobile robots often rely on LiDAR (Light Detection and Ranging) sensors to perform SLAM tasks to achieve high-precision positioning and navigation. This high-precision localization capability not only improves the robot's efficiency in downstream tasks such as navigation, detection, and perception, but also significantly enhances its environmental adaptability. However, due to the characteristics of large-scale mapping, SLAM systems often face the challenge of cumulative errors in outdoor environments, which can seriously affect the robot's positioning accuracy.

[0003] The cumulative error in SLAM mainly comes from the inaccuracy of pose estimation in the odometry, which is aggravated by the gradual accumulation of alignment errors between adjacent frames. To solve this problem, a variety of solutions have been proposed in the prior art. For example, pioneering methods such as LOAM (LiDAR Odometry and Mapping) and KISS-ICP significantly improve the efficiency and accuracy of alignment by directly aligning point cloud data rather than relying on feature extraction. However, these methods also have limitations: LOAM performs poorly in environments with fewer curvature features, while KISS-ICP is prone to drift in the absence of closed-loop detection and back-end optimization. In addition, these methods still have shortcomings in meeting the needs of high-precision pose estimation.

[0004] Closed loop detection is considered one of the effective methods to reduce cumulative error. In the SLAM process, by detecting and correcting closed loops, the cumulative error from the first frame to the Nth frame can be reduced, thereby improving the overall system performance. For example, the Scan Context-based point cloud descriptor performs well in closed loop detection, and a series of algorithms (such as SC-LEGO-LOAM and SC-LIO-SAM) have been derived. These algorithms significantly reduce the cumulative error through closed loop detection. However, the Scan Context method is prone to produce erroneous closed loop detection results, which weakens its reliability.

[0005] Therefore, a technical solution is needed to improve the accuracy of the odometer in the SLAM process. Summary of the Invention

[0006] The embodiments of the present application provide an odometer method and device, an electronic device, and a storage medium based on a neural distance field to address the defect in the prior art that the odometer accuracy is low due to large cumulative errors in the point cloud SLAM process.

[0007] To achieve the above objectives, the present invention provides an odometer method based on a neural distance field, comprising:

[0008] Get the first frame of point cloud data;

[0009] dividing the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data, wherein the first point cloud sub-data is a plurality of beam point cloud data acquired from at least one laser beam reflected from a surface of a scanned object;

[0010] generating positive sample data for the first machine learning model based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on a neural radiation field, and the positive sample data includes the first point cloud sub-data and a first neural distance field value for each point in the first point cloud sub-data, and the first neural distance field value for each point in the first point cloud sub-data is equal to zero;

[0011] Training the first machine learning model using the positive sample data to generate a trained first machine learning model;

[0012] Inputting the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data;

[0013] Generate an initial odometry map using the first frame of point cloud data, wherein the initial odometry map uses a first coordinate system to represent the coordinates of each point;

[0014] For the second neural distance field value of each point in the second point cloud sub-data, respectively determine the gradients of the second neural distance field value in the three axis directions of the first coordinate system;

[0015] For each point in the second point cloud sub-data, using the gradients in the three axis directions to determine the moving direction of the point and using the second neural distance field value corresponding to the point to determine the moving distance of the point;

[0016] The position of each point in the initial odometer map is updated according to the determined moving direction and moving distance of each point to obtain an updated odometer map.

[0017] According to an embodiment of the present application, the odometer method further includes:

[0018] Selecting, from the historical frame point cloud data of the first frame point cloud data, a second frame point cloud data whose distance from the first frame point cloud data is less than a preset frame distance threshold;

[0019] Inputting the second frame of point cloud data into the trained first machine learning model to generate a third neural distance field value for each point in the second frame of point cloud data;

[0020] When a first difference between the third neural distance field value of each point in the second frame of point cloud data and the first neural distance field value and the second neural distance field value of the first frame of point cloud data is less than a preset neural distance threshold, generating position adjustment information for each point in the first frame of point cloud data using the posture data of each point in the second frame of point cloud data and the first difference;

[0021] The position adjustment information is used to update the position of each point in the initial odometer map to obtain an updated odometer map.

[0022] According to an embodiment of the present application, the first point cloud sub-data is determined in the following manner:

[0023] selecting a distal end point of at least one laser beam reflected from a surface of a scanned object as a target end point, wherein the distal end point refers to an end point far from an end of a sensor receiving the laser beam, and a first neural distance field value of the target end point is equal to zero;

[0024] A line r=p / ||P||2 is formed from the end point to the sensor, and the i-th sampling point on the line is represented by N i =dr, where p represents the coordinate vector of the endpoint and d represents the depth relative to the endpoint on the line;

[0025] At the end points along a Gaussian distribution at a first interval Sampling is performed to obtain the value corresponding to the endpoint N i N adjacent surface points s As each point corresponding to the first point cloud sub-data.

[0026] According to an embodiment of the present application, using the positive sample data to train the first machine learning model includes: training the first machine learning model based on the positive sample data using the following loss function:

[0027]

[0028] in,

[0029]

[0030] λ1, λ2 and λ3 are respectively related to the loss function and The corresponding preset coefficient, N represents the number of points in the first frame of point cloud data, s i is the second neural distance field value, is the target value, ρ(s i )and and s respectively i and The associated reflectivity,

[0031]

[0032] R is the measured distance of each point in the first frame of point cloud data, is the angle-corrected intensity I combined with the preset replacement value r I The obtained post-compensation intensity, where the replacement value r I is the preset intensity compensation value for points with zero intensity, n all is a preset constant.

[0033] According to an embodiment of the present application, in the odometer method, updating the position of each point in the initial odometer map according to the determined moving direction and moving distance of each point includes:

[0034] Determine a translation matrix and a transformation matrix for each point according to the gradient and the second neural distance field value;

[0035] An updated position of each point in the initial odometry map is determined using the translation matrix and the transformation matrix to update the position of each point in the initial odometry map.

[0036] According to an embodiment of the present application, in the odometry method, determining the translation matrix and the transformation matrix of each point according to the gradient and the second neural distance field value includes:

[0037] Construct the Hessian matrix H and the gradient vector g* of the objective function, where

[0038]

[0039] θ = log(R) and is the axial angle representation of the rotation matrix R, ω is the weight matrix,

[0040] And represents the distance gradient of point P, P represents the coordinates of point P,

[0041] g * =J T ωb, b is the residual of the neural distance field value between point P and the target function;

[0042] The translation matrix and transformation matrix of each point are calculated in an iterative manner in each round, and the increment of the current round of iteration is determined based on the translation matrix and transformation matrix of each point calculated in the current round of iteration δ∈=-(H+μ d diag(H)) -1 g *, and the gain ratio ρ is calculated to adjust the damping factor μ d ,

[0043] v is used to adjust μ d Factors of

[0044] When the loop increment is less than the preset threshold, the translation matrix and conversion matrix of the current iteration round are output.

[0045] The present application also provides an odometer device based on a neural distance field, including:

[0046] Acquisition module, used to obtain the first frame of point cloud data;

[0047] a dividing module, configured to divide the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data, wherein the first point cloud sub-data is a plurality of beam point cloud data obtained from at least one laser beam reflected from a surface of a scanned object;

[0048] a sample generation module, configured to generate positive sample data for the first machine learning model based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on a neural radiation field, and the positive sample data includes the first point cloud sub-data and a first neural distance field value for each point in the first point cloud sub-data, and the first neural distance field value for each point in the first point cloud sub-data is equal to zero;

[0049] a training module, configured to train the first machine learning model using the positive sample data to generate a trained first machine learning model;

[0050] a first generating module, configured to input the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data;

[0051] an odometry map generating module, configured to generate an initial odometry map using the first frame of point cloud data, wherein the initial odometry map uses a first coordinate system to represent the coordinates of each point;

[0052] a first determination module configured to determine, for each point in the second point cloud sub-data, a second neural distance field value, and gradients of the second neural distance field value in the three axial directions of the first coordinate system; and for each point in the second point cloud sub-data, determine a movement direction of the point using the gradients in the three axial directions and determine a movement distance of the point using the second neural distance field value corresponding to the point;

[0053] An updating module is used to update the position of each point in the initial odometer map according to the determined moving direction and moving distance of each point to obtain an updated odometer map.

[0054] An embodiment of the present application further provides an electronic device, including:

[0055] Memory, used to store programs;

[0056] A processor is configured to run the program stored in the memory, wherein when the program is run, the odometer method based on the neural distance field provided in the embodiment of the present application is executed.

[0057] An embodiment of the present application further provides a computer-readable storage medium storing a computer program executable by a processor, wherein when the program is executed by the processor, the odometer method based on the neural distance field provided in the embodiment of the present application is implemented.

[0058] The neural distance field-based odometry method and apparatus, electronic device, and storage medium provided in the embodiments of the present application divide the collected point cloud data into two parts, one part being used as a positive sample for real-time training of a machine learning model using a neural radiance field (NeRF), and then the remaining point cloud data being input into the trained model to generate a signed distance function (SDF) value as a neural distance field value. Ideally, the signed distance function (SDF) values of points reflected from the surface of an object should all be zero. Therefore, the generated neural distance field values can be used to iteratively adjust toward zero to reduce or eliminate errors in the point cloud data. After the iteration is completed, the movement direction and corresponding movement distance of each point in the odometry map can be obtained, thereby obtaining an updated odometry map that more accurately represents the scanned environment, greatly improving the measurement accuracy of SLAM.

[0059] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0061] Figure 1 This is a flow chart of an embodiment of an odometer method based on a neural distance field according to an embodiment of the present application;

[0062] Figure 2 Schematic diagram of the loop detection process in the neural distance field-based odometer method according to an embodiment of the present application;

[0063] Figure 3 Schematic diagram of the structure of an embodiment of an odometer device based on a neural distance field according to an embodiment of the present application;

[0064] Figure 4 This is a schematic structural diagram of an electronic device embodiment provided in this application. DETAILED DESCRIPTION

[0065] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0066] Simultaneous Localization and Mapping (SLAM) technology is a key technology for large-scale map building for autonomous mobile robots. Especially in outdoor environments, mobile robots often rely on LiDAR (Light Detection and Ranging) sensors to perform SLAM tasks to achieve high-precision positioning and navigation. This high-precision localization capability not only improves the robot's efficiency in downstream tasks such as navigation, detection, and perception, but also significantly enhances its environmental adaptability. However, due to the characteristics of large-scale mapping, SLAM systems often face the challenge of cumulative errors in outdoor environments, which can seriously affect the robot's positioning accuracy.

[0067] The cumulative error in SLAM mainly comes from the inaccuracy of pose estimation in the odometry, which is aggravated by the gradual accumulation of alignment errors between adjacent frames. To solve this problem, a variety of solutions have been proposed in the prior art. For example, pioneering methods such as LOAM (LiDAR Odometry and Mapping) and KISS-ICP significantly improve the efficiency and accuracy of alignment by directly aligning point cloud data rather than relying on feature extraction. However, these methods also have limitations: LOAM performs poorly in environments with fewer curvature features, while KISS-ICP is prone to drift in the absence of closed-loop detection and back-end optimization. In addition, these methods still have shortcomings in meeting the needs of high-precision pose estimation.

[0068] Closed loop detection is considered one of the effective methods to reduce cumulative error. In the SLAM process, by detecting and correcting closed loops, the cumulative error from the first frame to the Nth frame can be reduced, thereby improving the overall system performance. For example, the Scan Context-based point cloud descriptor performs well in closed loop detection, and a series of algorithms (such as SC-LEGO-LOAM and SC-LIO-SAM) have been derived. These algorithms significantly reduce the cumulative error through closed loop detection. However, the Scan Context method is prone to produce erroneous closed loop detection results, which weakens its reliability.

[0069] In recent years, LiDAR SLAM technology based on Neural Radiance Field (NeRF) has received widespread attention. This type of method uses NeRF to generate a signed distance function (SDF) to represent the distance from a point to the nearest surface, thereby demonstrating excellent feature learning capabilities. For example, SHINE-Mapping and NeRF-LOAM achieve significant mapping effects in three-dimensional space. These methods supervise the generation of SDF values through depth information, such as TNDF-Fusion, which enhances ground point segmentation, and PIN-SLAM, which directly calculates SDF target values from depth data. However, these NeRF-based SLAM methods still face many challenges in pose estimation, including improvements in SDF representation, improvements in real-time alignment rates, effectiveness of loop closure detection, and accuracy of loop closure detection in large-scale scenes.

[0070] To this end, an embodiment of the present application provides an odometry method based on a neural distance field, which can divide a frame of acquired point cloud data into two parts. For example, the point cloud data corresponding to certain laser scan lines can be used as positive samples to train the NeRF-based machine learning model according to the present application, and after training, the remaining point cloud data is input into the trained machine learning model to generate the corresponding signed distance function SDF value. In an embodiment of the present application, since the neural radiation field NeRF is introduced into the machine learning model, the SDF value generated by such a model can be called the value of the neural distance field. And therefore, the signed distance function value (i.e., the neural distance field value) corresponding to a portion of the point cloud data selected as the positive sample can be set to zero, so that training data with input samples and sample values is obtained. After the machine learning model is trained using such training data, the remaining point cloud data can be input into the trained machine learning model to generate the corresponding neural distance field value. It should be noted that since the SDF value is defined as the distance from a point to the nearest surface, ideally, the SDF value of the point cloud data returned from the object surface by laser irradiation should be zero. However, in reality, due to complex environments and errors in the collected data, the collected point cloud data cannot truly reflect the conditions of the scanned object surface. Therefore, the calculated SDF value is usually not zero. In the embodiment of the present application, the position of each point projected into the odometry map can be adjusted by optimizing and adjusting such non-zero SDF values toward zero, thereby correcting the collected point cloud data to achieve a more accurate reflection of the scanned environment.

[0071] In addition, in the embodiment of the present application, a damping factor that is dynamically adjusted based on the intensity value is further introduced during the optimization process as a further constraint during iteration, thereby avoiding falling into a local optimal solution too early during the iteration process, thereby further improving the registration accuracy. In addition, in addition to adjusting the position of the points in the odometer map based on the optimization results of the SDF, the embodiment of the present application further introduces a loop detection step, which can be based on the comparison of the neural distance field values of the current frame point cloud data and the historical frame point cloud data. When the values of the two are close enough, the loop is considered to be established, thereby confirming that the optimization results of the current frame are usable.

[0072] The odometer solution based on neural distance field of the present application will be described in detail below with reference to specific embodiments.

[0073] Example 1

[0074] Figure 1 FIG. 1 is a flow chart of an embodiment of an odometer method based on a neural distance field according to an embodiment of the present application. Figure 1 As shown in , the odometer method based on neural distance field according to an embodiment of the present application may include:

[0075] S101, obtaining the first frame of point cloud data.

[0076] In an embodiment of the present application, point cloud data can be extracted from reflected signals received by a laser radar scanning the surrounding environment. After acquiring the point cloud data, the point cloud data can be pre-processed, for example, by performing distortion correction processing and / or filtering and downsampling processing to remove noise.

[0077] S102: Divide the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data.

[0078] In step S102, the point cloud data obtained in step S101 or the point cloud data obtained after preprocessing in step S101 can be divided into two parts: first point cloud sub-data and second point cloud sub-data. In the embodiment of the present application, since the point cloud data is generated based on the laser radar radiating a laser beam outward and receiving the reflection signal of the laser beam on the surrounding objects, one or more laser beams can be randomly or specifically selected in step S102 and the point cloud data corresponding to the selected laser beam can be used as the first point cloud sub-data, and the remaining point cloud data can be used as the second point cloud sub-data. Of course, in the embodiment of the present application, the method of dividing the point cloud data is not limited to this, and other methods can also be used for division.

[0079] For example, in the embodiment of the present application, the first point cloud sub-data may be determined in the following manner:

[0080] The distal end point of at least one laser beam reflected from the surface of the scanned object is selected as the target end point, wherein the distal end point refers to an end point at one end away from the sensor receiving the laser beam, and the first neural distance field value of the target end point is equal to zero, and a line r=p / ||P||2 is formed from the end point to the sensor, and the i-th sampling point on the line is represented by N i =dr, where p represents the coordinate vector of the endpoint, d represents the depth relative to the endpoint on the line, and the Gaussian distribution is carried out at the endpoint with the first interval. Sampling is performed to obtain the value corresponding to the endpoint N i N adjacent surface points s As each point corresponding to the first point cloud sub-data.

[0081] S103: Generate positive sample data for the first machine learning model based on the first point cloud sub-data.

[0082] In step S103, the first point cloud data obtained after segmenting the first frame of point cloud data in step S102 can be used to generate positive sample data for the machine learning model. For example, in an embodiment of the present application, a machine learning model based on a neural radiation field (NeRF) can be used to generate the value of a signed distance function (SDF) of the point cloud data. In an embodiment of the present application, the SDF value generated using such a NeRF-based machine learning model can also be referred to as a neural distance field (NDF) value.

[0083] When generating positive sample data, the neural distance field value of each point in the first point cloud sub-data can be set to zero, that is, each point in the first point cloud sub-data is considered to be point cloud data generated by reflection on the surface of an object in the environment, and therefore its NDF value generated by the model should be zero, so that the first point cloud sub-data with such a preset zero NDF value can be used as positive sample data.

[0084] S104: Use positive sample data to train the first machine learning model to generate a trained first machine learning model.

[0085] In step S104, the positive sample data generated in step S103 can be used to train the machine learning model based on the neural radiation field NeRF, that is, the first point cloud sub-data obtained after the division in step S102 can be used as input, and the NDF value preset for each point in the first point cloud sub-data in step S103 can be used as the positive sample value for training, so that a trained first machine learning model can be obtained.

[0086] For example, in an embodiment of the present application, the first machine learning model may be trained based on the positive sample data generated in step S103 using a loss function as shown in equation (1):

[0087]

[0088] in,

[0089]

[0090] λ1, λ2 and λ3 are respectively related to the loss function and The corresponding preset coefficient, N represents the number of points in the first frame of point cloud data, s i is the second neural distance field value, is the target value, ρ(s i )and and s respectively i and The associated reflectivity,

[0091]

[0092] R is the measured distance of each point in the first frame of point cloud data, is the angle-corrected intensity I combined with the preset replacement value r I The obtained post-compensation intensity, where the replacement value r I is the preset intensity compensation value for points with zero intensity, n all is a preset constant. In the embodiment of the present application, n all It can be a value of 20 or more.

[0093] In the embodiment of the present application, the laser intensity I of each point actually depends on the incident angle and the measured distance R. In the prior art, the measured distance is usually used to calibrate the intensity. In the embodiment of the present application, the incident angle is calibrated to adjust the laser intensity of the point in the point cloud data. In addition, during the scanning process, due to the low reflectivity of the target object, various interferences including strong light interference or occlusion, the intensity of the point captured by the LiDAR may be zero. In the embodiment of the present application, a preset replacement value r is used. I To recalculate the intensity value to zero, thus obtaining the adjusted compensation intensity

[0094] S105 , inputting the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data.

[0095] S106 , generating an initial odometry map using the first frame of point cloud data.

[0096] In step S105, the second point cloud sub-data can be input into the first machine learning model trained in step S104 to calculate the NDF value of each point, and at the same time or thereafter, the first frame point cloud data can be used to generate an initial odometry map.

[0097] Due to environmental factors and other reasons, the point cloud data generated based on the reflected signal of the laser beam cannot accurately reflect the shape of the environmental objects, that is, at least some of the points in the returned point cloud data are not actually on the surface of the object, so the NDF value of such points obtained after calculation by the machine learning model must be non-zero. For example, in an embodiment of the present application, the NDF value of the first point cloud sub-data in the first frame of point cloud data has been set to zero, and the second point cloud sub-data must contain such points with NDF values that are not zero, and such points with NDF values that are not zero can become the objects of optimization and adjustment, that is, they can be optimized in the direction of zero, and the adjusted pose of the corresponding points in the odometer map can be obtained accordingly, thereby improving the accuracy of the odometer map generated based on the point cloud data.

[0098] S107 : For the second neural distance field value of each point in the second point cloud sub-data, determine the gradients of the second neural distance field value in the three axis directions of the first coordinate system respectively.

[0099] S108 , for each point in the second point cloud sub-data, determine the moving direction of the point using the gradients in the three axis directions and determine the moving distance of the point using the second neural distance field value corresponding to the point.

[0100] In step S107, the second NDF value generated by the second point cloud sub-data in step S105 can be used to calculate the gradient in the three axis directions, such as xyz, and in step S108, the gradient and the second NDF value can be used to determine the moving direction and distance of the corresponding point in the initial odometry map.

[0101] For example, in step S108, the translation matrix and transformation matrix of each point can be determined according to the gradient of each point and the second neural distance field value, and then the updated position of each point in the initial odometry map is determined using the translation matrix and the transformation matrix to update the position of each point in the initial odometry map.

[0102] Specifically, in step S108 , the translation matrix and the transformation matrix of each point may be determined in an iterative manner.

[0103] For example, we can first construct the Hessian matrix H and the gradient vector g* of the objective function, where H = J T ωJ, θ=log(R) and θ is the axial angle representation of the rotation matrix R, ω is the weight matrix, And g represents the distance gradient of point P, P represents the coordinates of point P, g * =J T ωb, b is the residual of the neural distance field value between point P and the target function. Then, the translation matrix and transformation matrix of each point can be calculated in each iteration, and the increment of this iteration δ∈=-(H+μ d diag(H)) -1 g * , and the gain ratio ρ is calculated to adjust the damping factor μ d ,

[0104] v is used to adjust μ d Finally, when the loop increment is less than the preset threshold, the translation matrix and conversion matrix of the current iteration are output as the final transformation matrix used to adjust the position of each point in the odometry map.

[0105] S109 , updating the position of each point in the initial odometer map according to the determined moving direction and moving distance of each point to obtain an updated odometer map.

[0106] In step S109 , the transformation matrix calculated and determined in step S108 , ie, the translation matrix and the conversion matrix, may be used to adjust the position of each point in the odometer map.

[0107] In addition, in an embodiment of the present application, since cumulative errors may occur in large-scale point cloud mapping, after the transformation matrix is generated in step S108, at the same time or after the update processing in step S109, the distance between the adjusted posture determined by the current frame point cloud data and the posture of the historical frame can be used to determine the historical frame that may have a closed loop with the current frame, and when it is determined that such a historical frame exists, the point cloud data of such a historical frame can be further input into the first machine learning model to calculate the NDF value, and the difference between the NDF value of the historical frame and the NDF frame of the current frame is calculated to confirm whether it is less than a preset threshold.

[0108] For example, Figure 2 As shown in Figure 2 : is a flow chart illustrating loop closure detection in the odometer method of an embodiment of the present application. In loop closure detection, a second frame of point cloud data whose distance from the first frame of point cloud data is less than a preset frame distance threshold can be selected from the historical frame of point cloud data of the first frame of point cloud data. For example, as described above, the pose of the point cloud data of the current frame can be used to calculate the distance between the pose of the point cloud data of the historical frame. When the distance is less than the preset distance threshold, it can be considered that there may be a closed loop between the historical frame and the current frame, so that the point cloud data of the historical frame can be input into the trained first machine learning model trained in step S104 to generate a third neural distance field value for each point in the historical frame of point cloud data. When the first difference between the third neural distance field value of each point in the historical frame of point cloud data and the first neural distance field value and the second neural distance field value of the first frame of point cloud data is less than the preset neural distance threshold, it can be confirmed that the current frame and the historical frame have formed a closed loop, thereby confirming the accuracy of the pose adjustment of the current frame.

[0109] The neural distance field-based odometry method provided in an embodiment of the present application divides the collected point cloud data into two parts, one part is used as a positive sample for real-time training of a machine learning model using a neural radiance field (NeRF), and the remaining point cloud data is then input into the trained model to generate a signed distance function (SDF) value as a neural distance field value. Since, ideally, the signed distance function SDF values of points reflected from the surface of an object should all be zero, the neural distance field values generated in this way can be used to iteratively adjust toward zero to reduce or eliminate errors in the point cloud data. After the iteration is completed, the movement direction and corresponding movement distance of each point in the odometry map can be obtained, thereby obtaining an updated odometry map that can more accurately represent the scanned environment, greatly improving the measurement accuracy of SLAM.

[0110] Example 2

[0111] The present application also provides a sparse time fusion method for detecting three-dimensional objects in a laser radar point cloud. Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an embodiment of an odometer device based on a neural distance field provided by the present application. The odometer device of this embodiment of the present application may include an acquisition module 31, a division module 32, a sample generation module 33, a training module 34, a first generation module 35, an odometer map generation module 36, a first determination module 37, and an update module 38.

[0112] The acquisition module 31 can be used to acquire a first frame of point cloud data.

[0113] In the embodiment of the present application, the acquisition module 31 can extract point cloud data from the reflected signals received by the laser radar scanning the surrounding environment. After acquiring the point cloud data, the acquisition module 31 can pre-process the point cloud data, for example, by performing distortion correction processing and / or filtering and downsampling processing to remove noise.

[0114] The division module 32 may be configured to divide the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data.

[0115] The division module 32 can divide the point cloud data acquired by the acquisition module 31 or the point cloud data obtained after preprocessing by the acquisition module 31 into two parts: first point cloud sub-data and second point cloud sub-data. In the embodiment of the present application, since the point cloud data is generated based on the laser radar radiating a laser beam outward and receiving the reflection signal of the laser beam on the surrounding objects, the division module 32 can randomly or specifically select one or several laser beams and use the point cloud data corresponding to the selected laser beam as the first point cloud sub-data, and use the remaining point cloud data as the second point cloud sub-data. Of course, in the embodiment of the present application, the division method of the point cloud data is not limited to this, and other methods can also be used for division.

[0116] For example, in the embodiment of the present application, the segmentation module 32 may determine the first point cloud sub-data in the following manner:

[0117] The distal end point of at least one laser beam reflected from the surface of the scanned object is selected as the target end point, wherein the distal end point refers to an end point at one end away from the sensor receiving the laser beam, and the first neural distance field value of the target end point is equal to zero, and a line r=p / ||P||2 is formed from the end point to the sensor, and the i-th sampling point on the line is represented by N i =dr, where p represents the coordinate vector of the endpoint, d represents the depth relative to the endpoint on the line, and the Gaussian distribution is carried out at the endpoint with the first interval. Sampling is performed to obtain the value corresponding to the endpoint N i N adjacent surface points s As each point corresponding to the first point cloud sub-data.

[0118] The sample generation module 33 can be used to generate positive sample data for the first machine learning model based on the first point cloud sub-data.

[0119] In the sample generation module 33, the first point cloud data obtained by segmenting the first frame of point cloud data using the segmentation module 32 can be used to generate positive sample data for the machine learning model. For example, in an embodiment of the present application, a machine learning model based on a neural radiation field (NeRF) can be used to generate the value of the signed distance function (SDF) of the point cloud data. In an embodiment of the present application, the SDF value generated using such a NeRF-based machine learning model can also be referred to as the value of the neural distance field (NDF).

[0120] When generating positive sample data, the neural distance field value of each point in the first point cloud sub-data can be set to zero, that is, each point in the first point cloud sub-data is considered to be point cloud data generated by reflection on the surface of an object in the environment, and therefore its NDF value generated by the model should be zero, so that the first point cloud sub-data with such a preset zero NDF value can be used as positive sample data.

[0121] The training module 34 can be used to train the first machine learning model using positive sample data to generate a trained first machine learning model.

[0122] The training module 34 can use the positive sample data generated by the sample generation module 33 to train the machine learning model based on the neural radiation field NeRF, that is, the first point cloud sub-data obtained after the division by the division module 32 can be used as input, and the NDF value preset by the sample generation module 33 for each point in the first point cloud sub-data can be used as the positive sample value for training, so that a trained first machine learning model can be obtained.

[0123] For example, in an embodiment of the present application, the training module 34 may train the first machine learning model based on the positive sample data generated by the sample generation module 33 using the loss function shown in equation (1):

[0124]

[0125] in,

[0126]

[0127] λ1, λ2 and λ3 are respectively related to the loss function and The corresponding preset coefficient, N represents the number of points in the first frame of point cloud data, s i is the second neural distance field value, is the target value, ρ(s i )and and s respectively i and The associated reflectivity,

[0128]

[0129] R is the measured distance of each point in the first frame of point cloud data, is the angle-corrected intensity I combined with the preset replacement value r I The obtained post-compensation intensity, where the replacement value r I is the preset intensity compensation value for points with zero intensity, n all is a preset constant. In the embodiment of the present application, n all It can be a value of 20 or more.

[0130] In the embodiment of the present application, the laser intensity I of each point actually depends on the incident angle and the measured distance R. In the prior art, the measured distance is usually used to calibrate the intensity. In the embodiment of the present application, the incident angle is calibrated to adjust the laser intensity of the point in the point cloud data. In addition, during the scanning process, due to the low reflectivity of the target object, various interferences including strong light interference or occlusion, the intensity of the point captured by the LiDAR may be zero. In the embodiment of the present application, a preset replacement value r is used. I To recalculate the intensity value to zero, thus obtaining the adjusted compensation intensity

[0131] The first generating module 35 may be configured to input the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data.

[0132] The odometry map generation module 36 may be configured to generate an initial odometry map using the first frame of point cloud data.

[0133] The first generation module 35 can input the second point cloud sub-data into the first machine learning model trained by the training module 34 to calculate the NDF value of each point, and at the same time or thereafter, can use the first frame point cloud data to generate an initial odometry map.

[0134] Due to environmental factors and other reasons, the point cloud data generated based on the reflected signal of the laser beam cannot accurately reflect the shape of the environmental objects, that is, at least some of the points in the returned point cloud data are not actually on the surface of the object, so the NDF value of such points obtained after calculation by the machine learning model must be non-zero. For example, in an embodiment of the present application, the NDF value of the first point cloud sub-data in the first frame of point cloud data has been set to zero, and the second point cloud sub-data must contain such points with NDF values that are not zero, and such points with NDF values that are not zero can become the objects of optimization and adjustment, that is, they can be optimized in the direction of zero, and the adjusted pose of the corresponding points in the odometer map can be obtained accordingly, thereby improving the accuracy of the odometer map generated based on the point cloud data.

[0135] The first determination module 37 can be used to determine, for each point in the second point cloud sub-data, the second neural distance field value's gradients in the three axis directions of the first coordinate system, and, for each point in the second point cloud sub-data, determine the point's movement direction using the gradients in the three axis directions and determine the point's movement distance using the second neural distance field value corresponding to the point.

[0136] In the first determination module 37, the second NDF value generated by the second point cloud sub-data in the first generation module 35 can be used to calculate the gradient in the three axis directions such as xyz, and the gradient and the second NDF value can be used to determine the moving direction and distance of the corresponding point in the initial odometry map.

[0137] For example, the first determination module 37 can determine the translation matrix and the transformation matrix of each point based on the gradient of each point and the second neural distance field value, and then use the translation matrix and the transformation matrix to determine the updated position of each point in the initial odometry map to update the position of each point in the initial odometry map.

[0138] Specifically, the first determining module 37 may determine the translation matrix and the transformation matrix of each point in an iterative manner.

[0139] For example, we can first construct the Hessian matrix H and the gradient vector g* of the objective function, where H = J T ωJ, θ=log(R) and θ is the axial angle representation of the rotation matrix R, ω is the weight matrix, And g represents the distance gradient of point P, P represents the coordinates of point P, g * =J T ωb, b is the residual of the neural distance field value between point P and the target function. Then, the translation matrix and transformation matrix of each point can be calculated in each iteration, and the increment of this iteration δ∈=-(H+μ d diag(H)) -1 g * , and the gain ratio ρ is calculated to adjust the damping factor μ d ,

[0140] v is used to adjust μ d Finally, when the loop increment is less than the preset threshold, the translation matrix and conversion matrix of the current iteration are output as the final transformation matrix used to adjust the position of each point in the odometry map.

[0141] The updating module 38 may be configured to update the position of each point in the initial odometer map according to the determined moving direction and moving distance of each point to obtain an updated odometer map.

[0142] The updating module 38 may use the transformation matrix calculated and determined by the first determining module 37 , ie, the translation matrix and the conversion matrix, to adjust the position of each point in the odometer map.

[0143] In addition, in an embodiment of the present application, since cumulative errors may occur in large-scale point cloud mapping, after the first determination module 37 generates the transformation matrix, at the same time or after the update processing of the update module 38, the distance between the adjusted posture determined by the current frame point cloud data and the posture of the historical frame can be used to determine the historical frame that may have a closed loop with the current frame, and when it is determined that such a historical frame exists, the point cloud data of such a historical frame can be further input into the first machine learning model to calculate the NDF value, and the difference between the NDF value of the historical frame and the NDF frame of the current frame is calculated to confirm whether it is less than a preset threshold.

[0144] For example, a second frame of point cloud data can be selected from the historical frame point cloud data of the first frame point cloud data, the distance between which and the first frame point cloud data is less than a preset frame distance threshold. For example, as described above, the pose of the point cloud data of the current frame can be used to calculate the distance between the pose of the point cloud data of the historical frame and the pose of the point cloud data of the historical frame. When the distance is less than the preset distance threshold, it can be considered that there may be a closed loop between the historical frame and the current frame. Therefore, the point cloud data of the historical frame can be input into the trained first machine learning model trained in the training module 34 to generate a third neural distance field value for each point in the historical frame point cloud data. When the first difference between the third neural distance field value of each point in the historical frame point cloud data and the first neural distance field value and the second neural distance field value of the first frame point cloud data is less than the preset neural distance threshold, the pose data of each point in the historical frame point cloud data and the first difference are used to generate position adjustment information for each point in the first frame point cloud data, and the position adjustment information is used to update the position of each point in the odometry map to obtain an updated odometry map.

[0145] The neural distance field-based odometry device provided in an embodiment of the present application divides the collected point cloud data into two parts, one part is used as a positive sample for real-time training of a machine learning model using a neural radiance field (NeRF), and the remaining point cloud data is then input into the trained model to generate a signed distance function (SDF) value as a neural distance field value. Since, ideally, the signed distance function (SDF) values of points reflected from the surface of an object should all be zero, the neural distance field values generated in this way can be used to iteratively adjust toward zero to reduce or eliminate errors in the point cloud data. After the iteration is completed, the movement direction and corresponding movement distance of each point in the odometry map can be obtained, thereby obtaining an updated odometry map that can more accurately represent the scanned environment, greatly improving the measurement accuracy of SLAM.

[0146] Example 3

[0147] The above describes the internal functions and structure of the neural distance field based odometry method, which can be implemented as an electronic device. Figure 4This is a schematic diagram of the structure of an electronic device embodiment provided by this application. Figure 4 As shown, the electronic device includes a memory 41 and a processor 42 .

[0148] Memory 41 is used to store programs. In addition to the aforementioned programs, memory 41 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, images, videos, etc.

[0149] The memory 41 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] Processor 42 is not limited to a CPU and may also be a graphics processing unit (GPU), a field programmable gate array (FPGA), an embedded neural network processor (NPU), or an artificial intelligence (AI) chip. Processor 42 is coupled to memory 41 and executes the program stored in memory 41 to implement the neural distance field-based odometry method of the first embodiment.

[0151] Further, if Figure 4 As shown, the electronic device may further include: a communication component 43, a power component 44, an audio component 45, a display 46 and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 Components shown.

[0152] The communication component 43 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 43 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 43 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0153] The power supply assembly 44 provides power to various components of the electronic device. The power supply assembly 44 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0154] The audio component 45 is configured to output and / or input audio signals. For example, the audio component 45 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 41 or transmitted via the communication component 43. In some embodiments, the audio component 45 also includes a speaker for outputting audio signals.

[0155] The display 46 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0156] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for odometry based on neural distance field, characterized in that: include: Get the first frame of point cloud data; dividing the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data, wherein the first point cloud sub-data is a plurality of beam point cloud data acquired from at least one laser beam reflected from a surface of a scanned object; generating positive sample data for the first machine learning model based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on a neural radiation field, and the positive sample data includes the first point cloud sub-data and a first neural distance field value for each point in the first point cloud sub-data, and the first neural distance field value for each point in the first point cloud sub-data is equal to zero; Training the first machine learning model using the positive sample data to generate a trained first machine learning model; Inputting the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data; Generate an initial odometry map using the first frame of point cloud data, wherein the initial odometry map uses a first coordinate system to represent the coordinates of each point; For the second neural distance field value of each point in the second point cloud sub-data, respectively determine the gradients of the second neural distance field value in the three axis directions of the first coordinate system; For each point in the second point cloud sub-data, using the gradients in the three axis directions to determine the moving direction of the point and using the second neural distance field value corresponding to the point to determine the moving distance of the point; The position of each point in the initial odometer map is updated according to the determined moving direction and moving distance of each point to obtain an updated odometer map.

2. The odometer method based on neural distance field according to claim 1, characterized in that The method further comprises: Selecting, from the historical frame point cloud data of the first frame point cloud data, a second frame point cloud data whose distance from the first frame point cloud data is less than a preset frame distance threshold; Inputting the second frame of point cloud data into the trained first machine learning model to generate a third neural distance field value for each point in the second frame of point cloud data; When a first difference between the third neural distance field value of each point in the second frame point cloud data and the first neural distance field value and the second neural distance field value of the first frame point cloud data is less than a preset neural distance threshold, it is confirmed that the first frame point cloud data and the second frame point cloud data form a closed loop.

3. The odometer method based on neural distance field according to claim 1, characterized in that The first point cloud sub-data is determined by: selecting a distal end point of at least one laser beam reflected from a surface of a scanned object as a target end point, wherein the distal end point refers to an end point far from an end of a sensor receiving the laser beam, and a first neural distance field value of the target end point is equal to zero; A line r=p / ||P||2 is formed from the end point to the sensor, and the i-th sampling point on the line is represented by N i =dr, where p represents the coordinate vector of the endpoint and d represents the depth relative to the endpoint on the line; At the end points along a Gaussian distribution at a first interval Sampling is performed to obtain the value corresponding to the endpoint N i N adjacent surface points s As each point corresponding to the first point cloud sub-data.

4. The odometer method based on neural distance field according to claim 1, characterized in that The training of the first machine learning model using the positive sample data includes: training the first machine learning model based on the positive sample data using the following loss function: in, λ1, λ2 and λ3 are respectively related to the loss function and The corresponding preset coefficient, N represents the number of points in the first frame of point cloud data, s i is the second neural distance field value, is the target value, ρ(s i )and and s respectively i and The associated reflectivity, R is the measured distance of each point in the first frame of point cloud data, is the angle-corrected intensity I combined with the preset replacement value r I The obtained post-compensation intensity, where the replacement value r I is the preset intensity compensation value for points with zero intensity, n all is a preset constant.

5. The odometer method based on neural distance field according to claim 1, characterized in that The updating of the position of each point in the initial odometer map according to the determined moving direction and moving distance of each point includes: Determine a translation matrix and a transformation matrix for each point according to the gradient and the second neural distance field value; An updated position of each point in the initial odometry map is determined using the translation matrix and the transformation matrix to update the position of each point in the initial odometry map.

6. The odometer method based on neural distance field according to claim 5, characterized in that Determining the translation matrix and the transformation matrix of each point according to the gradient and the second neural distance field value includes: Construct the Hessian matrix H and the gradient vector g* of the objective function, where θ = log(R) and is the axial angle representation of the rotation matrix R, ω is the weight matrix, And represents the distance gradient of point P, P represents the coordinates of point P, g * =J T ωb, b is the residual of the neural distance field value between point P and the target function; The translation matrix and transformation matrix of each point are calculated in an iterative manner in each round of iteration, and the increment of the current round of iteration is determined based on the translation matrix and transformation matrix of each point calculated in the current round of iteration. d diag(H)) -1 g * , and the gain ratio ρ is calculated to adjust the damping factor μ d , v is used to adjust μ d Factors of When the loop increment is less than the preset threshold, the translation matrix and conversion matrix of the current iteration round are output.

7. An odometer device based on neural distance field, characterized in that: The device comprises: Acquisition module, used to obtain the first frame of point cloud data; a dividing module, configured to divide the first frame of point cloud data into first point cloud sub-data and second point cloud sub-data, wherein the first point cloud sub-data is a plurality of beam point cloud data obtained from at least one laser beam reflected from a surface of a scanned object; a sample generation module, configured to generate positive sample data for the first machine learning model based on the first point cloud sub-data, wherein the first machine learning model is a machine learning model based on a neural radiation field, and the positive sample data includes the first point cloud sub-data and a first neural distance field value for each point in the first point cloud sub-data, and the first neural distance field value for each point in the first point cloud sub-data is equal to zero; a training module, configured to train the first machine learning model using the positive sample data to generate a trained first machine learning model; a first generating module, configured to input the second point cloud sub-data into the trained first machine learning model to generate a second neural distance field value for each point in the second point cloud sub-data; an odometry map generating module, configured to generate an initial odometry map using the first frame of point cloud data, wherein the initial odometry map uses a first coordinate system to represent the coordinates of each point; a first determination module configured to determine, for each point in the second point cloud sub-data, a second neural distance field value, and gradients of the second neural distance field value in the three axial directions of the first coordinate system; and for each point in the second point cloud sub-data, determine a movement direction of the point using the gradients in the three axial directions and determine a movement distance of the point using the second neural distance field value corresponding to the point; An updating module is used to update the position of each point in the initial odometer map according to the determined moving direction and moving distance of each point to obtain an updated odometer map.

8. An electronic device, characterized in that: include: Memory, used to store programs; A processor, configured to run the program stored in the memory to perform the odometer method based on neural distance field according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program executable by a processor, characterized in that: When the program is executed by a processor, the odometer method based on neural distance field is implemented.

Images