A radiation field SLAM pose optimization method based on motion constraints

Abstract

The invention discloses a radiation field SLAM posture optimization method based on motion constraints, belongs to the field of SLAM positioning calculation technology, and is used for posture optimization, including obtaining an inertial measurement unit IMU data frame, an RGB image data frame, and a depth image data frame, performing pre-integration to obtain a motion increment, recording the Jacobian matrix of the motion increment relative to the zero bias, and recording the noise covariance matrix of the motion increment; initializing gravity and speed; calculating the residual of the estimated value and measured value of the posture and speed, optimizing the posture, speed, and zero bias in combination with the radiation field loss, and updating the motion increment with the Jacobian matrix. The invention provides an optimization function to optimize the interference of the IMU zero bias, initializes the motion parameters and gravity parameters compared to the radiation field SLAM method of fusing IMU data, so that the radiation field SLAM has higher robustness to the IMU data quality, and provides a solution for the practical application of the radiation field SLAM.

Description

Technical Field

[0001] The invention discloses a radiation field SLAM posture optimization method based on motion constraints, belonging to the technical field of posture calculation. Background Art

[0002] Simultaneous localization and mapping (SLAM) is an important research topic in computer vision, which involves exploring the environment and estimating the position in real time without prior map information. SLAM is divided into two types: dense and sparse according to the level of map detail. Dense SLAM builds detailed 3D models, while sparse SLAM only extracts key feature points. Traditional dense SLAM systems are adapted to large-scale scene applications such as autonomous driving, home robots, and augmented reality through high positioning accuracy and real-time processing capabilities. These advances have significantly promoted the autonomy and interaction capabilities of intelligent systems. In the study of high-fidelity 3D reconstruction, the radiation field theory has shown its excellent performance, and the neural radiation field (NeRF) and 3D Gaussian splash representation methods have shown significant success. In order to overcome the limitations faced by traditional SLAM technology, these methods are being introduced into the SLAM field to build more refined geometric field models. Compared with traditional map representation methods, these new methods have many advantages. They enable continuous surface modeling, reduce memory requirements, and handle noise and outliers more effectively. In addition, they show better hole filling and scene rendering capabilities when dealing with occlusion or sparse observations. These methods also generate more dense and compact maps and can be reconstructed into 3D meshes at arbitrary resolution, providing a high-quality and flexible 3D reconstruction solution.

[0003] The neural radiance field method uses a fully connected non-convolutional deep network to represent the scene. The network takes a continuous five-dimensional coordinate as input, including the spatial position and the viewing direction, and outputs the volume density of the corresponding spatial point and the radiation intensity related to the view. View synthesis is achieved by querying these five-dimensional coordinates along the camera ray, and the color and density information are projected into the final image using volume rendering technology. The spatial voxel-based NeRF is introduced, and only the part of the NeRF related to the current camera view space is updated. The 3D Gaussian method uses an explicit 3D Gaussian sphere to represent the scene. The 3D Gaussian sphere contains the center position, Gaussian covariance matrix, opacity, and spherical harmonic function parameters, where the center position and Gaussian covariance matrix represent the influence range of the explicit 3D Gaussian sphere, and the opacity and spherical harmonic parameters are used to control the color representation. Since NeRF and 3D Gaussian have the same actual form of tracking, they are only consistent in the tracking algorithm. NeRF and 3D Gaussian are collectively referred to as radiance field algorithms.

[0004] Although SLAM based on radiation fields can generate complete and dense maps, it still has shortcomings in posture estimation compared with traditional SLAM methods. Although there have been important advances in the optimization of positioning, there are still shortcomings. For example, once the loop detection fails, it will cause devastating reconstruction errors. SLAM based on sparse point clouds is prone to tracking failures. In the process of introducing additional data sources, it is necessary to face the interference of noise. Summary of the invention

[0005] The purpose of the present invention is to provide a radiation field SLAM posture optimization method based on motion constraints to solve the problem of insufficient posture estimation accuracy of radiation field-based SLAM in the prior art.

[0006] A radiation field SLAM posture optimization method based on motion constraints includes: S1: acquiring an inertial measurement unit IMU data frame and an image frame, wherein the image frame includes an RGB image frame and a depth image frame, wherein the frame rates of the RGB image data frame and the depth image data frame are consistent;

[0007] S2: Segment the data frames according to the timestamps of the image frames, and separate all IMU data frames between two image frames;

[0008] S3: Obstacle avoidance system VO prediction pose using radiation field loss;

[0009] S4: Pre-integrate the data frame to obtain the motion data of the current frame relative to the previous frame in the world coordinate system, including the motion increment of speed, position and posture;

[0010] S5: Record the Jacobian matrix of the current motion increment relative to the zero bias, and use the Jacobian matrix to update the motion increment each time the zero bias is updated;

[0011] S6: record the noise covariance matrix of the current motion increment;

[0012] S7: When the image frames exceed 200 frames and the number of key frames exceeds 10 frames, the global gravity and velocity of the IMU are corrected based on the pose predicted by the radiation field loss;

[0013] S8: Calculate the residual of the motion estimation value and the measurement value, and set the initial value of the pose to the pose of the previous frame;

[0014] S9: Calculate the total loss and complete the pose optimization.

[0015] S3 includes calculating the joint color depth loss of the SLAM image frame and the real image frame :

[0016] ;

[0017] In the formula, for norm, is the color of the sampling frame in the SLAM image frame, is the depth of the sampling frame in the SLAM image frame, is the color of the sampled frame in the real image frame, is the depth of the sampled frame in the real image frame;

[0018] Use the estimated pose of the previous frame as the input initial pose of the neural radiation field NeRF, use the multi-layer perceptron MLP as the decoder, and use the feature grid to represent the color of the sampled frame and occupancy value :

[0019] ; ;

[0020] In the formula, Represents a color decoder, To express attitude, represents the color feature grid, represents the occupancy decoder, Represents the occupancy feature grid.

[0021] S3 includes calculating the occupancy integral of the ray where the sampling ray is located along the sampling frame:

[0022] ;

[0023] In the formula, represents the first Locations The occupancy rate, Indicates along the ray The occupancy value, Indicates along the ray Occupancy value of

[0024] Perform comprehensive processing on the color and depth of the sampled frame:

[0025] ; ;

[0026] In the formula, Along the ray Color, Along the ray Depth.

[0027] S3 includes the use of an explicit 3D Gaussian sphere as a scene representation for SLAM :

[0028] }; ;

[0029] In the formula, It is 3D Gaussian scene representation of the frame, It is The position of the frame represents the center point of the 3D Gaussian sphere. Each frame corresponds to a 3D Gaussian sphere. It is The three-dimensional covariance matrix of a 3D Gaussian ball describes the shape and direction of the 3D Gaussian ball. It is The opacity of a 3D Gaussian sphere, It is The color of a 3D Gaussian sphere, is the total number of sampling frames, is a three-dimensional scale vector, representing the stretched shape, It is a rotation matrix, which represents the direction and uses quaternion to store the rotation matrix;

[0030] right Projection generates two-dimensional covariance :

[0031] ;

[0032] In the formula, is the Jacobian matrix of the affine approximation of the projection function, Represents a view transform;

[0033] After the projection is completed, a 2D Gaussian projection is formed. The center point of the 2D Gaussian projection is used as the center point of the corresponding 3D Gaussian sphere. The 2D Gaussian projection is depth-sorted along the ray direction. Opacity of a 2D Gaussian projection for:

[0034] ;

[0035] In the formula, represents exponential operation with natural constant as base, represents determinant operation;

[0036] The image generated after depth sorting is rendered using a method that comprehensively processes the color and depth of the sampled frame. and camera pose , and for:

[0037] ;

[0038] ;

[0039] In the formula, Representation and Different 2D Gaussian spheres, Indicates The depth of a 2D Gaussian projection.

[0040] S4 includes establishing the initial movement increment , where the initial and is a three-dimensional zero matrix, is the identity matrix, receiving Acceleration of the IMU frame , angular velocity , Time Input , combined with the acceleration noise zero bias of the previous frame and angular velocity noise bias calculate :

[0041] ; ;

[0042] ;

[0043] In the formula, For the Frame to The frame's pose increment, is the exponential mapping from Lie algebra to Lie group, For the Frame to The frame rate includes the increment of the gravity component, For the Frame to The frame's pose increment, For the Frame to The frame position contains the increment of the gravity component, For the Frame to The frame rate includes the increment of the gravity component.

[0044] S5 includes:

[0045] ; ;

[0046] ;

[0047] ;

[0048] ;

[0049] In the formula, the acceleration noise zero bias and angular velocity noise zero bias are constant between frames. is the inter-frame acceleration noise bias, is the inter-frame angular velocity noise zero bias, represents partial derivative, is the antisymmetric operation in Lie algebra, is the right Jacobian matrix in the Lie algebra space.

[0050] S6 includes:

[0051] ;

[0052] ;

[0053] ;

[0054] In the formula, and is the intermediate matrix, For the Frame to The rotation increment of the frame, is the identity matrix, for Frame , is the covariance matrix of the current zero bias, is the covariance matrix of the previous frame zero bias, is the zero-mean Gaussian noise covariance matrix of acceleration and angular velocity.

[0055] S7 includes S7.1: The gravity and speed to be optimized are ,in Indicates Frame rate, Represents gravity, and the optimization target is the residual between the predicted value of velocity and displacement between adjacent frames and the IMU measurement value;

[0056] Convert the velocity and displacement residual formulas into matrix forms respectively:

[0057] ;

[0058] ;

[0059] In the formula, For the The posture of the frame, For the Frame to frame The velocity of each image frame contains the increment of the gravity component. For the Frame to frame The position of each image frame contains the increment of the gravity component. For the The position of the frame;

[0060] Combined into the first joint form:

[0061] ;

[0062] Solve the matrix of unknown coefficients using the least squares method based on the first simultaneous equation ;

[0063] S7.2: Correct the direction of gravity, fix the modulus of the gravity vector as the magnitude of gravity, and use a pair of unit vectors orthogonal to the direction of gravity as the quantity to be optimized. The gravitational acceleration is:

[0064] ;

[0065] In the formula, is the modulus of gravity, is the direction of gravity, , is the coefficient to be determined, , is a pair of orthogonal unit vectors, , are perpendicular to the direction of gravity. for , The joint matrix of for , The joint matrix of

[0066] After each round of iterative update, the updated gravity direction is normalized to update ;

[0067] Substitute the calculation formula of gravitational acceleration into the first simultaneous equation to obtain the second simultaneous equation:

[0068] ;

[0069] Use the least squares method to solve the coefficient matrix based on the second simultaneous equation .

[0070] S8 includes:

[0071] ;

[0072] ;

[0073] ;

[0074] In the formula, is the residual between the estimated and measured values ​​of the attitude, is the residual between the velocity estimate and the measured value, is the residual error between the estimated position and the measured value, is the logarithmic mapping from Lie group to Lie algebra, For the The pose of each image frame, For the The frame rate of the image frame, For the The position of the frame image, is the zero bias increment of acceleration noise between frames, is the zero bias increment of the angular velocity noise between frames.

[0075] Calculate total loss :

[0076] ;

[0077] In the formula, for The diagonal matrix of ;

[0078] ;

[0079] In the formula, is the total loss of tracking, is the weight coefficient;

[0080] The gradient descent method is used to optimize the posture, speed and zero bias. When the total loss is minimized, the optimal posture is reached.

[0081] After each round of gradient descent, the Jacobian matrix of the current motion increment relative to the noise is used to update the pre-integrated terms of the attitude, velocity, and displacement:

[0082] ;

[0083] ;

[0084] ;

[0085] In the formula, , , are the attitude, velocity, and displacement after the integral term is updated.

[0086] Compared with the prior art, the present invention has the following beneficial effects: an optimization function is provided to optimize the interference of IMU zero bias, further improving the robustness of additional data sources, and compared with the previous inertial-based radiation field SLAM method, the motion parameters and gravity parameters are initialized, making the radiation field SLAM more adaptable to positioning, providing a solution for the practical application of radiation field SLAM. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is a technical flow chart of the present invention;

[0088] Figure 2 The x-direction tracking result diagram is shown without and with IMU added;

[0089] Figure 3 The y-direction tracking result diagram is shown without and with IMU added;

[0090] Figure 4 The following are the z-direction tracking result diagrams without and with IMU added;

[0091] Figure 5 This is the flow chart of pre-integration. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] A radiation field SLAM posture optimization method based on motion constraints includes: S1: acquiring an inertial measurement unit IMU data frame and an image frame, wherein the image frame includes an RGB image frame and a depth image frame, wherein the frame rates of the RGB image data frame and the depth image data frame are consistent;

[0094] S2: Segment the data frames according to the timestamps of the image frames, and separate all IMU data frames between two image frames;

[0095] S3: Obstacle avoidance system VO prediction pose using radiation field loss;

[0096] S4: Pre-integrate the data frame to obtain the motion data of the current frame relative to the previous frame in the world coordinate system, including the motion increment of speed, position and posture;

[0097] S5: Record the Jacobian matrix of the current motion increment relative to the zero bias, and use the Jacobian matrix to update the motion increment each time the zero bias is updated;

[0098] S6: record the noise covariance matrix of the current motion increment;

[0099] S7: When the image frames exceed 200 frames and the number of key frames exceeds 10 frames, the global gravity and velocity of the IMU are corrected based on the pose predicted by the radiation field loss;

[0100] S8: Calculate the residual of the motion estimation value and the measurement value, and set the initial value of the pose to the pose of the previous frame;

[0101] S9: Calculate the total loss and complete the pose optimization.

[0102] S3 includes calculating the joint color depth loss of the SLAM image frame and the real image frame :

[0103] ;

[0104] In the formula, for norm, is the color of the sampling frame in the SLAM image frame, is the depth of the sampling frame in the SLAM image frame, is the color of the sampled frame in the real image frame, is the depth of the sampled frame in the real image frame;

[0105] Use the estimated pose of the previous frame as the input initial pose of the neural radiation field NeRF, use the multi-layer perceptron MLP as the decoder, and use the feature grid to represent the color of the sampled frame and occupancy value :

[0106] ; ;

[0107] In the formula, Represents a color decoder, To express attitude, represents the color feature grid, represents the occupancy decoder, Represents the occupancy feature grid.

[0108] S3 includes calculating the occupancy integral of the ray where the sampling ray is located along the sampling frame:

[0109] ;

[0110] In the formula, represents the first Locations The occupancy rate, Indicates along the ray The occupancy value, Indicates along the ray Occupancy value of

[0111] Perform comprehensive processing on the color and depth of the sampled frame:

[0112] ; ;

[0113] In the formula, Along the ray Color, Along the ray Depth.

[0114] S3 includes the use of an explicit 3D Gaussian sphere as a scene representation for SLAM :

[0115] }; ;

[0116] In the formula, It is 3D Gaussian scene representation of the frame, It is The position of the frame represents the center point of the 3D Gaussian sphere. Each frame corresponds to a 3D Gaussian sphere. It is The three-dimensional covariance matrix of a 3D Gaussian ball describes the shape and direction of the 3D Gaussian ball. It is The opacity of a 3D Gaussian sphere, It is The color of a 3D Gaussian sphere, is the total number of sampling frames, is a three-dimensional scale vector, representing the stretched shape, It is a rotation matrix, which represents the direction and uses quaternion to store the rotation matrix;

[0117] right Projection generates two-dimensional covariance :

[0118] ;

[0119] In the formula, is the Jacobian matrix of the affine approximation of the projection function, Represents a view transform;

[0120] After the projection is completed, a 2D Gaussian projection is formed. The center point of the 2D Gaussian projection is used as the center point of the corresponding 3D Gaussian sphere. The 2D Gaussian projection is depth-sorted along the ray direction. Opacity of a 2D Gaussian projection for:

[0121] ;

[0122] In the formula, represents exponential operation with natural constant as base, represents determinant operation;

[0123] The image generated after depth sorting is rendered using a method that comprehensively processes the color and depth of the sampled frame. and camera pose , and for:

[0124] ;

[0125] ;

[0126] In the formula, Representation and Different 2D Gaussian spheres, Indicates The depth of a 2D Gaussian projection.

[0127] S4 includes establishing the initial movement increment , where the initial and is a three-dimensional zero matrix, is the identity matrix, receiving Acceleration of the IMU frame , angular velocity , Time Input , combined with the acceleration noise zero bias of the previous frame and angular velocity noise bias calculate :

[0128] ; ;

[0129] ;

[0130] In the formula, For the Frame to The frame's pose increment, is the exponential mapping from Lie algebra to Lie group, For the Frame to The frame rate includes the increment of the gravity component, For the Frame to The frame's pose increment, For the Frame to The frame position contains the increment of the gravity component, For the Frame to The frame rate includes the increment of the gravity component.

[0131] S5 includes:

[0132] ; ;

[0133] ;

[0134] ;

[0135] ;

[0136] In the formula, the acceleration noise zero bias and angular velocity noise zero bias are constant between frames. is the inter-frame acceleration noise bias, is the inter-frame angular velocity noise zero bias, represents partial derivative, is the antisymmetric operation in Lie algebra, is the right Jacobian matrix in the Lie algebra space.

[0137] S6 includes:

[0138] ;

[0139] ;

[0140] ;

[0141] In the formula, and is the intermediate matrix, For the Frame to The rotation increment of the frame, is the identity matrix, for Frame , is the covariance matrix of the current zero bias, is the covariance matrix of the previous frame zero bias, is the zero-mean Gaussian noise covariance matrix of acceleration and angular velocity.

[0142] S7 includes S7.1: The gravity and speed to be optimized are ,in Indicates Frame rate, Represents gravity, and the optimization target is the residual between the predicted value of velocity and displacement between adjacent frames and the IMU measurement value;

[0143] Convert the velocity and displacement residual formulas into matrix forms respectively:

[0144] ;

[0145] ;

[0146] In the formula, For the The posture of the frame, For the Frame to frame The velocity of each image frame contains the increment of the gravity component. For the Frame to frame The position of each image frame contains the increment of the gravity component. For the The position of the frame;

[0147] Combined into the first joint form:

[0148] ;

[0149] Solve the matrix of unknown coefficients using the least squares method based on the first simultaneous equation ;

[0150] S7.2: Correct the direction of gravity, fix the modulus of the gravity vector as the magnitude of gravity, and use a pair of unit vectors orthogonal to the direction of gravity as the quantity to be optimized. The gravitational acceleration is:

[0151] ;

[0152] In the formula, is the modulus of gravity, is the direction of gravity, , is the coefficient to be determined, , is a pair of orthogonal unit vectors, , are perpendicular to the direction of gravity. for , The joint matrix of for , The joint matrix of

[0153] After each round of iterative update, the updated gravity direction is normalized to update ;

[0154] Substitute the calculation formula of gravitational acceleration into the first simultaneous equation to obtain the second simultaneous equation:

[0155] ;

[0156] Use the least squares method to solve the coefficient matrix based on the second simultaneous equation .

[0157] S8 includes:

[0158] ;

[0159] ;

[0160] ;

[0161] In the formula, is the residual between the estimated and measured values ​​of the attitude, is the residual between the velocity estimate and the measured value, is the residual error between the estimated position and the measured value, is the logarithmic mapping from Lie group to Lie algebra, For the The pose of each image frame, For the The frame rate of the image frame, For the The position of the frame image, is the zero bias increment of acceleration noise between frames, is the inter-frame angular velocity noise zero bias increment.

[0162] Calculate total loss :

[0163] ;

[0164] In the formula, for The diagonal matrix of ;

[0165] ;

[0166] In the formula, is the total loss of tracking, is the weight coefficient;

[0167] The gradient descent method is used to optimize the posture, speed and zero bias. When the total loss is minimized, the optimal posture is reached.

[0168] After each round of gradient descent, the Jacobian matrix of the current motion increment relative to the noise is used to update the pre-integrated terms of the attitude, velocity, and displacement:

[0169] ;

[0170] ;

[0171] ;

[0172] In the formula, , , are the attitude, velocity, and displacement after the integral term is updated.

[0173] The technical process of the present invention is as follows Figure 1 As shown, a generated depth map and a generated color map are obtained from the scene generator and the current pose of the camera, and the input RGB stream and the input IMU stream are obtained from the input stream to obtain the input depth map and the input color map. The generated color map and the input color map are imported into the color loss (for tracking loss), and the generated depth map and the input depth map are imported into the depth loss (for minimization). The pose is estimated using the color loss and the depth loss before the motion parameters are initialized. The pre-integration result is obtained by inputting the IMU stream. When the judgment is greater than 200 frames and the key number is greater than 10, the motion parameters are initialized and updated. The measured pose is initialized by the pre-integration result, and the estimated pose is obtained by the motion parameters and the current pose of the camera. Both the measured pose and the estimated pose are input into the generated IMU loss.

[0174] The pre-integration process is as follows Figure 5 As shown, the IMU data stream is divided into image frames to obtain inter-frame IMU data, and pre-integration is performed to obtain the noise Jacobian matrix, velocity and displacement increments and noise covariance. The residuals of the velocity and displacement increments and noise covariance are calculated and combined with the radiation field loss to obtain the total loss. The total loss and the noise Jacobian matrix are iteratively updated.

[0175] NICE-SLAM is an RGB-DSLAM system that combines neural implicit radiation field and hierarchical scene representation, which is suitable for large-scale scenes. The embodiment provides a three-axis trajectory comparison experiment of NICE-SLAM before and after adding the improved algorithm of the present invention, using RTX3080Ti with 12G memory GPU on the office1-1 sequence on the open source dataset OpenLORIS, where Figure 2 This is the x-direction tracking result without and with IMU. Figure 3 It is the y-direction tracking result without adding IMU and after adding IMU. Figure 4The following are the z-tracking results without and with the IMU. Multiple ground robots equipped with Intel Real Sense D435i cameras and T265 cameras were used for data collection. The D435i camera provides aligned color and depth images and IMU measurements. The image frames have a frame rate of 30 Hz, the IMU gyroscope has a frame rate of 400 Hz, and the accelerometer has a frame rate of 250 Hz. The ground truth trajectory of the robot is obtained from the OptiTrack motion capture system, a full-body motion capture system.

[0176] In the experiment, the number of iterations of the gradient descent optimization of the tracking module was 50. In the original NICE-SLAM system, a large number of small drifts occurred in the tracking trajectory, which was more obvious on the y-axis and z-axis. These problems were significantly improved by fusing IMU data. In addition, on the x-axis and y-axis, the baseline of the original NICE-SLAM showed a slight deviation from the true value, which was effectively corrected by introducing IMU data. The improved algorithm proposed in the present invention has a constraint effect on the offset on each axis, effectively avoiding the non-physical drift phenomenon in the tracking process.

[0177] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A radiation field SLAM pose optimization method based on motion constraints, characterized in that: include, S1: Acquire an inertial measurement unit IMU data frame and an image frame, wherein the image frame includes an RGB image frame and a depth image frame, wherein the frame rates of the RGB image data frame and the depth image data frame are consistent; S2: Segment the data frames according to the timestamps of the image frames, and separate all IMU data frames between two image frames; S3: Obstacle avoidance system VO prediction pose using radiation field loss; S4: Pre-integrate the data frame to obtain the motion data of the current frame relative to the previous frame in the world coordinate system, including the motion increment of speed, position and posture; S5: Record the Jacobian matrix of the current motion increment relative to the zero bias, and use the Jacobian matrix to update the motion increment each time the zero bias is updated; S6: record the noise covariance matrix of the current motion increment; S7: When the image frames exceed 200 frames and the number of key frames exceeds 10 frames, the global gravity and velocity of the IMU are corrected based on the pose predicted by the radiation field loss; S8: Calculate the residual of the motion estimation value and the measurement value, and set the initial value of the pose to the pose of the previous frame; S9: Calculate the total loss and complete the pose optimization.

2. A radiation field SLAM pose optimization method based on motion constraints according to claim 1, characterized in that: S3 includes calculating the joint color depth loss of the SLAM image frame and the real image frame : ； In the formula, is the norm, is the color of the sampling frame in the SLAM image frame, is the depth of the sampling frame in the SLAM image frame, is the color of the sampled frame in the real image frame, is the depth of the sampled frame in the real image frame; Use the estimated pose of the previous frame as the input initial pose of the neural radiation field NeRF, use the multi-layer perceptron MLP as the decoder, and use the feature grid to represent the color of the sampled frame and occupancy value : ； ； In the formula, Represents a color decoder, To express attitude, represents the color feature grid, represents the occupancy decoder, Represents the occupancy feature grid.

3. A radiation field SLAM pose optimization method based on motion constraints according to claim 2, characterized in that: S3 includes calculating the occupancy integral of the ray where the sampling ray is located along the sampling frame: ； In the formula, represents the first Locations The occupancy rate, Indicates along the ray The occupancy value, Indicates along the ray Occupancy value of Perform comprehensive processing on the color and depth of the sampled frame: ； ； In the formula, Along the ray Color, Along the ray Depth.

4. A radiation field SLAM pose optimization method based on motion constraints according to claim 2, characterized in that: S3 includes the use of an explicit 3D Gaussian sphere as a scene representation for SLAM : }； ； In the formula, It is 3D Gaussian scene representation of the frame, It is The position of the frame represents the center point of the 3D Gaussian sphere. Each frame corresponds to a 3D Gaussian sphere. It is The three-dimensional covariance matrix of a 3D Gaussian ball describes the shape and direction of the 3D Gaussian ball. It is The opacity of a 3D Gaussian sphere, It is The color of a 3D Gaussian sphere, is the total number of sampling frames, is a three-dimensional scale vector, representing the stretched shape, It is a rotation matrix, which represents the direction and uses quaternion to store the rotation matrix; right Projection generates two-dimensional covariance : ； In the formula, is the Jacobian matrix of the affine approximation of the projection function, Represents a view transform; After the projection is completed, a 2D Gaussian projection is formed. The center point of the 2D Gaussian projection is used as the center point of the corresponding 3D Gaussian sphere. The 2D Gaussian projection is depth-sorted along the ray direction. Opacity of a 2D Gaussian projection for: ； In the formula, represents an exponential operation with a natural constant as the base, represents determinant operation; The image generated after depth sorting is rendered using a method that comprehensively processes the color and depth of the sampled frame. and camera pose , and for: ； ； In the formula, Representation and Different 2D Gaussian spheres, Indicates The depth of a 2D Gaussian projection.

5. A radiation field SLAM pose optimization method based on motion constraints according to any one of claims 3 or 4, characterized in that: S4 includes establishing the initial movement increment , where the initial and is a three-dimensional zero matrix, is the identity matrix, receiving Acceleration of the IMU frame , angular velocity , Time Input , combined with the acceleration noise zero bias of the previous frame and angular velocity noise bias calculate : ； ； ； In the formula, For the Frame to The frame's pose increment, is the exponential mapping from Lie algebra to Lie group, For the Frame to The frame rate includes the increment of the gravity component, For the Frame to The frame's pose increment, For the Frame to The frame position contains the increment of the gravity component, For the Frame to The frame rate includes the increment of the gravity component.

6. A radiation field SLAM pose optimization method based on motion constraints according to claim 5, characterized in that S5 include: ； ； ； ； ； In the formula, the acceleration noise zero bias and angular velocity noise zero bias are constant between frames. is the inter-frame acceleration noise bias, is the inter-frame angular velocity noise zero bias, represents partial derivative, is the antisymmetric operation in Lie algebra, is the right Jacobian matrix in the Lie algebra space.

7. A radiation field SLAM pose optimization method based on motion constraints according to claim 6, characterized in that S6 include: ； ； ； In the formula, and is the intermediate matrix, For the Frame to The rotation increment of the frame, is the identity matrix, for Frame , is the covariance matrix of the current zero bias, is the covariance matrix of the previous frame zero bias, is the zero-mean Gaussian noise covariance matrix of acceleration and angular velocity.

8. The method for optimizing the radiation field SLAM posture based on motion constraints according to claim 7, characterized in that: S7 includes S7.1: The gravity and speed to be optimized are ,in Indicates Frame rate, Represents gravity, and the optimization target is the residual between the predicted value of velocity and displacement between adjacent frames and the IMU measurement value; Convert the velocity and displacement residual formulas into matrix forms respectively: ； ； In the formula, For the The posture of the frame, For the Frame to frame The velocity of each image frame contains the increment of the gravity component. For the Frame to frame The position of each image frame contains the increment of the gravity component. For the The position of the frame; Combined into the first joint form: ； Solve the matrix of unknown coefficients using the least squares method based on the first simultaneous equation ; S7.2: Correct the direction of gravity, fix the modulus of the gravity vector as the magnitude of gravity, and use a pair of unit vectors orthogonal to the direction of gravity as the quantity to be optimized. The gravitational acceleration is: ； In the formula, is the modulus of gravity, is the direction of gravity, , is the coefficient to be determined, , is a pair of orthogonal unit vectors, , are perpendicular to the direction of gravity. for , The joint matrix of for , The joint matrix of After each round of iterative update, the updated gravity direction is normalized to update ; Substitute the calculation formula of gravitational acceleration into the first simultaneous equation to obtain the second simultaneous equation: ； Use the least squares method to solve the coefficient matrix based on the second simultaneous equation .

9. The method for optimizing the radiation field SLAM posture based on motion constraints according to claim 8, wherein S8 include: ； ； ； In the formula, is the residual between the estimated and measured values ​​of the attitude, is the residual between the velocity estimate and the measured value, is the residual error between the estimated position and the measured value, is the logarithmic mapping from Lie group to Lie algebra, For the The pose of each image frame, For the The frame rate of the image frame, For the The position of the frame image, is the zero bias increment of acceleration noise between frames, is the inter-frame angular velocity noise zero bias increment.

10. The method for optimizing the radiation field SLAM posture based on motion constraints according to claim 9, characterized in that: Calculate total loss : ； In the formula, for The diagonal matrix of ; ； In the formula, is the total loss of tracking, is the weight coefficient; The gradient descent method is used to optimize the posture, speed and zero bias. When the total loss is minimized, the optimal posture is reached. After each round of gradient descent, the Jacobian matrix of the current motion increment relative to the noise is used to update the pre-integrated terms of the attitude, velocity, and displacement: ； ； ； In the formula, , , are the attitude, velocity, and displacement after the integral term is updated.

Images