Automatic driving boundary-free scene implicit reconstruction method considering geometric information enhancement

Abstract

The invention relates to the field of automatic driving scene reconstruction, discloses an automatic driving boundary-free scene implicit reconstruction method and system considering geometric information enhancement, and adopts an automatic driving geometric perception grid-based neural rendering system. The system reconstructs and renders an accurate driving environment from sparse sensor data using a perspective deformed hash grid and a signed distance function (SDF). The system comprises a Hash pool, 16 levels are included, 219 two-dimensional feature vectors are held in each level, the feature vectors are input into an MLP network through spatial interpolation, scene features and SDF are extracted, finally, high-quality scene reconstruction is achieved through optimization of RGB images, depth and normal, and RGB input and monocular depth and normal observation are used as supervision during joint optimization. According to the system, the scene reconstruction quality is improved, a boundless scene can be effectively processed, the performance is improved in a viewpoint sparse environment, and higher robustness and accuracy are shown when a complex scene in an actual driving environment is processed.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving scene reconstruction, and in particular to an autonomous driving boundaryless scene implicit reconstruction method and system considering geometric information enhancement. Background Art

[0002] Neural radiation field is a novel 3D reconstruction technology. In autonomous driving, 3D reconstruction technology has become an indispensable key technology. Autonomous driving systems require accurate environmental perception capabilities, such as recognition and tracking of roads, static scenes, dynamic objects, as well as path planning and high-quality scene 3D modeling. 3D reconstruction technology is used to assist autonomous driving in achieving these tasks and improve the safety and reliability of autonomous driving. First, neural radiation field technology can reconstruct 2D images into 3D scenes, and then produce high-precision maps, achieve high-precision vehicle positioning and map matching, and promote the research and development of downstream tasks of autonomous driving; second, neural radiation field technology can synthesize complex autonomous driving scenes, and then enrich the training data of autonomous driving, helping the autonomous driving system to perform efficient data enhancement; third, neural radiation field technology can simulate harsh scenes such as extreme weather and serious traffic accidents, so as to restore real harsh scenes with simulated data and improve the safety of autonomous driving. In short, neural radiation field technology in 3D reconstruction has a wide range of applications in autonomous driving. Combining it with autonomous driving scenes will help promote the development and application of autonomous driving technology.

[0003] Due to the limited viewing angle of the autonomous driving scene, the quality of NeRF reconstruction will decline. How to synthesize high-quality views under limited viewing angles is a problem that researchers need to face; due to the large amount of data in the autonomous driving scene and the limited storage capacity of the NeRF model, how to reconstruct large-scale and large-scale scenes in limited model storage is a challenge that researchers need to face; due to the changes in lighting appearance and dynamic objects in the autonomous driving scene, which exceed the assumptions of the original NeRF model, how to deal with dynamic objects in the scene needs to be solved urgently; finally, it is necessary to consider how to improve the training and rendering speed of the model and speed up the rendering of the autonomous driving scene. High-quality reconstruction of autonomous driving scenes is a prerequisite for the large-scale commercialization of autonomous driving and other applications. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings existing in the above-mentioned prior art, and to provide a method and system for implicit reconstruction of autonomous driving borderless scenes considering geometric information enhancement, which uses a geometric-aware grid-based neural rendering system for autonomous driving. The system uses a perspective-deformed hash grid and a signed distance function (SDF) to reconstruct and render an accurate driving environment from sparse sensor data, thereby improving the quality of scene reconstruction, being able to effectively process borderless scenes, and improving performance in a viewpoint-sparse environment.

[0005] On the one hand, the present invention provides an implicit reconstruction method for an autonomous driving boundaryless scene considering geometric information enhancement, comprising the following steps:

[0006] S1: The boundless open scene is spatially divided by an octree structure to generate several local small scenes. A perspective deformation function is used on each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and a multi-resolution hash grid is used to store the encoded features. The hash code is used to index the local grid corresponding to the feature of any point in the space, and the vertices of the local grid are used to perform trilinear interpolation to obtain the features of the spatial point.

[0007] S2: Input the features of the spatial points into a neural implicit rendering network, output a signed distance function SDF field and a color field, and use differentiable geometric enhanced feature rendering to obtain predicted values ​​of scene color, scene depth, and scene normal vector;

[0008] S3: The scene color, scene depth and scene normal vector are extracted through the monocular depth model. By introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the true values ​​of the scene color, scene depth and scene normal vector;

[0009] S4: Calculate the loss function based on the predicted value of step S2 and the true value of step S3, and perform joint optimization based on the loss function to achieve scene reconstruction.

[0010] Wherein, in step S1, spatially dividing the borderless open scene by using the octree structure to generate a number of local small scenes further includes:

[0011] Initialize the root node size of the octree to be 32 times the bounding box including all input camera trajectories;

[0012] For each tree node of the octree, whether to further subdivide is determined according to the camera visibility and the distance from the node center to form a leaf node set.

[0013] Further, in step S1, the perspective deformation function is constructed by the principal component analysis PCA method, and the perspective deformation function is used on each local small scene to map the space at infinity to a finite distance, specifically including:

[0014] First, the point cloud data or mesh vertices of the three-dimensional model are taken as a data set. The three-dimensional coordinates of each point constitute the original feature vector. The main change direction of the data set is found through the principal component analysis (PCA) method, and a new two-dimensional coordinate system is defined based on this. The perspective deformation function is: F(x)=MW(x), where W(x) is the two-dimensional coordinate of projecting point x to all visible cameras, and M is the projection matrix constructed by the principal component analysis (PCA) method.

[0015] Furthermore, in step S1, storing the encoded features using a multi-resolution hash grid further includes:

[0016] The features corresponding to each leaf node are mapped to a hash pool through a hash function. The hash pool includes 16 levels, each level holds 219 two-dimensional feature vectors, and the feature vectors are obtained by the following spatial interpolation calculation:

[0017]

[0018] Among them, o and d represent the center of the camera and the direction of the light, respectively, j i is the perspective deformation function at x i The Jacobian matrix at , l is a hyperparameter that controls the sampling interval.

[0019] Furthermore, in step S2, the feature of the spatial point is input into a neural implicit rendering network, and outputting a signed distance function SDF field and a color field further includes:

[0020] The signed distance function SDF is learned through a multi-layer perceptron MLP network and used as an intermediary variable for density modeling. The signed distance function SDF is defined as the zero level set of the object surface. The signed distance function SDF is converted into volume density through the cumulative distribution function of the Laplace distribution. The density is expressed by the following formula:

[0021] σ(x)=αΨ β (-d Ω (x)),

[0022] Among them, α and β are learnable parameters, Ψ β represents the cumulative distribution function of the Laplace distribution, d Ω (x) is the signed distance function SDF field representation function of point x.

[0023] Preferably, in step S2, using differentiable geometric enhancement feature rendering to obtain predicted values ​​of scene color, scene depth, and scene normal vector further comprises:

[0024] The gradient of the signed distance function SDF is calculated by numerical differentiation method to obtain the surface normal. The initial step size is the size of the leaf node and is gradually reduced to capture local details. The gradient calculation formula of the signed distance function SDF is expressed as:

[0025]

[0026] Where ∈ is a perturbation x i The minimum vector of ;

[0027] Then, the predicted values ​​of scene color, scene depth and scene normal vector are finally calculated through volume rendering technology.

[0028] Furthermore, in step S3, by introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field, and the true values ​​of the scene color, scene depth, and scene normal vector are obtained, which further includes:

[0029] The relative depth value is generated by using the pre-trained monocular estimation model, the scale and deviation of each batch are learned by the least squares criterion, and the relative depth value and the actual depth value are aligned by the depth consistency loss function. The depth consistency loss function is expressed as:

[0030]

[0031] Ensure the consistency of the rendered normal and the predicted monocular normal in the same coordinate space, and optimize it through the normal consistency loss function, which is calculated by L1 norm loss and angle loss. The normal consistency loss function is expressed as follows:

[0032]

[0033] Where k and b are learnable parameters for aligning depth values, R is the set of rays in the training batch, and r is an element in the set R. represents the predicted value of the scene depth of variable r, is the true value of the scene depth of variable r, is the predicted value of the scene normal vector of variable r, is the true value of the scene normal vector of variable r.

[0034] Preferably, step S4 further comprises:

[0035] Set the scene color reconstruction loss function to connect the 3D environment and its corresponding 2D observation by minimizing the difference between the input image and the rendered image through color loss. The scene color reconstruction loss function is defined as:

[0036]

[0037] Where R is the set of rays in the training batch, r is the element in the set R, represents the predicted value of the scene color of variable r, is the true value of the scene color of variable r;

[0038] The regularized loss function is set, the Eikonal term is introduced to standardize the signed distance function SDF field, and the parallax loss is used to constrain the parallax and reduce floating artifacts. The regularized loss function is defined as:

[0039]

[0040] in, is the gradient of the signed distance function SDF at the point x.

[0041] Preferably, in step S4, performing joint optimization according to the loss function to achieve scene reconstruction further includes:

[0042] All loss functions are jointly optimized through the Adam optimizer until the predetermined reconstruction quality and accuracy are achieved. The final loss function is expressed as:

[0043] L=L rgb +λ depth L depth +λ normal L normal +λ eikonal L eikonal +λ disp Lx isp ,

[0044] Among them, λ depth is the depth consistency loss weight, λ eikonal is the normal consistency loss weight, λ eikonal is the regular loss weight, L disp is the disparity map loss, λ disp is the disparity map loss weight.

[0045] On the other hand, the present invention provides an automatic driving boundaryless scene implicit reconstruction system considering geometric information enhancement, comprising:

[0046] A spatial division characterization module is used to perform spatial division of a borderless open scene through an octree structure to generate a number of local small scenes, use a perspective deformation function on each local small scene to map the space at infinity to a finite distance, perform spatial encoding on the local small scene, use a multi-resolution hash grid to store the encoded features, use the hash code to index to the local grid corresponding to the feature of any point in the space, and use the vertices of the local grid to perform trilinear interpolation to obtain the features of the spatial point;

[0047] The spatial rendering module is used to input the features of the spatial points into the neural implicit rendering network, output the signed distance function SDF field and the color field, and use the differentiable geometric enhanced feature rendering to obtain the predicted values ​​of the scene color, scene depth, and scene normal vector.

[0048] The multi-consistent loss function scene reconstruction module is used to extract the scene color, scene depth and scene normal vector through a monocular depth model. By introducing the geometric prior generated by a pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the true value of the scene color, scene depth and scene normal vector. At the same time, the loss function is calculated based on the predicted value and the true value, and the scene reconstruction is achieved by joint optimization based on the loss function.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] (1) By introducing geometric enhancement features and perspective deformation hash grids, the present invention significantly improves the geometric details and global consistency of the reconstructed driving scene. The reconstruction quality on KITT I, Free-dataset and self-collected urban road datasets is better than the existing state-of-the-art methods.

[0051] (2) The present invention utilizes SDF fields and geometric prior enhancement features, so that the system can still maintain high accuracy and high stability when processing complex, sparse viewpoint open scenes. In particular, the reconstruction effect of the system is still excellent in low-texture areas and sparse viewpoint conditions, significantly reducing the mismatch and blur in the reconstruction;

[0052] (3) Through the spatial division and feature storage of perspective deformed hash grids, the present invention achieves efficient sampling and calculation in data processing and feature extraction, significantly reducing the computational complexity and time cost of the system. Under the same training rounds, the method of the present invention can obtain higher quality reconstruction results in a shorter time than the traditional multi-view stereo reconstruction method;

[0053] (4) By using a pre-trained monocular depth and normal estimation model, the system can extract geometric priors from relatively simple and low-cost image data, thereby improving the feasibility and economy of reconstruction; BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0055] In the attached picture:

[0056] Figure 1This is a flow chart of an implicit reconstruction method for autonomous driving without boundary scenes considering geometric information enhancement according to the present invention;

[0057] Figure 2 This is a schematic diagram of scene color, normal vector, and depth rendering effects of the present invention;

[0058] Figure 3 This is a structural block diagram of an implicit reconstruction system for autonomous driving without boundaries that takes into account geometric information enhancement according to the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0061] The present invention and system include a hash pool, which includes 16 levels, each level holds 219 two-dimensional feature vectors. The feature vectors are input to the MLP network through spatial interpolation to extract scene features and SDF. Finally, high-quality scene reconstruction is achieved by optimizing RGB images, depth and normals.

[0062] The specific implementation of the present invention is described below with reference to the accompanying drawings and embodiments.

[0063] Example 1

[0064] like Figure 1 As shown, this embodiment provides an implicit reconstruction method for an autonomous driving boundaryless scene considering geometric information enhancement, and the technical solution includes the following steps:

[0065] S1: The boundless open scene is spatially divided by an octree structure to generate several local small scenes. A perspective deformation function is used on each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and a multi-resolution hash grid is used to store the encoded features. The hash code is used to index the local grid corresponding to the feature of any point in the space, and the vertices of the local grid are used to perform trilinear interpolation to obtain the features of the spatial point.

[0066] S2: Input the features of the spatial points into a neural implicit rendering network, output a signed distance function SDF field and a color field, and use differentiable geometric enhanced feature rendering to obtain predicted values ​​of scene color, scene depth, and scene normal vector;

[0067] S3: The scene color, scene depth and scene normal vector are extracted through the monocular depth model. By introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the true values ​​of the scene color, scene depth and scene normal vector;

[0068] S4: Calculate the loss function based on the predicted value of step S2 and the true value of step S3, and perform joint optimization based on the loss function to achieve scene reconstruction.

[0069] In step S1, we use the perspective deformation function to spatially transform the sparse sensor data and use the hash grid to store the encoded features to ensure efficient data sampling and processing. In the spatial division, the octree structure is used to spatially divide the boundless open scene to generate several local small scenes, which further includes:

[0070] Initialize the root node size of the octree to be 32 times the bounding box including all input camera trajectories;

[0071] For each tree node of the octree, whether to further subdivide is determined according to the camera visibility and the distance from the node center to form a leaf node set.

[0072] In addition, in terms of perspective deformation, we construct the perspective deformation function through the principal component analysis (PCA) method to ensure the fidelity of information after dimension reduction. Specifically, the perspective deformation function is used on each local small scene to map the space at infinity to a finite distance, which specifically includes:

[0073] First, the point cloud data or mesh vertices of the 3D model are used as a data set. The 3D coordinates of each point constitute the original feature vector. The main change direction of the data set is found through the principal component analysis (PCA) method, and a new 2D coordinate system is defined based on this. The perspective deformation function is: F(x) = MW(x), where W(x) is the 2D coordinate of projecting point x to all visible cameras, and M is the projection matrix constructed by the principal component analysis (PCA) method. This deformation process can ensure uniform sampling in the deformation space and improve the accuracy and efficiency of sampling.

[0074] Furthermore, in step S1, storing the encoded features using a multi-resolution hash grid further includes:

[0075] The features corresponding to each leaf node are mapped to a hash pool through a hash function. The hash pool includes 16 levels, each level holds 219 two-dimensional feature vectors, and the feature vectors are obtained by the following spatial interpolation calculation:

[0076]

[0077] Among them, o and d represent the center of the camera and the direction of the light, respectively, j i is the perspective deformation function at x i The Jacobian matrix at , l is a hyperparameter that controls the sampling interval.

[0078] Furthermore, in step S2, the feature of the spatial point is input into a neural implicit rendering network, and outputting a signed distance function SDF field and a color field further includes:

[0079] The signed distance function SDF is learned through a multi-layer perceptron MLP network and used as an intermediary variable for density modeling. The signed distance function SDF is defined as the zero level set of the object surface. The signed distance function SDF is converted into volume density through the cumulative distribution function of the Laplace distribution. The density is expressed by the following formula:

[0080] σ(x)=αΨ β (-d Ω (x)),

[0081] Among them, α and β are learnable parameters, Ψ β represents the cumulative distribution function of the Laplace distribution, d Ω (x) is the signed distance function SDF field representation function of point x.

[0082] In addition, in step S2, the predicted values ​​of scene color, scene depth, and scene normal vector obtained by rendering using differentiable geometric enhancement features further include:

[0083] The gradient of the signed distance function SDF is calculated by numerical differentiation method to obtain the surface normal. The initial step size is the size of the leaf node and is gradually reduced to capture local details. The gradient calculation formula of the signed distance function SDF is expressed as:

[0084]

[0085] Where ∈ is a perturbation x i The minimum vector of ;

[0086] Then, the predicted values ​​of scene color, scene depth and scene normal vector are finally calculated through volume rendering technology.

[0087] Further, in step S3, by introducing the geometric prior generated by the pre-trained monocular estimation model, the geometric prior includes depth and normal estimation, and applying additional constraints to the signed distance function SDF field, the true values ​​of the scene color, scene depth, and scene normal vector are obtained, which further includes:

[0088] The relative depth value is generated by using the pre-trained monocular estimation model, the scale and deviation of each batch are learned by the least squares criterion, and the relative depth value and the actual depth value are aligned by the depth consistency loss function. The depth consistency loss function is expressed as:

[0089]

[0090] Ensure the consistency of the rendered normal and the predicted monocular normal in the same coordinate space, and optimize it through the normal consistency loss function, which is calculated by L1 norm loss and angle loss. The normal consistency loss function is expressed as follows:

[0091]

[0092] Where k and b are learnable parameters for aligning depth values, R is the set of rays in the training batch, and r is an element in the set R. represents the predicted value of the scene depth of variable r, is the true value of the scene depth of variable r, is the predicted value of the scene normal vector of variable r, is the true value of the scene normal vector of variable r.

[0093] In step S4, the RGB input and the monocular depth and normal observation are combined for joint optimization, further comprising:

[0094] Set the scene color reconstruction loss function to connect the 3D environment and its corresponding 2D observation by minimizing the difference between the input image and the rendered image through color loss. The scene color reconstruction loss function is defined as:

[0095]

[0096] Where R is the set of rays in the training batch, r is the element in the set R, represents the predicted value of the scene color of variable r, is the true value of the scene color of variable r;

[0097] The regularized loss function is set, the Eikonal term is introduced to standardize the signed distance function SDF field, and the parallax loss is used to constrain the parallax and reduce floating artifacts. The regularized loss function is defined as:

[0098]

[0099] in, is the gradient of the signed distance function SDF at the point x.

[0100] Preferably, in step S4, performing joint optimization according to the loss function to achieve scene reconstruction further includes:

[0101] All loss functions are jointly optimized through the Adam optimizer until the predetermined reconstruction quality and accuracy are achieved. The final loss function is expressed as:

[0102] L=L rgb +λ depth L depth +λ normal L normal +λ eikonal L eikonal +λ disp L disp ,

[0103] Among them, λ depth is the depth consistency loss weight, λ normal is the normal consistency loss weight, λ eikonal is the regular loss weight, L disp is the disparity map loss, λ disp is the disparity map loss weight.

[0104] Through the above specific implementation methods, the present invention can achieve efficient and accurate reconstruction of autonomous driving scenes, and provide reliable technical support for efficient training and testing of autonomous driving systems.

[0105] We experimentally verified and compared the performance of the present invention with existing methods on multiple datasets. The results show that in the "sky" and "stairs" scenes of the Free-dataset, the present invention achieves better performance in terms of PSNR, SSIM, LPIPS and other indicators. On the KITTI and self-collected FMAD datasets, the present invention shows higher robustness and accuracy when processing complex scenes in actual driving environments.

[0106] The specific experimental data are as follows:

[0107] In the "sky" scene of Free-dataset, the PSNR of this method reaches 25.93, the SSIM reaches 0.827, and the LPIPS is 0.336, all of which are better than the existing methods.

[0108] In the "highway" scene of the self-collected FMAD dataset, the PSNR of this method reaches 24.13, the SSIM reaches 0.825, and the LPIPS is 0.347, which are significantly better than other methods.

[0109] In summary, the present invention has been proved through theoretical analysis and experimental data that, compared with the prior art, it significantly improves the reconstruction quality and robustness of autonomous driving scenes, reduces costs, improves efficiency, and has important practical application value.

[0110] Example 2

[0111] This embodiment provides an automatic driving boundaryless scene implicit reconstruction system considering geometric information enhancement, including:

[0112] A spatial division characterization module is used to perform spatial division of a borderless open scene through an octree structure to generate a number of local small scenes, use a perspective deformation function on each local small scene to map the space at infinity to a finite distance, perform spatial encoding on the local small scene, use a multi-resolution hash grid to store the encoded features, use the hash code to index to the local grid corresponding to the feature of any point in the space, and use the vertices of the local grid to perform trilinear interpolation to obtain the features of the spatial point;

[0113] The spatial rendering module is used to input the features of the spatial points into the neural implicit rendering network, output the signed distance function SDF field and the color field, and use the differentiable geometric enhanced feature rendering to obtain the predicted values ​​of the scene color, scene depth, and scene normal vector.

[0114] The multi-consistent loss function scene reconstruction module is used to extract the scene color, scene depth and scene normal vector through a monocular depth model. By introducing the geometric prior generated by a pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the true value of the scene color, scene depth and scene normal vector. At the same time, the loss function is calculated based on the predicted value and the true value, and the scene reconstruction is achieved by joint optimization based on the loss function.

[0115] Finally, it should be noted that the above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0116] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An implicit reconstruction method for autonomous driving boundless scenes considering geometric information enhancement, characterized in that: The steps include: S1: The boundless open scene is spatially divided by an octree structure to generate several local small scenes. A perspective deformation function is used on each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and a multi-resolution hash grid is used to store the encoded features. The hash code is used to index the local grid corresponding to the feature of any point in the space, and the vertices of the local grid are used to perform trilinear interpolation to obtain the features of the spatial point. S2: Input the features of the spatial points into a neural implicit rendering network, output a signed distance function SDF field and a color field, and use differentiable geometric enhanced feature rendering to obtain predicted values ​​of scene color, scene depth, and scene normal vector; S3: The scene color, scene depth and scene normal vector are extracted through the monocular depth model. By introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the true values ​​of the scene color, scene depth and scene normal vector; S4: Calculate the loss function based on the predicted value of step S2 and the true value of step S3, and perform joint optimization based on the loss function to achieve scene reconstruction.

2. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 1, characterized in that: In step S1, spatially dividing the borderless open scene by using an octree structure to generate a number of local small scenes further includes: Initialize the root node size of the octree to be 32 times the bounding box including all input camera trajectories; For each tree node of the octree, whether to further subdivide is determined according to the camera visibility and the distance from the node center to form a leaf node set.

3. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 1, characterized in that: In step S1, the perspective deformation function is constructed by the principal component analysis PCA method, and the perspective deformation function is used on each local small scene to map the space at infinity to a finite distance, specifically including: First, the point cloud data or mesh vertices of the three-dimensional model are taken as a data set. The three-dimensional coordinates of each point constitute the original feature vector. The main change direction of the data set is found through the principal component analysis (PCA) method, and a new two-dimensional coordinate system is defined based on this. The perspective deformation function is: F(x)=MW(x), where W(x) is the two-dimensional coordinate of projecting point x to all visible cameras, and M is the projection matrix constructed by the principal component analysis (PCA) method.

4. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 2 or 3, characterized in that: In step S1, storing the encoded features using a multi-resolution hash grid further comprises: The features corresponding to each leaf node are mapped to a hash pool through a hash function. The hash pool includes 16 levels, each level holds 219 two-dimensional feature vectors, and the feature vectors are obtained by the following spatial interpolation calculation: Among them, o and d represent the center of the camera and the direction of the light, respectively, and ji is the perspective deformation function at x i The Jacobian matrix at , l is a hyperparameter that controls the sampling interval.

5. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 1, characterized in that: In step S2, the feature of the spatial point is input into a neural implicit rendering network, and the output of the signed distance function SDF field and the color field further includes: The signed distance function SDF is learned through a multi-layer perceptron MLP network and used as an intermediary variable for density modeling. The signed distance function SDF is defined as the zero level set of the object surface. The signed distance function SDF is converted into volume density through the cumulative distribution function of the Laplace distribution. The density is expressed by the following formula: σ(x)=αψβ(-d Ω (x)), Among them, α and β are learnable parameters, ψβ represents the cumulative distribution function of the Laplace distribution, and d Ω (x) is the signed distance function SDF field representation function of point x.

6. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 5, characterized in that: In step S2, the prediction values ​​of scene color, scene depth, and scene normal vector obtained by rendering using differentiable geometric enhancement features further include: The gradient of the signed distance function SDF is calculated by numerical differentiation method to obtain the surface normal. The initial step size is the size of the leaf node and is gradually reduced to capture local details. The gradient calculation formula of the signed distance function SDF is expressed as: Where ∈ is a perturbation x i The minimum vector of ; Then, the predicted values ​​of scene color, scene depth and scene normal vector are finally calculated through volume rendering technology.

7. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 1, characterized in that: In step S3, by introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field, and the true values ​​of the scene color, scene depth, and scene normal vector are obtained, which further includes: The relative depth value is generated by using the pre-trained monocular estimation model, the scale and deviation of each batch are learned by the least squares criterion, and the relative depth value and the actual depth value are aligned by the depth consistency loss function. The depth consistency loss function is expressed as: Ensure the consistency of the rendered normal and the predicted monocular normal in the same coordinate space, and optimize it through the normal consistency loss function, which is calculated by L1 norm loss and angle loss. The normal consistency loss function is expressed as follows: Where k and b are learnable parameters for aligning depth values, R is the set of rays in the training batch, and r is an element in the set R. represents the predicted value of the scene depth of variable r, is the true value of the scene depth of variable r, is the predicted value of the scene normal vector of variable r, is the true value of the scene normal vector of variable r.

8. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 6 or 7, characterized in that: Step S4 further comprises: Set the scene color reconstruction loss function to connect the 3D environment and its corresponding 2D observation by minimizing the difference between the input image and the rendered image through color loss. The scene color reconstruction loss function is defined as: Where R is the set of rays in the training batch, r is the element in the set R, represents the predicted value of the scene color of variable r, is the true value of the scene color of variable r; The regularized loss function is set, the Eikonal term is introduced to standardize the signed distance function SDF field, and the parallax loss is used to constrain the parallax and reduce floating artifacts. The regularized loss function is defined as: in, is the gradient of the signed distance function SDF at the point x.

9. The method for implicitly reconstructing a boundaryless scene for autonomous driving considering geometric information enhancement according to claim 8, characterized in that: In step S4, performing joint optimization according to the loss function to achieve scene reconstruction further includes: All loss functions are jointly optimized through the Adam optimizer until the predetermined reconstruction quality and accuracy are achieved. The final loss function is expressed as: L=L rgb +λ depth L depth +λ normal L normal +λ eikonal L eikonal +λ disp L disp , Among them, λ depth is the depth consistency loss weight, λ normal is the normal consistency loss weight, λ eikonal is the regular loss weight, L disp is the disparity map loss, λ disp is the disparity map loss weight.

10. An implicit reconstruction system for autonomous driving without boundaries based on geometric information enhancement, characterized in that: include: A spatial division characterization module is used to perform spatial division of a borderless open scene through an octree structure to generate a number of local small scenes, and use a perspective deformation function on each local small scene to map the space at infinity to a finite distance, perform spatial encoding on the local small scene, use a multi-resolution hash grid to store the encoded features, and use the hash code to index to the local grid corresponding to the feature of any point in the space, and use the vertices of the local grid to perform trilinear interpolation to obtain the features of the spatial point; The spatial rendering module is used to input the features of the spatial points into the neural implicit rendering network, output the signed distance function SDF field and the color field, and use the differentiable geometric enhanced feature rendering to obtain the predicted values ​​of the scene color, scene depth, and scene normal vector. The multi-consistent loss function scene reconstruction module is used to extract the scene color, scene depth and scene normal vector through a monocular depth model. By introducing the geometric prior generated by a pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the true value of the scene color, scene depth and scene normal vector. At the same time, the loss function is calculated based on the predicted value and the true value, and the scene reconstruction is achieved by joint optimization based on the loss function.

Images