Geometric information enhanced implicit reconstruction method for autonomous driving unbounded scene

Abstract

The present application relates to the field of automatic driving scene reconstruction, and discloses a kind of automatic driving borderless scene implicit reconstruction method and system considering geometric information enhancement, adopt a kind of automatic driving geometric perception grid-based neural rendering system, the system utilizes perspective deformation hash grid and signed distance function SDF from sparse sensor data reconstruction and rendering accurate driving environment.System includes a hash pool, contains 16 levels, each level holds 219 2-dimensional feature vectors, feature vectors are input to MLP network by spatial interpolation, extract scene features and SDF, finally, through the optimization of RGB image, depth and normal, realize high-quality scene reconstruction, when joint optimization, use RGB input and monocular depth, normal observation as supervision.This system improves the quality of scene reconstruction, can effectively handle borderless scene, and improve performance in sparse viewpoint environment, when dealing with complex scenes in actual driving environment, show higher robustness and accuracy.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving scene reconstruction, and in particular to an implicit reconstruction method and system for autonomous driving borderless scenes that considers geometric information enhancement. Background Technology

[0002] Neural radiation field (NRF) is a novel 3D reconstruction technology. In autonomous driving, 3D reconstruction has become an indispensable key technology. Autonomous driving systems require accurate environmental perception capabilities, such as the identification and tracking of road surfaces, static scenes, and dynamic objects, as well as path planning and high-quality 3D scene modeling. Utilizing 3D reconstruction technology to assist autonomous driving in achieving these tasks improves the safety and reliability of autonomous driving. First, NRF technology can reconstruct 3D scenes from 2D images, thereby creating high-precision maps, achieving high-precision vehicle positioning and map matching, and promoting downstream task research and development in autonomous driving. Second, NRF technology can synthesize complex autonomous driving scenes, thereby enriching the training data for autonomous driving and helping autonomous driving systems perform efficient data augmentation. Third, NRF technology can simulate harsh scenarios such as extreme weather and severe traffic accidents to simulate data and recreate realistic harsh scenarios, improving the safety of autonomous driving. In conclusion, NRF technology in 3D reconstruction has wide applications in autonomous driving, and combining it with autonomous driving scenarios will help promote the development and application of autonomous driving technology.

[0003] Due to the limited field of view in autonomous driving scenarios, the quality of NeRF reconstruction will decrease. How to synthesize high-quality views under limited field of view is a problem researchers need to address. Given the massive data volume of autonomous driving scenarios and the limited storage capacity of NeRF models, how to reconstruct large-scale scenes within limited model storage is a challenge researchers need to overcome. Furthermore, autonomous driving scenarios involve changes in lighting and appearance, as well as dynamic changes in objects, exceeding the assumptions of the original NeRF model; how to handle dynamic objects in the scene urgently needs to be addressed. Finally, it is necessary to consider how to improve model training and rendering speed to accelerate the rendering of autonomous driving scenes. High-quality autonomous driving scene reconstruction is a prerequisite for the large-scale commercialization and application of autonomous driving. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the existing technology and provide an implicit reconstruction method and system for borderless autonomous driving scenes that considers geometric information enhancement. It utilizes a geometric perception mesh-based neural rendering system for autonomous driving. This system uses perspective deformation hash mesh and signed distance function (SDF) to reconstruct and render an accurate driving environment from sparse sensor data, thereby improving the quality of scene reconstruction, effectively handling borderless scenes, and enhancing performance in viewpoint-sparse environments.

[0005] On the one hand, the present invention provides an implicit reconstruction method for borderless scenes in autonomous driving that considers geometric information enhancement, comprising the following steps:

[0006] S1: The boundless open scene is spatially divided into several local small scenes through an octree structure. A perspective deformation function is applied to each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and the encoded features are stored using a multi-resolution hash grid. The local grid corresponding to any point feature in the space is indexed by the hash encoding. The features of the spatial point are obtained by trilinear interpolation using the vertices of the local grid.

[0007] S2: Input the features of the spatial points into the neural implicit rendering network, output a signed distance function SDF field and a color field, and use differentiable geometric enhancement features to render the predicted values ​​of scene color, scene depth and scene normal vector.

[0008] S3: Extract scene color, scene depth, and scene normal vector through a monocular depth model. By introducing geometric priors generated by a pre-trained monocular estimation model, additional constraints are applied to the signed distance function SDF field to obtain the true values ​​of scene color, scene depth, and scene normal vector.

[0009] S4: Calculate the loss function based on the predicted value in step S2 and the true value in step S3, and perform joint optimization based on the loss function to achieve scene reconstruction.

[0010] In step S1, the process of spatially partitioning the unbounded open scene using an octree structure to generate several local smaller scenes further includes:

[0011] The root node size of the octree is initialized to 32 times the size of the bounding box that includes all input camera trajectories;

[0012] For each node of the octree, the decision to further subdivide it based on the camera's visibility and the distance from the node's center is made to form a set of leaf nodes.

[0013] Further, in step S1, the perspective distortion function is constructed using principal component analysis (PCA). Specifically, applying the perspective distortion function to each local small scene to map the space at infinity to a finite distance includes:

[0014] First, the point cloud data or mesh vertices of the 3D model are used as the dataset. The 3D coordinates of each point constitute the original feature vector. The main direction of change of the dataset is found by the principal component analysis (PCA) method, and a new 2D coordinate system is defined accordingly. The perspective distortion function is: F(x) = MW(x), where W(x) is the 2D coordinate of the point x projected onto all visible cameras, and M is the projection matrix constructed by the principal component analysis (PCA) method.

[0015] Furthermore, in step S1, storing the encoded features using a multi-resolution hash grid further includes:

[0016] Each leaf node's corresponding feature is mapped to a hash pool via a hash function. The hash pool comprises 16 levels, each holding 219 two-dimensional feature vectors. The feature vectors are calculated using the following spatial interpolation:

[0017]

[0018] Where o and d represent the center of the camera and the direction of the light rays, respectively, j i For the perspective distortion function in x i The Jacobian matrix at point l is the hyperparameter controlling the sampling interval.

[0019] Further, in step S2, the features of the spatial points are input into the neural implicit rendering network, and the output of the signed distance function SDF field and color field further includes:

[0020] A signed distance function SDF is learned through a multilayer perceptron (MLP) network and used as an intermediary variable for density modeling. The signed distance function SDF is defined as the zero-level set of the object surface. The signed distance function SDF is transformed into volume density through the cumulative distribution function of the Laplace distribution, and the density is expressed by the following formula:

[0021] σ(x)=αΨ β (-d Ω (x)),

[0022] Where α and β are learnable parameters, Ψ β Let d represent the cumulative distribution function of the Laplace distribution. Ω (x) is the signed distance function of point x, represented by the SDF field function.

[0023] Preferably, in step S2, the process of obtaining predicted values ​​for scene color, scene depth, and scene normal vectors using differentiable geometric enhancement features further includes:

[0024] The gradient of the signed distance function SDF is calculated using numerical differentiation to obtain the surface normal. The initial step size is the size of the leaf node, which is gradually reduced to capture local details. The formula for calculating the gradient of the signed distance function SDF is expressed as follows:

[0025]

[0026] Where ∈ is a perturbation for x i The smallest vector;

[0027] Then, volume rendering technology is used to calculate the predicted values ​​of scene color, scene depth, and scene normal vectors.

[0028] Furthermore, in step S3, by introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the ground truth values ​​of scene color, scene depth, and scene normal vector, which further includes:

[0029] Relative depth values ​​are generated using a pre-trained monocular estimation model. The scale and bias of each batch are learned using the least squares criterion, and optimized using a depth consistency loss function to align the relative depth values ​​with the actual depth values. The depth consistency loss function is expressed as:

[0030]

[0031] To ensure consistency between the rendered normal and the predicted monocular normal in the same coordinate space, optimization is performed using a normal consistency loss function. This loss function is derived by calculating L1 norm loss and angle loss, and is expressed as follows:

[0032]

[0033] Where k and b are learnable parameters for the alignment depth value, R is the set of rays in the training batch, and r is an element in set R. This represents the predicted value of scene depth for variable r. Let r be the truth value of the scene depth. Let r be the predicted value of the scene normal vector for variable r. Let r be the truth value of the scene normal vector for variable r.

[0034] Preferably, step S4 further includes:

[0035] A scene color reconstruction loss function is set up to minimize the difference between the input image and the rendered image by color loss, connecting the 3D environment with its corresponding 2D observation. The scene color reconstruction loss function is defined as follows:

[0036]

[0037] Where R is the set of rays in the training batch, and r is an element in set R. This represents the predicted value of the scene color for variable r. Let r be the truth value of the scene color;

[0038] A regularized loss function is set up, introducing an Eikonal term to normalize the signed distance function SDF field, and disparity loss constrains disparity to reduce floating artifacts. The regularized loss function is defined as follows:

[0039]

[0040] in, Let x be the gradient of the signed distance function SDF at the point x.

[0041] More preferably, in step S4, the joint optimization based on the loss function to achieve scene reconstruction further includes:

[0042] All loss functions are jointly optimized by the Adam optimizer until the predetermined reconstruction quality and accuracy are achieved. The final loss function is expressed as follows:

[0043] L = L rgb +λ depth L depth +λ normal L normal +λ eikonal L eikonal +λ disp Lx isp ,

[0044] Where, λ depth λ is the weight of the deep consistency loss. eikonal λ is the weight for the normal consistency loss. eikonal For the regularization loss weight, L disp For disparity map loss, λ disp The loss weights are applied to the disparity map.

[0045] On the other hand, the present invention provides an implicit reconstruction system for boundless scenes in autonomous driving with geometric information enhancement, comprising:

[0046] The spatial partitioning and representation module is used to divide the boundless open scene into several local small scenes using an octree structure. A perspective deformation function is applied to each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and the encoded features are stored using a multi-resolution hash grid. The hash encoding indexes the local grid corresponding to any point feature in the space. The features of the spatial point are obtained by trilinear interpolation using the vertices of the local grid.

[0047] The spatial rendering module is used to input the features of the spatial points into the neural implicit rendering network, output a signed distance function (SDF) field and a color field, and use differentiable geometric enhancement features to render predicted values ​​of scene color, scene depth, and scene normal vector.

[0048] The multi-consistent loss function scene reconstruction module is used to extract scene color, scene depth, and scene normal vector through a monocular depth model. By introducing geometric priors generated by a pre-trained monocular estimation model, additional constraints are applied to the signed distance function SDF field to obtain the ground truth values ​​of scene color, scene depth, and scene normal vector. At the same time, the loss function is calculated based on the predicted values ​​and the ground truth values, and joint optimization is performed based on the loss function to achieve scene reconstruction.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] (1) By introducing geometric enhancement features and perspective deformation hash grid, the reconstruction of driving scene is significantly improved in terms of geometric details and global consistency. The reconstruction quality on KITT I, Free-dataset and self-collected urban road datasets is better than the state-of-the-art methods.

[0051] (2) This invention utilizes SDF fields and geometric prior enhancement features to enable the system to maintain high accuracy and high stability when processing complex, sparse viewpoint open scenes. Especially in low-texture areas and sparse viewpoint conditions, the reconstruction effect of the system is still excellent, significantly reducing mismatch and blurring in reconstruction.

[0052] (3) Through the spatial partitioning and feature storage of the perspective deformation hash grid, this invention achieves efficient sampling and calculation in data processing and feature extraction, which significantly reduces the computational complexity and time cost of the system. Under the same training rounds, the method of this invention can obtain higher quality reconstruction results in a shorter time compared with the traditional multi-view stereo reconstruction method.

[0053] (4) By using a pre-trained monocular depth and normal estimation model, the system is able to extract geometric priors from relatively simple and low-cost image data, which improves the feasibility and economy of reconstruction. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0055] In the attached diagram:

[0056] Figure 1This is a flowchart of an implicit reconstruction method for borderless scenes in autonomous driving that considers geometric information enhancement, according to the present invention.

[0057] Figure 2 This is a schematic diagram illustrating the scene color, normal vector, and depth rendering effect of the present invention;

[0058] Figure 3 This is a block diagram of an implicit reconstruction system for borderless scenes in autonomous driving that considers geometric information enhancement, according to the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0061] The invention and system include a hash pool with 16 levels, each holding 219 two-dimensional feature vectors. These feature vectors are spatially interpolated and fed into an MLP network to extract scene features and SDF (Single Element Function). Finally, high-quality scene reconstruction is achieved through optimization of the RGB image, depth, and normals.

[0062] The specific embodiments of the present invention will be described below with reference to the accompanying drawings and examples.

[0063] Example 1

[0064] like Figure 1 As shown in the figure, this embodiment provides an implicit reconstruction method for borderless scenes in autonomous driving that considers geometric information enhancement. The technical solution includes the following steps:

[0065] S1: The boundless open scene is spatially divided into several local small scenes through an octree structure. A perspective deformation function is applied to each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and the encoded features are stored using a multi-resolution hash grid. The local grid corresponding to any point feature in the space is indexed by the hash encoding. The features of the spatial point are obtained by trilinear interpolation using the vertices of the local grid.

[0066] S2: Input the features of the spatial points into the neural implicit rendering network, output a signed distance function SDF field and a color field, and use differentiable geometric enhancement features to render the predicted values ​​of scene color, scene depth and scene normal vector.

[0067] S3: Extract scene color, scene depth, and scene normal vector through a monocular depth model. By introducing geometric priors generated by a pre-trained monocular estimation model, additional constraints are applied to the signed distance function SDF field to obtain the true values ​​of scene color, scene depth, and scene normal vector.

[0068] S4: Calculate the loss function based on the predicted value in step S2 and the true value in step S3, and perform joint optimization based on the loss function to achieve scene reconstruction.

[0069] In step S1, we spatially transform the sparse sensor data using a perspective deformation function and utilize a hash grid to store encoded features, ensuring efficient data sampling and processing. Specifically, during spatial partitioning, the unbounded open scene is spatially divided into several local smaller scenes using an octree structure, further including:

[0070] The root node size of the octree is initialized to 32 times the size of the bounding box that includes all input camera trajectories;

[0071] For each node of the octree, the decision to further subdivide it based on the camera's visibility and the distance from the node's center is made to form a set of leaf nodes.

[0072] Furthermore, regarding perspective distortion, we construct the perspective distortion function using Principal Component Analysis (PCA) to ensure information fidelity after dimensionality reduction. Specifically, the perspective distortion function is applied to each local small scene to map the space at infinity to a finite distance, including:

[0073] First, the point cloud data or mesh vertices of the 3D model are used as the dataset. The 3D coordinates of each point constitute the original feature vector. Principal Component Analysis (PCA) is used to find the main direction of change in the dataset, and a new 2D coordinate system is defined accordingly. The perspective deformation function is: F(x) = MW(x), where W(x) is the 2D coordinate of point x projected onto all visible cameras, and M is the projection matrix constructed using PCA. This deformation process ensures uniform sampling in the deformation space, improving the accuracy and efficiency of sampling.

[0074] Furthermore, in step S1, storing the encoded features using a multi-resolution hash grid further includes:

[0075] Each leaf node's corresponding feature is mapped to a hash pool via a hash function. The hash pool comprises 16 levels, each holding 219 two-dimensional feature vectors. The feature vectors are calculated using the following spatial interpolation:

[0076]

[0077] Where o and d represent the center of the camera and the direction of the light rays, respectively, j i For the perspective distortion function in x i The Jacobian matrix at point l is the hyperparameter controlling the sampling interval.

[0078] Further, in step S2, the features of the spatial points are input into the neural implicit rendering network, and the output of the signed distance function SDF field and color field further includes:

[0079] A signed distance function SDF is learned through a multilayer perceptron (MLP) network and used as an intermediary variable for density modeling. The signed distance function SDF is defined as the zero-level set of the object surface. The signed distance function SDF is transformed into volume density through the cumulative distribution function of the Laplace distribution, and the density is expressed by the following formula:

[0080] σ(x)=αΨ β (-d Ω (x)),

[0081] Where α and β are learnable parameters, Ψ β Let d represent the cumulative distribution function of the Laplace distribution. Ω (x) is the signed distance function of point x, represented by the SDF field function.

[0082] Furthermore, in step S2, the predicted values ​​of scene color, scene depth, and scene normal vector obtained by rendering differentiable geometric enhancement features further include:

[0083] The gradient of the signed distance function SDF is calculated using numerical differentiation to obtain the surface normal. The initial step size is the size of the leaf node, which is gradually reduced to capture local details. The formula for calculating the gradient of the signed distance function SDF is expressed as follows:

[0084]

[0085] Where ∈ is a perturbation for x i The smallest vector;

[0086] Then, volume rendering technology is used to calculate the predicted values ​​of scene color, scene depth, and scene normal vectors.

[0087] Further, in step S3, by introducing the geometric prior generated by the pre-trained monocular estimation model, which includes depth and normal estimates, additional constraints are imposed on the signed distance function SDF field to obtain the ground truth values ​​of scene color, scene depth, and scene normal vectors. This further includes:

[0088] Relative depth values ​​are generated using a pre-trained monocular estimation model. The scale and bias of each batch are learned using the least squares criterion, and optimized using a depth consistency loss function to align the relative depth values ​​with the actual depth values. The depth consistency loss function is expressed as:

[0089]

[0090] To ensure consistency between the rendered normal and the predicted monocular normal in the same coordinate space, optimization is performed using a normal consistency loss function. This loss function is derived by calculating L1 norm loss and angle loss, and is expressed as follows:

[0091]

[0092] Where k and b are learnable parameters for the alignment depth value, R is the set of rays in the training batch, and r is an element in set R. This represents the predicted value of scene depth for variable r. Let r be the truth value of the scene depth. Let r be the predicted value of the scene normal vector for variable r. Let r be the truth value of the scene normal vector for variable r.

[0093] In step S4, joint optimization is performed by combining RGB input with monocular depth and normal observation, which further includes:

[0094] A scene color reconstruction loss function is set up to minimize the difference between the input image and the rendered image by color loss, connecting the 3D environment with its corresponding 2D observation. The scene color reconstruction loss function is defined as follows:

[0095]

[0096] Where R is the set of rays in the training batch, and r is an element in set R. This represents the predicted value of the scene color for variable r. Let r be the truth value of the scene color;

[0097] A regularized loss function is set up, introducing an Eikonal term to normalize the signed distance function SDF field, and disparity loss constrains disparity to reduce floating artifacts. The regularized loss function is defined as follows:

[0098]

[0099] in, Let x be the gradient of the signed distance function SDF at the point x.

[0100] Preferably, in step S4, performing joint optimization based on the loss function to achieve scene reconstruction further includes:

[0101] All loss functions are jointly optimized by the Adam optimizer until the predetermined reconstruction quality and accuracy are achieved. The final loss function is expressed as follows:

[0102] L = L rgb +λ depth L depth +λ normal L normal +λ eikonal L eikonal +λ disp L disp ,

[0103] Where, λ depth λ is the weight of the deep consistency loss. normal λ is the weight for the normal consistency loss. eikonal For the regularization loss weight, L disp For disparity map loss, λ disp The loss weights are applied to the disparity map.

[0104] Through the above specific implementation methods, the present invention can achieve efficient and accurate autonomous driving scene reconstruction, providing reliable technical support for the efficient training and testing of autonomous driving systems.

[0105] We experimentally verified and compared the performance of our invention with existing methods on multiple datasets. The results show that in the "sky" and "stairs" scenarios of the free-dataset, our invention achieves better performance in metrics such as PSNR, SSIM, and LPIPS. On the KITTI and self-collected FMAD datasets, our invention demonstrates higher robustness and accuracy in handling complex scenarios in real-world driving environments.

[0106] The specific experimental data are as follows:

[0107] In the "sky" scenario of the free-dataset, our method achieves a PSNR of 25.93, an SSIM of 0.827, and an LPIPS of 0.336, all of which are superior to existing methods.

[0108] In the "highway" scenario of the self-collected FMAD dataset, our method achieved a PSNR of 24.13, an SSIM of 0.825, and an LPIPS of 0.347, significantly outperforming other methods.

[0109] In summary, through theoretical analysis and experimental data, this invention demonstrates that, compared with existing technologies, it significantly improves the reconstruction quality and robustness of autonomous driving scenarios, reduces costs, and increases efficiency, thus possessing significant practical application value.

[0110] Example 2

[0111] This embodiment provides an implicit reconstruction system for boundless scenes in autonomous driving that considers geometric information enhancement, including:

[0112] The spatial partitioning and representation module is used to divide the boundless open scene into several local small scenes using an octree structure. A perspective deformation function is applied to each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and the encoded features are stored using a multi-resolution hash grid. The hash encoding indexes the local grid corresponding to any point feature in the space. The features of the spatial point are obtained by trilinear interpolation using the vertices of the local grid.

[0113] The spatial rendering module is used to input the features of the spatial points into the neural implicit rendering network, output a signed distance function (SDF) field and a color field, and use differentiable geometric enhancement features to render predicted values ​​of scene color, scene depth, and scene normal vector.

[0114] The multi-consistent loss function scene reconstruction module is used to extract scene color, scene depth, and scene normal vector through a monocular depth model. By introducing geometric priors generated by a pre-trained monocular estimation model, additional constraints are applied to the signed distance function SDF field to obtain the ground truth values ​​of scene color, scene depth, and scene normal vector. At the same time, the loss function is calculated based on the predicted values ​​and the ground truth values, and joint optimization is performed based on the loss function to achieve scene reconstruction.

[0115] Finally, it should be noted that the above description is only a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be pointed out that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An implicit reconstruction method for boundless scenes in autonomous driving considering geometric information enhancement, characterized in that, Includes the following steps: S1: The boundless open scene is spatially divided into several local small scenes through an octree structure. A perspective deformation function is applied to each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and the encoded features are stored using a multi-resolution hash grid. The local grid corresponding to any point feature in the space is indexed by the hash encoding. The features of the spatial point are obtained by trilinear interpolation using the vertices of the local grid. S2: Input the features of the spatial points into the neural implicit rendering network, output a signed distance function SDF field and a color field, and use differentiable geometric enhancement features to render the predicted values ​​of scene color, scene depth and scene normal vector. S3: Extract scene color, scene depth, and scene normal vector through a monocular depth model. By introducing geometric priors generated by a pre-trained monocular estimation model, additional constraints are applied to the signed distance function SDF field to obtain the true values ​​of scene color, scene depth, and scene normal vector. S4: Calculate the loss function based on the predicted value in step S2 and the true value in step S3, and perform joint optimization based on the loss function to achieve scene reconstruction.

2. The implicit reconstruction method for borderless scenes in autonomous driving considering geometric information enhancement according to claim 1, characterized in that, In step S1, the process of spatially partitioning the unbounded open scene using an octree structure to generate several local smaller scenes further includes: The root node size of the octree is initialized to 32 times the size of the bounding box that includes all input camera trajectories; For each node of the octree, the decision to further subdivide it based on the camera's visibility and the distance from the node's center is made to form a set of leaf nodes.

3. The implicit reconstruction method for borderless scenes in autonomous driving considering geometric information enhancement according to claim 1, characterized in that, In step S1, the perspective distortion function is constructed using principal component analysis (PCA). Specifically, applying the perspective distortion function to each local small scene to map the space at infinity to a finite distance includes: First, the point cloud data or mesh vertices of the 3D model are used as the dataset. The 3D coordinates of each point constitute the original feature vector. The main direction of change of the dataset is found by the principal component analysis (PCA) method, and a new 2D coordinate system is defined accordingly. The perspective distortion function is: F(x) = MW(x), where W(x) is the 2D coordinate of the point x projected onto all visible cameras, and M is the projection matrix constructed by the principal component analysis (PCA) method.

4. The implicit reconstruction method for borderless scenes of autonomous driving considering geometric information enhancement according to claim 2 or 3, characterized in that, In step S1, storing the encoded features using a multi-resolution hash grid further includes: The feature corresponding to each leaf node is mapped to a hash pool through a hash function. The hash pool includes 16 levels, and each level holds 219 two-dimensional feature vectors. The feature vectors are calculated through the following spatial interpolation: Where o and d represent the center of the camera and the direction of the light rays, respectively, and ji is the perspective distortion function in x i The Jacobian matrix at point l is the hyperparameter controlling the sampling interval.

5. The implicit reconstruction method for borderless scenes of autonomous driving considering geometric information enhancement according to claim 1, characterized in that, In step S2, the features of the spatial points are input into the neural implicit rendering network, and the output of the signed distance function SDF field and color field further includes: A signed distance function SDF is learned through a multilayer perceptron (MLP) network and used as an intermediary variable for density modeling. The signed distance function SDF is defined as the zero-level set of the object surface. The signed distance function SDF is transformed into volume density through the cumulative distribution function of the Laplace distribution, and the density is expressed by the following formula: σ(x)=αψβ(-d Ω (x)), Where α and β are learnable parameters, ψβ represents the cumulative distribution function of the Laplace distribution, and d Ω (x) is the signed distance function of point x, represented by the SDF field function.

6. The implicit reconstruction method for borderless scenes of autonomous driving considering geometric information enhancement according to claim 5, characterized in that, In step S2, the predicted values ​​of scene color, scene depth, and scene normal vector obtained by rendering differentiable geometric enhancement features further include: The gradient of the signed distance function SDF is calculated using numerical differentiation to obtain the surface normal. The initial step size is the size of the leaf node, which is gradually reduced to capture local details. The formula for calculating the gradient of the signed distance function SDF is expressed as follows: Where ∈ is a perturbation for x i The smallest vector; Then, volume rendering technology is used to calculate the predicted values ​​of scene color, scene depth, and scene normal vectors.

7. The implicit reconstruction method for borderless scenes in autonomous driving considering geometric information enhancement according to claim 1, characterized in that, In step S3, by introducing the geometric prior generated by the pre-trained monocular estimation model, additional constraints are imposed on the signed distance function SDF field to obtain the ground truth values ​​of scene color, scene depth, and scene normal vector. This further includes: Relative depth values ​​are generated using a pre-trained monocular estimation model. The scale and bias of each batch are learned using the least squares criterion, and optimized using a depth consistency loss function to align the relative depth values ​​with the actual depth values. The depth consistency loss function is expressed as: To ensure consistency between the rendered normal and the predicted monocular normal in the same coordinate space, optimization is performed using a normal consistency loss function. This loss function is derived by calculating L1 norm loss and angle loss, and is expressed as follows: Where k and b are learnable parameters for the alignment depth value, R is the set of rays in the training batch, and r is an element in set R. This represents the predicted value of scene depth for variable r. Let r be the truth value of the scene depth. Let r be the predicted value of the scene normal vector for variable r. Let r be the truth value of the scene normal vector.

8. The implicit reconstruction method for borderless scenes of autonomous driving considering geometric information enhancement according to claim 6 or 7, characterized in that, Step S4 further includes: A scene color reconstruction loss function is set up to minimize the difference between the input image and the rendered image by color loss, connecting the 3D environment with its corresponding 2D observation. The scene color reconstruction loss function is defined as follows: Where R is the set of rays in the training batch, and r is an element in set R. This represents the predicted value of the scene color for variable r. Let r be the truth value of the scene color. A regularized loss function is set up, introducing an Eikonal term to normalize the signed distance function SDF field, and disparity loss constrains disparity to reduce floating artifacts. The regularized loss function is defined as follows: in, Let x be the gradient of the signed distance function SDF at the point x.

9. The implicit reconstruction method for borderless scenes of autonomous driving considering geometric information enhancement according to claim 8, characterized in that, In step S4, the joint optimization based on the loss function to achieve scene reconstruction further includes: All loss functions are jointly optimized by the Adam optimizer until the predetermined reconstruction quality and accuracy are achieved. The final loss function is expressed as follows: L=L rgb +λ depth L depth +λ normal L normal +λ eikonal L eikonal +λ disp L disp , Where, λ depth λ is the weight for the deep consistency loss. normal λ is the weight for the normal consistency loss. eikonal For the regularization loss weight, L disp For disparity map loss, λ disp The loss weights are applied to the disparity map.

10. An implicit reconstruction system for boundless scenes in autonomous driving, considering geometric information enhancement, characterized in that: include: The spatial partitioning and representation module is used to divide the boundless open scene into several local small scenes using an octree structure. A perspective deformation function is applied to each local small scene to map the space at infinity to a finite distance. The local small scene is spatially encoded, and the encoded features are stored using a multi-resolution hash grid. The hash encoding indexes the local grid corresponding to any point feature in the space. The features of the spatial point are obtained by trilinear interpolation using the vertices of the local grid. The spatial rendering module is used to input the features of the spatial points into the neural implicit rendering network, output a signed distance function (SDF) field and a color field, and use differentiable geometric enhancement features to render predicted values ​​of scene color, scene depth, and scene normal vector. The multi-consistent loss function scene reconstruction module is used to extract scene color, scene depth, and scene normal vector through a monocular depth model. By introducing geometric priors generated by a pre-trained monocular estimation model, additional constraints are applied to the signed distance function SDF field to obtain the ground truth values ​​of scene color, scene depth, and scene normal vector. At the same time, the loss function is calculated based on the predicted values ​​and the ground truth values, and joint optimization is performed based on the loss function to achieve scene reconstruction.

Images