Sparse view angle three-dimensional reconstruction method and system based on voxel grid constraint

Abstract

The invention discloses a sparse visual angle three-dimensional reconstruction method and system based on voxel grid constraint, and belongs to the technical field of visual three-dimensional reconstruction. Constructing a three-dimensional voxel grid of a self-adaptive scene scale based on point cloud distribution, and dividing the point cloud to the corresponding voxel grid; fusing the geometric features of the fast point feature histogram and the confidence score in the grid, generating a geometric confidence comprehensive measure, and screening key points to initialize Gaussian primitives; designing a voxel grid constrained gradient clipping strategy, limiting Gaussian primitive error diffusion through a distance attenuation coefficient, and adaptively optimizing grid distribution in combination with a dynamic grid deletion and addition mechanism; and finally, carrying out iterative training by using a 3D Gaussian splash radiation field loss function to realize high-fidelity static scene reconstruction under a sparse view angle. According to the method, scene geometric priori is introduced, an optimization strategy based on voxel grid constraint is designed to effectively control excessive diffusion or drift of Gaussian primitives, generation of artifacts is reduced, and meanwhile, the situation that robustness is reduced due to the influence of priori quality is avoided.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of visual three-dimensional reconstruction, and particularly relates to a sparse view three-dimensional reconstruction method and system based on voxel grid constraint. BACKGROUND

[0002] Three-dimensional reconstruction is an important part of computer vision, and has important application value in the fields of robot perception and virtual reality technology. In recent years, three-dimensional reconstruction methods based on radiance field have developed rapidly, and three-dimensional Gaussian splatting based reconstruction methods have shown great advantages in high-fidelity reconstruction and fast rendering. However, similar to traditional methods, 3D Gaussian splatting relies on a large number of input images to obtain multi-view information, while in actual applications, only a limited number of cameras or acquisition devices can be used to capture and reconstruct the scene. The performance of 3D Gaussian splatting will be severely degraded with the decrease of input views, and in the sparse view area, the reconstruction will be wrong due to the lack of constraints, and floating objects or artifacts will appear.

[0003] Existing research mainly solves the sparsity problem of 3D Gaussian splatting from two aspects. The first is to introduce spatial regularization or geometric constraints to limit the blind diffusion and unreasonable splitting of Gaussian primitives, and to reduce floating objects and geometric inconsistency caused by the lack of sufficient constraints. However, this kind of method usually needs to manually design regularization constraint terms, and the robustness for complex scenes still needs to be further improved. The other is to introduce additional base models to provide additional prior information, such as using pre-trained depth estimation models or multi-view stereo geometry models to generate initial scene geometry or depth maps as prior information to assist Gaussian optimization under sparse views, thereby enhancing the generalization ability of 3D Gaussian splatting to sparse view areas. However, the optimization process of fusing prior information and 3D Gaussian often needs to design a general and efficient alignment mechanism, and the results are also easily affected by the quality of the prior information and unstable, which brings challenges to the generalizable 3D Gaussian splatting sparse reconstruction.

[0004] The patent application No. CN202411072863.X, entitled "Sparse view three-dimensional reconstruction method based on deep prior information", combines scene depth prior information with sparse view three-dimensional Gaussian splatting method to achieve high-quality three-dimensional reconstruction of limited view input scene. Specifically, the method uses a motion recovery structure algorithm to obtain camera pose and sparse point cloud, uses a depth prediction large model to obtain monocular depth prior information, and uses the prior to expand the sparse point cloud, further constructs a three-dimensional Gaussian model and optimizes the rendering result. However, the reconstruction accuracy of this method is significantly dependent on the quality and accuracy of the monocular depth prior information. If there is a large deviation between the prior information and the actual scene, it may cause structural disorder or floating artifacts in the reconstructed scene. In addition, although object-level depth consistency constraints are introduced to improve the rationality of the scene structure, the generalization ability of the method still needs to be further improved in complex structures or scenes with single texture. SUMMARY

[0005] The present application provides a sparse view three-dimensional reconstruction method and system based on voxel grid constraints, aiming to solve the problem that the prior art cannot achieve 3D Gaussian splatting reconstruction under sparse view. The core of the present application is to introduce scene geometry prior by using a multi-view stereo geometry large model, and to design a 3D Gaussian primitive optimization strategy based on voxel grid constraints to effectively control the over-dispersion or drift of Gaussian primitives, and to realize high-quality static scene three-dimensional reconstruction under sparse view.

[0006] The sparse view three-dimensional reconstruction method based on voxel grid constraints comprises:

[0007] First step: obtain the camera internal and external parameters corresponding to all images; obtain the dense three-dimensional point cloud of the current scene and the confidence score corresponding to the point cloud;

[0008] Second step: based on the dense three-dimensional point cloud, construct a three-dimensional voxel grid, and divide the point cloud into different voxel grids;

[0009] Third step: calculate the normalized geometric feature score of all point clouds in each voxel grid, and combine the confidence score carried by the point itself to obtain the geometric confidence comprehensive metric of each point cloud. According to the geometric confidence comprehensive metric, retain multiple key points in each voxel grid to initialize the three-dimensional Gaussian primitive of the 3D Gaussian splatting radiation field;

[0010] Fourth step: establish a 3D Gaussian splatting radiation field loss function, and further optimize the 3D Gaussian splatting radiation field according to the set adjustment strategy;

[0011] Fifth step: iteratively optimize and train the 3D Gaussian splatting radiation field until the preset stopping condition is reached, and complete the three-dimensional reconstruction of the static scene under sparse view.

[0012] Preferably, in the fourth step, the method for establishing the 3D Gaussian splash radiation field loss function comprises:

[0013] In step 401, a 3D Gaussian splash radiation field loss function L is established, and the calculation formula is:

[0014] L = (1 - λ) L L1 + λ L D-SSIM

[0015] Wherein, L L1 is the normalized L1 loss of the 3D Gaussian splash radiation field rendering image and the original image, L D-SSIM is the normalized D-SSIM loss of the 3D Gaussian splash radiation field rendering image and the original image, and λ is a set weight coefficient.

[0016] The set adjustment strategy comprises a voxel grid constraint gradient attenuation strategy, and specifically comprises:

[0017] In step 402, the position of the Gaussian cell is P g , the center position of the voxel grid is P v , and the Euclidean distance between the Gaussian cell and the center position of the voxel grid is d g .

[0018] In step 403, the Gaussian cell gradient attenuation coefficient λ g is constructed according to the Euclidean distance d g , and is specifically defined as:

[0019]

[0020] Wherein, γ is a gradient attenuation rate parameter, which is used to control the attenuation speed of the gradient with the increase of the distance; L voxel is the size of the voxel grid edge length;

[0021] In step 404, the voxel grid constraint gradient attenuation strategy is established, and when the distance between the Gaussian cell and the center of the voxel grid exceeds the grid size, the position gradient is gradually attenuated, and is specifically defined as:

[0022]

[0023] Wherein, represents the partial derivative of the loss function L with respect to the position P g of the Gaussian cell, that is, the original position gradient before attenuation, represents the attenuated position gradient.

[0024] The position of the Gaussian cell is updated by using the attenuated position gradient.

[0025] Preferably, in the fourth step, the adjustment strategy further includes a grid control strategy, specifically:

[0026] Step 405, Mesh Reduction Strategy: When the number of Gaussian elements contained in any voxel mesh is less than a set threshold τ del At that time, delete the voxel mesh and all Gaussian elements within it;

[0027] Step 406, Mesh addition strategy: When the position gradient of any Gaussian element exceeds a set threshold τ grad The updated new position is recorded as when If it does not belong to any existing voxel mesh, create a new voxel mesh V. new V new The expression for the center position is Specifically defined as:

[0028]

[0029] Among them, X min Y min and Z min Represents the minimum coordinate values ​​on the X, Y, and Z axes of a dense point cloud;

[0030] The position gradient exceeds the set threshold τ grad And the new location Gausky elements that do not belong to any existing voxel mesh are placed into a new voxel mesh V. new middle.

[0031] Preferably, the second step specifically includes:

[0032] Step 201: Construct a scene-space aligned bounding box based on the dense point cloud, with a bounding box size of [L]. x ,L y ,L z The calculation formula is:

[0033] L x =X max -X min

[0034] L y =Y max -Y min

[0035] L z =Z max -Z min

[0036] Among them, X max Y max and Z maxRepresents the maximum coordinate values ​​on the X, Y, and Z axes of a dense point cloud;

[0037] Step 202, determine the voxel mesh side length L voxel The calculation formula is:

[0038]

[0039] Among them, L max =max(L x ,L y ,L z ) represents the maximum side length of the scene, and N represents the number of partitions in the specified dimension;

[0040] Step 203: Construct a 3D voxel mesh for dividing the dense point cloud, with N meshes on the X, Y, and Z axes respectively. x N y and N z The calculation formula is:

[0041]

[0042] Step 204: Adjust the scene boundaries so that the dimensions of each dimension of the scene are exactly integer multiples of the voxel mesh size. The calculation formula is:

[0043] X′ max =X min +N x ·L voxel

[0044] Y′ max =Y min +N y ·L voxel

[0045] Z′ max =Z min +N z ·L voxel

[0046] Where, X′ max 、Y′ max and Z′ max The maximum coordinate values ​​of the adjusted scene boundary on the X, Y, and Z axes;

[0047] Step 205, divide the dense point cloud into [X] min Y min Z min ;X′ max ,Y′ max Z′ max Within different voxel mesh cells at the boundary, construct the mapping relationship between point cloud and voxel mesh;

[0048] Given a point cloud p i =(x i ,y i ,z i The indices of the voxel mesh to which it belongs on the X, Y, and Z coordinate axes are i, j, and k, respectively, and the calculation formula is:

[0049]

[0050] Preferably, in the third step, within each voxel grid, the point cloud geometric confidence synthesis metric is sorted from largest to smallest, and the top k points are retained as key points, where k is a set number.

[0051] Preferably, in the third step, the method for obtaining the point cloud geometric confidence synthesis metric specifically includes:

[0052] Step 301: Calculate the fast point feature histogram geometric features f for all point clouds within each voxel mesh. n ;

[0053] Step 302: Calculate the fast point feature histogram geometric features f for all point clouds within each voxel mesh. n average

[0054] Step 303, define the geometric feature score S for each point cloud. geo (p n The calculation formula is:

[0055]

[0056] Geometric feature score S for each point cloud geo (p n Normalization is performed to obtain the normalized geometric characteristic score S. geo_norm (p n );

[0057] Step 304, obtain the confidence score S of the point cloud. conf Combined with normalized geometric characteristic fraction S geo_norm (p n The geometric confidence synthesis metric for each point cloud is obtained, calculated using the following formula:

[0058] GGM(p n ) = S geo_norm (p n )·S conf .

[0059] Preferably, in the first step, the camera intrinsic and extrinsic parameters of all images are obtained using the motion reconstruction structure algorithm based on the scene images.

[0060] Preferably, in the first step, the scene image and camera intrinsic and extrinsic parameters are obtained, and a dense 3D point cloud and point cloud confidence score S of the current scene are obtained using a multi-view stereo geometric model. conf .

[0061] A sparse-view 3D reconstruction system based on voxel mesh constraints includes a first module, a second module, a third module, a fourth module, and a fifth module;

[0062] The first module is used to perform the first step: obtain the camera intrinsic and extrinsic parameters corresponding to all images; obtain the dense 3D point cloud of the current scene and the confidence score corresponding to the point cloud;

[0063] The second module is used to perform the second step: constructing a three-dimensional voxel mesh based on the dense three-dimensional point cloud, and dividing the point cloud into different voxel meshes;

[0064] The third module is used to perform the third step: calculate the normalized geometric feature score of all point clouds in each voxel grid, and combine it with the confidence score carried by the point itself to obtain the geometric confidence comprehensive metric of each point cloud. Based on the geometric confidence comprehensive metric, multiple key points are retained in each voxel grid to initialize the three-dimensional Gaussian primitives of the 3D Gaussian splash radiation field.

[0065] The fourth module is used to perform the fourth step: establish the 3D Gaussian splash radiation field loss function, and further optimize the 3D Gaussian splash radiation field according to the set adjustment strategy;

[0066] The fifth module is used to perform the fifth step: iteratively optimize and train the 3D Gaussian splash radiation field until the preset stopping condition is reached, and complete the 3D reconstruction of the static scene under the sparse view.

[0067] The present invention has the following beneficial effects:

[0068] This invention discloses a sparse-view 3D reconstruction method and system based on voxel mesh constraints. Addressing the problem of degradation in reconstruction quality and the generation of floating objects or artifacts due to insufficient constraints in existing 3D Gaussian splash methods under sparse views, this invention proposes an optimization strategy combining geometric priors and voxel mesh control. The core of the method includes: obtaining camera parameters, dense 3D point clouds, and point cloud confidence scores through a motion recovery structure algorithm and a multi-view stereo geometric model; constructing an adaptive scene-scale 3D voxel mesh based on the point cloud distribution, dividing the point cloud into corresponding voxel meshes; fusing Fast Point Feature Histogram (FPFH) geometric features and confidence scores within the mesh to generate a geometric confidence comprehensive metric, and selecting key points to initialize Gaussian primitives; designing a gradient pruning strategy for voxel mesh constraints, limiting Gaussian primitive error propagation through a distance decay coefficient, and adaptively optimizing the mesh distribution by combining dynamic mesh pruning and addition mechanisms; and finally, iteratively training with a 3D Gaussian splash radiation field loss function to achieve high-fidelity static scene reconstruction under sparse views. The innovation of this invention lies in introducing scene geometric priors using a large multi-view 3D geometric model, and designing a 3D Gaussian primitive optimization strategy based on voxel mesh constraints to effectively control excessive diffusion or drift of Gaussian primitives, reduce artifact generation, and avoid reducing robustness due to the influence of prior quality. Experiments show that this method significantly improves reconstruction accuracy under limited input viewpoints and is applicable to application scenarios such as robot perception and virtual reality, which have high requirements for the number of viewpoints and reconstruction quality. Attached Figure Description

[0069] Figure 1 A flowchart illustrating one embodiment of the method of the present invention;

[0070] Figure 2(a) is a schematic diagram of dense point cloud, and Figure 2(b) is a schematic diagram of key point cloud;

[0071] Figure 3(a) is a scene image; Figure 3(b) is a traditional 3DGS optimization effect diagram; Figure 3(c) is a schematic diagram of the 3D Gaussian element optimization effect based on the voxel mesh constraint of the present invention. Detailed Implementation

[0072] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0073] A sparse-view 3D reconstruction method based on voxel mesh constraints includes the following steps:

[0074] Step 1: Obtain camera intrinsic and extrinsic parameters using the structure-of-motion-reconstruction algorithm based on the scene images; obtain the dense 3D point cloud and point cloud confidence score S of the current scene using a multi-view stereo geometric model. conf ;

[0075] Step 101: Obtain the camera intrinsic and extrinsic parameters of all images using the structure-of-motion-reconstruction algorithm based on the scene images;

[0076] Step 102: Obtain the scene image and camera intrinsic and extrinsic parameters from Step 101, and use a multi-view stereo geometric model to obtain the dense 3D point cloud and point cloud confidence score S of the current scene. conf ;

[0077] Step 2: Construct an adaptive scene-scale 3D voxel mesh based on the dense point cloud, and divide the point cloud into different voxel mesh units;

[0078] Step 201: Construct a scene-space aligned bounding box based on the dense point cloud, with a bounding box size of [L]. x ,L y ,L z The calculation formula is:

[0079] L x =X max -X min #(1)

[0080] L y =Y max -Y min #(2)

[0081] L z =Z max -Z min #(3)

[0082] Among them, X max Y max and Z max This represents the maximum coordinate value on the X, Y, and Z axes of a dense point cloud, where X represents the maximum coordinate value on the X, Y, and Z axes. min Y min and Z min Represents the minimum coordinate values ​​on the X, Y, and Z axes of a dense point cloud;

[0083] Step 202, determine the voxel mesh side length L voxel The calculation formula is:

[0084]

[0085] Among them, L max =max(L x ,L y ,L z ) represents the maximum side length of the scene, and N represents the number of partitions in the specified dimension.

[0086] Step 203: Construct a 3D voxel mesh for dividing the dense point cloud, with N meshes on the X, Y, and Z axes respectively. x N y and N z The calculation formula is:

[0087]

[0088] Step 204: Adjust the scene boundaries so that the dimensions of each dimension of the scene are exactly integer multiples of the voxel mesh size. The calculation formula is:

[0089] X′ max =X min +N x ·L voxel #(8)

[0090] Y′ max =Y min +N y ·L voxel #(9)

[0091] Z′ max =Z min +N z ·L voxel #(10)

[0092] Where, X′ max 、Y′ max and Z′ max The maximum coordinate values ​​of the adjusted scene boundary on the X, Y, and Z axes;

[0093] Step 205, divide the dense point cloud into [X] min Y min Z min ;X′ max ,Y′ max Z′ max Within different voxel mesh cells at the boundary, construct the mapping relationship between the point cloud and the voxel mesh. Given a point cloud p i =(x i ,y i ,z i The indices of the voxel mesh to which it belongs on the X, Y, and Z coordinate axes are i, j, and k, respectively, and the calculation formula is:

[0094]

[0095] Step 3: Calculate the normalized geometric feature score of all point clouds in each voxel grid, and combine it with the confidence score carried by the point itself to obtain the geometric confidence comprehensive metric of each point. Based on this comprehensive metric, retain a number of key points in each grid to initialize the three-dimensional Gaussian primitives.

[0096] Step 301: Calculate the Fast Point Feature Histogram (FPFH) geometric features for all point clouds within each voxel mesh. For point p... iIts Fast Point Feature Histogram (FPFH) geometric features are f i The calculation formula is:

[0097]

[0098] Among them, f i It is a 33-dimensional feature vector that describes the statistical characteristics of the local geometric structure of a point.

[0099] Step 302: Calculate the average FPFH geometric features of all point clouds within each voxel mesh. The calculation formula is:

[0100]

[0101] Among them, |V (i,j,k) | Represents a voxel grid V (i,j,k) The number of point clouds contained within;

[0102] Step 303, define the geometric feature score S for each point. geo (p n The calculation formula is:

[0103]

[0104] Normalized geometric characteristic score S for each point geo_norm (p n The calculation formula is:

[0105]

[0106] Among them, S geo_min S is the minimum geometric characteristic score of all points within the current voxel. geo_max This represents the maximum geometric feature score of all points within the current voxel.

[0107] Step 304: Take the confidence score S of the point cloud obtained in step 102. conf Combined with normalized geometric characteristic fraction S geo_norm (p n The Geometric Confidence Metric (GGM) for each point is obtained, calculated using the following formula:

[0108] GGM(p n ) = S geo_norm (p n )·S conf #(18)

[0109] Step 305: Within each voxel unit, sort the voxel units in descending order according to the geometric confidence synthesis metric (GGM) score, and retain the top k points as key points to initialize the three-dimensional Gaussian units.

[0110] Step 4: Establish the loss function for the 3D Gaussian splash radiation field, and establish a gradient decay strategy and a grid control strategy based on voxel grid constraints to further optimize the 3D Gaussian splash radiation field;

[0111] Step 401: Establish the 3D Gaussian splash radiation field loss function L for backpropagation gradient, and calculate it using the following formula:

[0112] L=(1-λ)L L1 +λL D-SSIM #(19)

[0113] Among them, L L1 The normalized L1 loss for the rendered image of the 3D Gaussian splash radiation field and the original image, L D-SSIM The normalized D-SSIM loss is used to compare the rendered image of the 3D Gaussian splash radiation field with the original image, where λ is the weighting coefficient.

[0114] Step 402, the position P of the Gaussian element g =(x g ,y g ,z g The center position P of the voxel grid to which it belongs. v =(x v ,y v ,z v ), calculate the Euclidean distance d between the Gaussian element and the center of its corresponding voxel mesh. g The calculation formula is:

[0115]

[0116] Step 403, based on d calculated in step 401 g Construct the Gaussian gradient decay coefficient λ g Specifically defined as:

[0117]

[0118] Wherein, γ is the gradient decay rate parameter, which is used to control the rate at which the gradient decays with increasing distance;

[0119] Step 404: Establish a gradient decay strategy. When the distance between a Gaussian element and the center of its corresponding voxel mesh exceeds the mesh size, the position gradient gradually decays, specifically defined as:

[0120]

[0121] in, The loss L represents the position P of the Gaussian element. g The partial derivative of , i.e., the original position gradient before decay, This represents the decayed position gradient; the position of the Gaussian element is updated using the decayed position gradient.

[0122] Step 405: Establish a mesh reduction strategy. When the number of Gaussian elements contained in any voxel mesh is less than a fixed threshold τ... del At that time, delete the voxel mesh and all Gaussian elements within it;

[0123] Step 406: Establish a grid addition strategy. When the gradient of any Gaussian element exceeds a fixed threshold τ... grad Meanwhile, its updated new position is recorded as when If it does not belong to any existing voxel mesh, create a new voxel mesh V. new V new The expression for the center position is Specifically defined as:

[0124]

[0125] The gradient mentioned above exceeds a fixed threshold τ. grad And the new location after the update Gausky elements that do not belong to any existing voxel mesh are placed into a new voxel mesh V. new middle;

[0126] Step 5: Iteratively optimize and train the 3D Gaussian splash radiation field until the preset stopping condition is reached, completing the 3D reconstruction of the static scene under sparse viewpoint, specifically:

[0127] Step 501: Repeat all steps in step four to train the 3D Gaussian splash radiation field, setting the maximum training iteration T. max The training iteration t reaches the maximum training iteration T. max Training is stopped at a certain point; a 3D Gaussian splash radiation field is obtained, enabling 3D reconstruction of static scenes from a sparse perspective.

[0128] Based on the above-mentioned sparse view 3D reconstruction method based on voxel mesh constraints, the present invention also provides a sparse view 3D reconstruction system based on voxel mesh constraints, including a first module, a second module, a third module, a fourth module, and a fifth module.

[0129] The first module is used to perform the first step: obtain the camera intrinsic and extrinsic parameters; obtain the dense 3D point cloud of the current scene and the corresponding confidence score of the point cloud;

[0130] The second module is used to perform the second step: constructing a three-dimensional voxel mesh based on the dense three-dimensional point cloud, and dividing the point cloud into different voxel mesh units;

[0131] The third module is used to perform the third step: calculate the normalized geometric feature score of all point clouds in each voxel grid, and combine it with the confidence score carried by the point itself to obtain the geometric confidence comprehensive metric of each point. Based on this comprehensive metric, multiple key points are retained in each grid to initialize the three-dimensional Gaussian primitives.

[0132] The fourth module is used to perform the fourth step: establish the 3D Gaussian splash radiation field loss function, and establish a voxel grid-constrained gradient decay strategy and grid control strategy to further optimize the 3D Gaussian splash radiation field;

[0133] The fifth module is used to perform the fifth step: iteratively optimize and train the 3D Gaussian splash radiation field until the preset stopping condition is reached, and complete the 3D reconstruction of the static scene under the sparse view.

[0134] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A sparse-view 3D reconstruction method based on voxel mesh constraints, characterized in that, include: Step 1: Obtain the camera intrinsic and extrinsic parameters corresponding to all images; Obtain the dense 3D point cloud of the current scene and the corresponding confidence score of the point cloud; Step 2: Construct a three-dimensional voxel mesh based on the dense three-dimensional point cloud, and divide the point cloud into different voxel meshes; Step 3: Calculate the normalized geometric feature score of all point clouds in each voxel grid, and combine it with the confidence score carried by the point itself to obtain the geometric confidence comprehensive metric of each point cloud. Based on this geometric confidence comprehensive metric, retain multiple key points in each voxel grid to initialize the three-dimensional Gaussian splash radiation field. Step 4: Establish the 3D Gaussian splash radiation field loss function, and further optimize the 3D Gaussian splash radiation field according to the set adjustment strategy; Step 5: Iteratively optimize and train the 3D Gaussian splash radiation field until the preset stopping condition is reached to complete the 3D reconstruction of the static scene under sparse perspective.

2. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 1, characterized in that, The method for establishing the 3D Gaussian splash radiation field loss function in the fourth step includes: Step 401, establish the 3D Gaussian splash radiation field loss function L, the calculation formula is: L=(1-λ)L L1 +λL D-SSIM Among them, L L1 The normalized L1 loss for the rendered image of the 3D Gaussian splash radiation field and the original image, L D-SSIM The normalized D-SSIM loss is used to compare the rendered image of the 3D Gaussian splash radiation field with the original image, where λ is the set weight coefficient. The established adjustment strategy includes a voxel grid-constrained gradient decay strategy, specifically: Step 402, let the position of the Gaussian element be P. g The center position of the voxel grid is P. v The Euclidean distance between the Gaussian element and the center of its corresponding voxel mesh is d. g ; Step 403, based on the Euclidean distance d g Construct the Gaussian gradient decay coefficient λ g Specifically defined as: Where γ is the gradient decay rate parameter, used to control the rate at which the gradient decays with increasing distance; L voxel This refers to the side length dimensions of the voxel mesh. Step 404: Establish a voxel grid constraint gradient decay strategy. When the distance between a Gaussian element and the center of its voxel grid exceeds the grid size, the position gradient gradually decays, specifically defined as: in, The loss function L represents the Gaussian cell position P. g The partial derivative of , i.e., the original position gradient before decay, This represents the position gradient after decay; The position of the Gaussian unit is updated using the decayed position gradient.

3. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 2, characterized in that, The adjustment strategy set in the fourth step also includes a grid control strategy, specifically: Step 405, Mesh Reduction Strategy: When the number of Gaussian elements contained in any voxel mesh is less than a set threshold τ del At that time, delete the voxel mesh and all Gaussian elements within it; Step 406, Mesh addition strategy: When the position gradient of any Gaussian element exceeds a set threshold τ grad The updated new position is recorded as when If it does not belong to any existing voxel mesh, create a new voxel mesh V. new V new The expression for the center position is Specifically defined as: Among them, X min Y min and Z min Represents the minimum coordinate values ​​on the X, Y, and Z axes of a dense point cloud; The position gradient exceeds the set threshold τ grad And the new location Gausky elements that do not belong to any existing voxel mesh are placed into a new voxel mesh V. new middle.

4. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 3, characterized in that, The second step specifically includes: Step 201: Construct a scene-space aligned bounding box based on the dense point cloud, with a bounding box size of [L]. x ,L y ,L z The calculation formula is: L x =X max -X min L y =Y max -AND min L z =Z max -WITH min Among them, X max Y max and Z max Represents the maximum coordinate values ​​on the X, Y, and Z axes of a dense point cloud; Step 202, determine the voxel mesh side length L voxel The calculation formula is: Among them, L max =max(L x ,L y ,L z ) represents the maximum side length of the scene, and N represents the number of partitions in the specified dimension; Step 203: Construct a 3D voxel mesh for dividing the dense point cloud, with N meshes on the X, Y, and Z axes respectively. x N y and N z The calculation formula is: Step 204: Adjust the scene boundaries so that the dimensions of each dimension of the scene are exactly integer multiples of the voxel mesh size. The calculation formula is: X′ max =X min +N x ·L voxel AND' max =Y min +N y ·L voxel Z′ max =Z min +N z ·L voxel Where, X′ max Y′ max and Z′ max The maximum coordinate values ​​of the adjusted scene boundary on the X, Y, and Z axes; Step 205, divide the dense point cloud into [X] min Y min Z min ;X′ max ,Y′ max Z′ max Within different voxel mesh cells at the boundary, construct the mapping relationship between point cloud and voxel mesh; Given a point cloud p i =(x i ,y i ,z i The indices of the voxel mesh to which it belongs on the X, Y, and Z axes are i, j, and k, respectively, and the calculation formula is as follows:

5. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 3 or 4, characterized in that, In the third step, within each voxel grid, the geometric confidence synthesis metric of the point cloud is sorted from largest to smallest, and the top k points are retained as key points, where k is a set number.

6. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 5, characterized in that, The method for obtaining the geometric confidence synthesis metric of point clouds in the third step specifically includes: Step 301: Calculate the fast point feature histogram geometric features f for all point clouds within each voxel mesh. n ; Step 302: Calculate the fast point feature histogram geometric features f for all point clouds within each voxel mesh. n average Step 303, define the geometric feature score S for each point cloud. geo (p n The calculation formula is: Geometric feature fraction S for each point cloud geo (p n Normalization is performed to obtain the normalized geometric characteristic score S. geo_norm (p n ); Step 304, obtain the confidence score S of the point cloud. conf Combined with normalized geometric characteristic fraction S geo_norm (p n The geometric confidence synthesis metric for each point cloud is obtained, and the calculation formula is as follows: GGM(p n )=S geo_norm (p n )·S conf 。 7. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 3, characterized in that, In the first step, the camera intrinsic and extrinsic parameters of all images are obtained using the motion reconstruction structure algorithm based on the scene images.

8. The sparse viewpoint 3D reconstruction method based on voxel mesh constraints according to claim 7, characterized in that, In the first step, scene images and camera intrinsic and extrinsic parameters are obtained, and a dense 3D point cloud and point cloud confidence score S of the current scene are obtained using a multi-view stereo geometric model. conf .

9. A sparse-view 3D reconstruction system based on voxel mesh constraints, characterized in that, It includes Module 1, Module 2, Module 3, Module 4, and Module 5; The first module is used to perform the first step: obtain the camera intrinsic and extrinsic parameters corresponding to all images; obtain the dense 3D point cloud of the current scene and the confidence score corresponding to the point cloud; The second module is used to perform the second step: constructing a three-dimensional voxel mesh based on the dense three-dimensional point cloud, and dividing the point cloud into different voxel meshes; The third module is used to perform the third step: calculate the normalized geometric feature score of all point clouds in each voxel grid, and combine it with the confidence score carried by the point itself to obtain the geometric confidence comprehensive metric of each point cloud. Based on the geometric confidence comprehensive metric, multiple key points are retained in each voxel grid to initialize the three-dimensional Gaussian primitives of the 3D Gaussian splash radiation field. The fourth module is used to perform the fourth step: establish the 3D Gaussian splash radiation field loss function, and further optimize the 3D Gaussian splash radiation field according to the set adjustment strategy; The fifth module is used to perform the fifth step: iteratively optimize and train the 3D Gaussian splash radiation field until the preset stopping condition is reached, and complete the 3D reconstruction of the static scene under the sparse view.

Images