Scene three-dimensional reconstruction method based on density-gradient dual drive

Abstract

The invention discloses a scene three-dimensional reconstruction method based on density-gradient dual-drive, which performs multi-round reconstruction to obtain a three-dimensional model, introduces a space density sensing mechanism on the basis of gradient feedback, and effectively solves the problem of scarcity of anchor points in a low-gradient region. Through dynamic density threshold scheduling and anchor point budget control, self-adaptive balance of the number of anchor points is realized; according to the method, the state reconstruction frequency is obviously reduced and the reconstruction stability and the calculation efficiency are improved by performing dynamic cutting and then performing adjustment through an optimizer instead of performing adjustment directly through the optimizer in combination with hash value de-duplication and anchor point growth, so that the state reconstruction frequency is obviously reduced, and the reconstruction stability and the calculation efficiency are improved; a dynamic pre-cutting strategy is adopted, so that the training process is more stable and the occupation of a video memory is controllable; on the whole, the method can keep high geometric continuity and rendering precision under the sparse view angle, the shielding boundary and the complex geometric structure, and double improvement of the three-dimensional reconstruction quality and the calculation efficiency is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of three-dimensional reconstruction, and particularly relates to a scene three-dimensional reconstruction method based on density-gradient double driving. BACKGROUND

[0002] With the development of three-dimensional scene rendering technology, three-dimensional Gaussian Splatting (3DGS) exhibits excellent rendering fidelity and efficiency. However, the explicit 3DGS representation has very high requirements for computing and memory resources, mainly because each Gaussian needs to explicitly store its parameters (such as position, shape, color and transparency, etc.), and needs to be calculated and updated in real time during the rendering process. In addition, in order to accurately fit each view and present details, 3DGS often generates a large number of redundant Gaussian distributions. These redundant Gaussians repeatedly represent the same geometric features, greatly increasing the memory occupation and computing burden. This is particularly significant when dealing with complex scenes or high-resolution rendering.

[0003] Scaffold-GS overcomes the above limitations by anchoring neural Gaussians, Scaffold-GS generates anchor points from sparse point clouds, constructs a voxel grid, and trains neural Gaussians using a multi-layer perceptron (MLP), significantly reducing memory consumption. Since the neural Gaussians on the anchor points only need to store a small number of parameters such as mean and variance, and the calculation mainly depends on the data distribution within the local voxel grid, Scaffold-GS avoids global rendering and calculation for the entire scene. Therefore, even when dealing with complex buildings or large open spaces and other scenes, it can still run with low and stable memory consumption.

[0004] However, the existing Scaffold-GS anchor point growth mainly relies on the gradient intensity signal. In high gradient areas, anchor points are increased, and in low gradient areas, they remain unchanged. This "error-oriented" strategy often has problems such as missing anchor points or local geometric discontinuity when dealing with low gradient but structurally critical scenes (such as occlusion boundaries, sparse textures, low contrast backgrounds). SUMMARY

[0005] The purpose of the present application is to solve the problems raised in the background art, and to provide a scene three-dimensional reconstruction method based on density-gradient double driving.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is:

[0007] The scene three-dimensional reconstruction method based on density-gradient double driving proposed by the present application carries out multi-round reconstruction to obtain a three-dimensional model, which includes:

[0008] In one round of reconstruction process:

[0009] Step 1, obtaining camera poses and sparse point clouds of multi-view images of an object to be reconstructed, voxelizing each sparse point cloud, taking the center position of each voxel as the initial coordinates of an anchor point, initializing the neural Gaussian parameters of each anchor point, and all the anchor points form a first anchor point set;

[0010] Step 2, calculating the gradient of the neural Gaussian parameters of each anchor point in the current first anchor point set, calculating the gradient amplitude of each anchor point in the current first anchor point set, dividing the current first anchor point set into grids, and calculating the spatial density of the anchor points in each grid;

[0011] Step 3, generating a second anchor point set according to the gradient amplitude and a third anchor point set according to the spatial density;

[0012] Step 4, constructing a candidate anchor point set from the second anchor point set and the third anchor point set, and performing an anchor point deduplication operation on the candidate anchor point set to obtain an added anchor point set;

[0013] Step 5, initializing the neural Gaussian parameters of each anchor point in the added anchor point set, and concatenating the neural Gaussian parameters of each anchor point in the added anchor point set with the neural Gaussian parameters of each anchor point in the first anchor point set to obtain a neural Gaussian parameter set;

[0014] Step 6, dynamically clipping all anchor points corresponding to the neural Gaussian parameter set according to the anchor point importance budget to obtain an optimized anchor point set, calculating the gradient of the neural Gaussian parameters of each anchor point in the optimized anchor point set, and updating the neural Gaussian parameter set;

[0015] Step 7, the optimizer adjusts the neural Gaussian parameters of each anchor point in the optimized anchor point set according to the gradient of the neural Gaussian parameters of each anchor point in the optimized anchor point set;

[0016] Step 8, determining whether the reconstruction converges, when the reconstruction converges, performing three-dimensional rendering on the adjusted neural Gaussian parameters of each anchor point in the optimized anchor point set to obtain a three-dimensional model, realizing three-dimensional reconstruction of the object to be reconstructed, if the reconstruction does not converge, entering the next round of reconstruction, and taking the current optimized anchor point set as the first anchor point set in the next round of reconstruction, returning to step 2 and iterating.

[0017] Preferably, the neural Gaussian parameters include a position vector, a scale parameter, a rotation quaternion, an eigenvector, an offset parameter, and an opacity.

[0018] Preferably, calculating the gradient of the neural Gaussian parameters of each anchor point includes:

[0019] The three-dimensional rendering is performed on all anchor points in the current anchor point set to obtain a predicted three-dimensional model, and the predicted three-dimensional model is subtracted from the real three-dimensional model of the object to be reconstructed to obtain a reconstruction error, and the gradient of the neural Gaussian parameter of each anchor point is calculated by back propagation according to the reconstruction error.

[0020] Preferably, the gradient amplitude of each anchor point is calculated as the L2 norm of the gradient of the offset parameter of each anchor point.

[0021] Preferably, the second anchor point set is generated according to the gradient amplitude, comprising:

[0022] For all anchor points in the current first anchor point set, when the gradient amplitude of the anchor point is greater than a preset value and the anchor point sampling is sufficient, a preset number of fourth anchor points are generated in all neighboring grids of the grid where the anchor point is located, and all fourth anchor points constitute the second anchor point set.

[0023] Preferably, the third anchor point set is generated according to the spatial density, comprising:

[0024] For all anchor points in the current first anchor point set, when the spatial density of the anchor point in the grid is less than the current adaptive threshold value, a preset number of fifth anchor points are generated in the current grid, and all fifth anchor points constitute the third anchor point set.

[0025] Wherein, the adaptive threshold value The adaptive threshold value is updated as the reconstruction round progresses, and the formula is as follows:

[0026] ;

[0027] Wherein,

[0028] ;

[0029] Wherein, is the first reconstruction round, is the decay factor, is the minimum value of the adaptive threshold value, is the initial value of the adaptive threshold value, is the adjustment factor, is the intermediate parameter, is the average value of the difference in the gradient amplitude of all corresponding anchor point offset parameters between adjacent two reconstruction rounds, and when the anchor point has no difference in the gradient amplitude, then the gradient amplitude of the corresponding anchor point current offset parameter is taken, is the standard deviation of the difference in the gradient amplitude of all corresponding anchor point offset parameters between adjacent two reconstruction rounds.

[0030] Preferably, the anchor point deduplication operation is performed on the candidate anchor point set, comprising:

[0031] Hash values of each anchor point in the candidate anchor point set and the first anchor point set are calculated respectively, and it is judged whether the Hash values of each anchor point in the candidate anchor point set are repeated with the Hash values of each anchor point in the first anchor point set, if yes, the corresponding anchor point in the candidate anchor point set is directly eliminated;

[0032] The calculation formula of the Hash value is:

[0033]

[0034] The calculation formula of the anchor point importance budget is: The coordinates of each anchor point, , and are all preset values.

[0035] Preferably, the all anchor points corresponding to the neural Gaussian parameter set are dynamically pruned according to the anchor point importance budget to obtain an optimized anchor point set, including:

[0036] The all anchor points corresponding to the neural Gaussian parameter set are sorted according to the anchor point importance budget from high to low, and the first preset number of anchor points are retained to form the optimized anchor point set, wherein the calculation formula of the anchor point importance budget is:

[0037]

[0038] The calculation formula of the anchor point importance budget is: The number of anchor point access statistics, is the cumulative opacity, which represents the visibility and rendering contribution.

[0039] Preferably, the condition of reconstruction convergence is that when the reconstruction round reaches a preset round or the global error is lower than a set threshold, the reconstruction converges, wherein the global error is the pixel-level loss or the perception-level loss between the three-dimensional model obtained by the three-dimensional rendering of the adjusted neural Gaussian parameters of each anchor point in the current optimized anchor point set and the real three-dimensional model of the object to be reconstructed.

[0040] Compared with the prior art, the beneficial effects of the present application are:

[0041] ​​​​The scene three-dimensional reconstruction method based on the density-gradient double driving introduces a space density perception mechanism on the basis of gradient feedback, effectively making up for the problem of anchor point scarcity in low gradient areas. Through dynamic density threshold scheduling and anchor point budget control, adaptive balance of anchor point quantity is realized; combined with hash value deduplication, and after anchor point growth, instead of directly adjusting through the optimizer, the optimizer is adjusted after dynamic clipping, thereby significantly reducing the state reconstruction frequency, improving reconstruction stability and computational efficiency; a dynamic pre-clipping strategy is used to make the training process more stable and the memory occupancy controllable; overall, the method can maintain high geometric continuity and rendering accuracy under sparse view angles, occluded boundaries and complex geometric structures, achieving dual improvement of three-dimensional reconstruction quality and computational efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 FIG. 1 is a flowchart of the scene three-dimensional reconstruction method based on the density-gradient double driving of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the specification of the present application is only for the purpose of describing specific embodiments and is not intended to limit the present application.

[0045] In one embodiment, as shown in Figure 1 a scene three-dimensional reconstruction method based on density-gradient double driving is provided, comprising:

[0046] The scene three-dimensional reconstruction method based on density-gradient double driving is used for multi-round reconstruction to obtain a three-dimensional model (i.e., reconstructing a three-dimensional model from multi-view images of an object to be reconstructed), comprising:

[0047] In one round of reconstruction process:

[0048] Step 1, (Structure from Motion (SfM)) obtaining camera poses and sparse point clouds of multi-view images of an object to be reconstructed, voxelizing each sparse point cloud, taking the center position of each voxel as the initial coordinates of an anchor point, initializing the neural Gaussian parameters of each anchor point (through a multi-layer perception machine (MLP)), and all current anchor points constitute a first anchor point set;

[0049] The neural Gaussian parameters include: a position vector, a scale parameter (set by log(initial voxel scale) at initialization), a rotation quaternion, an eigenvector (copied from the existing anchor point feature mean), an offset parameter, and an opacity (initialized by inverse_sigmoid(0.1)).

[0050] Step 2, calculating the gradient of each anchor point neural Gaussian parameter in the current first anchor point set, performing gradient amplitude calculation on each anchor point in the current first anchor point set, performing grid division on the current first anchor point set, and performing spatial density calculation on the anchor points in each grid (the spatial density is the number of anchor points in the grid / the volume of the grid);

[0051] The gradient of each anchor point neural Gaussian parameter is calculated, including:

[0052] The three-dimensional rendering is performed on all anchor points in the current anchor point set to obtain a predicted three-dimensional model, and the predicted three-dimensional model is subtracted from the real three-dimensional model of the object to be reconstructed to obtain a reconstruction error, and the gradient of each anchor point neural Gaussian parameter is calculated by back propagation according to the reconstruction error.

[0053] The gradient amplitude of each anchor point is calculated, including: calculating the L2 norm of the gradient of each anchor point offset parameter to obtain the gradient amplitude.

[0054] Step 3, generating a second anchor point set according to the gradient amplitude and generating a third anchor point set according to the spatial density;

[0055] The second anchor point set is generated according to the gradient amplitude, including:

[0056] For all anchor points in the current first anchor point set, when the gradient amplitude of the anchor point is greater than a preset value and the anchor point sampling is sufficient (offset_mask is true, wherein offset_mask is a Boolean variable used to mark whether the anchor point is used by a sufficient number of effective pixels in the current training period; the effective pixel refers to a pixel that actually participates in the contribution calculation of the anchor point and transmits the gradient to it during the rendering process), a preset number of fourth anchor points (the specific number is randomly set) are generated in all neighboring grids of the grid where the anchor point is located, and all fourth anchor points constitute the second anchor point set.

[0057] The third anchor point set is generated according to the spatial density, including:

[0058] For all anchor points in the current first anchor point set, when the spatial density of the anchor point in the grid is less than the current adaptive threshold , a preset number of fifth anchor points (the specific number is randomly set) are generated in the current grid, and all fifth anchor points constitute the third anchor point set;

[0059] Among them, adaptive threshold Updated with each reconstruction round, and the formula is as follows:

[0060] ;

[0061] in,

[0062] ;

[0063] in, For the reconstruction of the first Rounds, As the attenuation factor, This is the minimum value of the adaptive threshold (the lowest protection value of the adaptive threshold to avoid excessive growth in the later stages). This is the initial value for the adaptive threshold (used to quickly fill in sparse regions during the initial reconstruction phase). This is an adjustment factor (controlling the extent to which the adaptive threshold decreases with the number of reconstruction rounds). Intermediate parameters (based on local error plot) The statistic, denoted by mean / std, measures the overall error fluctuation and is used to dynamically adjust the adaptive threshold. This is the average of the differences in gradient magnitudes of all corresponding anchor point offset parameters between two adjacent reconstruction rounds. If there is no difference in gradient magnitude between anchor points, then... Take the gradient magnitude of the current offset parameter of the corresponding anchor point. It is the standard deviation of the difference in gradient magnitude of all corresponding anchor point offset parameters between two adjacent reconstruction rounds.

[0064] Step 4: Combine the second anchor point set and the third anchor point set to form a candidate anchor point set, and perform anchor point deduplication on the candidate anchor point set to obtain the new anchor point set;

[0065] The anchor point deduplication operation on the candidate anchor point set includes:

[0066] Calculate the hash value of each anchor in the candidate anchor set and the first anchor set respectively, and determine whether the hash value of each anchor in the candidate anchor set is the same as the hash value of each anchor in the first anchor set. If so, directly remove the corresponding anchor in the candidate anchor set.

[0067] Hash value The calculation formula is:

[0068] ;

[0069] in, Here are the coordinates of each anchor point. , and All are preset values ​​( , and These are coprime large prime constants (such as 73856093, 19349663, and 83492791), used to distinguish the influence of each coordinate dimension and prevent hash collisions.

[0070] Step 5: Initialize the neural Gaussian parameters of each anchor point in the newly added anchor point set, and concatenate the neural Gaussian parameters of each anchor point in the newly added anchor point set with the neural Gaussian parameters of each anchor point in the first anchor point set (this process only expands the number of anchor points, without changing the neural Gaussian parameter structure dimension or feature space dimension), to obtain the neural Gaussian parameter set (this realizes the growth of anchor points, but after the anchor points are grown, they are not directly adjusted by the optimizer, but are adjusted by the optimizer after being pruned in step 6, thereby significantly reducing the state reconstruction frequency and improving reconstruction stability and computational efficiency).

[0071] Step 6: Dynamically prune all anchor points corresponding to the set of neural Gaussian parameters according to the anchor point importance budget to obtain the optimized anchor point set, and calculate the gradient of the neural Gaussian parameters of each anchor point in the optimized anchor point set (as in step 2, which calculates the gradient of the neural Gaussian parameters of each anchor point), and update the set of neural Gaussian parameters (that is, remove the neural Gaussian parameters corresponding to the dynamically pruned anchor points from the set of neural Gaussian parameters).

[0072] Specifically, all anchor points corresponding to the neural Gaussian parameter set are dynamically pruned according to anchor point importance estimation to obtain an optimized anchor point set, including:

[0073] All anchor points corresponding to the neural Gaussian parameter set are sorted from high to low according to their anchor point importance budget. A predetermined number of anchor points are retained to form the optimized anchor point set, where the anchor point importance budget... The calculation formula is:

[0074] ;

[0075] in, The number of times the anchor point is visited ( The statistics are updated jointly during the rendering and gradient backpropagation phases, recording the frequency of anchor point usage and gradient accumulation during reconstruction. When an anchor point participates in pixel contribution, structural representation, or generates effective gradients on the offset parameters, its... The value increases, thus accurately reflecting the true contribution of the anchor point rather than the instantaneous error. Cumulative opacity represents the contribution of visibility to rendering.

[0076] Step 7: The optimizer adjusts the neural Gaussian parameters of each anchor point in the set of optimized anchor points based on the gradient of the neural Gaussian parameters of each anchor point. (The optimizer's role is to perform gradient updates on the learnable parameters of the anchor points, so that the anchor points gradually fit the real 3D structure and rendering contribution. The optimizer is only responsible for adjusting the anchor point's position vector, scale parameter, rotation quaternion, feature vector, offset parameter, and opacity based on the gradient of the neural Gaussian parameters obtained from backpropagation, thereby continuously reducing the reconstruction error; the optimizer does not determine the generation or pruning of the number of anchor points, and the increase or decrease of the number of anchor points is entirely controlled by the dual-drive strategy and importance budget mechanism.)

[0077] Step 8: Determine if the reconstruction has converged (the condition for reconstruction convergence is: when the reconstruction rounds reach the preset number of rounds or the global error is lower than the set threshold, the reconstruction converges, where the global error is the pixel-level or perceptual-level loss between the 3D model obtained by 3D rendering of the adjusted neural Gaussian parameters of each anchor point in the current optimized anchor point set and the real 3D model of the object to be reconstructed). When converged, the adjusted neural Gaussian parameters of each anchor point in the optimized anchor point set are used to 3D render the 3D model to achieve the 3D reconstruction of the object to be reconstructed. If it has not converged, proceed to the next round of reconstruction, and use the current optimized anchor point set as the first anchor point set in the next round of reconstruction, return to step 2, and iterate.

[0078] In another embodiment, to verify the effectiveness of the technical solution, training and testing were performed on three public datasets: Mip-NeRF360, Tanks & Temples, and Deep Blending. Similar to the 3DGS method, for each scene's input images, one-eighth of the total number of images was used as the test set, and the remaining images were used as the training set. To objectively evaluate visual fidelity, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Patch Similarity (LPIPS) were used as evaluation metrics. These metrics can compare the differences between images rendered by different methods and their corresponding real frames, thus providing a quantitative analysis of the rendering effect. Specific experimental results are shown in Table 1.

[0079] Table 1

[0080]

[0081] As shown in Table 1, the rendering quality of this application outperforms other methods in all three datasets mentioned above. As evaluation criteria, higher PSNR and SSIM values ​​indicate higher image similarity, while lower LPIPS values ​​indicate a smaller visual difference between the rendered image and the real image. Further analysis of the rendering results reveals that this application can preserve fine geometric details in complex scenes. For example, in the STUMP scene of the Mip-NeRF360 dataset, it accurately renders small tree branches; in the PLAYROOM scene of the Deep Blending dataset, it meticulously depicts the switches on the wall; and in the TRAIN scene of the Tanks & Temples dataset, it well reproduces the details of the truck license plate and windshield reflections.

[0082] This density-gradient dual-driven scene 3D reconstruction method introduces a spatial density-aware mechanism on top of gradient feedback, effectively compensating for the scarcity of anchor points in low-gradient regions. Through dynamic density threshold scheduling and anchor point budget control, an adaptive balance in the number of anchor points is achieved. Combined with hash value deduplication and the fact that after anchor point growth, adjustments are not directly made through the optimizer but rather dynamically pruned before being adjusted by the optimizer, significantly reducing the state reconstruction frequency and improving reconstruction stability and computational efficiency. A dynamic pre-pruning strategy is employed to make the training process more stable and memory usage controllable. Overall, this method maintains high geometric continuity and rendering accuracy under sparse viewpoints, occluded boundaries, and complex geometric structures, achieving a dual improvement in 3D reconstruction quality and computational efficiency.

[0083] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0084] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for 3D scene reconstruction based on density-gradient dual-drive, characterized in that: The density-gradient dual-driven scene 3D reconstruction method performs multiple rounds of reconstruction to obtain a 3D model, including: During the first round of reconstruction: Step 1: Obtain the camera pose and sparse point cloud of the multi-view image of the object to be reconstructed. Perform voxelization on each sparse point cloud, take the center position of each voxel as the initial coordinate of an anchor point, and initialize the neural Gaussian parameters of each anchor point. All current anchor points constitute the first anchor point set. Step 2: Calculate the gradient of the neural Gaussian parameters of each anchor point in the current first anchor point set, calculate the gradient magnitude of each anchor point in the current first anchor point set, divide the current first anchor point set into a grid, and calculate the spatial density of anchor points in each grid. Step 3: Generate a second set of anchor points based on the gradient magnitude, and a third set of anchor points based on the spatial density; Step 4: Combine the second anchor point set and the third anchor point set to form a candidate anchor point set, and perform anchor point deduplication on the candidate anchor point set to obtain the new anchor point set; Step 5: Initialize the neural Gaussian parameters of each anchor point in the newly added anchor point set, and concatenate the neural Gaussian parameters of each anchor point in the newly added anchor point set with the neural Gaussian parameters of each anchor point in the first anchor point set to obtain the neural Gaussian parameter set. Step 6: Dynamically prune all anchor points corresponding to the neural Gaussian parameter set according to the anchor point importance budget to obtain the optimized anchor point set, calculate the gradient of the neural Gaussian parameters of each anchor point in the optimized anchor point set, and update the neural Gaussian parameter set. Step 7: The optimizer adjusts the Gaussian parameters of each anchor point in the set of anchor points according to the gradient of the Gaussian parameters of each anchor point in the set of anchor points. Step 8: Determine if the reconstruction has converged. If it has converged, perform 3D rendering on the adjusted neural Gaussian parameters of each anchor point in the optimized anchor point set to obtain a 3D model and realize the 3D reconstruction of the object to be reconstructed. If it has not converged, proceed to the next round of reconstruction and use the current optimized anchor point set as the first anchor point set in the next round of reconstruction. Return to step 2 and iterate.

2. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: The neural Gaussian parameters include: position vector, scale parameter, rotation quaternion, eigenvector, offset parameter, and opacity.

3. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: Calculate the gradients of the Gaussian parameters of the nerve at each anchor point, including: A 3D rendering is performed on all anchor points in the current anchor point set to obtain a predicted 3D model. The difference between the predicted 3D model and the real 3D model of the object to be reconstructed is calculated to obtain the reconstruction error. The gradient of the neural Gaussian parameters of each anchor point is calculated based on the backpropagation of the reconstruction error.

4. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 2, characterized in that: The gradient magnitude of each anchor point is obtained by calculating the L2 norm of the gradient of the offset parameter of each anchor point.

5. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: The step of generating the second anchor point set based on the gradient magnitude includes: For all anchor points in the current first anchor point set, when the gradient magnitude of the anchor point is greater than a preset value and the anchor point is sufficiently sampled, a preset number of fourth anchor points are generated in all neighboring grids of the grid where the anchor point is located, and all fourth anchor points constitute the second anchor point set.

6. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: The generation of the third anchor point set based on spatial density includes: For all anchor points in the current first anchor point set, if the spatial density of anchor points in the grid is less than the current adaptive threshold, then a preset number of fifth anchor points are generated in the current grid, and all fifth anchor points constitute the third anchor point set. Among them, adaptive threshold Updated with each reconstruction round, and the formula is as follows: ； in, ； in, For the reconstruction of the first Rounds, As the attenuation factor, This represents the minimum value of the adaptive threshold. This is the initial value for the adaptive threshold. As a regulating factor, For intermediate parameters, This is the average of the differences in gradient magnitudes of all corresponding anchor point offset parameters between two adjacent reconstruction rounds. If there is no difference in gradient magnitude between anchor points, then... Take the gradient magnitude of the current offset parameter of the corresponding anchor point. It is the standard deviation of the difference in gradient magnitude of all corresponding anchor point offset parameters between two adjacent reconstruction rounds.

7. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: The anchor point deduplication operation on the candidate anchor point set includes: Calculate the hash value of each anchor in the candidate anchor set and the first anchor set respectively, and determine whether the hash value of each anchor in the candidate anchor set is the same as the hash value of each anchor in the first anchor set. If so, directly remove the corresponding anchor in the candidate anchor set. Hash value The calculation formula is: ； in, Here are the coordinates of each anchor point. , and All are preset values.

8. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: The process of dynamically pruning all anchor points corresponding to the neural Gaussian parameter set according to anchor point importance budget to obtain an optimized anchor point set includes: All anchor points corresponding to the neural Gaussian parameter set are sorted from high to low according to their anchor point importance budget. A predetermined number of anchor points are retained to form the optimized anchor point set, where the anchor point importance budget... The calculation formula is: ； in, This is a statistic representing the number of times the anchor point is accessed. Cumulative opacity represents the contribution of visibility to rendering.

9. The scene 3D reconstruction method based on density-gradient dual-drive as described in claim 1, characterized in that: The conditions for reconstruction convergence are: when the number of reconstruction rounds reaches the preset number of rounds or the global error is lower than the set threshold, the reconstruction converges. The global error is the pixel-level or perceptual-level loss between the 3D model obtained by 3D rendering the neural Gaussian parameters of each anchor point in the current set of optimized anchor points and the real 3D model of the object to be reconstructed.

Images