Reconstruction method and reconstruction system for three-dimensional scene

Abstract

A method and system for reconstructing a 3D scene. The 3D scene reconstruction method includes the following steps: Training a number of 3D Gaussian Splats (3DGS) for a 3D scene. Expanding the 3D Gaussian Splats based on several positional gradients of these 3DGS. The number of expanded 3D Gaussian Splats is below a predetermined upper limit. Reducing the number of 3D Gaussian Splats based on several opacities of these 3DGS. After performing the steps of expanding and reducing the 3D Gaussian Splats, the steps of training the 3D Gaussian Splats for the 3D scene are repeated.

Claims

1. A method for reconstructing a three-dimensional scene, wherein the method, after loading a program onto a computer, performs the following steps: training a plurality of three-dimensional Gaussian points of a three-dimensional scene; expanding the three-dimensional Gaussian points based on a plurality of position gradients of the three-dimensional Gaussian points, wherein the expanded three-dimensional Gaussian points are below a predetermined upper limit; and reducing the three-dimensional Gaussian points based on a plurality of opacities of the three-dimensional Gaussian points; wherein after the steps of expanding and reducing the three-dimensional Gaussian points are performed, the step of training the three-dimensional Gaussian points of the three-dimensional scene is repeated; wherein the step of expanding the three-dimensional Gaussian points includes: Simulate the amplification quantity of one of these three-dimensional Gaussian points; determine whether the sum of the current quantity of one of these three-dimensional Gaussian points and the simulated amplification quantity is greater than the predetermined upper limit; And if the sum of the current number and the simulated amplification number is greater than the predetermined upper limit, then a portion of the three-dimensional Gaussian points are selected for amplification in descending order of the positional gradients, so as to keep the amplified three-dimensional Gaussian points below the predetermined upper limit.

2. The method for reconstructing a three-dimensional scene as described in claim 1, wherein the number of simulated augmentations includes the number of simulated segments of the three-dimensional Gaussian point segments from the portion.

3. The method for reconstructing a three-dimensional scene as described in claim 2, wherein each of the segmented three-dimensional Gaussian points has a position gradient higher than a gradient threshold and a size higher than a size threshold.

4. The method for reconstructing a three-dimensional scene as described in claim 1, wherein the number of simulated augmentations includes the number of simulated copies of the portion of the three-dimensional Gaussian points.

5. The method for reconstructing a three-dimensional scene as described in claim 4, wherein each of the copied three-dimensional Gaussian points has a position gradient higher than a gradient threshold and a size less than or equal to a size threshold.

6. The method for reconstructing a three-dimensional scene as described in Request 1, wherein the upper limit of the predetermined number is K times the current number.

7. The method for reconstructing a 3D scene as described in Request 6, wherein K is 2.

8. The method for reconstructing a three-dimensional scene as described in claim 1, wherein in the step of reducing the three-dimensional Gaussian points, a portion of the three-dimensional Gaussian points are selected for deletion to maintain the reduced three-dimensional Gaussian points above a predetermined lower limit.

9. The method for reconstructing a three-dimensional scene as described in claim 1, wherein each of the deleted three-dimensional Gaussian points has an opacity below an opacity threshold.

10. A three-dimensional scene reconstruction system, comprising: A training module for training a plurality of 3D Gaussian points in a 3D scene; an augmentation module for augmenting the 3D Gaussian points based on a plurality of position gradients of the 3D Gaussian points, such that the augmented 3D Gaussian points are below a predetermined upper limit; and a reduction module for reducing the 3D Gaussian points based on a plurality of opacities of the 3D Gaussian points; wherein after the augmentation module augments the 3D Gaussian points and the reduction module reduces the 3D Gaussian points, the training module repeatedly trains the 3D Gaussian points in the 3D scene; wherein the reduction module selects a portion of the 3D Gaussian points for deletion to maintain the reduced 3D Gaussian points above a predetermined lower limit.

11. A three-dimensional scene reconstruction system as described in claim 10, wherein the augmentation module comprises: A simulation unit is used to simulate the amplification quantity of one of these three-dimensional Gaussian points; A judgment unit for determining whether the sum of the current number of the three-dimensional Gaussian points and the simulated amplification number is greater than the predetermined upper limit; an amplification unit; and a selection unit, wherein if the sum of the current number and the simulated amplification number is greater than the predetermined upper limit, the selection unit selects a portion of the three-dimensional Gaussian points for the amplification unit to amplify in order of the position gradient from high to low, so as to maintain the amplified three-dimensional Gaussian points below the predetermined upper limit.

12. The reconstruction system of a three-dimensional scene as described in claim 11, wherein the number of simulated augmentations includes the number of three-dimensional Gaussian point segments from the portion.

13. A three-dimensional scene reconstruction system as described in claim 12, wherein each of the segmented three-dimensional Gaussian points has a position gradient above a gradient threshold and a size above a size threshold.

14. A three-dimensional scene reconstruction system as described in claim 11, wherein the number of simulation augmentations includes the number of copies of the three-dimensional Gaussian points from the portion.

15. A three-dimensional scene reconstruction system as described in claim 14, wherein each of the replicated three-dimensional Gaussian points has a position gradient above a gradient threshold and a size less than or equal to a size threshold.

16. The three-dimensional scene reconstruction system as described in claim 11, wherein the upper limit of the predetermined number is K times the current number.

17. A three-dimensional scene reconstruction system as described in claim 16, wherein K is 2.

18. A three-dimensional scene reconstruction system as described in claim 10, wherein each of the deleted three-dimensional Gaussian points has an opacity below an opacity threshold.

Images