Large-scale scene three-dimensional reconstruction method based on three-dimensional Gaussian

Abstract

The invention belongs to the technical field of scene three-dimensional reconstruction, and discloses a large-scene three-dimensional reconstruction method based on three-dimensional Gaussian, and the method comprises the steps: firstly obtaining an initial scene point cloud of an original input image, and dividing a scene into a plurality of adjacent cube blocks according to a space coordinate range of the initial scene point cloud; aiming at the coordinate and the visual angle of a given target observation point, frequency coding is carried out on the center coordinate of each block, the coordinate of the observation point and the visual angle, and the obtained feature vectors are input into a pre-trained multi-layer perceptron (MLP). The MLP outputs the visibility probability value of each block under the target observation point, the blocks exceeding a preset threshold value are judged as visible blocks, and only three-dimensional Gaussian model parameters defined in the visible blocks are loaded to the GPU for rendering, so that a target view is generated. According to the method, the requirement of large-scale scene training for hardware resources is greatly reduced while the quality is ensured, and efficient training and real-time rendering can be realized under the condition of single GPU computing power.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of scene three-dimensional reconstruction, and particularly relates to a large scene three-dimensional reconstruction method based on three-dimensional Gauss. BACKGROUND

[0002] Novel view synthesis and three-dimensional reconstruction are core topics in the field of computer vision and computer graphics, and have wide application value in virtual reality, augmented reality, digital twinning, and automatic driving simulation scenarios. In recent years, implicit neural representation methods, especially neural radiance fields (NeRF) and its variants, have made significant breakthroughs in the quality of novel view synthesis. However, the standard NeRF method has inherent bottlenecks of extremely slow training and rendering speed due to the need for dense sampling of coordinate points throughout the scene and calculation through a neural network, making it difficult to apply to large-scale urban scenes and other practical situations.

[0003] To meet the needs of large scene rendering, existing technologies have proposed a series of improved schemes based on block or divide-and-conquer strategies. For example, Block-NeRF divides the scene into multiple blocks by artificially preset rules (such as using crossroads as boundaries), and trains a NeRF model for each block independently, and fuses the results of adjacent blocks during rendering. Although this method decomposes and processes large scenes, the partition strategy relies on prior knowledge and lacks flexibility, and the block boundary fusion effect is not good. Mega-NeRF attempts to assign the training image set to different NeRF expert modules for parallel training through a geometric clustering algorithm, and introduces a foreground-background separation sampling strategy. This method achieves data parallelism to some extent, but its structure is complex, and it does not fundamentally solve the problem of slow rendering speed, making it difficult to achieve real-time interaction. Switch-NeRF uses a gating network to dynamically select expert models, and its goal is more focused on model capacity and rendering quality, with limited improvement in training and rendering efficiency. The above methods are all built on the NeRF framework, and the inherent computational overhead of volume rendering makes their training time often take tens of hours, which cannot meet the needs of real-time applications.

[0004] The emerging three-dimensional Gaussian splatting (3D Gaussian Splatting, 3DGS) technology represents a scene by an explicit and optimizable Gaussian ellipsoid, and combines a tile-based rasterizer to achieve high-quality real-time rendering, bringing new dawn to the reconstruction and rendering of large-scale scenes. However, when directly applying 3DGS to a large scene, it needs to load millions of Gaussian units of the entire scene into the video memory for rendering at one time, which puts extremely high requirements on the video memory capacity of the graphics processing unit (GPU). The recently proposed DoGaussian scheme attempts to solve this problem by distributed computing, which partitions the scene and collaboratively trains a global model on multiple computing nodes (GPUs). Although this scheme can handle large scene data, it relies on expensive multi-GPU hardware platforms, greatly limiting its application universality in ordinary hardware environments.

[0005] In summary, although the NeRF-based method can handle large scenes after improvement, it cannot balance training efficiency and real-time rendering capability due to its underlying principles; while the 3DGS-based method can achieve real-time rendering, but it faces extremely high hardware resource thresholds when processing large scenes. SUMMARY

[0006] In view of the real-time rendering of large scenes under limited computing power in the prior art, a three-dimensional Gaussian-based large scene three-dimensional reconstruction method is provided. By designing an adaptive scene blocking and visibility selection mechanism driven by an MLP, the demand for hardware resources for large scene model training is significantly reduced while ensuring rendering quality, thereby realizing efficient training and real-time rendering of large scene data under the condition of limited computing power of a single GPU, and effectively solving the core contradiction between training efficiency and real-time rendering capability.

[0007] In order to achieve the above technical purpose, the following technical solutions are specifically adopted in the present application: In one aspect of the present application, a three-dimensional Gaussian-based large scene three-dimensional reconstruction method is provided, comprising the following steps: Step S1: obtaining an initial scene point cloud of an original input image, and dividing the entire three-dimensional scene into a plurality of adjacent cubic blocks according to the spatial coordinate range of the initial scene point cloud; Step S2: for a given coordinate and viewing angle of a target observation point, the center coordinates of each cubic block, the coordinate and viewing angle of the target observation point are respectively frequency-encoded, and the encoded feature vectors are input into a pre-trained multi-layer perception MLP; the multi-layer perception MLP outputs a visibility probability value of each cubic block under the target observation point; Step S3: determining the cube block whose visibility probability value exceeds the preset threshold as a visible block, and loading only the three-dimensional Gaussian model parameters defined in the visible block to a graphics processing unit (GPU) for rendering to generate a view corresponding to the target observation point.

[0008] In an embodiment, the scene division step specifically comprises: determining axial boundary points (x min ,y min ,z min ) and (x max ,y max ,z max ) of the scene according to the initial scene point cloud; calculating max(x max -x min ,y max -y min ,z max -z min ) as a reference length, and dividing the reference length by 128 to obtain the edge length r of the cube block; dividing the scene space into multiple cube blocks with (x min ,y min ,z min ) as the starting point and the edge length r as the step length.

[0009] In an embodiment, the function form of the frequency encoding is:

[0010] wherein, p is the input coordinate value or angle value, L is the order of frequency encoding.

[0011] In an embodiment, when frequency encoding the center coordinates of the cube block and the coordinates of the target observation point L 10 is taken, and when encoding the view angle L 6 is taken.

[0012] In an embodiment, the multi-layer perception machine (MLP) is obtained through the following training process: obtaining a training image set of a target scene and calculating the corresponding camera pose and point cloud by using COLMAP; for each training image, frequency encoding the corresponding camera viewpoint coordinates, view angle direction, and center coordinates of each cube block and inputting them into the multi-layer perception machine (MLP) to obtain the visibility prediction value of each cube block; based on the prediction value, screening the visible block for rendering to obtain a predicted image; Calculate the rendering loss between the predicted image and the real training image, and calculate the block selection loss to penalize incorrect block selection; The parameters of the multilayer perceptron (MLP) and the 3D Gaussian model are updated by combining the rendering loss and the block selection loss through backpropagation.

[0013] In one implementation, the block selection loss is calculated as follows:

[0014] in, This represents the proportion of Gaussian units that did not contribute to the rendering process out of the total number of Gaussian units loaded in this rendering. This represents 1 minus the proportion of the number of Gaussian units loaded this time to the estimated total number of Gaussian units in the scene. This represents hyperparameters.

[0015] In one implementation, when backpropagation flows through the threshold determination step for filtering blocks, a pass-through estimator method is used, which approximates the gradient of the step function using the gradient of the sigmoid function: .

[0016] In another aspect of this application, a large-scene 3D reconstruction device based on 3D Gaussian is provided, comprising: The block segmentation module is used to divide the entire 3D scene into several adjacent cube blocks based on the spatial coordinate range of the initial scene point cloud of the original input image. The encoding and judgment module is used to perform frequency encoding on the center coordinates of each cube block, the coordinates of the target observation point, and the viewpoint for a given target observation point, and input the encoded feature vector into a pre-trained multilayer perceptron (MLP); the MLP outputs the visibility probability value of each cube block under the target observation point, and determines the cube block whose visibility probability value exceeds a preset threshold as a visible block; The rendering module is used to load the three-dimensional Gaussian model parameters defined in the visible block into the graphics processing unit (GPU) for rendering, and generate the view corresponding to the target observation point. The training optimization module is used to update the model parameters of the multilayer perceptron (MLP) during the training phase by performing backpropagation using the rendering loss function and the block selection loss function.

[0017] In another aspect of this application, an electronic device is provided, including a processor and a memory storing a computer program, which, when executed by the processor, performs the large-scene 3D reconstruction method based on 3D Gaussian described above.

[0018] In another aspect of this application, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, performs the aforementioned method for large-scene 3D reconstruction based on 3D Gaussian.

[0019] The beneficial effects of this application are as follows: Significantly improved training and rendering efficiency: By intelligently selecting visible blocks through MLP, only a portion of the scene data is loaded to the GPU, greatly reducing memory usage and computational load. This enables rapid training of large-scene models within 1-3 hours and achieves a real-time rendering frame rate of over 30 frames per second.

[0020] The hardware threshold has been significantly reduced: the innovative adaptive segmentation and selection mechanism enables high-quality large-scene 3D reconstruction and real-time rendering to be completed on a single consumer-grade graphics card with 24GB of video memory without relying on expensive multi-GPU distributed computing platforms, greatly improving the practicality and universality of the method.

[0021] Rendering quality is guaranteed: Combining the rendering advantages of 3D Gaussian sputtering with a dedicated loss function for block selection, this application significantly improves efficiency while achieving visual quality comparable to or better than existing advanced methods in objective indicators such as PSNR and SSIM, effectively balancing the core contradiction between efficiency and quality. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the large-scene 3D reconstruction method according to an embodiment of this application; Figure 2 These are some experimental renderings of embodiments of this application. Detailed Implementation

[0023] The technical solution of this application will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art will understand that the embodiments described below are only some embodiments of this application, not all embodiments, and are only used to illustrate this application, and should not be regarded as limiting the scope of this application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In one embodiment of this application, a method for large-scene 3D reconstruction based on 3D Gaussian is provided, referring to... Figure 1 As shown, it includes the following steps: Step S1: Obtain the initial scene point cloud of the original input image. Based on the spatial coordinate range of the initial scene point cloud, divide the entire 3D scene into several adjacent cube blocks.

[0025] Specifically, it includes the following steps: 1) Determine the axial boundary points (x, y, y) of the scene based on the initial scene point cloud. min ,y min ,z min ) and (x max ,y max ,z max ).

[0026] The initial scene point cloud was obtained from the original input image using tools such as COLMAP (SfM). The minimum and maximum values ​​of all points in the initial scene point cloud on the 3D coordinate axes were obtained, and the axial boundary points of the scene were defined, denoted as (x...). min ,y min ,z min ) and (x max ,y max ,z max This forms an axially aligned bounding box that completely surrounds all point cloud data.

[0027] 2) Calculate max(x) max -x min ,y max -y min ,z max -z min ), which is used as the reference length. Dividing the reference length by 128 gives the side length r of the cube block.

[0028] Calculate the length of the bounding box in the three-dimensional coordinates X, Y, Z directions, i.e., (x max -x min ), (y max -y min ), (z max -z min The maximum value among the three lengths is taken as the baseline length of the scene, and this baseline length is divided by a predetermined number of divisions. In this embodiment, the number of pollen is 128, thereby obtaining the side length r of each cube block.

[0029] 3) with (x) min ,y min ,z min Starting from 0 and with side length r as the step size, the scene space is divided into multiple cube blocks.

[0030] With boundary point (x min ,y min ,z minStarting from the x, y, and z directions, and using the calculated side length r of the cube blocks as the step size, cube blocks are sequentially filled to uniformly divide the scene into a maximum of 128×128×128 grid units. Each grid unit is a cube block with a side length r. These cube blocks are adjacent to each other and seamlessly constitute a discretized representation of the entire scene.

[0031] Step S2: For a given target observation point coordinates and viewing angle, the center coordinates of each cube block, the coordinates of the target observation point, and the viewing angle are frequency encoded respectively, and the encoded feature vectors are input into a pre-trained multilayer perceptron (MLP); the MLP outputs the visibility probability value of each cube block under the target observation point.

[0032] In order for the Multilayer Perceptron (MLP) to effectively learn and understand the implicit information of three-dimensional spatial position and viewing direction, this embodiment performs frequency encoding on the following three inputs: the center coordinates p of each cube block. i =(x i ,y i ,z i The coordinates of the target observation point, p. v =(x v ,y v ,z v ), the viewpoint of the target observation point v=(θ) v ,φ v ), where θ v φ is the counterclockwise rotation angle with the positive x-axis as 0°. v Let p be the angle between the line of sight about the 0° plane formed by the X and Y axes and the angle of elevation about the 0° plane. i p v Substituting and v into the following formula for frequency encoding:

[0033] in, p For input coordinate or angle values; L The order of frequency encoding determines the dimensionality of the encoded features and the highest frequency details that can be represented. When encoding the center coordinates of the cube block and the coordinates of the target observation point, L When encoding the viewpoint direction, the value is 10. L It is 6.

[0034] After encoding, three feature tensors are obtained with dimensions (1,60), (1,60), and (1,24), respectively. The three feature tensors are stacked along their feature dimensions to form a comprehensive feature vector vec with dimension (1,144). The feature vector vec is used as the input of a multilayer perceptron (MLP). After forward computation, a scalar value in the interval [0,1] is output for each cube block as the visibility probability value of the block at a given target observation point.

[0035] In some embodiments, the multilayer perceptron (MLP) is trained through the following supervised learning process: a. Obtain the training image set of the target scene, as well as the corresponding camera pose, i.e., the camera viewpoint coordinates and viewing direction when each training image was captured; b. For each training image, the corresponding camera viewpoint coordinates, viewing direction, and center coordinates of each cube block in the scene are frequency encoded and input into the multilayer perceptron. The frequency encoding method is the same as above. c. The multilayer perceptron (MLP) outputs the visibility prediction values ​​of all cube blocks, and determines the cube blocks with prediction values ​​greater than or equal to λ as visible based on a preset threshold λ. d. Load the 3D Gaussian ellipsoid (point cloud) inside the visible cube block into the GPU, perform 3D Gaussian sputtering rendering, and obtain the predicted image from this viewpoint.

[0036] The training objective is guided by a composite loss function, which comprises two parts: rendering loss and block selection loss. The total loss is a weighted sum of the rendering loss and the block selection loss. The rendering loss is used to calculate the difference between the predicted image and the real training image. This rendering loss function employs the loss from a standard 3D Gaussian sputtering model, such as a combination of L1 loss and SSIM loss. The block selection loss directly guides the Multilayer Perceptron (MLP) to make correct block selection decisions, and its calculation method is as follows:

[0037] in, ; n 1 represents the number of Gaussian units with a radius of 0 after being mapped to the imaging plane, indicating that they are loaded into the GPU in this rendering but ultimately do not contribute to the imaging. n slice This indicates the total number of Gaussian units loaded into the GPU this time.

[0038] ; n This represents the total number of point clouds in the scene. Hyperparameters are used for balancing. and The weights between them.

[0039] In some embodiments, after calculating the total loss, the parameters of the multilayer perceptron (MLP) model and the three-dimensional Gaussian model itself are updated using the backpropagation algorithm, such as the position, color, and transparency of the Gaussian units.

[0040] In forward propagation, the cube block selection based on the threshold λ is a non-differentiable step function that blocks gradient flow. To address this issue, this embodiment employs the Straight-Through Estimator (STE) method, which approximates the gradient of the step function using the gradient of the sigmoid function σ during this step in backpropagation.

[0041] Specifically, a portion of the Gaussian units from the output of the Multilayer Perceptron (MLP) is extracted and fed into the GPU for rendering. This is equivalent to setting the opacity of the Gaussian units not yet passed in to 0 before inputting them. ; in, , m This is the output of the MLP.

[0042] According to the strain through esitator method, the gradient of the sigmoid function is used instead. The gradient. Then, during backpropagation, the gradient is obtained. Then the gradient of the total loss with respect to the MLP output can be obtained:

[0043] .

[0044] Step S3: Determine the cube blocks whose visibility probability value exceeds a preset threshold as visible blocks, and load only the three-dimensional Gaussian model parameters defined within the visible blocks into the graphics processing unit (GPU) for rendering to generate the view corresponding to the target observation point.

[0045] After obtaining the visibility probability value (a scalar between 0 and 1) output by the multilayer perceptron (MLP) for each cube block, it needs to be binarized according to a preset threshold λ to finally determine the visibility state of the cube block, such as visible or invisible.

[0046] From the entire scene's 3D Gaussian model, only the model parameters of the 3D Gaussian ellipsoids contained within the visible blocks are selected. These parameters typically include, but are not limited to: the 3D position of the Gaussian units, covariance (used to define the ellipsoid's shape and orientation), color, and opacity. Subsequently, the graphics processing unit (GPU) rendering pipeline loads only this small subset of selected model parameters and executes the 3D Gaussian sputtering rendering algorithm. This algorithm efficiently projects each Gaussian ellipsoid onto a 2D imaging plane and rasterizes it, then synthesizes it using alpha blending calculations to generate the final view (i.e., image) corresponding to the target observation point.

[0047] To verify the effectiveness and advancement of the proposed method, a thorough experimental evaluation was conducted on a series of standard datasets, and a systematic comparison was made with current mainstream methods.

[0048] As shown in Table 1, the proposed method was tested on three publicly available datasets: Rubble, Trans-America, and Sci-Art. Evaluation metrics included PSNR (measuring image fidelity), SSIM (structural similarity), and LPIPS (perceptual quality). Experimental data show that the proposed method outperforms the compared methods in most scenarios, especially on the Rubble and Trans-America datasets, demonstrating its superior view synthesis quality. Furthermore, the proposed method requires only 1 to 3 hours to train the entire large-scene model and can maintain a stable frame rate above 30 frames per second during rendering, achieving true real-time synthesis. This efficient training and rendering process relies solely on a single NVIDIA GeForce RTX 3090 graphics card with 24GB of VRAM, demonstrating the method's friendliness and practicality in terms of hardware resource requirements.

[0049] Table 1

[0050] Reference Figure 2 The image shows a portion of the experimental renderings. This figure illustrates the rendering results on three datasets: Rubble, Trans-America, and Sci-Art. Each dataset occupies two rows, with the "ground truth" column representing the original real image and the "render" column representing the corresponding view generated by this method. Visual comparison reveals that, aside from minor differences caused by inherent characteristics of the training data (such as dynamic objects briefly appearing from different viewpoints) or inherent flaws in the datasets (such as the fixed watermark in the Trans-America dataset), the method in this application can reconstruct and render most of the scene content with high quality. The generated images closely match the real images in both detail and overall appearance.

[0051] In another aspect of this application, a large-scene 3D reconstruction device based on 3D Gaussian is provided, including a block segmentation module, an encoding and judgment module, a rendering module, and a training and optimization module.

[0052] The segmentation module is connected to the data preprocessing unit (not shown in the figure, such as a subsystem running COLMAP), receives the initial scene point cloud data output by the preprocessing unit, and its output is connected to the input of the judgment module. The segmentation module is used to divide the entire 3D scene into several adjacent cubic blocks based on the spatial coordinate range of the initial scene point cloud of the original input image.

[0053] Specifically, the segmentation module parses the three-dimensional coordinates of all points in the initial scene point cloud and calculates the axial boundary points (x, y, y) of the scene. min ,y min ,z min ) and (x max ,y max ,z max The process calculates the lengths of the bounding box in the X, Y, and Z directions based on the axial boundary points, and takes the maximum value as the reference length. This reference length is then divided by a predetermined number of divisions, 128, to obtain the side length *r* of each regular cube block. Using *r* as a fixed step size, the cubes are filled along the three coordinate axes until the entire bounding box is covered, dividing the scene into a maximum of 128×128×128 tightly packed cube blocks, and outputting a structured set of scene blocks.

[0054] The encoding and judgment module is used to perform frequency encoding on the center coordinates of each cube block, the coordinates of the target observation point, and the viewpoint for a given target observation point, and input the encoded feature vector into a pre-trained multilayer perceptron (MLP).

[0055] Specifically, the encoding and judgment module receives the center coordinates of each cube block generated by the block segmentation module, as well as the coordinates and viewing direction of the target observation point specified by the user. An integrated frequency encoder encodes the received block center coordinates, target observation point coordinates, and observation point viewing direction. These three encoded feature tensors are concatenated along their feature dimensions to form a comprehensive feature vector. This vector is then input to a pre-trained multilayer perceptron (MLP). The MLP outputs a visibility probability value between 0 and 1 for each block. The MLP output value is compared to a preset threshold λ; blocks with visibility probability values ​​greater than or equal to λ are classified as visible blocks, and a list of visible blocks is output.

[0056] The rendering module is used to load the three-dimensional Gaussian model parameters defined within the visible block into the graphics processing unit (GPU) for rendering, thereby generating a view corresponding to the target observation point.

[0057] Specifically, the rendering module receives the list of visible blocks and the complete 3D Gaussian model parameters stored in memory. Based on the list of visible blocks, it selectively extracts all point cloud (Gaussian ellipsoid) parameters located within the visible blocks from the complete 3D Gaussian model, including position, covariance, color, and opacity. The selected Gaussian model parameters are loaded into the video memory of the graphics processing unit (GPU). The 3D Gaussian sputtering rendering pipeline is invoked on the GPU to rasterize and blend the loaded visible point clouds, ultimately synthesizing and outputting the view corresponding to the target observation point, thereby producing a high-quality rendered image.

[0058] The training optimization module is used to perform backpropagation using a rendering loss function and a block selection loss function during the training phase, and to update the model parameters of the multilayer perceptron (MLP) and the three-dimensional Gaussian model parameters.

[0059] Specifically, the training optimization module receives a training image set of the target scene and its corresponding camera poses (coordinates and viewpoints). For each training sample, it simulates the workflow of the encoding and judgment module and the rendering module to generate a predicted image. It calculates the total loss function, a weighted sum of rendering loss and block selection loss, and uses the backpropagation algorithm to calculate the gradient of the total loss with respect to the MLP parameters and the 3D Gaussian model parameters. When the gradient flows through the non-differentiable threshold judgment step, it is approximated by the gradient of the Sigmoid function, thereby achieving effective updates to the MLP parameters and the 3D Gaussian model parameters. The output is the trained, parameter-optimized MLP model and the 3D Gaussian model parameters, which can be integrated into the encoding and judgment module for use.

[0060] Although the embodiments of this application have been described above in conjunction with the accompanying drawings, this application is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of this application, and these are all within the scope of protection of this application.

Claims

1. A method for large-scene 3D reconstruction based on 3D Gaussian, characterized in that, include: S1: Obtain the initial scene point cloud of the original input image, and divide the entire 3D scene into several adjacent cube blocks according to the spatial coordinate range of the initial scene point cloud. S2: For a given target observation point coordinates and viewpoint, the center coordinates of each cube block, the coordinates of the target observation point and the viewpoint are frequency encoded respectively, and the encoded feature vectors are input into a pre-trained multilayer perceptron (MLP); the multilayer perceptron (MLP) outputs the visibility probability value of each cube block under the target observation point. S3: The cube blocks whose visibility probability value exceeds the preset threshold are determined as visible blocks, and only the three-dimensional Gaussian model parameters defined in the visible blocks are loaded into the graphics processor GPU for rendering to generate the view corresponding to the target observation point.

2. The method for large-scene 3D reconstruction based on 3D Gaussian as described in claim 1, characterized in that, The scene division includes: Determine the axial boundary points (x) of the scene based on the initial scene point cloud. min ,y min ,z min ) and (x max ,y max ,z max ); Calculate max(x) max -x min ,y max -y min ,z max -z min ), which is used as the reference length. Divide the reference length by 128 to obtain the side length r of the cube block; With (x min ,y min ,z min Starting from 0 and with side length r as the step size, the scene space is divided into multiple cube blocks.

3. The method for large-scene 3D reconstruction based on 3D Gaussian as described in claim 1, characterized in that, The frequency coding function is in the form of: in, p For the input coordinate or angle values, L The order of the frequency coding.

4. The method for large-scene 3D reconstruction based on 3D Gaussian as described in claim 3, characterized in that, When performing frequency encoding on the center coordinates of the cube block and the coordinates of the target observation point L When encoding the viewpoint, select 10. L Take 6.

5. The method for large-scene 3D reconstruction based on 3D Gaussian as described in claim 1, characterized in that, The Multilayer Perceptron (MLP) is obtained through the following training process: Obtain the training image set of the target scene and use COLMAP to calculate its corresponding camera pose and point cloud; For each training image, the corresponding camera viewpoint coordinates, viewing direction, and center coordinates of each cube block are frequency encoded and input into the multilayer perceptron (MLP) to obtain the visibility prediction value of each cube block; based on this prediction value, visible blocks are selected for rendering to obtain the predicted image. Calculate the rendering loss between the predicted image and the real training image, and calculate the block selection loss to penalize incorrect block selection; The parameters of the multilayer perceptron (MLP) and the 3D Gaussian model are updated by combining the rendering loss and the block selection loss through backpropagation.

6. The method for large-scene 3D reconstruction based on 3D Gaussian as described in claim 5, characterized in that, The block selection loss is calculated as follows: in, This represents the proportion of Gaussian units that did not contribute to the rendering process out of the total number of Gaussian units loaded in this rendering. This represents 1 minus the proportion of the number of Gaussian units loaded this time to the estimated total number of Gaussian units in the scene. This represents hyperparameters.

7. The method for large-scene 3D reconstruction based on 3D Gaussian as described in claim 5, characterized in that, When backpropagating through the threshold judgment step used to filter blocks, a pass-through estimator method is used, which approximates the gradient of the step function with the gradient of the sigmoid function: 。 8. A large-scene 3D reconstruction device based on 3D Gaussian, comprising: The block segmentation module is used to divide the entire 3D scene into several adjacent cube blocks based on the spatial coordinate range of the initial scene point cloud of the original input image. The encoding and judgment module is used to perform frequency encoding on the center coordinates of each cube block, the coordinates of the target observation point, and the viewpoint for a given target observation point, and input the encoded feature vector into a pre-trained multilayer perceptron (MLP); the MLP outputs the visibility probability value of each cube block under the target observation point, and determines the cube block whose visibility probability value exceeds a preset threshold as a visible block; The rendering module is used to load the three-dimensional Gaussian model parameters defined in the visible block into the graphics processing unit (GPU) for rendering, and generate the view corresponding to the target observation point. The training optimization module is used to update the model parameters of the multilayer perceptron (MLP) during the training phase by performing backpropagation using the rendering loss function and the block selection loss function.

9. An electronic device comprising a processor and a memory storing a computer program, said computer program being executed by the processor, characterized in that, The processor executes the large-scene 3D reconstruction method based on 3D Gaussian as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is run by a processor, the processor performs the large-scene 3D reconstruction method based on 3D Gaussian as described in any one of claims 1 to 7.

Images