A sparse view three-dimensional gaussian reconstruction method based on context recovery

Abstract

The application discloses a sparse view three-dimensional Gaussian reconstruction method based on context recovery, comprising the following steps: S1, acquiring multi-view input images and performing pretreatment; S2, generating an initial color object point cloud with geometric consistency based on the pretreated images; S3, performing point cloud refinement and density control on the generated point cloud by using a method guided by structure prior; S4, converting the refined point cloud into an anisotropic three-dimensional Gaussian representation; S5, introducing ICRM based on the obtained three-dimensional Gaussian representation, capturing the visual correlation across views through three-dimensional appearance consistency learning based on cross-validation, and improving the rendering quality decline caused by the internal limitations of the three-dimensional Gaussian model optimization process; and S6, further training the three-dimensional Gaussian model by using an iterative training method with controlled view perturbation, thereby guaranteeing the consistency of the reconstruction result and improving the integrity, and finally realizing high structural consistency and visual fidelity in the three-dimensional Gaussian reconstruction.

Description

Technical Field

[0001] This application relates to the fields of computer vision and 3D reconstruction technology, specifically to a sparse-view 3D Gaussian reconstruction method based on context recovery. Background Technology

[0002] With the development of applications such as digital content production, augmented reality, virtual reality, and e-commerce 3D display, reconstructing high-quality 3D models from multi-view images has become a key technical challenge. Traditional photogrammetry methods rely on dense viewpoint input, recovering 3D structures through structured bundle adjustment (SfM) and multi-view stereo matching (MVS). However, these methods struggle to guarantee reconstruction quality when there are only a few viewpoints. In recent years, 3D Gaussian splatting has achieved efficient real-time rendering and continuous geometric representation by representing scenes using an anisotropic 3D Gaussian distribution. However, this method is prone to optimization collapse under sparse viewpoint conditions, resulting in floating artifacts, surface fractures, and severe overfitting. Therefore, further research is needed. Summary of the Invention

[0003] The purpose of this application is to provide a sparse-view 3D Gaussian reconstruction method based on context recovery, and the specific technical solution is as follows:

[0004] A sparse-view 3D Gaussian reconstruction method based on context recovery includes: S1, acquiring and preprocessing multi-view input images; S2, generating an initial colored object point cloud with geometric consistency based on the preprocessed image in S1; S3, refining and controlling the density of the point cloud generated in S2 using a structure prior-guided method; S4, converting the refined point cloud in S3 into anisotropic 3D Gaussian representation; S5, introducing ICRM based on the 3D Gaussian representation obtained in S4, capturing cross-view visual correlations through 3D appearance consistency learning based on cross-validation, thereby improving the rendering quality degradation caused by the inherent limitations of the 3D Gaussian model optimization process; S6, further training the 3D Gaussian model based on S5 using an iterative training method with controlled viewpoint perturbation, improving integrity while ensuring the consistency of the reconstruction results, and ultimately achieving high structural consistency and visual fidelity in 3D Gaussian reconstruction.

[0005] The preprocessing in S1 includes: S1.1, using a segmentation model to segment the foreground of the N input images and separate the main subject from the complex background; S1.2, performing scale normalization on the main subject separated in S1.1, uniformly scaling and centering the main subject to a fixed resolution canvas.

[0006] The initial point cloud generation in S2 includes: S2.1, using the MASt3R model to simultaneously predict the camera pose, monocular depth, and pixel-aligned point map; S2.2, integrating the point map, pose matrix, and scale factor predicted in S2.1 into a unified global coordinate system to obtain an initial colored object point cloud with geometric consistency.

[0007] The point cloud refinement and density control in S3 includes: S3.1, constructing visual shell constraints; S3.2, calculating the Euclidean distance transformation of the foreground mask based on the visual shell constraints constructed in S3.1; S3.3, defining a relaxation mask based on the Euclidean distance transformation in S3.2; S3.4, filtering the point cloud obtained in S2.2 using the relaxation mask defined in S3.3 to remove outliers whose projections fall outside the relaxation mask; S3.5, applying small perturbations along the six directions of x, y, and z axes at the corresponding 3D positions in discontinuous or sparse regions within the relaxation mask area to generate additional local points, thereby enhancing the point cloud density and surface continuity.

[0008] When capturing cross-viewpoint visual relevance in S5, the steps include: S5.1, constructing degraded-real image pairs for training ICRM using a leave-one-out strategy; S5.2, based on the degraded-real image pairs constructed in S5.1, capturing viewpoint-related correlation information through the context recovery model, and reasonably completing the missing geometric cues in single-viewpoint observations.

[0009] The process of constructing the degraded-real image pairs for training ICRM in S5.1 includes: 5.11 Dividing the input set of N real images into N subsets, each subset containing N-1 reference images and one real image as the preserved viewpoint; 5.12 Using the initial point cloud obtained in S2.2, training N coarse 3D Gaussian models on the corresponding subsets; 5.13 Performing minimization Gaussian splitting and pruning operations on the coarse 3D Gaussian models trained in S5.12 to obtain refined coarse 3D Gaussian models; 5.14 Based on the refined coarse 3D Gaussian models obtained in S5.13, for each model, pairing the degraded rendered image from the preserved viewpoint with its corresponding real image to form a degraded-real image pair for training ICRM.

[0010] The training of the context restoration model in S5.2 includes: S5.21, inputting the viewpoint-preserving degraded rendered image obtained in 5.14 and its corresponding real image into the diffusion image generation model; S5.22, during training, using the degraded image as the input image to be restored and the corresponding real image as the supervised target image; S5.23, finally obtaining a context restoration module that can use reference view information to perform structural correction and texture completion on the degraded image.

[0011] S6 employs an iterative training method with controlled viewpoint perturbation to further train the 3D Gaussian model, including: S6.1 Constructing a spherical trajectory around the subject and uniformly arranging virtual cameras on this sphere. Applying small perturbations to the pitch and azimuth angles of the reference camera pose, sampling new viewpoints within its neighborhood to ensure that the newly generated viewpoints remain within a controllable angle range; S6.2 Rendering the degraded image of the perturbated viewpoint generated in S6.1; S6.3 Restoring the degraded image generated in S6.2 using ICRM to obtain a pseudo-realistic image; S6.4 Updating the 3D Gaussian parameters using the pseudo-realistic image obtained in S6.3; S6.5 Repeating steps S6.1-S6.4, gradually expanding the perturbation range along the spherical trajectory to cover all virtual camera viewpoints; S6.6 Finally, obtaining a highly consistent 3D Gaussian model that can be rendered in real time.

[0012] The beneficial effects of this application are as follows: by combining the visual shell and density control mechanism with the context recovery mechanism, the true structure of the object is maintained; the context recovery mechanism significantly improves texture quality and object details; a leave-one-out strategy is used to establish a self-supervised closed loop, effectively alleviating the identity drift problem; high resolution, consistent identity, and continuous structure are achieved in 3D reconstruction; the entire training process can be completed in about 40 minutes, which has high engineering feasibility. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the application process.

[0014] Figure 2 This is a schematic diagram illustrating the use of this method to generate a real-time renderable 3D Gaussian model in a specific application of this application.

[0015] Figure 3 This is a schematic diagram of the comparative experiment in this application. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this application. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0017] like Figure 1 As shown, a sparse-view 3D Gaussian reconstruction method based on context recovery includes:

[0018] S1. Acquire multi-view input images and perform preprocessing. Specifically, the preprocessing includes: S1.1. Using a segmentation model to perform foreground segmentation on the N input images, separating the main subject from the complex background; S1.2. Performing scale normalization on the main subject separated in S1.1, uniformly scaling and centering the main subject to a fixed resolution canvas.

[0019] In practical applications, segmentation models such as Segment Anything (SAM) are used to separate the main object from the complex background. Scale normalization is then performed to uniformly scale the object and center it on a 1024*768 canvas. This process improves the stability of the MASt3R camera pose estimation, ensures consistent geometric constraints in the subsequent visual shell stage, and guarantees compatibility with the input resolution required by Kontext.

[0020] S2. Based on the preprocessed image from S1, generate an initial colored object point cloud with geometric consistency. Specifically, generating the initial point cloud includes: S2.1, using the MASt3R model to simultaneously predict camera pose, monocular depth, and pixel-aligned point maps; S2.2, integrating the predicted point maps, pose matrix, and scale factor from S2.1 into a unified global coordinate system to obtain an initial colored object point cloud with geometric consistency.

[0021] In practical applications, MASt3R is used as the implementation model for dense geometric priors, which can simultaneously predict camera pose, monocular depth, and pixel-aligned point maps. MASt3R was chosen because its multi-view inference mechanism based on a cross-view Transformer allows for reliable stereo point map regression even in the presence of foreground masks. Following its processing flow, the predicted point maps, transformation matrices, and scale factors are integrated into a unified global coordinate system, thereby obtaining an initial colored object point cloud with strong geometric consistency.

[0022] S3. The point cloud generated in S2 is refined and its density controlled using a structure-prior-guided method. Specifically, the point cloud refinement and density control include: S3.1. Constructing a visual shell constraint; S3.2. Calculating the Euclidean distance transform of the foreground mask based on the visual shell constraint constructed in S3.1; S3.3. Defining a relaxation mask based on the Euclidean distance transform in S3.2; S3.4. Filtering the point cloud obtained in S2.2 using the relaxation mask defined in S3.3 to remove outliers whose projections fall outside the relaxation mask; S3.5. Applying small perturbations along the x, y, and z axes in the corresponding 3D positions of discontinuous or sparse regions within the relaxation mask area to generate additional local points, thereby enhancing the point cloud density and surface continuity.

[0023] In practical applications, a point cloud refinement and density control method guided by structural priors is proposed. To address the outlier problem, a visual hull constraint is introduced. Let the 3D object (the main object) be defined from a reference viewpoint set. Observed in, where each viewpoint They all have contours obtained from their masks. Its internal pixels represent the area covered by the object. For each A cone-shaped volume can be formed by projecting rays from the center of the camera along all the directions of its internal pixels. Real objects necessarily contain It lies within, and therefore at the intersection of all such volumes, that is:

[0024] .

[0025] In actual observation, it was found that the point cloud region generated by MASt3R... Near the boundary of an object, it often does not fall strictly on the ground. Inside. Therefore, the mask is moderately relaxed to tolerate small-range projection errors. Let... Indicates perspective The set of foreground pixels in the image. Calculate the Euclidean distance transformation map. Each pixel The value of represents its distance to the nearest foreground pixel, specifically expressed as:

[0026] .

[0027] Based on this distance map, a relaxation mask is defined. Distance threshold The pixels within a certain range are included in the foreground region, expressed as:

[0028] .

[0029] This strategy allows the mask to be moderately extended near the object boundaries, thus tolerating small errors in the point cloud projection. Subsequently, we utilize... Point cloud A filtering process is performed, retaining only points whose projections fall within the relaxed mask. This process removes outliers while preserving valid points near the contour boundaries, thus improving the stability and integrity of the coarse 3D Gaussian scene representation. Within the masked region, we further perform adaptive encryption based on local depth consistency in 3D space. When the depth difference between a pixel and its neighboring pixels exceeds a preset threshold, the region is identified as a potential geometric discontinuity or sparse region. Small perturbations are applied along the six directions of the x, y, and z axes at the corresponding 3D location to generate additional local points, thereby enhancing point cloud density and surface continuity.

[0030] S4. Convert the refined point cloud from S3 into anisotropic 3D Gaussian representation.

[0031] S5. Based on the 3D Gaussian representation obtained in S4, ICRM is introduced. Through 3D appearance consistency learning based on cross-validation, cross-view visual correlations are captured to improve the rendering quality degradation caused by the inherent limitations of the 3D Gaussian model optimization process. Specifically, capturing cross-view visual correlations includes: S5.1, constructing degenerate-real image pairs for training ICRM using a leave-one-out strategy; S5.2, based on the degenerate-real image pairs constructed in S5.1, capturing view-related correlation information through the context recovery model and reasonably completing the missing geometric cues in single-view observations. The process of constructing the degraded-real image pairs for training ICRM includes: 5.11. Dividing the input set of N real images into N subsets, each subset containing N-1 reference images and one real image as the preserved viewpoint; 5.12. Using the initial point cloud obtained in S2.2, training N coarse 3D Gaussian models on the corresponding subsets; 5.13. Performing minimization Gaussian splitting and pruning operations on the coarse 3D Gaussian models trained in S5.12 to obtain refined coarse 3D Gaussian models; 5.14. Based on the refined coarse 3D Gaussian models obtained in S5.13, for each model, pairing the degraded rendered image from the preserved viewpoint with its corresponding real image to form a degraded-real image pair for training ICRM. The training of the context restoration model in S5.2 includes: S5.21, inputting the viewpoint-preserving degraded rendered image obtained in 5.14 and its corresponding real image into the diffusion image generation model; S5.22, during training, using the degraded image as the input image to be restored and the corresponding real image as the supervised target image; S5.23, finally obtaining a context restoration module that can use reference view information to perform structural correction and texture completion on the degraded image.

[0032] In practical applications, we will first explain how the model input data is constructed. Following the leave-one-out strategy, the input set containing N images is divided into N subsets, each subset... It contains N-1 reference images and one reserved image. Using the initial point cloud obtained in S2.2, in the corresponding Train N coarse 3D Gaussian models on a subset To avoid overfitting, for Only minimal Gaussian splitting or pruning operations are performed, relying primarily on the prior-corrected point cloud to achieve fast geometric convergence. After a few training iterations, a refined coarse 3D Gaussian model is obtained. In this process, a Gaussian distribution outside the visual shell is humorously cropped according to the object mask to prevent excessive surface expansion and provide a clearer structural representation for understanding ICRM. For each model, a degraded render image from the preserved viewpoint is used. Its corresponding real image Pairing to form control-target image pairs The ICRM was trained using a FLUX.1 Kontext network. Subsequently, FLUX.1 Kontext was used as the recovery backbone network. This model has built-in LoRA support, enabling lightweight fine-tuning of the degradation rendering domain while keeping the core diffusion parameters constant. During the inference phase... Encoded as a latent representation and injected into a diffuse backbone network, it guides the denoising process to generate perceptually consistent representations. In this method, FLUX.1 Kontext acts not only as an image enhancer but also as a cross-viewpoint structural refiner. By operating in the latent space, the model can capture viewpoint-related correlation information and reasonably complete the geometric cues missing in single-view observations. This makes the generated pseudo-GT output superior to traditional image restoration models in terms of both 3D consistency and perceptual realism.

[0033] S6. Based on S5, the three-dimensional Gaussian model is further trained using an iterative training method with controlled viewpoint perturbation. This method improves the integrity of the reconstruction results while ensuring consistency, ultimately achieving high structural consistency and visual fidelity in the three-dimensional Gaussian reconstruction. Specifically, the iterative training method using controlled viewpoint perturbation to further train the 3D Gaussian model includes: S6.1, constructing a spherical trajectory around the main body, and uniformly arranging virtual cameras on the sphere, applying small perturbations to the pitch and azimuth angles of the reference camera pose, and sampling new viewpoints within its neighborhood to ensure that the newly generated viewpoints remain within a controllable angle range; S6.2, rendering the degraded image of the perturbated viewpoint generated in S6.1; S6.3, restoring the degraded image generated in S6.2 using ICRM to obtain a pseudo-real image; S6.4, updating the 3D Gaussian parameters using the pseudo-real image obtained in S6.3; S6.5, repeating steps S6.1-S6.4, gradually expanding the perturbation range along the spherical trajectory to cover all virtual camera viewpoints; S6.6, finally obtaining a highly consistent 3D Gaussian model that can be rendered in real time.

[0034] In practical applications, to ensure the consistency of object structure, we construct a spherical trajectory around the object and uniformly arrange a set of virtual cameras on this sphere. Then, we apply small perturbations to the pitch and azimuth angles of these reference cameras, prioritizing sampling new viewpoints within their neighborhoods. This process can be viewed as a geometrically constrained spherical jitter strategy, ensuring that all newly generated viewpoints are within a controlled angular range. The degenerate rendering results from these viewpoints are then input into the ICRM, and the generated pseudo-GT images are used to further train the 3D Gaussian model. In this way, we progressively expand the perturbation range along the spherical trajectory until it covers all virtual camera viewpoints. The entire process is iteratively expanded in a progressive manner, improving the integrity of the reconstruction results while maintaining consistency. Through this closed-loop "rendering-restoration-reconstruction" mechanism, our framework achieves high structural consistency and visual fidelity in the final 3D Gaussian reconstruction.

[0035] like Figure 2 As shown, when using this method to generate a real-time rendered 3D Gaussian model in a specific application, the following steps are taken: For object A, take four pictures with a mobile phone, approximately at the front, back, left, and right angles of the object.

[0036] First, the SAM model is used to segment the foreground of the four input images to separate the target object from the background.

[0037] The object region was then scaled and centered within a 1024×768 canvas. The multi-view geometric inference model MASt3R was used to perform joint inference on the four images, outputting: the camera extrinsic matrix for each image; and a monocular depth pixel-aligned 3D point map. These point maps were then unified into the global coordinate system, resulting in an initial colored 3D point cloud of 180,000 points.

[0038] To improve structural stability, a visual shell constraint was used to filter the point cloud. Specifically, a visual shell was constructed based on the foreground masks of four images; the Euclidean distance transformation of the mask was calculated; a relaxation mask was generated by setting a distance threshold δ=3 pixels; 3D points whose projections fall outside the relaxation mask were removed; after filtering, the number of point clouds was approximately 150,000. Adaptive density enhancement was then performed: when the depth difference between adjacent pixels was greater than 0.02 (normalized depth units), small perturbation points were added along the positive and negative x, y, and z axes at the corresponding 3D positions, with a perturbation amplitude of 5% of the original point spacing. The final number of point clouds was approximately 210,000.

[0039] The refined point cloud is converted into an initial 3D Gaussian representation, each Gaussian containing the following parameters: μ: 3D center coordinates, Σ: covariance matrix, α: opacity, c: color parameter.

[0040] For the four input images, construct four subsets:

[0041] Group 1: Use images 2, 3, and 4 to train a coarse model; image 1 is for preserving the viewpoint.

[0042] Group 2: Use images 1, 3, and 4 to train a coarse model, while image 2 preserves the viewpoint;

[0043] Group 3: Use images 1, 2, and 4 to train a coarse model; image 3 is for preserving the viewpoint.

[0044] Group 4: Use images 1, 2, and 3 to train a coarse model, and image 4 to preserve the viewpoint.

[0045] The coarse model training is based on the initial 3D Gaussian generated in step 4. Each group is trained 600 times, and finally, scene 1, scene 2, scene 3, and scene 4 are generated.

[0046] For each scene, render the corresponding viewpoint-preserving degraded image, and finally construct 4 training pairs: (degraded image i, real image i), i∈[1,2,3,4]

[0047] Kontext was used as the context recovery module, and the pre-trained FLUX.1 Kontext model was selected as the base model. The training data consisted of four training pairs: degraded images (input) and corresponding real images (supervised targets).

[0048] The training parameters are as follows: number of training steps: 1000 steps; fine-tuning method: update only the adaptation layer; batch size: 1; learning rate: 5e-5.

[0049] After training is complete, the context recovery module for object A is obtained.

[0050] Iterative training with controlled viewpoint perturbation involves setting virtual camera angles on the circumscribed sphere of the object, based on the initial four viewpoints.

[0051] Initial disturbance range: azimuth ±5°, elevation ±5°, gradually increasing the disturbance range to ±30°.

[0052] The process for each round is as follows:

[0053] 1) Render the perturbed viewpoint image using the current 3D Gaussian model;

[0054] 2) Input these perturbed viewpoint images into the context recovery module trained in step 6 to generate pseudo-realistic images;

[0055] 3) Add pseudo-realistic images and angles to the current 3D Gaussian model to update the 3D Gaussian model parameters;

[0056] 4) Each training round consists of 500 iterations;

[0057] A total of 120 pseudo-real images were generated during the entire iteration phase. The total number of training iterations was 28,000.

[0058] To make this application easier to understand, the following explanation is based on actual experiments.

[0059] Experiments were conducted on two benchmark datasets: OmniObject3D and Mip-NeRF 360. These datasets were chosen because they both provide 360° object images, suitable for 360° reconstruction tasks with sparse viewpoints. Each scene is represented by four images.

[0060] We compare the sparse-view 3D Gaussian reconstruction method based on context recovery (SOCon-GS) disclosed in this application with 3D Gaussian and existing few-shot novel view synthesis methods, including FSGS, GaussianObject, and CoR-GS. We reimplemented and reproduced the results of these baseline methods under the same experimental configuration. It should be noted that these methods use a maximum resolution of 779*520 when using the original image, instead of cropping the object from the original image and adapting it to 1024*768 as our method does. To ensure fairness, we also downsampled the results of our own method to 779*520 for comparison. The comparison results are summarized in Table 1. The reconstruction results are as follows: Figure 3 As shown.

[0061] Table 1

[0062]

[0063] We used three standard quantitative metrics to evaluate the results: PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index Measure), and LPIPS (Learned Perceptual Image Patch Similarity), which measure pixel-level fidelity, structural similarity, and perceptual realism, respectively. PSNR measures the pixel-level error between the reconstructed image and the real image, and is essentially a metric based on mean squared error (MSE). SSIM assesses the similarity between two images in terms of structural information, brightness, and contrast, more closely resembling the human visual system's judgment of image quality. LPIPS measures the difference between two images in the perceptual feature space, and is a perceptual metric based on deep neural network features.

Claims

1. A sparse-view 3D Gaussian reconstruction method based on context recovery, characterized in that, include: S1. Acquire multi-view input images and perform preprocessing; S2. Based on the preprocessed image in S1, generate an initial colored object point cloud with geometric consistency; S3. The point cloud generated in S2 is refined and its density is controlled using a structure priori-guided method. S4. Convert the refined point cloud in S3 into an anisotropic three-dimensional Gaussian representation; S5. Based on the 3D Gaussian representation obtained in S4, ICRM is introduced. Through cross-validation-based 3D appearance consistency learning, cross-view visual correlation is captured to improve the rendering quality degradation caused by the inherent limitations of the 3D Gaussian model optimization process. S6. Based on S5, the three-dimensional Gaussian model is further trained using an iterative training method with controlled viewpoint perturbation. This improves the integrity of the reconstruction results while ensuring consistency, ultimately achieving high structural consistency and visual fidelity in the three-dimensional Gaussian reconstruction.

2. The sparse viewpoint 3D Gaussian reconstruction method based on context recovery as described in claim 1, characterized in that, The preprocessing in S1 includes: S1.

1. Use a segmentation model to perform foreground segmentation on the N input images to separate the main subject from the complex background; S1.2, Perform scale normalization on the main body separated in S1.1, and uniformly scale and center the main body to a fixed resolution canvas.

3. The sparse viewpoint 3D Gaussian reconstruction method based on context recovery as described in claim 2, characterized in that, The generation of the initial point cloud in S2 includes: S2.

1. The MASt3R model is used to simultaneously predict camera pose, monocular depth, and pixel-aligned point maps. S2.2 Integrate the point map, pose matrix and scale factor predicted in S2.1 into a unified global coordinate system to obtain an initial colored object point cloud with geometric consistency.

4. The sparse viewpoint 3D Gaussian reconstruction method based on context recovery as described in claim 3, characterized in that, The point cloud refinement and density control process in S3 includes: S3.1 Construct visual shell constraints; S3.

2. Based on the visual hull constraints constructed in S3.1, calculate the Euclidean distance transform of the foreground mask; S3.3, Define a relaxation mask based on the Euclidean distance transform in S3.2; S3.

4. The point cloud obtained in S2.2 is filtered using the relaxation mask defined in S3.3 to remove outliers whose projections fall outside the relaxation mask. S3.

5. In the discontinuous or sparse regions inside the relaxed mask region, apply small perturbations along the six directions of x, y and z axes at the corresponding three-dimensional positions to generate additional local points, thereby enhancing the point cloud density and surface continuity.

5. The sparse viewpoint 3D Gaussian reconstruction method based on context recovery as described in claim 4, characterized in that, When capturing cross-view visual correlation in S5, the following are included: S5.

1. A leave-one-out strategy is used to construct degraded-real image pairs for training ICRM; S5.2 Based on the degraded-real image pair constructed in S5.1, the context recovery model captures the viewpoint-related correlation information and reasonably completes the missing geometric clues in the single-view observation.

6. The sparse viewpoint 3D Gaussian reconstruction method based on context recovery as described in claim 5, characterized in that, The process of constructing the degraded-real image pairs for training ICRM in S5.1 includes: 5.

11. Divide the input set of N real images into N subsets, each subset containing N-1 reference images and one real image as a preserved viewpoint; 5.

12. Using the initial point cloud obtained in S2.2, train N coarse 3D Gaussian models on the corresponding subset; 5.

13. Perform minimization Gaussian splitting and pruning operations on the coarse 3D Gaussian model trained in S5.12 to obtain a refined coarse 3D Gaussian model; 5.

14. Based on the refined coarse 3D Gaussian model obtained in S5.13, for each model, the degraded rendered image from the preserved viewpoint is paired with its corresponding real image to form a degraded-real image pair for training ICRM.

7. The sparse-view 3D Gaussian reconstruction method based on context recovery as described in claim 6, characterized in that, The training of the context recovery model in S5.2 includes: S5.

21. Input the viewpoint-preserving degraded rendering image obtained in 5.14 and its corresponding real image into the diffusion image generation model; S5.

22. During the training process, the degraded image is used as the input image to be restored, and the corresponding real image is used as the supervision target image. S5.23 Finally, a context restoration module is obtained that can use reference view information to perform structural correction and texture completion on degraded images.

8. The sparse viewpoint 3D Gaussian reconstruction method based on context recovery as described in claim 7, characterized in that, The iterative training method with controlled viewpoint perturbation used in S6 to further train the 3D Gaussian model includes: S6.1 Construct a spherical trajectory around the main body and evenly arrange virtual cameras on the sphere. Apply small perturbations to the pitch and azimuth angles of the reference camera pose and sample new viewpoints within its neighborhood to ensure that the newly generated viewpoints remain within a controllable angle range. S6.2 Render the degraded image of the perturbation viewpoint generated in S6.1; S6.

3. Restore the degraded image generated in S6.2 using ICRM to obtain a pseudo-real image; S6.4 Update the three-dimensional Gaussian parameters using the pseudo-real image obtained in S6.3; S6.5 Repeat steps S6.1-S6.4, gradually expanding the disturbance range along the spherical trajectory until it covers the viewpoints of all virtual cameras; S6.6 Finally, a highly consistent 3D Gaussian model that can be rendered in real time is obtained.

Images