Sparse view angle pose-free scene reconstruction method and system based on 3DGS

Abstract

The invention relates to a sparse view angle pose-free scene reconstruction method and system based on 3DGS, belongs to the field of computer vision and three-dimensional reconstruction, and solves the problems of easy failure and poor geometric consistency in a sparse view angle or weak texture environment due to dependence on accurate camera pose priori. Constructing a double-flow sensing module containing semantic flow and geometric flow, extracting semantic features by using a visual basic model, and extracting an explicit geometric corresponding relation by using a dense feature matching network, so as to regress relative camera pose under pose-free priori; a geometric guidance depth refinement module combining a potential diffusion model architecture and Pluecker ray coding is introduced, and scale fuzziness of monocular depth estimation is eliminated through a depth residual prediction mechanism; and based on the micronizable Gaussian rasterization, performing end-to-end optimization by using a self-supervised loss function including rendering consistency, reprojection and epipolar geometric constraint. According to the method, high-fidelity three-dimensional reconstruction is realized without supervision of external parameter true values, and geometric stability and rendering quality in a complex scene are improved.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, deep learning, and 3D reconstruction technology, specifically relating to a sparse viewpoint no-pose scene reconstruction method and system based on 3DGS (3D Gaussian Splatting). This invention is particularly suitable for complex scenes where precise camera pose cannot be obtained, views are sparse, and there are large baseline differences. It utilizes a combination of implicit neural representations and explicit geometric constraints to achieve high-quality 3D scene reconstruction and novel viewpoint synthesis. Background Technology

[0002] 3D scene reconstruction is a fundamental task in computer vision, augmented reality, and autonomous robot navigation. In this field, how to efficiently and effectively represent and render 3D scenes has always been a core research focus. In recent years, implicit representation methods, such as Neural Radiation Field (NeRF), have achieved photorealistic rendering quality, but their high training and inference costs limit practical applications. 3D Gaussian Splatting (3DGS), as an emerging explicit scene representation method, leverages the characteristic of modeling scenes using anisotropic Gaussian ellipsoids, maintaining extremely high rendering fidelity while achieving real-time rendering speed through an efficient rasterization pipeline. This combination of high quality and high efficiency has rapidly made it the mainstream technology in the current field of 3D scene reconstruction and new perspective compositing.

[0003] However, despite 3DGS's excellent performance in controlled environments, its standard training process has a very high barrier to entry, heavily relying on accurate camera extrinsic parameters as prior input. In routine operations, these camera poses are typically pre-calculated from image sequences using Structure from Motion (SfM) or Simultaneous Localization and Mapping (SLAM) algorithms. But in real-world open-world scenarios, facing weakly textured regions, sparse shooting perspectives, or complex dynamic environments, traditional SfM algorithms often encounter feature matching failures or convergence difficulties, resulting in the inability to obtain accurate initial pose values, thus directly hindering subsequent 3DGS reconstruction processes.

[0004] To overcome the reliance on pre-calculated poses, researchers have explored various solutions for 3D reconstruction under unknown pose conditions. Existing technologies mainly fall into two categories: One approach attempts to jointly optimize camera pose during neural field training. This method's solution process is highly dependent on the initial estimate of the camera pose. Without precise initial values, the optimization algorithm is prone to getting stuck in local convergence, leading to distorted scene geometry in the reconstructed image. The other approach, based on deep learning, attempts to directly regress the pose through an encoder. While this improves inference speed, it typically relies on difficult-to-obtain ground truth poses for supervised training. Furthermore, due to the lack of explicit multi-view geometric constraints in the network design, it often fails to guarantee geometric consistency across views when handling large parallax or sparse viewpoint data. This lack of geometric constraints directly results in unstable depth estimation, blurred object edges, ghosting, and even structural collapse in the reconstruction results, severely impacting the accuracy and robustness of 3D reconstruction.

[0005] In summary, how to fully utilize the efficient expressive power of 3DGS and achieve high-quality 3D scene reconstruction with geometric consistency and robustness under the conditions of no camera pose prior and no ground truth supervision is a key technical problem that urgently needs to be solved in the field of 3D computer vision. Summary of the Invention

[0006] The purpose of this invention is to provide a sparse viewpoint no-pose scene reconstruction method and system based on three-dimensional Gaussian representation, which aims to solve the problems of poor three-dimensional reconstruction effect, easy collapse of geometric structure and strong dependence on ground truth data in the existing technology under the conditions of lack of accurate camera pose, sparse viewpoint or weak texture.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A sparse viewpoint poseless scene reconstruction method based on 3D Gaussian representation includes the following steps:

[0009] S1: Data Acquisition and Preprocessing. A multi-frame sequence of 2D images of the scene to be reconstructed is acquired as input. These images can be sparsely captured, and the camera's extrinsic parameters (rotation and translation) do not need to be pre-calibrated. The system assumes that the camera's intrinsic parameters are known or can be initialized using default parameters.

[0010] S2: Construct a two-stream sensing module. This module contains two parallel feature extraction paths:

[0011] Semantic Flow: This approach extracts pixel-level high-dimensional semantic features from images using pre-trained visual foundational models. These features possess strong generalization capabilities and can capture global contextual information of the scene.

[0012] Geometric Flow: This approach utilizes a detector-independent dense feature matching network to extract explicit correspondences between views. Unlike traditional implicit feature encoding, this flow directly outputs point-to-point geometric matching information, providing robust physical constraints for pose estimation.

[0013] S3: Pose estimation based on explicit matching. Using the matched point pairs and their confidence scores from the geometry flow output as input, a lightweight pose regression network predicts the six-DOF pose of the context view relative to the target view. By introducing explicit geometry matching, this method avoids the instability of pure implicit regression when dealing with large parallax.

[0014] S4: Geometric-Guided Depth Refinement. To address the scale ambiguity and inaccurate boundary issues in monocular depth estimation, this invention constructs a U-Net depth refinement network based on an LDM architecture. This module does not perform random sampling generation but instead employs a deterministic residual prediction mechanism. Specifically, the relative pose, semantic features, and geometric ray features encoded using Plück coordinates predicted in step S2 are concatenated and input as multimodal conditions into the U-Net network. The network outputs a depth residual term, which is added to the initial depth map to recover refined depth information that conforms to multi-view geometric consistency.

[0015] S5: Dense 3D Gaussian Generation. Based on the optimized depth map and semantic features, the 3D Gaussian primitive attributes corresponding to each pixel are regressed through a Gaussian head network, including its center position in the world coordinate system, the covariance matrix describing the spatial morphology, the spherical harmonic coefficient reflecting the material color, and the opacity controlling visibility.

[0016] S6: Self-Supervised Joint Optimization. A rendering pipeline based on differentiable Gaussian rasterization is constructed. During training, instead of relying on any ground truth pose, a composite loss function is designed, incorporating rendering consistency loss, reprojection loss, and epipolar geometry constraint loss. By minimizing the difference between the rendered image and the real image, and by enforcing the physical constraints of epipolar geometry, end-to-end self-supervised optimization of network parameters and camera pose is achieved. Attached Figure Description

[0017] Figure 1 The diagram shows the overall process architecture of the method of this invention, illustrating the end-to-end pipeline of dual-stream sensing, pose estimation, depth optimization, and Gaussian rendering.

[0018] Figure 2This presents experimental results comparing the quality of novel perspective synthesis and camera pose estimation accuracy of the proposed method on the RealEstate10K and ACID datasets. The final performance was measured using five key evaluation metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Perceptual Patch Similarity (LPIPS), and rotation and translation errors of the camera pose.

[0019] Among them, PSNR represents Peak Signal-to-Noise Ratio, which is used to evaluate the error between the rendered image and the real image at the pixel level (the higher the value, the less distortion and the clearer the image); SSIM is the Structural Similarity Index, which is used to comprehensively evaluate the degree of preservation of brightness, contrast and structural information in an image (the closer the value is to 1, the more complete the structure); LPIPS reflects the feature differences that conform to human visual perception, and is used to measure the realism of an image in terms of texture details and perceptual level (the lower the value, the more it conforms to human visual habits).

[0020] Furthermore, to verify geometric consistency, pose error metrics were introduced: Rot and Trans represent the rotation and translation errors of the camera relative to the pose (in degrees), respectively, used to quantify the geometric solution accuracy in the 3D reconstruction process (lower values ​​indicate more accurate pose estimation and stronger spatial perception capabilities of the system). By comparing these metrics, the high-fidelity rendering capability and robust geometric perception performance of the proposed method on different datasets can be comprehensively verified.

[0021] Figure 3 This is a visual comparison of the reconstruction results of the present invention on the RealEstate10K dataset and the ACID dataset.

[0022] Figure 4 This is a system block diagram of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment proposes an end-to-end 3D reconstruction network architecture that does not require prior camera pose information. This architecture is stored in memory as computer program instructions and executed by a processor (such as a computing platform configured with a high-performance GPU). It performs feature extraction, geometric matching, and depth optimization processing on image data through the following specific modules, the specific implementation logic of which is as follows:

[0024] Example 1: System Architecture and Process

[0025] like Figure 1As shown, this invention provides a sparse viewpoint poseless scene reconstruction system based on 3D Gaussian representation, mainly composed of a data input interface, a semantic-geometric dual-stream perception module, a depth refinement module, a 3D Gaussian generation module, and a self-supervised rendering optimization module. The system runs on a computing platform equipped with a high-performance GPU and is implemented using the PyTorch framework and a CUDA-accelerated Gaussian rasterization library.

[0026] Example 2: Two-stream sensing and feature extraction

[0027] To achieve robust reconstruction in the absence of pose priors, this invention designs a parallel two-stream architecture.

[0028] Semantic Flow:

[0029] A pre-trained DINOv2 (ViT-L / 14) algorithm is used as the feature extractor. DINOv2 extracts features of dimension 14.

[0030] The input image (where Image pixel height, The image is divided into patches (where 3 represents the RGB color channel and the image pixel width is 3). Features are extracted using a Transformer encoder.

[0031] The output dimension is Feature map (in and These represent the height and width of the feature map, respectively. The feature channels contain rich semantic information. To fuse multi-view information, a cross-view attention layer is introduced. Assume there are two context views. and Their characteristics are as follows: and .

[0032] The calculation formula is:

[0033]

[0034] in From The query vector obtained by projection. From The key and value vectors obtained by projection This is a scaling factor for the feature dimension. This step ensures that each view not only contains its own semantics but also incorporates complementary information from another viewpoint, which helps infer the geometry of the occluded area.

[0035] Geometric Flow:

[0036] LoFTR (Detector-Free Local Feature Transformer) is used as the matching backbone. LoFTR utilizes the self-attention and mutual attention mechanisms of the Transformer to capture long-distance dependencies.

[0037] Input image pairs LoFTR outputs a dense matching probability matrix. This invention extracts the Top-N (this invention takes...) Construct a geometric matching set from the 10 explicit point pairs with the highest confidence:

[0038]

[0039] in For pixel coordinates, , where is the confidence level. and They represent the first The pixel coordinates of each matching point pair in the context view and the target view; For the first Confidence of each matching point pair; The index number for matching point pairs (the range of values ​​is...) arrive These explicit matching points, acting as "geometric anchor points," directly constrain the relative positional relationship between the two views, effectively preventing pose drift in areas with weak texture.

[0040] Example 3: Unsupervised Pose Estimation

[0041] This invention does not rely on external SfM tools (such as COLMAP), but instead directly regresses pose from image content.

[0042] Pose Regressor: Construct a lightweight multilayer perceptron. The input is the set of matches output by the geometry flow. The flattened vector.

[0043] Output: Predicted context view Relative to the target view Relative rotation (in quaternions) Output in the form of a rotation matrix. ) and relative translation .

[0044] To ensure the orthogonality of the rotation matrices, the output quaternions are normalized and then converted into rotation matrices using the Rodriguez formula. The final transformation matrix... Used for subsequent geometric transformations.

[0045] Example 4: Geometry-guided depth optimization

[0046] This is a key step in resolving scale ambiguity in monocular depth estimation.

[0047] Initial Depth: First, a simple convolutional head network is used. Based on the fused semantic features Predicting the initial depth map .

[0048] Plücker ray coding: To explicitly inject geometric priors, the representation of the imaging ray corresponding to each pixel in the Plücker coordinate system is calculated. For pixels... Combined with the predicted pose and camera internal reference Calculate the direction vector (where the symbol is superscript) The transpose operation represents a matrix or vector, used to convert a row vector of pixel coordinates into a column vector, and the torque vector. (where, symbol) (representing the cross product of vectors), ray coding As a geometric condition embedding.

[0049] LDM-Unet Depth Refinement: Employs a UNet architecture based on a latent diffusion model. Input includes the initial depth. semantic features and ray coding The network performs a deterministic deep refinement process to predict deep residuals. Thus, the optimized depth is obtained. This process fully leverages the advantages of the LDM-optimized UNet network architecture in multi-scale feature extraction and context modeling. This architecture treats the input semantic and geometric features as strong constraints, and through a residual prediction mechanism, effectively corrects for local noise and global scale bias present in the initial depth map.

[0050] Example 5: 3D Gaussian Generation and Rendering

[0051] Gaussian parametric regression: For each pixel, the optimized depth is back-projected onto 3D space to obtain the Gaussian center. The formula for calculating the center position of a three-dimensional Gaussian element is:

[0052]

[0053] in, The optimized depth value. For pixel coordinates, For the camera intrinsic parameter matrix, This is the predicted relative pose matrix;

[0054] Simultaneously, the covariance matrix is ​​regressed using a Gaussian head network. (Parameterized by scaling factor and rotation quaternion), color (Spherical harmonic coefficient) and opacity .

[0055] The aforementioned attribute parameters, together with the center position, construct a Gaussian point cloud model representing the target 3D scene. Specifically, these parameters explicitly digitize the scene from three dimensions: geometry, surface texture, and material transmittance, thus providing a complete 3D scene description for subsequent synthesis of high-quality images from any viewpoint.

[0056] Differentiable rasterization: Projects millions of generated Gaussian pixels onto the target view plane. The rasterizer sorts them by depth and performs alpha blending to generate the final rendered image. The process is fully differentiable and supports gradient backpropagation from the rendered image to Gaussian properties and camera pose.

[0057] Example 6: Design of Self-Supervised Loss Function

[0058] To achieve unsupervised training, a composite loss function was designed. :

[0059]

[0060] Rendering consistency loss ( ): Calculate the L2 distance and LPIPS perceptual distance between the rendered image and the real image.

[0061]

[0062] in, This is a predicted rendered image generated based on 3D Gaussian rasterization. The actual image corresponding to the target's viewpoint; L2 norm (Euclidean distance) is used to measure color differences at the pixel level. This represents a function for calculating the similarity of image patches, used to measure the differences in features that conform to human visual perception.

[0063] Reprojection loss Using the predicted depth and pose, the target view is warped onto the context view, and the photometric error is calculated. This constrains the geometric consistency among multiple views, as shown in the formula:

[0064]

[0065] in, This represents the image transformation sampling function based on depth and pose. L1 norm

[0066] Epipolar geometric constraint loss ( This is crucial for preventing pose estimation degradation. Constructing the fundamental matrix:

[0067]

[0068] in, and Camera intrinsic matrix for the target view and context view, respectively; symbol This represents the transpose of the matrix inverse; The rotation matrix is ​​the prediction. Represents the predicted translation vector The antisymmetric matrix is ​​used to perform matrix-based operations on vector cross products. Its normalized minimum singular value is minimized. This loss forces the predicted pose to satisfy the rank-2 constraint of epipolar geometry, eliminating scale drift.

[0069] In the training process of this embodiment, in order to balance the contribution of each loss term to network optimization, the preferred range or specific value of each weight coefficient can be set as follows, based on experimental verification:

[0070] Weights of rendering consistency loss It is recommended to set it to around 1.1;

[0071] Weights of bidirectional reprojection loss It is recommended to set it to around 0.1;

[0072] Weights of epipolar geometric constraint loss It is recommended to set it to around 0.01.

[0073] It should be noted that the above values ​​are only one specific embodiment of the present invention. Those skilled in the art can make adaptive adjustments to the above weight coefficients according to the characteristics of the actual dataset (such as scene scale, lighting changes, etc.), and such adjustments are still within the protection scope of the present invention.

[0074] Through the joint optimization of the above steps, this invention can quickly converge and generate high-quality 3D scene models with only sparse image sequences. To verify the effectiveness and advancement of the proposed method, this embodiment was comprehensively tested on two widely used public datasets: the RealEstate10K dataset, which contains large-scale complex indoor layouts, and the ACID dataset, which contains large-scale outdoor natural scenes. The experimental results were evaluated from two dimensions: the quality of the new perspective synthesis and the accuracy of camera pose estimation.

[0075] 1. The new perspective on image quality assessment demonstrates significant advantages in all key metrics reflecting reconstructed image quality. Experimental data shows that:

[0076] On the RealEstate10K dataset, our method achieves a peak signal-to-noise ratio (PSNR) of 25.95 dB, a structural similarity (SSIM) of 0.843, and a perceptual distance metric (LPIPS) reduced to 0.150. Compared to existing state-of-the-art poseless methods, this invention demonstrates significant improvements in both image sharpness and detail reproduction.

[0077] On the more challenging outdoor ACID dataset, our method achieves a PSNR of 26.71 dB, which is also superior to the 25.12 dB of the comparative method. This demonstrates the strong robustness of the dual-stream sensing module in this invention when handling unbounded natural scenes.

[0078] 2. Camera pose estimation accuracy evaluation: In terms of pose metrics reflecting geometric consistency, thanks to the introduction of explicit geometric matching and epipolar geometric constraints, this invention significantly reduces estimation errors:

[0079] Rotation error: On RealEstate10K, the average rotation error of our method is only 0.645 degrees, which is better than the existing mainstream methods; on the ACID dataset, the error is further reduced to 0.518 degrees.

[0080] Translation error: On the ACID dataset, the translation error of this method is reduced from 15.327 degrees in the prior art to 14.273 degrees.

[0081] 3. Overall Technical Effects In summary, by introducing semantic-geometric dual-stream perception and geometry-guided depth optimization, this invention effectively overcomes the artifacts and geometric collapse problems caused by existing technologies under sparse viewpoints. Whether in textured indoor environments or outdoor environments with varying lighting, this invention can achieve high-fidelity 3D scene reconstruction without pose ground truth supervision, possessing extremely high practical application value.

[0082] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A sparse viewpoint poseless scene reconstruction method based on 3DGS, characterized in that, Includes the following steps: S1: Obtain a sparse multi-view 2D image sequence of the scene to be reconstructed. The image sequence includes a context view and a reference view of unknown camera extrinsic parameters. The images are preprocessed to unify the resolution and format. S2: Construct a dual-stream perception module that includes semantic and geometric flows. Use a pre-trained visual base model to extract pixel-level semantic features of the context view, and use a pixel-level dense feature matching network to extract the explicit geometric correspondence between the context view and the target view. S3: Based on the explicit geometric correspondence, a lightweight pose regression network is used to predict the relative camera pose of the context view relative to the reference view. The relative camera pose includes a rotation matrix and a translation vector, and semantic features are used for auxiliary correction. S4: Using the pixel-level semantic features and the predicted initial depth map, construct a geometry-guided depth optimization module. This module is based on the U-Net depth refinement network, injects the predicted relative camera pose and multi-view features as conditions, and optimizes the depth map through the residual prediction mechanism to eliminate the scale ambiguity of monocular depth estimation. S5: Based on the optimized depth map and semantic features, the attribute parameters of the three-dimensional Gaussian primitives are regressed through the Gaussian generator head network. The attribute parameters include the three-dimensional center position, anisotropic covariance matrix, color coefficient and opacity, thereby constructing a dense three-dimensional Gaussian field representation of the scene. S6: The three-dimensional Gaussian field is projected onto the target view plane using differentiable Gaussian rasterization technology to generate a rendered image. The self-supervised loss function is calculated based on the difference between the rendered image and the real target image. The network parameters are jointly optimized through backpropagation to achieve end-to-end poseless scene reconstruction. The self-supervised loss function includes rendering consistency loss, reprojection loss, and epipolar geometry constraint loss.

2. The method according to claim 1, characterized in that, The specific implementation of the dual-stream sensing module in step S2 includes: In the semantic flow, a pre-trained ViT (Vision Transformer) architecture is used as the encoder to extract high-dimensional semantic features of the image. Information between context views is interacted through cross-view attention layers, and the feature representation is updated using a Softmax-normalized dot product attention mechanism to infer the implicit three-dimensional geometric relationships. In the geometric flow, a LoFTR (Local Feature Transformer) network is used as the backbone to establish a dense correspondence between the context view and the target view on the coarse-grained feature map. The top-N explicit matching point pairs are selected based on the matching confidence to form a geometric matching set. This is used for subsequent pose estimation.

3. The method according to claim 2, characterized in that, The prediction process relative to the camera pose in step S3 is as follows: The geometric matching set Flattened into vectors, combined with global feature descriptors from the semantic flow, and input into a pose regressor composed of a multilayer perceptron; Quaternion representation of output rotation parameters Translation vector ; Normalize the quaternions and convert them into rotation matrices. Combining the translation vectors to form the transformation matrix It is used to describe rigid body transformations from the context view coordinate system to the target view coordinate system, thereby replacing the traditional implicit pose regression.

4. The method according to claim 1, characterized in that, The specific implementation of the geometry-guided depth optimization module in step S4 includes: A deep refinement network based on the LDM-optimized UNet architecture is constructed, with the input being the initial depth map, the fused semantic features, and the ray features encoded by Plück coordinates; The ray feature is obtained by combining pixel coordinates with predicted camera intrinsics. Given the relative pose, calculate the direction vector of each imaging ray. and torque vector ; The depth residual prediction mechanism is executed in the latent feature space. It utilizes the network's contextual understanding of multimodal features to directly regress the depth residual term and superimpose it onto the initial depth map, thereby correcting the local geometric errors and global scale inconsistencies of the initial depth map. In this process, the ray features are injected as explicit 3D geometric priors into the feature decoding process based on the latent diffusion model U-Net network architecture, forcing the output fine depth map to satisfy the multi-view geometric consistency constraint.

5. The method according to claim 1, characterized in that, Self-supervised loss function in step S6 Defined as: in, The rendering consistency loss is composed of a weighted average of L2 pixel loss and LPIPS perceptual loss, and is used to constrain the visual similarity between the rendered image and the real image. As a bidirectional reprojection loss, the target view pixels are back-projected to the context view according to the predicted depth and pose, and the consistency of the geometry is constrained by photometric error. To reduce scale drift in pose estimation, a fundamental matrix is ​​constructed to mitigate epipolar geometric constraint loss. These are the adjustable weighting coefficients for each loss term.

6. The method according to claim 1, characterized in that, Step S5 calculates the center position of the three-dimensional Gaussian element, specifically including: Using the inverse of the camera intrinsic parameter matrix, pixel coordinates are converted into normalized planar coordinates. Based on the optimized depth value, the normalized planar coordinates are extended to 3D points in the camera coordinate system. Using the predicted relative pose matrix, the 3D points in the camera coordinate system are transformed to the target coordinate system to obtain the center position of the 3D Gaussian primitives. Simultaneously, the Gaussian generator head network also regresses and outputs the covariance matrix. Color coefficient and opacity The aforementioned center position and attribute parameters together constitute the three-dimensional Gaussian representation of the scene, which is used to construct the three-dimensional scene to be reconstructed.

7. A system for implementing the method of claim 1, characterized in that, include: The data acquisition and preprocessing module is used to acquire sparse multi-view image sequences of the scene to be reconstructed and perform preprocessing. The dual-stream feature extraction module includes a semantic feature extraction unit and a geometric matching extraction unit, which are used to extract the DINOv2 semantic features and LoFTR explicit geometric correspondence of the image, respectively. The pose estimation module is used to regress the relative camera pose between views based on explicit geometric correspondences and semantic features; The geometry-guided depth optimization module integrates LDM-UNet and Plücker ray encoder to perform geometrically guided fine-tuning of the initial depth map based on predicted pose and semantic features. The 3D Gaussian point cloud generation module is used to generate spatial and attribute parameters of 3D Gaussian primitives based on the corrected depth map and image feature regression, and to aggregate the spatial and attribute parameters to generate a Gaussian point cloud model that expresses the complete 3D scene. The results acquisition module is used to obtain the final 3D scene model or synthesize a new perspective image based on the generated 3D Gaussian primitives.

8. The system according to claim 7, characterized in that, The system also includes a standardized data interface module, which automatically initializes pinhole camera model parameters in the absence of camera calibration parameters, and converts the predicted pose sequence and reconstructed point cloud data into standard triplet data formats, including cameras.bin, images.txt, and points3D.ply, to ensure compatibility with subsequent 3D visualization or editing tools.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

Images