Multi-view image three-dimensional reconstruction method

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to a multi-view image three-dimensional reconstruction method, which comprises the following steps of: firstly, inputting a small amount of two-dimensional images, generating a color point cloud through depth estimation, initializing a three-dimensional Gaussian model, and performing coarse-grained reconstruction by combining luminosity loss and geometric regularization; secondly, planning a camera track based on geometric distribution of the rough model, and collecting a multi-view image sequence; thirdly, performing fine adjustment and repair on the image by using a diffusion model fusing the camera pose and the visual features; and finally, iteratively optimizing the three-dimensional Gaussian model by using the repaired image, and outputting a high-precision three-dimensional reconstruction result. According to the method, high-quality reconstruction can be achieved only through sparse view angle images, and dependence of a traditional method on a large amount of data is broken through. Through composite condition coding and dynamic trajectory optimization, geometric consistency and texture authenticity are effectively improved, artifacts are avoided, and calculation efficiency and reconstruction precision are considered at the same time.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically a method for three-dimensional reconstruction of multi-view images. Background Technology

[0002] With the development of computer vision and computational graphics, 3D reconstruction technology has shown broad application prospects in many fields such as virtual reality, augmented reality, film and television production, industrial inspection, and cultural heritage protection. Multi-view image 3D reconstruction, as one of the core technologies, aims to obtain the 3D geometric information of a scene or object from multiple 2D images, thereby restoring its true structure.

[0003] From a data dependency perspective, existing technologies generally have a strong dependence on the number of input images, typically requiring dozens of viewpoint images to ensure reconstruction accuracy. Using only four or fewer images can easily lead to a coarse reconstruction model with missing details, failing to restore the target's fine geometric features and making it difficult to balance the practical needs of limited data input and high-precision output. Regarding the handling of missing regions, unavoidable occlusion, lighting variations, and viewpoint limitations in complex scenes can cause localized missing data in the reconstruction. Traditional algorithms often rely on simple rule-based assumptions or interpolation methods for missing data completion, failing to fully integrate the target's texture features and geometric logic. The completed results often exhibit artifacts or structural breaks, deviating significantly from the true shape.

[0004] To address camera pose errors, existing reconstruction processes lack effective adaptive adjustment mechanisms. Even minor camera pose deviations during shooting (such as position shifts or angular tilts) directly impact reconstruction accuracy, and these errors cannot be corrected in real time. This results in subsequent model optimization being limited by initial data defects, significantly restricting the ability to represent details. In terms of computational efficiency, existing methods often employ full-parameter iteration in the model detail optimization and defect completion stages, which is computationally intensive and time-consuming, making it difficult to meet the efficiency requirements of real-time reconstruction scenarios such as industrial online inspection or large-scale reconstruction scenarios such as the digitization of ancient buildings.

[0005] Furthermore, some generative model-based completion algorithms only focus on local restoration of a single frame image, failing to fully utilize the spatial correlation and semantic consistency between multiple perspectives. This results in stylistic variations in completion results across different perspectives, making it impossible to form a unified 3D structural logic. Meanwhile, existing image acquisition relies heavily on manual experience to design camera paths, making it impossible to adaptively plan trajectories based on the geometric features of the initial coarse-grained model. This easily leads to uneven viewpoint coverage and insufficient sampling of key areas, further limiting the quality of subsequent reconstructed data and the model's ability to restore details.

[0006] Based on the shortcomings of the existing technologies, there is an urgent need to design a technical solution that can combine a small number of multi-view image inputs to achieve efficient and high-precision 3D reconstruction, so as to overcome the limitations of the existing technologies and promote the in-depth application of 3D reconstruction technology in various fields. Summary of the Invention

[0007] The purpose of this invention is to provide a multi-view image 3D reconstruction method to solve the problems of insufficient accuracy, poor missing area completion effect, lack of effective adaptive adjustment of camera pose error, and low computational efficiency of existing multi-view image 3D reconstruction techniques mentioned in the background art when there are few image inputs.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for 3D reconstruction of multi-view images includes the following steps: Coarse-grained model reconstruction: Receive multiple 2D images from different perspectives as input, perform coarse-grained model reconstruction based on the 2D images, obtain image depth information through depth estimation, fuse color information to generate a color point cloud and initialize a 3D Gaussian model, and optimize model parameters using a multi-objective function with photometric loss and geometric regularization. Controllable camera trajectory image acquisition: Based on the geometric distribution planning of a coarse-grained 3D model, camera trajectories are planned, and multi-view image sequences are acquired along the camera trajectory; Diffusion model fine-tuning and image inpainting: The diffusion model is used to fine-tune and inpaint the acquired image sequence. The camera pose coding and input image visual coding are fused to construct a composite conditional coding to guide the model to complete the filling of missing areas. Model Modification and Reconstruction: The 3D Gaussian model is iteratively optimized based on the repaired image sequence to output a high-precision 3D reconstruction model.

[0009] According to the above technical solution, in coarse-grained model reconstruction, the photometric loss function of coarse-grained model reconstruction... The specific expression is:

[0010] in, Indicates the absolute error between pixels. It is an image semantic similarity index. is the weight hyperparameter; this loss function is used to ensure that the model-rendered image is highly consistent with the real image at both the pixel and semantic levels, promoting the restoration of color and detail.

[0011] According to the above technical solution, in coarse-grained model reconstruction, the geometric regularization term... The specific expression is:

[0012] in, Indicates depth map in Gradient of position, The constraint, which defines the unit normal vector at the corresponding location, effectively avoids sharp depth jumps, ensuring a smooth and realistic model surface.

[0013] According to the above technical solution, in the controllable camera trajectory image acquisition, the camera trajectory is a closed uniformly distributed path, and the number of acquisition points is set to 12 to 18 to achieve continuous coverage of the viewpoint.

[0014] According to the above technical solution, during the controllable camera trajectory acquisition process, the trajectory sampling density is dynamically optimized for areas with complex textures or insufficient coverage based on the real-time evaluation results of image data quality.

[0015] According to the above technical solution, the diffusion model adopts the stable video diffusion (SVD) model, and the input image is mapped to the latent space through the variational autoencoder (VAE) for denoising and repair.

[0016] According to the above technical solution, in the fine-tuning of the diffusion model and image inpainting, the composite conditional coding is fused through a multi-head cross-attention mechanism, as shown in the following formula:

[0017] in Visual encoding of the input image, Encode the camera pose. This is for fine-tuning deviations in pose.

[0018] Based on the above technical solution, the optimization objective of the diffusion model is to minimize the noise prediction error, as shown in the following formula:

[0019] in , For images without defects, for Image with added noise at any time They are respectively The noise figure and standard deviation at any given time. and Controlling the intensity of diffused noise, For time step, The sampling noise is a standard normal distribution.

[0020] Based on the above technical solution, global optimization updates the Gaussian kernel parameters through gradient descent, as shown in the following equation:

[0021] in For the Gaussian kernel synthesis parameter set, and These are the photometric and geometric losses, respectively. For the number of viewpoints, These are the geometric regularization weight coefficients. This represents the restored image. Indicates by parameters The rendered image, This represents geometric loss.

[0022] Based on the above technical solution, local optimization is performed on the Gaussian kernel subset of the error set, as shown in the following equation:

[0023] in These are residual weights, used to increase the intensity of local optimization. It is a set of Gaussian kernel indexes with relatively large errors; The geometric loss for the i-th sample is represented by the constraint parameters. The corresponding geometric structure rationality, For geometric loss terms, This represents the loss of realism in the photo of the i-th sample.

[0024] Compared with the prior art, the present invention has the following beneficial effects: The method in this invention achieves high-precision reconstruction under sparse perspectives, completing the 3D reconstruction of the target with only a small number of initial images. This breaks through the dependence of existing technologies on multiple data inputs, significantly reducing data acquisition costs while ensuring the restoration of fine textures and microstructures. The method in this invention also fully integrates multi-view semantic features and camera pose information through a combination of composite conditional coding and SVD diffusion model, so that the completion result has both geometric consistency and texture realism, effectively avoiding the artifact problem of traditional interpolation methods, and accurately restoring the missing areas.

[0025] In this invention, by introducing a pose fine-tuning deviation ΔC and a dynamic trajectory optimization mechanism, minute pose errors during the shooting process can be corrected in real time, which greatly improves the spatial consistency of multi-view images and significantly reduces the impact of initial data defects on reconstruction accuracy. Furthermore, by adopting a phased process of coarse-grained modeling and fine-grained optimization, the basic structure can be quickly constructed in a short time. The fine-grained stage only optimizes key parameters and local areas, and the overall computational efficiency is significantly improved compared with existing full-scale optimization methods, which can meet the requirements of near real-time reconstruction. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the workflow of multi-view image completion and 3D reconstruction of the present invention; Figure 2 This is a schematic diagram of the coarse-grained model reconstruction workflow of the present invention; Figure 3 This is a schematic diagram of the diffusion model fine-tuning and image repair process of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Example 1 like Figure 1 As shown, to achieve the above objectives, the high-precision 3D reconstruction method based on sparse viewpoint input provided by this invention completes 3D reconstruction through a phased collaborative optimization technical path. The specific technical process is as follows: First, the input data acquisition operation is performed, receiving four two-dimensional images from different perspectives as initial input. The images must cover the main geometric contours of the target (such as the front, side, top, and bottom surfaces), and the image resolution must be no less than 960×540 and no more than 1920×1080 to ensure that the basic texture and structural features of the target are included, providing a reliable initial data foundation for subsequent modeling.

[0029] After acquiring the input data, the process moves to the coarse-grained model reconstruction stage (e.g., ...). Figure 2 As shown in the diagram, this stage constructs a coarse-grained 3D Gaussian model based on the initial input image: First, a depth estimation model based on a convolutional neural network (CNN) (such as the DPT depth estimation model) is used to predict the depth of each input image, obtaining pixel-level depth values ​​to establish a mapping relationship between pixel coordinates and depth; then, the RGB color information of each image is fused with the depth information of the corresponding pixels to convert it into a color point cloud in 3D space, and the point cloud density is not less than 100 points / square centimeter to ensure coverage of the entire visible area of ​​the target; next, the color point cloud is mapped to a set of 3D Gaussian kernels, and the parameters of each Gaussian kernel include position, weight, and color, where the position is determined based on the clustering of point cloud coordinates, and the weight and color are initialized through the statistical features (such as mean and variance) of the local region of the point cloud; finally, a multi-objective function containing photometric loss and geometric regularization is used to perform 100-200 iterations to optimize the 3D Gaussian model, and the expression of this multi-objective function is:

[0030] in The absolute error between pixels in the input image and the rendered image from the model. This is the semantic similarity index between the two (with a value ranging from 0 to 1, where the closer to 1, the stronger the semantic consistency). This is the photometric loss weighting coefficient (range 0.6-0.8). The geometric regularization term is expressed as:

[0031] in, Depth map in Gradient of position, (The unit normal vector at the corresponding position). The geometric regularization weights (ranging from 0.2 to 0.4) are minimized using the gradient descent algorithm. This ensures that the geometry of the coarse-grained model is highly consistent with the photometric and semantic features of the input image.

[0032] After the coarse-grained model is constructed, the controllable camera trajectory planning and image acquisition process is initiated: First, a closed, uniformly distributed trajectory around the target is generated based on the geometric contour of the coarse-grained model (such as the maximum circumscribed sphere radius of the target). The trajectory radius is set to 1.5-2 times the maximum circumscribed sphere radius of the target, and 12 acquisition points are evenly set on the trajectory to ensure that the viewing angle between adjacent acquisition points is 30° to achieve full-view coverage of the target. Then, the camera is controlled to move along the planned trajectory, and one image with the same resolution as the initial input image is captured at each acquisition point to obtain a sequence of 12 consecutive viewing angles. At the same time, the quality of the acquired images (such as texture clarity and key structural integrity) is evaluated in real time. If a certain area (such as the depression of the target or the fine texture area) is found to be undersampled or the image is blurry, the acquisition point density in that area is automatically increased (such as reducing the distance between adjacent acquisition points to 1 / 2 of the original distance) to ensure that the image data quality of the key areas meets the requirements of subsequent processing.

[0033] After the image sequence was acquired, diffusion model fine-tuning and image inpainting were immediately carried out (e.g. Figure 3 As shown), this stage uses a geometrically consistent diffusion model to complete the missing parts of the acquired image sequence: First, a stable video diffusion (SVD) model is selected as the base model, and the acquired image sequence is mapped to the latent space through a variational autoencoder (VAE). The latent space dimension is set to 512×512 to reduce the computational cost of the model; then, the visual encoding of the initial 4 input images ( (semantic feature vectors extracted by CNN) and camera pose encoding ( (6-dimensional vectors containing camera position and angle) are fused, introducing pose fine-tuning bias. (Initially set to 0, adaptively optimized through model training), and employs a multi-head cross-attention mechanism (with 8 heads) to achieve deep feature fusion. The constructed composite conditional encoding satisfies the expression to ensure the effective combination of semantic and geometric features. The composite conditional encoding satisfies the expression shown below:

[0034] in Visual encoding of the input image, Encode the camera pose. This is for fine-tuning deviations in pose.

[0035] Then, the diffusion model is trained with the goal of minimizing the noise prediction error, and the objective function expression is optimized as follows:

[0036] in , ( For images without defects, for Image with added noise at any time , They are respectively The noise figure and standard deviation at each time point vary. (Increase linear variation) The time step (range 100-1000 steps). For noise sampled from a standard normal distribution, The noise prediction function for the model is set, and the training epochs are set to 50-100 epochs to ensure that the model masters the ability to complete the missing regions based on composite conditional coding; finally, the acquired sequence of missing images is input into the trained diffusion model, and through the back-diffusion process (from... Step by step down (Step 1) Denoise the latent space vectors and then output a sequence of images without defects and with consistent viewpoints through a VAE decoder.

[0037] After image inpainting is completed, the final stage of model repair and reconstruction begins. Based on the inpainted image sequence, a coarse-grained 3D Gaussian model undergoes joint global and local optimization: In the global optimization phase, using the inpainted image sequence as a reference, the photometric loss between the rendered model image and the reference image is calculated. With geometric loss The parameters of all Gaussian kernels are updated using the gradient descent algorithm. The optimization objective satisfies the expression:

[0038] in For the Gaussian kernel synthesis parameter set, and These are the photometric and geometric losses, respectively. To determine the number of viewpoints, the number of iterations is set to 200-500 to ensure that the global structure of the model is consistent with the reference image. In the local optimization phase, by analyzing the residual distribution between the rendered image and the reference image (residual = |rendered pixel value - reference pixel value|), error concentration regions with residuals greater than 0.1 are located. A Gaussian kernel subset corresponding to this region is extracted, and a residual-weighted local optimization strategy is used to adjust the subset parameters. The optimization objective satisfies the expression:

[0039] in The residual weights, ranging from 0 to 1, satisfy the following conditions: =Residual / Maximum Residual, to correct local detail loss and texture discontinuity issues; finally, the 3D Gaussian model after global and local optimization is converted into a triangular mesh model (mesh resolution of not less than 10,000 triangular faces), outputting a high-precision 3D reconstruction model with average symmetric surface distance (ASSD) ≤0.5mm and texture similarity (SSIM) ≥0.9.

[0040] This invention employs a dual optimization mechanism that, while ensuring global model consistency, effectively fills in detail gaps and corrects depth deviations and color distortions, thereby achieving high-quality 3D point cloud reconstruction with a reasonable structure and rich visuals. This strategy has significant advantages in overcoming local blurring and artifacts caused by viewpoint sparsity and depth estimation errors, promoting the realism and stability of the final model under multiple viewpoints, thus overcoming the blurring and folding problems commonly found in traditional reconstruction.

[0041] Meanwhile, this invention employs an iterative feedback mechanism, utilizing both visual and geometric cues to guide the optimization process, gradually reducing surface inconsistencies and enhancing the smoothness and naturalness of the 3D surface. During reconstruction, the system also specifically considers the coordination of lighting and texture across multiple viewpoints, enabling the generated 3D model to more realistically and naturally present diverse lighting effects and detailed representations.

[0042] Through in-depth optimization of this module, the entire 3D reconstruction pipeline has finally transformed from a coarse structure to a fine model. This not only improves the completeness of the scene but also greatly enhances the richness of visual details and the coherence of the visual reproduction. As a result, the generated 3D Gaussian scenes and models possess high-quality model characteristics suitable for multi-angle free viewpoint browsing and immersive interaction, fully meeting the needs of various practical applications such as virtual reality and digitization of cultural heritage.

[0043] Example 2 This embodiment is a further refinement of Embodiment 1. To make the technical solution of the present invention clearer and easier to understand, the technical process of the present invention will be described in detail below with reference to specific implementation cases to verify the feasibility and effectiveness of the technical solution. The hardware environment configuration of this embodiment case is as follows: the processor is an Intel Core i9-13900K to ensure data processing efficiency, the memory configuration is 64GB to meet the needs of large-scale data storage, and the graphics card is an NVIDIA RTX 4090 (24GB VRAM) to support computationally intensive tasks such as depth estimation and diffusion model training; the software environment is built on the Ubuntu 22.04 operating system, the deep learning framework is PyTorch 2.5, the depth estimation model is the pre-trained Depth Anything model to ensure depth prediction accuracy, the diffusion model is based on Stable Diffusion v2.1 with adaptive modifications, and the optimization algorithm is AdamW (learning rate set to 1e-4) to improve the model training convergence speed.

[0044] This implementation case selects a Tang Dynasty terracotta figurine as the reconstruction target. The figurine is about 30cm tall and has fine carvings on its surface. It needs to be digitized in three dimensions from all angles using the method of this invention for long-term archiving and digital display of cultural heritage. Therefore, the accuracy and completeness of the detail restoration of the reconstruction model are required to be high. The specific implementation includes the following steps: First, input data was acquired by taking four images of the terracotta figurine from different perspectives using a Canon EOS R6 SLR camera: front (0° directly in front of the figurine), left side (90° to the left of the figurine), right side (90° to the right of the figurine), and back (0° directly behind the figurine). The image resolution was downsampled to 1920×1080 to clearly capture the details of the patterns on the surface of the terracotta figurine, ensuring the quality of the initial input data, while reducing the computational load and improving the inference speed of the model.

[0045] The next step is coarse-grained model reconstruction: four input images are fed into the pre-trained Depth Anything model, which outputs a pixel-level depth map with depth values ​​controlled between 0.5-1.5m (corresponding to the actual shooting distance between the camera and the terracotta figure); the "RGB image + depth map" is converted into a color point cloud using the Open3D library, with approximately 500,000 points and a density of approximately 200 points / square centimeter, ensuring coverage of the entire visible surface of the terracotta figure; the K-Means clustering algorithm (100,000 clusters) is used to cluster the color point cloud, with each cluster center serving as the location of the Gaussian kernel. The weights of the Gaussian kernel Set as the percentage of cluster points, color Set the RGB mean of the clustering region to complete the initialization of the 3D Gaussian model; set =0.8、 =0.2, and the AdamW algorithm was used to perform 5000 iterations of optimization on the multi-objective function to finally obtain a coarse-grained three-dimensional Gaussian model.

[0046] After the coarse-grained model was constructed, controllable camera trajectory planning and image acquisition were initiated: First, based on the maximum circumscribed sphere radius of the terracotta figurine (approximately 15cm), the trajectory radius was set to 30cm (twice the circumscribed sphere radius), and the trajectory shape was a circle with the same height as the center of the terracotta figurine. Twelve acquisition points were evenly set, with an angle of 30° between adjacent acquisition points. By moving a virtual camera designed in the rendering environment along the planned trajectory, one image was captured at each of the 12 acquisition points, initially obtaining a sequence of 12 consecutive viewpoint images of the terracotta figurine. Through real-time evaluation, it was found that the image of the patterned area on the back of the terracotta figurine (due to insufficient initial sampling caused by occlusion) had a blur problem. Therefore, three additional acquisition points were added to the trajectory segment corresponding to this area (including 3 initial acquisition points), ultimately obtaining a sequence of 15 images to ensure sufficient sampling of the patterned area.

[0047] Next, we performed diffusion model fine-tuning and image inpainting: We loaded the modified Stable Video Diffusionv2.1 model and encoded 15 acquired images into a 32×32-dimensional latent space using the Diffusion encoder. The latent space vector dimensions were 8×32×32. Similarly, we used the Diffusion encoder to extract the visual codes from the initial 4 images. (Dimension 2048), Camera pose encoding A 6-dimensional vector (containing x, y, z coordinates and roll, pitch, and yaw angles) is used. The number of multi-head cross-attention heads is set to 8, and ΔC is initialized to 0. A composite conditional code C is constructed. The time step is set to t = 500 steps. It decreased linearly from 0.99 to 0.1. The diffusion model was trained for 80 rounds, linearly increasing from 0.1 to 0.99. After training, the noise prediction error (MSE) of the model was reduced to below 0.01. Fifteen acquired images were input into the trained model, and a reverse diffusion process was performed from t=500 to t=0, outputting a sequence of 15 undamaged terracotta figurine images. The LPSIS of the repaired images remained below 0.10.

[0048] Finally, model repair and reconstruction were performed: using 15 repaired images as references, with 300 iterations, the Gaussian kernel parameters were updated using the gradient descent algorithm. After global optimization, the ASSD of the rendered model image and the reference image was reduced to 0.3mm. A residual threshold of 0.1 was set to locate the terracotta figurine pattern area (the residual concentration area), and a subset of 20,000 corresponding Gaussian kernels was extracted. =Residual / 0.5 (the maximum residual in this area is 0.5), and perform 50 iterations of optimization on this subset, finally reducing the ASSD of the patterned area to 0.2mm; the optimized 3D Gaussian model is converted into a triangular mesh model (150,000 meshes) using Open3D and saved in PLY format. This model can be directly used for VR browsing (supporting 360° rotation viewing) and high-precision measurement (dimensional error ≤0.1mm), fully meeting the needs of digital archiving of cultural heritage.

[0049] The effectiveness of this implementation case was verified through multiple dimensions of indicators: In terms of accuracy, the average symmetrical surface distance (ASSD) between the reconstructed model and the actual terracotta figurine was 0.25 mm, and the texture similarity (SSIM) was 0.93, which is far higher than the accuracy requirements for the digitization of cultural heritage; in terms of efficiency, the total time from data input to model output was 12 minutes, which is 2.75 times more efficient than the existing reconstruction method based on 100 images (which takes 45 minutes); in terms of integrity, there were no defects in the occluded areas and fine pattern areas on the back of the terracotta figurine, and the model had no artifacts from all angles, with excellent visual coherence, which fully verified the effectiveness and practicality of the technical solution of this invention.

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0051] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for three-dimensional reconstruction of multi-view images, characterized in that: Includes the following steps: Coarse-grained model reconstruction: Receive multiple 2D images from different perspectives as input, perform coarse-grained model reconstruction based on the 2D images, obtain image depth information through depth estimation, fuse color information to generate a color point cloud and initialize a 3D Gaussian model, and optimize model parameters using a multi-objective function with photometric loss and geometric regularization. Controllable camera trajectory image acquisition: Based on the geometric distribution planning of a coarse-grained 3D model, camera trajectories are planned, and multi-view image sequences are acquired along the camera trajectory; Diffusion model fine-tuning and image inpainting: The diffusion model is used to fine-tune and inpaint the acquired image sequence. The camera pose coding and input image visual coding are fused to construct a composite conditional coding to guide the model to complete the filling of missing areas. Model Modification and Reconstruction: The 3D Gaussian model is iteratively optimized based on the repaired image sequence to output a high-precision 3D reconstruction model.

2. The multi-view image three-dimensional reconstruction method according to claim 1, characterized in that: In coarse-grained model reconstruction, the photometric loss function of coarse-grained model reconstruction The specific expression is: in, Indicates the absolute error between pixels. It is an image semantic similarity index. is the weight hyperparameter; this loss function is used to ensure that the model-rendered image is highly consistent with the real image at both the pixel and semantic levels, promoting the restoration of color and detail.

3. The multi-view image three-dimensional reconstruction method according to claim 2, characterized in that: In coarse-grained model reconstruction, geometric regularization term The specific expression is: in, Indicates depth map in Gradient of position, This is the unit normal vector at the corresponding position.

4. The multi-view image three-dimensional reconstruction method according to claim 1, characterized in that: In the controllable camera trajectory image acquisition, the camera trajectory is a closed uniformly distributed path, and the number of acquisition points is set to 12 to 18 to achieve continuous coverage of the viewpoint.

5. The multi-view image three-dimensional reconstruction method according to claim 1, characterized in that: During the controllable camera trajectory acquisition process, the trajectory sampling density is dynamically optimized for areas with complex textures or insufficient coverage based on the real-time evaluation results of image data quality.

6. The multi-view image three-dimensional reconstruction method according to claim 1, characterized in that: The diffusion model adopts the stable video diffusion (SVD) model, which uses a variational autoencoder (VAE) to map the input image to the latent space for denoising and repair.

7. The multi-view image three-dimensional reconstruction method according to claim 6, characterized in that: In diffusion model fine-tuning and image inpainting, composite conditional coding achieves fusion through a multi-head cross-attention mechanism, as shown in the following equation: in Visual encoding of the input image, Encode the camera pose. This is for fine-tuning deviations in pose.

8. The method for three-dimensional reconstruction of multi-view images according to claim 7, characterized in that: The optimization objective of the diffusion model is to minimize the noise prediction error, as shown in the following equation: in , For images without defects, for Image with added noise at any time They are respectively The noise figure and standard deviation at any given time. and Controlling the intensity of diffused noise, For time step, The sampling noise is a standard normal distribution.

9. The multi-view image three-dimensional reconstruction method according to claim 1, characterized in that: Global optimization updates the Gaussian kernel parameters through gradient descent, as shown in the following equation: in For the Gaussian kernel synthesis parameter set, and These are the photometric and geometric losses, respectively. For the number of viewpoints, These are the geometric regularization weight coefficients. This represents the restored image. Indicates by parameters The rendered image, This represents geometric loss.

10. A method for three-dimensional reconstruction of multi-view images according to claim 9, characterized in that: Local optimization is performed on a subset of Gaussian kernels within the error set, as shown in the following equation: in These are residual weights, used to increase the intensity of local optimization. It is a set of Gaussian kernel indexes; The geometric loss for the i-th sample is represented by the constraint parameters. The corresponding geometric structure rationality, For geometric loss terms, This represents the loss of realism in the photo of the i-th sample.

Images