Method for reconstructing color three-dimensional scene from low-resolution grayscale image

Abstract

The invention discloses a method for reconstructing a color three-dimensional scene from a low-resolution grayscale image, belongs to the technical field of computer vision and image processing, and aims to solve the problem that high-resolution three-dimensional reconstruction cannot be simultaneously realized under the condition of low-resolution grayscale input in the prior art. The method comprises the following steps: firstly, carrying out three-dimensional Gaussian scene initialization on an input low-resolution grayscale multi-view image set; constructing a physical imaging degradation model, predicting the difference between the degradation model and real input, and reconstructing a high-resolution three-dimensional Gaussian scene in an unsupervised manner; then providing color knowledge for the multi-view rendering grey-scale map by using a pre-trained color generation model, and obtaining a scene-specific multi-view coloring model through knowledge distillation; then, a palette color representation mechanism is introduced, each three-dimensional Gaussian element color is represented as a weighted combination of a plurality of primary colors, color optimization is carried out in combination with a color migration model, and multi-view color consistency is ensured.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and specifically relates to a method for reconstructing a color three-dimensional scene from a low-resolution grayscale image. Background Technology

[0002] In recent years, with the development of 3D vision technology, 3D scene reconstruction has been widely used in fields such as virtual reality, digital cultural relic preservation, film and television production, and robot vision. Existing 3D reconstruction technologies mainly include traditional methods based on geometric principles, deep learning methods based on Neural Radiance Fields (NeRF), and lightweight rendering methods based on 3D Gaussian distributions.

[0003] 1. Three-dimensional reconstruction technology based on traditional geometric methods

[0004] These methods typically rely on principles such as multi-view stereo vision (MVS), structured light scanning, and LiDAR scanning to obtain 3D point cloud data through image matching and triangulation. While these methods offer advantages in geometric accuracy and physical consistency, they suffer from high computational complexity, are sensitive to external conditions such as lighting and reflection, and lack the ability to model color details with high quality.

[0005] 2. Three-dimensional reconstruction technology based on neural radiation fields

[0006] Neural radiation field (NeRF) methods learn the radiation distribution function in three-dimensional space through neural networks, generating high-quality 3D reconstruction results from multi-view images. These methods do not require explicit geometry and can directly learn the scene's lighting and color information from image data. However, their training and rendering processes are computationally intensive and inefficient, exhibiting significant performance bottlenecks, especially when handling high-resolution and complex scenes. Furthermore, existing NeRF methods struggle to recover high-resolution color 3D scenes from degraded low-resolution grayscale inputs.

[0007] 3. Three-dimensional reconstruction method based on three-dimensional Gaussian distribution

[0008] In recent years, representing 3D scenes using 3D Gaussian splatting has become an emerging trend. This method represents the geometric and color information of a scene by distributing Gaussian spheres in 3D space, offering advantages such as compact parameters, differentiable rendering speed, and fast rendering speed. However, existing methods mainly rely on high-quality color input images for modeling, and there are still no effective solutions for high-resolution reconstruction and colorization of degraded or grayscale images. Furthermore, current 3D Gaussian methods lack flexible color control and interactive editing mechanisms. Summary of the Invention

[0009] To address the problem that existing technologies cannot simultaneously achieve high-resolution 3D reconstruction under low-resolution grayscale input conditions, this invention provides a method for reconstructing a color 3D scene from a low-resolution grayscale image. This method reconstructs a high-resolution grayscale 3D scene through degenerate physical modeling and combines color palette representation and a controllable color optimization mechanism to achieve unified modeling from degenerate input to a high-quality color 3D scene.

[0010] The present invention discloses a method for reconstructing a color 3D scene from a low-resolution grayscale image, comprising the following steps:

[0011] S1. 3D Gaussian Scene Initialization: Based on a set of low-resolution grayscale images from multiple input perspectives. An initial 3D Gaussian scene representation is obtained through image enhancement and 3D geometry estimation. ;

[0012] S2. 3D Reconstruction Based on Degenerate Physical Model: Based on the degenerate physical model, using the initial 3D Gaussian scene representation... Starting from this point, optimization is performed to reconstruct a high-resolution 3D Gaussian scene representation. ;

[0013] S3. Colored Model Training: Utilizing the aforementioned high-resolution 3D Gaussian scene representation Rendered as a collection of grayscale images from multiple perspectives Color knowledge is acquired based on a pre-trained color generation model, and a scene-specific multi-view coloring model is trained. ;

[0014] S4. Palette-based 3D color optimization: Employing a palette-based color representation mechanism that includes primary colors, combined with a scene-specific multi-view coloring model. The three-dimensional Gaussian scene representation Perform color optimization to obtain a three-dimensional color scene with consistent colors. ;

[0015] S5. Rendering and Editing: This involves rendering and editing the 3D color scene. The rendering output is performed, and the scene colors can be edited in a controllable manner by adjusting the base colors of the color palette.

[0016] Preferably, step S1 includes:

[0017] S11, for the set of low-resolution grayscale images Perform single-image super-resolution processing to obtain an enhanced set of high-resolution grayscale images;

[0018] S12. Based on the enhanced high-resolution grayscale image set, the camera pose parameters are estimated and a sparse three-dimensional point cloud is generated by using a multi-view stereo matching and motion recovery structure method.

[0019] S13. Using the points in the sparse 3D point cloud as centers, initialize the 3D Gaussian parameter set to obtain the initial 3D Gaussian scene representation. The first in a 3D Gaussian scene A Gaussian ball use The four parameters represent, For location, Let covariance matrix be the variance matrix. For transparency, is the spherical harmonic coefficient.

[0020] Preferably, step S2 includes:

[0021] S21, Establish the current 3D Gaussian scene representation The rendering result is compared with the input set of low-resolution grayscale images. The physical degradation relationship between them is expressed by the degradation physical model as follows:

[0022]

[0023] in, For the degraded physical model The processed set of predicted low-resolution images, To perform the current 3D Gaussian splash rendering operation, For fuzzy kernel, This indicates a downsampling operation. Noise term;

[0024] S22, by minimizing the predicted low-resolution image set A collection of low-resolution grayscale images compared to real input The differences between them are used to construct a degradation loss function, which is then used to optimize the geometric parameters of the 3D Gaussian scene. The degradation loss function is as follows:

[0025]

[0026] By iteratively optimizing and minimizing the loss, a high-resolution 3D Gaussian scene representation is obtained. .

[0027] Preferably, step S3 includes:

[0028] S31, representing the high-resolution three-dimensional Gaussian scene. Rendered as a collection of grayscale images from multiple perspectives , For the first Aspect Ratio Grayscale Image;

[0029] S32, utilizing a pre-trained color generation model Each image in the multi-view grayscale image set is colorized to obtain the corresponding color image set. , For the first Aspect Ratio Grayscale Image The corresponding color image;

[0030] S33, with paired data As a supervisory signal, a lightweight scene-specific multi-view colorization model is trained using the knowledge distillation method. It takes a grayscale image as input and predicts the corresponding color image.

[0031] Preferably, step S4 includes:

[0032] S41, Initialization includes A global color palette for each primary color. , For the first A primary color is used to represent the high-resolution 3D Gaussian scene. Each Gaussian sphere Assign the corresponding color palette weight vector , The weight vector is the first The weight of each primary color;

[0033] S42, based on the color palette weights With base color Calculate each Gaussian sphere color The color The spherical harmonic coefficients used to determine or optimize the color characteristics related to the viewpoint of this Gaussian sphere. ;

[0034] S43, using the scene-specific multi-view coloring model To Rendered collection of multi-view grayscale images Perform coloring and generate color monitoring signals. ;

[0035] S44, representing the high-resolution three-dimensional Gaussian scene. Render a color image using the current color parameters. By minimizing and The differences between them optimize the weights of the color palette. and the spherical harmonic coefficients Its color reconstruction loss function is:

[0036]

[0037] After optimization, a three-dimensional color scene with consistent colors is obtained. .

[0038] Preferably, in step S42, the Gaussian sphere color Calculated by the linear combination of the color palette weights and the primary colors:

[0039] .

[0040] Preferably, step S5 includes:

[0041] S51, for the three-dimensional color scene Render the image to obtain a high-resolution color image.

[0042] S52, the primary colors of the color palette are adjusted interactively. The values ​​are then used to re-render the 3D color scene, achieving overall scene tone changes, style transfer, or local color enhancement.

[0043] The beneficial effects of this invention: Compared with the prior art, this invention provides a 3D super-resolution and coloring method based on Gaussian splashing, which can reconstruct high-resolution color 3D scenes from low-resolution grayscale images, and has the following significant and excellent beneficial effects:

[0044] 1. Significant High-Resolution Reconstruction Results: This invention reconstructs high-quality 3D Gaussian scenes from low-resolution grayscale images using a degenerate physics modeling method, without requiring additional supervisory signals to constrain the optimization of high-resolution scenes. This method effectively improves the resolution and quality of 3D reconstruction, resulting in more detailed and realistic reconstruction effects, suitable for various high-precision 3D reconstruction needs.

[0045] 2. Strong Coloring Consistency: This invention utilizes a trained single-image coloring model to colorize multi-view images, and integrates the coloring results into a small, scene-specific colorist through distillation technology. This ensures consistency across multiple views, avoiding color inconsistencies that may occur in traditional coloring methods, thereby significantly improving the visual effect and realism of 3D scenes.

[0046] 3. Flexible Color Editing Capabilities: This invention uses a color palette to represent the color of each Gaussian sphere, enabling convenient adjustment of the colors in a 3D scene. After coloring, the 3D scene can be quickly edited by simply changing the colors on the palette. This method not only improves the efficiency of color adjustment but also enhances the customizability of the 3D scene, adapting to the needs of different users and application scenarios.

[0047] 4. Broad Application Prospects: Due to its strong robustness and universality, the method of this invention can be applied to a variety of complex 3D reconstruction and coloring tasks. Whether it is industrial design and cultural relic protection requiring high-resolution 3D models, or film and television production and game development requiring high-quality coloring, this invention can provide reliable technical support and demonstrates broad application prospects. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the high-resolution 3D reconstruction based on physical modeling of the present invention.

[0049] Figure 2 A schematic diagram of training a scene-specific multi-view colorization model;

[0050] Figure 3 This is a schematic diagram of the three-dimensional coloring method based on the color palette of the present invention.

[0051] Figure 4 This is a visual comparison of the model results on simulation data for this invention.

[0052] Figure 5 This is a visual demonstration of the 3D scene color editing of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0056] Specific Implementation Method 1: The following is combined with... Figures 1 to 5 This embodiment describes a method for reconstructing a color 3D scene from a low-resolution grayscale image, such as... Figure 1As shown, this method mainly includes the following five core steps:

[0057] Step S1: Initialize the 3D Gaussian scene

[0058] Step S2: 3D Reconstruction Based on Degenerate Physical Model

[0059] Step S3: Colored Model Training

[0060] Step S4: Palette-based 3D color optimization

[0061] Step S5: Rendering and Editing

[0062] The following will explain in detail the specific implementation of each step.

[0063] Step S1: Initialize the 3D Gaussian scene

[0064] The goal of this step is to construct an initial, geometrically coarse 3D Gaussian scene representation based on the input set of low-resolution grayscale multi-view images.

[0065] Implementation process:

[0066] 1. Single-image super-resolution enhancement (corresponding to step S11):

[0067] Input: A set of low-resolution grayscale images from multiple viewpoints, denoted as Each image has a low resolution and may contain blur and noise.

[0068] Processing: Each image in the ensemble is processed independently using a pre-trained single-image super-resolution model (e.g., ESRGAN, Real-ESRGAN). This model learns the mapping from low resolution to high resolution.

[0069] Output: Enhanced high-resolution grayscale image sequence With the resolution increased (e.g., from 256×256 to 1024×1024), details are clearer.

[0070] 2. Structure of Motion (SfM) and Point Cloud Generation (corresponding to step S12):

[0071] Input: The above high-resolution grayscale image sequence .

[0072] Processing: Use open-source SfM tools (such as COLMAP) for processing.

[0073] a. Feature extraction and matching: Extract feature points such as SIFT or SuperPoint from each image and establish feature matching relationships between images.

[0074] b. Incremental reconstruction: Through iterative filtering and triangulation, the camera pose parameters (rotation matrix and translation vector) of each image are gradually recovered.

[0075] c. Point cloud generation: Generate a sparse 3D point cloud based on multi-view geometric constraints.

[0076] Output: Camera pose set and sparse 3D point cloud.

[0077] 3. Initialize the 3D Gaussian scene (corresponding to step S13):

[0078] Input: Sparse 3D point cloud.

[0079] Processing: For each point in the point cloud, the position is the center point of the Gaussian sphere, and a 3D Gaussian model in the 3D scene is initialized.

[0080] The position of each Gaussian sphere is directly taken from the point cloud coordinates; the covariance matrix is ​​initialized to an isotropic small-scale matrix; the transparency is initialized to a value close to 1 (e.g., 0.9); and the spherical harmonic coefficients are initialized to zero or the default value, indicating that there is no color information initially.

[0081] Output: Initial 3D Gaussian scene representation , The number is the Gaussian sphere number.

[0082] Step S2: 3D reconstruction based on the degenerate physical model:

[0083] Objective: To utilize a physical imaging degradation model as a supervisory signal to improve the performance of a rough initial scene. Optimized for accurate high-resolution 3D geometry This is one of the core innovative steps of this invention.

[0084] Implementation process:

[0085] 1. Construction of physical degradation model (corresponding to step S21):

[0086] Input: The 3D Gaussian scene representation of the current iteration (Initial time) ).

[0087] Forward rendering and degradation:

[0088] a. Rendering: Using a differentiable 3D Gaussian splash renderer, the current scene is rendered. From each training perspective Render a high-resolution grayscale image .

[0089] b. Simulated Degradation: Actively applying a known degradation process to the rendered high-resolution grayscale image to simulate its degradation into a low-resolution observation image.

[0090]

[0091] The degradation process includes blurring, downsampling, and noise addition. Blurring involves using a Gaussian blur kernel. Perform convolution to simulate camera or motion blur; downsampling: use bilinear or bicubic downsampling operations. (e.g., scaling factor of 4) to simulate sensor resolution limitations; noise addition: add Gaussian white noise. Simulate sensor noise.

[0092] Output: A set of predicted low-resolution images One image from each perspective, combined into a set for output.

[0093] 2. Loss Calculation and Parameter Optimization (S22):

[0094] Loss calculation: Calculate the prediction for the low-resolution image set. With real input low-resolution image set The mean square error between them is used as the degradation loss:

[0095]

[0096] Backpropagation and parameter update:

[0097] a. Calculate the loss Relative to each optimizable parameter (position) Covariance matrix (represented by scaling and rotation quaternions), transparency The gradient.

[0098] b. Update these parameters using a gradient descent optimizer (such as Adam).

[0099] Adaptive density control: During the optimization process, Gaussian elements are periodically cloned, split, or pruned based on gradient information, opacity, and other indicators to better fit the scene geometry.

[0100] Iterative loop: Repeat steps S21 and S22 until loss occurs. Convergence or reaching the preset number of iterations.

[0101] Output: Optimized high-resolution 3D Gaussian scene representation Its geometric details (shape, surface) have been significantly enhanced.

[0102] Step S3: Color model training (color knowledge transfer):

[0103] Objective: To distill and train a lightweight, scene-specific, multi-view color model tailored to the current scene, leveraging the color priors provided by a powerful pre-trained 2D color model. .

[0104] Implementation process:

[0105] 1. Multi-view grayscale image rendering (corresponding to step S31):

[0106] Input: Optimized high-resolution geometry scene (At this point, the color parameters are still at their default values).

[0107] Processing: Using the same renderer as S2, render the corresponding grayscale images from all training views.

[0108] Output: A collection of grayscale images from multiple perspectives .

[0109] 2. Coloring the pre-trained model (corresponding to step S32):

[0110] Input: a collection of grayscale images

[0111] Processing: Use a large, high-performance pre-trained color generation model For example, DDColor, DeOldify, or Colorization Transformer for each image It performs independent coloring. This model, trained on large-scale data, is able to predict reasonable and visually realistic colors for grayscale images.

[0112] Output: The corresponding set of color images ,in .

[0113] 3. Knowledge distillation training scenario-specific colorist (corresponding to step S33):

[0114] Training data: Paired data , As the output of the "teacher model", it provides a high-quality color supervision signal.

[0115] Student model: A lightweight convolutional neural network (such as a small UNet) with far fewer parameters than the teacher model;

[0116] Training process:

[0117] a. Forward propagation: Input the student model to obtain a predicted color image;

[0118] b. Loss Calculation: Calculate the loss between the predicted color plot and the teacher plot:

[0119] c. Backpropagation and update: Minimize the loss to update the parameters of the student model.

[0120] Output: A trained, lightweight, scene-specific, multi-view colorization model tailored to the current scene. The model "learned" how to consistently and with high quality color the multi-view grayscale image of the current specific object.

[0121] Step S4: 3D color optimization based on color palette

[0122] Objective: To "bake" 2D color knowledge into a 3D Gaussian scene, optimize color parameters using a color palette mechanism, and ensure consistent and high-quality colors from any viewpoint.

[0123] Implementation process:

[0124] 1. Initialize the color palette and weights (corresponding to step S41):

[0125] Global color palette: Initialization includes A global color palette for each primary color. Each of them This is an RGB vector. The primary colors can be randomly initialized or derived from the teacher model. The results are obtained by clustering the colorization results of a keyframe.

[0126] Gaussian sphere weights: for the scene Each Gaussian sphere Assign a weight vector ,satisfy and The weights can be initialized to a uniform distribution or simply assigned based on the spatial location of the Gaussian sphere.

[0127] 2. Relationship between basic colors and spherical harmonic coefficients (S42):

[0128] Basic color calculation: for each Gaussian sphere Its basic color Determined by a linear combination of the primary colors on the color palette and their weights:

[0129]

[0130] Spherical harmonics represent: In a 3D Gaussian scene, color changes with viewing angle due to spherical harmonics. Encoding. We will use the basic colors. This serves as a constraint or initialization target for the zeroth-order component of the spherical harmonic function. Higher-order spherical harmonic coefficients are initialized to zero, indicating that there are initially no view-dependent color changes.

[0131] 3. Color monitoring signal generation (corresponding to step S43):

[0132] Input: will Render grayscale images from various perspectives. ;

[0133] Processing: Use a trained scene-specific colorist Colorize these grayscale images.

[0134] Output: Color monitoring signal , as a reference color image.

[0135] 4. Color parameter optimization (corresponding to step S44):

[0136] Color rendering: Representing the high-resolution 3D Gaussian scene Render a color image using the current color parameters. ;

[0137] Loss Calculation: Calculation and Differences between them:

[0138]

[0139] Parameter optimization: Optimize the color palette weights. and the spherical harmonic coefficients ;

[0140] Iteration: Repeat S43 and S44 until the color loss converges.

[0141] Output: A color-optimized 3D color scene .

[0142] Step S5: Rendering and Editing

[0143] Objective: To provide high-quality output and support flexible, interactive color editing.

[0144] Implementation process:

[0145] 1. Final rendering (corresponding to step S51):

[0146] Input: Optimized 3D color scene .

[0147] Processing: Real-time rendering is performed using a 3D Gaussian splash renderer from any desired viewpoint (either a training viewpoint or a new viewpoint).

[0148] Output: High-resolution, high-quality color images or video sequences. Thanks to the efficiency of 3DGS, this process can achieve real-time frame rates.

[0149] 2. Palette-based interactive editing (corresponding to step S52):

[0150] Editing interface: Displays the global color palette to the user. The interface, each primary color It is displayed as a color block and allows users to modify its RGB values.

[0151] Real-time updates: By adjusting the colors in the color palette, the 3D scene is rendered again to generate a high-resolution color scene with different colors.

[0152] Application scenarios: It can achieve style transfer such as "retro brown tone", "cool blue tone", "seasonal change" (such as green in spring and yellow in autumn), or change the color of specific objects (such as clothes and car body).

[0153] Example: Reconstructing Lego Toys

[0154] The implementation process of this invention is illustrated by reconstructing a color 3D model of a LEGO toy from a set of low-resolution, black-and-white photos taken from multiple perspectives.

[0155] Input: Collect approximately 50 black and white photos (256x256 pixels) taken around LEGO toys. .

[0156] S1 Initialization: Each image is super-resolutioned to 1024×1024 pixels using the ESRGAN model. COLMAP is run to obtain the camera pose and sparse point cloud, and approximately 500,000 Gaussian elements are initialized to obtain... .

[0157] S2 geometric reconstruction: Constructing a physical degradation model (7×7 fuzzy kernel size, downsampling factor 4, and a small amount of Gaussian noise). After approximately 30,000 iterations of optimization, a high-resolution geometric model with clear details is obtained. The bumps and textures on the LEGO bricks were restored.

[0158] S3 Color Shift: From Render 50 grayscale images. Use the DDColor model to colorize them, obtaining vibrant and appropriate color reference images. Train a lightweight UNet model using these paired data. .

[0159] S4 Color Optimization: Initialize a color palette containing 8 primary colors (including common LEGO colors such as red, yellow, blue, and white). After approximately 5,000 color optimization iterations, a colored 3D model is obtained. The colors remain consistent and realistic from any viewing angle.

[0160] S5 Rendering and Editing:

[0161] Rendering: Generate a 360-degree rotating high-definition video of a LEGO toy.

[0162] Editor's Note: The user changed the "bright red" base color in the color palette to "dark blue," and the original red parts of the entire LEGO model instantly turned blue, while the other colors remained unchanged, generating a "new color scheme" version of the LEGO 3D model.

[0163] Implementation effect verification and demonstration:

[0164] To verify the effectiveness of the method of the present invention, a comparative experiment was conducted on a standard simulation dataset, and the color editing function was visualized.

[0165] 1. Comparison of reconstruction quality and color consistency (see attached document) Figure 4 )

[0166] Figure 4 The results show a visual comparison of the method of the present invention with existing technologies on simulation data.

[0167] The first column contains two low-resolution grayscale images from different perspectives.

[0168] The second column shows the rendering results of the existing technical solution. This solution first uses the existing 3D super-resolution method SRGS to obtain a high-resolution grayscale scene, and then independently colors and optimizes the colors of the images from each viewpoint based on the DDColor model. The results show that the texture reconstructed by this method is relatively blurry, and the same object is rendered with inconsistent colors under different viewpoints, which seriously affects the realism and usability of the 3D experience.

[0169] The third column shows the rendering results of the method proposed in this invention. As can be seen from the comparison, the method of this invention not only reconstructs finer geometric textures and details, but also strictly guarantees color consistency across multiple viewpoints. This is thanks to the invention's application of color knowledge to a unified 3D representation through a color palette mechanism, fundamentally avoiding color conflicts between viewpoints.

[0170] 2. Demonstration of color controllable editing function (see attached document) Figure 5 )

[0171] Figure 5 This demonstrates the ability to edit colors in a 3D scene based on a color palette. The color palette parameters, optimized in step S4, together with other parameters of the Gaussian model, define a colored 3D scene.

[0172] The first line shows a flower rendered using optimized original parameters, in pink color, and displays the corresponding initial color palette.

[0173] The second line shows the result of the system re-rendering in real time after the user lightened the first base color in the color palette through the interactive interface; the flower color changed to a light pinkish-purple.

[0174] The third line shows how the flower color turns blue when the base color is further lightened to a certain blue hue.

[0175] This process demonstrates that the method of the present invention can conveniently and coherently change the color style of the entire 3D scene by adjusting only the values ​​of a few primary colors in the global color palette, without the need for time-consuming 3D reconstruction or color optimization, thus greatly improving the efficiency of art design and workflow.

[0176] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A method of reconstructing a color three-dimensional scene from a low resolution gray scale image, characterized by, The method comprises the following steps: S1, three-dimensional Gaussian scene initialization: based on input multi-view low-resolution grayscale image set , through image enhancement and three-dimensional geometry estimation, obtain initial three-dimensional Gaussian scene representation ; S2, three-dimensional reconstruction based on a degenerate physical model: based on a degenerate physical model, reconstructing the initial three-dimensional Gaussian scene representation Optimization from scratch, reconstructing a high-resolution three-dimensional Gaussian scene representation ; S3, color model training: using the high-resolution three-dimensional Gaussian scene representation rendering into a multi-view grayscale image set , obtaining color knowledge based on a pre-trained color generation model, and training to obtain a scene-specific multi-view colorization model ; S4. Palette-based three-dimensional color optimization: employing a palette color representation mechanism containing base colors, in combination with the scene-specific multi-view shading model performing color optimization on the three-dimensional Gaussian scene representation to obtain a color-consistent three-dimensional color scene ; S5, rendering and editing: rendering output of the three-dimensional color scene and controllable editing of the scene colors by adjusting the base colors of the color wheel.

2. The method of claim 1, wherein, The step S1 comprises: S11, for the set of low-resolution grayscale images Perform single-image super-resolution processing to obtain an enhanced set of high-resolution grayscale images; S12, based on the enhanced high-resolution gray image set, estimating camera pose parameters and generating a sparse three-dimensional point cloud through multi-view stereo matching and motion structure recovery method; S13. Using the points in the sparse 3D point cloud as centers, initialize the 3D Gaussian parameter set to obtain the initial 3D Gaussian scene representation. The first in a 3D Gaussian scene A Gaussian ball use The four parameters represent, For location, Let covariance matrix be the variance matrix. For transparency, is the spherical harmonic coefficient.

3. The method of claim 2, wherein, The step S2 comprises: S21, establishing a current three-dimensional Gaussian scene representation between the rendered result and the input set of low-resolution grayscale images between the rendered result and the input set of low-resolution grayscale images wherein, is a degenerated physical model a processed set of predicted low resolution images, is a current three-dimensional Gaussian splatting rendering operation, is a blur kernel, denotes a down-sampling operation, is a noise term; S22, by minimizing the predicted low-resolution image set A collection of low-resolution grayscale images compared to real input The differences between them are used to construct a degradation loss function, which is then used to optimize the geometric parameters of the 3D Gaussian scene. The degradation loss function is as follows: minimizing the loss by iterative optimization to obtain an optimized high-resolution three-dimensional Gaussian scene representation .

4. The method of claim 1, wherein The step S3 comprises: S31, rendering the high-resolution three-dimensional Gaussian scene representation as a multi-view grayscale image set rendering as a multi-view grayscale image set , for the first frame view grayscale image S32, using the pre-trained color generation model coloring each image in the multi-view grayscale image set to obtain a corresponding color image set , for the first view grayscale image corresponding color image S33, with the pairing data As a supervision signal, a lightweight scene-specific multi-view colorization model is trained by a knowledge distillation method which takes a grayscale image as input and predicts the corresponding color image.

5. The method of claim 4, wherein, The step S4 comprises: S41, initializing a global palette containing a number of bases , a number of bases for the high resolution three-dimensional Gaussian scene representation and assigning a corresponding palette weight vector to each Gaussian sphere in the high resolution three-dimensional Gaussian scene representation , a weight of the bases in the weight vector; S42, based on the palette weights with the primary colors , compute a color for each of the Gaussian spheres of the color , the color used to determine or optimize the spherical harmonic coefficients of the Gaussian sphere view-dependent color characteristics ; S43, using the scene-specific multi-view colorization model to the rendered multi-view grayscale image set to the rendered multi-view grayscale image set to generate a color supervision signal ; S44, optimizing the high-resolution three-dimensional Gaussian scene representation rendering as a color image with current color parameters by minimizing the difference between and the palette weights and the spherical harmonic coefficients with a color reconstruction loss function: Optimizing a three-dimensional color scene to obtain a color consistent three-dimensional color scene .

6. The method of claim 5, wherein, In the step S42, the Gaussian sphere of colors is calculated from the linear combination of the palette weights and the primaries. 。 7. The method of claim 5, wherein: The step S5 comprises: S51, rendering the three-dimensional color scene to obtain a high-resolution color image; S52, adjusting the base colors of the palette interactively the values of the colors and re-rendering the three-dimensional color scene, achieving overall color tone change, style transfer or local color enhancement of the scene.

Images