Optimization method for 3D scene recoloring based on Gaussian splatting and deep learning

Abstract

The present invention discloses a three-dimensional scene recoloring optimization method based on Gaussian splash and deep learning, comprising the following steps: S1, constructing a multi-view data set of a given scene to generate an initial point cloud; S2, based on the initial point cloud, extracting a representative color set from all input pixels, generating a color palette and using it as an initial setting; S3, rendering a multi-view result image using the color palette and Gaussian splash algorithm, calculating a loss function, optimizing parameters by back propagation, and generating an optimized Gaussian radiation field; S4, re-rendering the scene and generating a recoloring result based on the optimized Gaussian radiation field and the palette color modified by the user. The present invention combines the Gaussian splash algorithm with deep learning technology to achieve efficient recoloring optimization of three-dimensional scenes, with the advantages of intuitive editing, precise illumination expression, and realistic rendering.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and three-dimensional scene editing, and in particular to a three-dimensional scene recoloring optimization method based on Gaussian splatting and deep learning. Background Art

[0002] In the field of computer graphics and computer vision, the editing of three-dimensional content is an important direction of current research. With the continuous advancement of technology, people's demand for the appearance editing of three-dimensional scenes is growing. How to edit the appearance of scenes in an efficient and accurate way has become an important research topic. In traditional three-dimensional scene editing, the appearance of the scene is usually adjusted by directly manipulating the geometric model, material parameters and lighting model. However, these methods are not only complex to operate, but also difficult to adapt to the needs of rapid generation and diversification of modern three-dimensional content. Therefore, exploring more automated and intelligent three-dimensional scene editing methods has become an important research direction in the field of computer graphics.

[0003] In recent years, 3D scene representation methods based on radiance fields have had a profound impact on related research. Such methods can achieve photorealistic new perspective synthesis by reconstructing complex 3D scenes from multi-view images. Among them, neural radiance fields (NeRF) and its variants are a representative technology, which encodes the geometric structure and appearance of the scene through a multi-layer perceptron (MLP), providing powerful expression capabilities for scene generation and rendering. However, NeRF still faces many challenges in scene appearance editing. Since the scene appearance information of NeRF is encoded in a multi-layer perceptron network, the appearance parameters are difficult to directly decode and adjust, which makes local editing of the scene very complex and unintuitive. In addition, NeRF has a weak ability to express high-frequency lighting information, which further limits its application in high-precision editing scenes.

[0004] Compared with NeRF, explicit point-based radiation field methods provide a more intuitive way to represent scenes. Among them, the 3D Gaussian splattering algorithm (3D-GS) provides the possibility of local editing of scene appearance by modeling the scene using explicit Gaussian kernels. However, 3D-GS still faces many technical difficulties. First, 3D-GS lacks a method to accurately identify the target Gaussian kernel, which makes local editing of specific areas difficult to achieve. Second, 3D-GS uses spherical harmonics (SH) to model anisotropic colors for each Gaussian kernel. Although this method can theoretically support the expression of high-frequency lighting information, in practice, its ability to capture high-frequency lighting details is limited. In addition, the parameter adjustment of spherical harmonics is relatively complex and not intuitive enough in practical applications, which limits its user operability.

[0005] In response to the above problems, academia and industry have proposed a variety of improvement methods to enhance the appearance editing capabilities of three-dimensional scenes. At present, these methods can be mainly divided into two categories: methods based on 3D style transfer and methods based on text-driven editing. The method based on 3D style transfer extracts texture or lighting features from the reference style and attempts to transfer these features to the target scene. This method has a good performance in achieving the overall stylization effect, but it is powerless when it comes to achieving fine-grained control. The method based on text-driven editing attempts to guide scene editing through natural language descriptions, such as using a pre-trained text-based diffusion model to generate or modify specific areas of the scene. Although this method has certain innovations in terms of interaction, it often requires a large amount of GPU memory due to its reliance on large pre-trained models, which greatly increases the implementation cost. In addition, text-driven editing still has difficulties in achieving precise control, such as adjusting the color of an object in the scene to a specific RGB value, which is difficult to achieve through pure text descriptions in actual operations.

[0006] In view of the shortcomings of the above-mentioned existing methods, the palette-based recoloring method has achieved a certain balance between editing granularity and ease of use. By introducing the intermediate representation of the palette, the colors in the scene can be adjusted more intuitively. However, the existing palette method still faces the problem of "color pollution" in practical applications, that is, a certain palette color is too widely distributed in the radiation field. When the user modifies the color, a large range of color changes will appear in the rendering result, which is far beyond the user's expectations. In addition, the existing methods also have shortcomings in generating palettes and optimizing rendering quality. For example, there is a lack of a mechanism for fine control of palette colors, making it difficult to balance fine-grained editing and global effects.

[0007] Therefore, how to provide a three-dimensional scene recoloring optimization method based on Gaussian splatting and deep learning is a problem that technicians in this field need to solve urgently. Summary of the invention

[0008] One purpose of the present invention is to propose a three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning. The present invention combines the Gaussian splashing algorithm with deep learning technology, introduces palette generation, rasterization rendering and multi-layer perceptron network optimization, solves the problems of insufficient local control ability and color pollution in three-dimensional scene editing, and realizes efficient three-dimensional scene recoloring optimization. The method supports users to fine-tune the appearance of the scene, improves the intuitiveness and ease of editing, and significantly enhances the expression of lighting details and the realism of rendering effects through sparsity loss function and palette offset optimization. It has the advantages of high efficiency, accuracy and high interactivity, and is suitable for a variety of three-dimensional content creation scenes.

[0009] According to an embodiment of the present invention, a three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning includes the following steps:

[0010] S1. Build a multi-view dataset for a given scene and generate an initial point cloud;

[0011] S2. Based on the initial point cloud, extract a representative color set from all input pixels, generate a color palette and use it as an initial setting;

[0012] S3, using the color palette and Gaussian splash algorithm to render the multi-view result image, calculate the loss function, optimize the parameters by back propagation, and generate an optimized Gaussian radiation field;

[0013] S4. Re-render the scene and generate a reshading result based on the optimized Gaussian radiation field and the palette color modified by the user.

[0014] Optionally, the S1 specifically includes:

[0015] S11. Take multi-view images at different angles of a given scene, and the spatial distribution of the viewpoints is uniform, to construct a multi-view dataset;

[0016] S12, use COLMAP to perform camera calibration and obtain the camera intrinsic parameters and extrinsic parameters corresponding to each multi-view image;

[0017] S13. Use a structured light model algorithm to match the correspondence between pixels of the multi-view images. The structured light model algorithm is used to calculate the geometric relationship between the multi-view images through feature point matching of the multi-view images, and generate an initial point cloud using incremental 3D reconstruction technology.

[0018] Optionally, the S2 specifically includes:

[0019] S21, extracting the pixel colors of all multi-view images in the multi-view dataset, and performing L2 normalization processing on all RGB color vectors;

[0020] S22, taking all extracted pixel colors as points in the RGB space and constructing a convex hull;

[0021] S23, using the edge collapse algorithm to iteratively simplify the convex hull and continuously reduce the number of convex hull vertices;

[0022] S24, setting an error threshold, and determining whether to stop simplification based on an iterative error judgment condition; the iterative error includes that the point error outside the convex hull is the minimum projection distance to the convex hull surface, and the point error inside the convex hull is the distance to the nearest convex hull vertex;

[0023] S25, when the sum of the point error outside the convex hull and the point error inside the convex hull is less than the error threshold, stop iterative simplification;

[0024] S26. Use the final simplified convex hull vertex colors as the initial settings of the palette.

[0025] Optionally, the S3 specifically includes:

[0026] S31. Define the expression of Gaussian kernel:

[0027] ;

[0028] in, represents the Gaussian kernel, x represents the position in the three-dimensional scene, represents the center position of the Gaussian kernel, represents the covariance matrix of the Gaussian kernel;

[0029] S32. Generate pixel color values ​​using the rasterization rendering formula of the Gaussian kernel:

[0030] ;

[0031] in, Represents a two-dimensional pixel position The color value of represents the color value of the i-th Gaussian kernel, represents the transparency of the i-th Gaussian kernel, represents the transparency of the jth Gaussian kernel, Indicates the i-th Gaussian kernel at the two-dimensional pixel position The contribution value of Indicates the jth Gaussian kernel at the two-dimensional pixel position The contribution value of , N represents the set of all Gaussian kernels;

[0032] S33. The sight direction vectors of each Gaussian kernel are used as the input of the multi-layer perceptron network, and a set of basis functions are output to represent the anisotropic offset of the palette color:

[0033] ;

[0034] in, represents the offset function, d represents the line of sight direction, Indicates the offset value of the palette color in the L dimension in the CIE LAB color space:

[0035] ;

[0036] in, Represents the final offset value, represents the basis coefficient of the i-th basis function, represents the offset value of the i-th basis function, and n represents the number of basis functions used to describe the color offset of the palette;

[0037] S34. Use the alpha blend formula to convert the transparency of the Gaussian kernel into a weight:

[0038] ;

[0039] in, represents the weight of the i-th Gaussian kernel;

[0040] S35. Combine the weight with the offset palette color to calculate the final color of the Gaussian kernel:

[0041] ;

[0042] in, represents the final color of the Gaussian kernel, W represents the weight matrix, Indicates the initial color of the palette;

[0043] S36, using a differentiable block rasterizer to render the calculated final color of the Gaussian kernel into a two-dimensional image to obtain a rendered image;

[0044] S37. Use the weighted average of the difference between the rendered image and the actual image pixel color and the SSIM score as the main loss function:

[0045] ;

[0046] in, represents the main loss function, Represents the weight coefficient of SSIM loss, Indicates the loss of pixel color difference between the rendered image and the actual image. represents the SSIM score loss;

[0047] Add smoothed weight L2 norm as sparsity loss function:

[0048] ;

[0049] in, represents the sparsity loss function, M represents the number of colors in the palette, represents a smooth approximation to the 0-1 indicative function;

[0050] Define the palette offset loss function:

[0051] ;

[0052] in, represents the palette offset value loss function, and P represents the current color of the palette;

[0053] S38. Optimize the Gaussian radiation field parameters and the multi-layer perceptron network parameters through the back propagation algorithm to generate an optimized Gaussian radiation field.

[0054] Optionally, the S4 specifically includes:

[0055] S41, the user edits one or more colors in the color palette to specified values;

[0056] S42, calculating the difference between the edited color palette and the original color palette;

[0057] S43, applying the difference to the color palette of each Gaussian kernel shifted in the CIE LAB color space;

[0058] S44. Re-render the scene and generate the reshading result.

[0059] The beneficial effects of the present invention are:

[0060] First, the present invention achieves accurate recognition of the target Gaussian kernel by introducing an explicit Gaussian splashing algorithm, thereby enabling fine editing of local areas of the scene. This method effectively makes up for the problem of insufficient local control ability of traditional explicit methods, allowing users to modify the appearance of the three-dimensional scene more intuitively while improving the accuracy of editing.

[0061] Secondly, the present invention uses a multi-layer perceptron network to generate anisotropic offsets of palette colors, significantly enhancing the ability to express high-frequency lighting information. Compared with the traditional spherical harmonic function method, the present invention not only performs better in capturing lighting details, but also can dynamically adjust the palette color to adapt to different editing needs, ensuring that the final rendering effect is more realistic. By introducing a smooth sparsity loss function, the present invention also solves the common "color pollution" problem in existing palette methods, effectively avoiding large-scale unreasonable changes in rendering results caused by modifying the palette color.

[0062] In addition, the present invention introduces mechanisms such as L2 normalization, edge collapse algorithm, and error threshold control in the palette generation process, which greatly improves the accuracy and robustness of palette color extraction. Combined with the sparsity loss function and the palette offset loss function, the present invention optimizes the Gaussian radiation field parameters and the multi-layer perceptron network parameters to make color adjustment more stable and ensure the controllability of color changes during editing. At the same time, the present invention uses a differentiable block rasterizer to efficiently render the optimized Gaussian kernel color into a two-dimensional image, further improving the interactivity and response speed of editing.

[0063] Finally, the present invention makes the editing of the appearance of three-dimensional scenes more intuitive and easy to use by introducing a user-editable palette. Users only need to simply modify the palette color to quickly adjust the appearance of the scene without relying on complex model training or large-scale computing resources. This efficient and low-threshold operation method significantly reduces the difficulty of three-dimensional content creation and opens up new application areas for three-dimensional scene editing technology. At the same time, by combining the optimized rendering process, the present invention achieves a good balance between local adjustment and global consistency, so that the final rendering result has a high degree of visual realism and meets user expectations. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0065] Figure 1 This is an overall flow chart of the three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning proposed by the present invention;

[0066] Figure 2 This is a comparison chart of the effects of the three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning proposed in the present invention and the 3D Gaussian splashing algorithm. DETAILED DESCRIPTION

[0067] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0068] refer to Figure 1 and Figure 2 , a three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning includes the following steps:

[0069] S1. Build a multi-view dataset for a given scene and generate an initial point cloud;

[0070] S2. Based on the initial point cloud, extract a representative color set from all input pixels, generate a color palette and use it as an initial setting;

[0071] S3, using the color palette and Gaussian splash algorithm to render the multi-view result image, calculate the loss function, optimize the parameters by back propagation, and generate an optimized Gaussian radiation field;

[0072] S4. Re-render the scene and generate a reshading result based on the optimized Gaussian radiation field and the palette color modified by the user.

[0073] In this implementation manner, the S1 specifically includes:

[0074] S11. Take multi-view images at different angles of a given scene, and the spatial distribution of the viewpoints is uniform, to construct a multi-view dataset;

[0075] S12, use COLMAP to perform camera calibration and obtain the camera intrinsic parameters and extrinsic parameters corresponding to each multi-view image;

[0076] S13. Use a structured light model algorithm to match the correspondence between pixels of the multi-view images. The structured light model algorithm is used to calculate the geometric relationship between the multi-view images through feature point matching of the multi-view images, and generate an initial point cloud using incremental 3D reconstruction technology.

[0077] In this implementation, S2 specifically includes:

[0078] S21, extracting the pixel colors of all multi-view images in the multi-view dataset, and performing L2 normalization processing on all RGB color vectors;

[0079] S22, taking all extracted pixel colors as points in the RGB space and constructing a convex hull;

[0080] S23, using the edge collapse algorithm to iteratively simplify the convex hull and continuously reduce the number of convex hull vertices;

[0081] S24, setting an error threshold, and determining whether to stop simplification based on an iterative error judgment condition; the iterative error includes that the point error outside the convex hull is the minimum projection distance to the convex hull surface, and the point error inside the convex hull is the distance to the nearest convex hull vertex;

[0082] S25, when the sum of the point error outside the convex hull and the point error inside the convex hull is less than the error threshold, stop iterative simplification;

[0083] S26. Use the final simplified convex hull vertex colors as the initial settings of the palette.

[0084] In this implementation, S3 specifically includes:

[0085] S31. Define the expression of Gaussian kernel:

[0086] ;

[0087] in, represents the Gaussian kernel, x represents the position in the three-dimensional scene, represents the center position of the Gaussian kernel, represents the covariance matrix of the Gaussian kernel;

[0088] S32. Generate pixel color values ​​using the rasterization rendering formula of the Gaussian kernel:

[0089] ;

[0090] in, Represents a two-dimensional pixel position The color value of represents the color value of the i-th Gaussian kernel, represents the transparency of the i-th Gaussian kernel, represents the transparency of the jth Gaussian kernel, Indicates the i-th Gaussian kernel at the two-dimensional pixel position The contribution value of Indicates the jth Gaussian kernel at the two-dimensional pixel position The contribution value of , N represents the set of all Gaussian kernels;

[0091] S33. The sight direction vectors of each Gaussian kernel are used as the input of the multi-layer perceptron network, and a set of basis functions are output to represent the anisotropic offset of the palette color:

[0092] ;

[0093] in, represents the offset function, d represents the line of sight direction, Indicates the offset value of the palette color in the L dimension in the CIE LAB color space:

[0094] ;

[0095] in, Represents the final offset value, represents the basis coefficient of the i-th basis function, represents the offset value of the i-th basis function, and n represents the number of basis functions used to describe the color offset of the palette;

[0096] S34. Use the alpha blend formula to convert the transparency of the Gaussian kernel into a weight:

[0097] ;

[0098] in, represents the weight of the i-th Gaussian kernel;

[0099] S35. Combine the weight with the offset palette color to calculate the final color of the Gaussian kernel:

[0100] ;

[0101] in, represents the final color of the Gaussian kernel, W represents the weight matrix, Indicates the initial color of the palette;

[0102] S36, using a differentiable block rasterizer to render the calculated final color of the Gaussian kernel into a two-dimensional image to obtain a rendered image;

[0103] S37. Use the weighted average of the difference between the rendered image and the actual image pixel color and the SSIM score as the main loss function:

[0104] ;

[0105] in, represents the main loss function, Represents the weight coefficient of SSIM loss, Indicates the loss of pixel color difference between the rendered image and the actual image. represents the SSIM score loss;

[0106] Add smoothed weight L2 norm as sparsity loss function:

[0107] ;

[0108] in, represents the sparsity loss function, M represents the number of colors in the palette, represents a smooth approximation to the 0-1 indicative function;

[0109] Define the palette offset loss function:

[0110] ;

[0111] in, represents the palette offset value loss function, and P represents the current color of the palette;

[0112] S38. Optimize the Gaussian radiation field parameters and the multi-layer perceptron network parameters through the back propagation algorithm to generate an optimized Gaussian radiation field.

[0113] In this implementation manner, the S4 specifically includes:

[0114] S41, the user edits one or more colors in the color palette to specified values;

[0115] S42, calculating the difference between the edited color palette and the original color palette;

[0116] S43, applying the difference to the color palette of each Gaussian kernel shifted in the CIE LAB color space;

[0117] S44. Re-render the scene and generate the reshading result.

[0118] Embodiment 1:

[0119] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the lego scene in the Synthetic NeRF data, and the present invention performs a recoloring task on it.

[0120] Read the multi-view images in the lego scene training set, call COLMAP to calibrate the camera position, and obtain the camera intrinsic and extrinsic parameters. Reconstruct the scene into a sparse point cloud representation through the feature matching algorithm, and use it as the initial Gaussian kernel position of the Gaussian splash algorithm.

[0121] The palette corresponding to the training set image is extracted according to the palette extraction algorithm of the present invention, and the error threshold is set to 0.008.

[0122] The position and shape of the Gaussian kernel are controlled using the following formula:

[0123] ;

[0124] in, represents the Gaussian kernel, x represents the position in the three-dimensional scene, represents the center position of the Gaussian kernel, represents the covariance matrix of the Gaussian kernel;

[0125] The sight direction vectors of each Gaussian ball are used as the input of the multi-layer perceptron network, and a set of basis functions are output to represent the anisotropic shift of the palette color, where n is set to 8:

[0126] ;

[0127] The dot product of the basis coefficient vector f contained in each Gaussian kernel and the basis function obtains the offset value of the palette color in the L dimension of the CIE LAB color space:

[0128] ;

[0129] The offset only needs to be applied to the palette color with the largest weight.

[0130] Use the alpha blend formula to convert the transparency of each layer corresponding to each color in the palette contained in each Gaussian kernel into a weight:

[0131] ;

[0132] in, represents the weight of the i-th Gaussian kernel;

[0133] The dot product of the weight vector and the palette color with the offset applied is used to get the final color of the Gaussian kernel:

[0134] ;

[0135] in, represents the final color of the Gaussian kernel, W represents the weight matrix, Indicates the initial color of the palette;

[0136] Use a differentiable block rasterizer to render all Gaussian kernel colors into a two-dimensional image. The rasterization rendering formula of the Gaussian kernel is defined as:

[0137] ;

[0138] in, Represents a two-dimensional pixel position The color value of represents the color value of the i-th Gaussian kernel, represents the transparency of the i-th Gaussian kernel, represents the transparency of the jth Gaussian kernel, Indicates the i-th Gaussian kernel at the two-dimensional pixel position The contribution value of Indicates the jth Gaussian kernel at the two-dimensional pixel position The contribution value of , N represents the set of all Gaussian kernels;

[0139] The weighted average of the difference between the rendered image and the actual image pixel color and the SSIM score is used as the main loss function, where Set to 0.2;

[0140] ;

[0141] in, represents the main loss function, Represents the weight coefficient of SSIM loss, Indicates the loss of pixel color difference between the rendered image and the actual image. represents the SSIM score loss;

[0142] Add smoothed weight L2 norm as sparsity loss function:

[0143] ;

[0144] in, represents the sparsity loss function, M represents the number of colors in the palette, represents a smooth approximation to the 0-1 indicative function;

[0145] As the number of training rounds increases, the The value also decreases continuously, defining the value sequence [1, 0.5, 0.2, 0.1, 0.05, 0.02, 0.01], The value is initially set to 1 and updated to the next value in the sequence after every 1000 iterations. The coefficient of this loss function is set to 1.

[0146] Add the offset value of the palette color as one of the loss functions, and set the coefficient of the loss function to 0.01:

[0147] ;

[0148] in, represents the palette offset value loss function, and P represents the current color of the palette.

[0149] The Gaussian radiation field parameters and multi-layer perceptron network parameters were optimized through the back-propagation algorithm, trained for 30,000 rounds, and the model and optimized color palette were saved.

[0150] Specify a color in the palette and update it to another color. The model parameters remain unchanged and re-render to get the recolored result.

[0151] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning, characterized in that: The steps include: S1. Build a multi-view dataset for a given scene and generate an initial point cloud; S2. Based on the initial point cloud, extract a representative color set from all input pixels, generate a color palette and use it as an initial setting; S3, using the color palette and Gaussian splash algorithm to render the multi-view result image, calculate the loss function, optimize the parameters by back propagation, and generate an optimized Gaussian radiation field; S4, re-rendering the scene and generating a reshading result based on the optimized Gaussian radiation field and the palette color modified by the user; The S3 specifically includes: S31. Define the expression of Gaussian kernel: ； in, represents the Gaussian kernel, x represents the position in the three-dimensional scene, represents the center position of the Gaussian kernel, represents the covariance matrix of the Gaussian kernel; S32. Generate pixel color values ​​using the rasterization rendering formula of the Gaussian kernel: ； in, Represents a two-dimensional pixel position The color value of represents the color value of the i-th Gaussian kernel, represents the transparency of the i-th Gaussian kernel, represents the transparency of the jth Gaussian kernel, Indicates the i-th Gaussian kernel at the two-dimensional pixel position The contribution value of Indicates the jth Gaussian kernel at the two-dimensional pixel position The contribution value of , N represents the set of all Gaussian kernels; S33. The sight direction vectors of each Gaussian kernel are used as the input of the multi-layer perceptron network, and a set of basis functions are output to represent the anisotropic offset of the palette color: ； in, represents the offset function, d represents the line of sight direction, Indicates the offset value of the palette color in the L dimension in the CIELAB color space: ； in, Represents the final offset value, represents the basis coefficient of the i-th basis function, represents the offset value of the i-th basis function, and n represents the number of basis functions used to describe the color offset of the palette; S34. Use the alpha blend formula to convert the transparency of the Gaussian kernel into a weight: ； in, represents the weight of the i-th Gaussian kernel; S35. Combine the weight with the offset palette color to calculate the final color of the Gaussian kernel: ； in, represents the final color of the Gaussian kernel, W represents the weight matrix, Indicates the initial color of the palette; S36, using a differentiable block rasterizer to render the calculated final color of the Gaussian kernel into a two-dimensional image to obtain a rendered image; S37. Use the weighted average of the difference between the rendered image and the actual image pixel color and the SSIM score as the main loss function: ； in, represents the main loss function, Represents the weight coefficient of SSIM loss, Indicates the loss of pixel color difference between the rendered image and the actual image. represents the SSIM score loss; Add smoothed weight L2 norm as sparsity loss function: ； in, represents the sparsity loss function, M represents the number of colors in the palette, represents a smooth approximation to the 0-1 indicative function; Define the palette offset loss function: ； in, represents the palette offset value loss function, and P represents the current color of the palette; S38. Optimize the Gaussian radiation field parameters and the multi-layer perceptron network parameters through the back propagation algorithm to generate an optimized Gaussian radiation field.

2. The three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning according to claim 1, characterized in that: The S1 specifically includes: S11. Take multi-view images at different angles of a given scene, and the spatial distribution of the viewpoints is uniform, to construct a multi-view dataset; S12, use COLMAP to perform camera calibration and obtain the camera intrinsic parameters and extrinsic parameters corresponding to each multi-view image; S13. Use a structured light model algorithm to match the correspondence between pixels of the multi-view images. The structured light model algorithm is used to calculate the geometric relationship between the multi-view images through feature point matching of the multi-view images, and generate an initial point cloud using incremental 3D reconstruction technology.

3. The three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning according to claim 1, characterized in that: The S2 specifically includes: S21, extracting the pixel colors of all multi-view images in the multi-view dataset, and performing L2 normalization processing on all RGB color vectors; S22, taking all extracted pixel colors as points in the RGB space and constructing a convex hull; S23, using the edge collapse algorithm to iteratively simplify the convex hull and continuously reduce the number of convex hull vertices; S24, setting an error threshold, and determining whether to stop simplification based on an iterative error judgment condition; the iterative error includes that the point error outside the convex hull is the minimum projection distance to the convex hull surface, and the point error inside the convex hull is the distance to the nearest convex hull vertex; S25, when the sum of the point error outside the convex hull and the point error inside the convex hull is less than the error threshold, stop iterative simplification; S26. Use the final simplified convex hull vertex colors as the initial settings of the palette.

4. The three-dimensional scene recoloring optimization method based on Gaussian splashing and deep learning according to claim 1, characterized in that: The S4 specifically includes: S41, the user edits one or more colors in the color palette to specified values; S42, calculating the difference between the edited color palette and the original color palette; S43, applying the difference to the color palette of each Gaussian kernel shifted in the CIE LAB color space; S44. Re-render the scene and generate the reshading result.

Images