A 3D gaussian sputtering style method, system, device and storage medium
By combining nonnegative matrix factorization and color orthogonal decoding models with multi-scale feature loss of VGG network, the problems of color inconsistency and geometric degradation in 3D Gaussian sputtering methods are solved, and efficient and stable 3D scene art style transfer is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV
- Filing Date
- 2026-03-05
- Publication Date
- 2026-06-05
AI Technical Summary
Existing 3D Gaussian sputtering methods suffer from problems such as color inconsistency and dirty colors, geometric degradation, low and unstable optimization efficiency, and limitations in optimization starting point and convergence, resulting in poor stylization effects.
Non-negative matrix factorization is used to extract the color basis of style images, and a color orthogonal decoding model is constructed. Combined with the multi-scale feature loss of the VGG network, geometric and color attributes are updated through iterative optimization. Brightness consistency and edge gradient loss are introduced, and the loss weights are dynamically adjusted to achieve global color consistency and geometric structure preservation.
It achieves high-fidelity and efficient 3D scene art style transfer, solves the problems of color inconsistency and dirty colors, reduces geometric structure degradation, improves optimization efficiency and stability, and ensures the quality of stylization effects.
Smart Images
Figure CN122156554A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a 3D Gaussian sputtering stylization method, system, device and storage medium. Background Technology
[0002] With the development of computer graphics, 3D style transfer has become a popular research area. Early techniques were mainly based on neural radiance fields (NeRFs), such as ARFs (Artistic Radiance Fields), which achieved high-quality style transfer through nearest neighbor feature matching (NNFM). However, due to the implicit representation of NeRFs, their training and rendering speeds were slow. To address the efficiency issue, stylization techniques based on 3D Gaussian sputtering (3DGS) have emerged.
[0003] Existing 3D Gaussian sputtering methods are mainly divided into two categories: one is the feedforward network-based method, such as StyleGaussian. The feedforward network-based method achieves fast style transfer through mechanisms such as AdaIN (Adaptive Instance Normalization), but it suffers from problems such as structural smoothing and loss of high-frequency details; the other is the optimization-based method, such as SGSST (Scene-Adaptive Gaussian Splatting Strategy), which optimizes color parameters through reverse rendering, but it is prone to color inconsistency, dirty color phenomenon and geometric degradation.
[0004] However, existing technologies have the following problems: (1) Color Disharmony: Existing methods such as SGSST perform independent and unconstrained optimization of the colors of millions of Gaussian spheres. The stylized scene exhibits severe "dirty colors," noise, and out-of-gamut colors that do not belong to the target art style. For example, when imitating a warm-toned oil painting, glaring fluorescent green or muddy gray may inexplicably appear in the scene. This is due to the discreteness of the parameter space: SGSST models the scene as independent Gaussian spheres. During the optimization process, the color coefficients of each Gaussian sphere... Each color is an independent degree of freedom. Furthermore, it lacks manifold constraints: the traditional VGG style loss (Gram Matrix) only constrains the statistical distribution at the feature level, not the specific range of color values. To mathematically minimize the loss function, the optimizer seeks "shortcuts," adjusting the colors of certain Gaussian spheres to extreme values to match the features. This ultimately results in the lack of a globally shared low-dimensional color manifold or palette constraint. The colors of millions of Gaussian spheres "go their own way," leading to a macroscopically chaotic color scheme that disrupts the original, strict color matching logic of the artwork.
[0005] (2) Structural Degradation: The rendering result exhibits an "over-smoothed" or "washed-out" appearance, with blurred object edges, making it difficult to reproduce fine brushstrokes (such as Van Gogh's short lines) or high-frequency textures. The visual experience is closer to a simple filter overlay than a deep redraw. This is due to the limitations of global statistics: feedforward techniques such as AdaIN mainly align the first-order statistics (mean) and second-order statistics (variance) of features. This global operation is essentially a low-frequency signal processing, which tends to smooth out high-frequency information in the image. It also encounters the feature resolution bottleneck: feedforward networks typically operate on downsampled feature maps and then upsample them back to the original resolution. This process inevitably leads to the loss of spatial information, making it impossible for the algorithm to accurately "sculpt" clear style textures in 3D space.
[0006] (3) Low optimization efficiency and instability: During strong stylization, the original geometric structure of the scene is destroyed. This manifests as straight edges becoming curved, object contours being interrupted by incorrect texture filling, or false geometric artifacts being generated to create a certain pattern. Because existing methods (whether SGSST or StyleGaussian) usually calculate the loss in RGB space or full-channel feature space, they do not explicitly decouple luminance information (carrying structure) from chrominance information (carrying style). Furthermore, when encountering style loss, they tend to change local pixel values to match the texture, which often conflicts with the depth constraints or luminance constraints that maintain the original geometric contours. In the absence of an adaptive weight adjustment mechanism, the "noise" of the style gradient will overwhelm the geometric gradient, causing structural information to be "swallowed" by the style texture.
[0007] (4) Limitations of Optimization Starting Point and Convergence: Methods such as SGSST typically start optimization directly based on the colors of the original photo. However, the original scene (e.g., photorealistic style) and the target artistic style (e.g., abstract painting) differ greatly in their color histogram distributions. This huge distribution shift makes the optimizer prone to getting trapped in local minima, resulting in slow convergence, and the final result often retains too much of the original scene's hue, failing to achieve a complete style transfer. Summary of the Invention
[0008] This application aims to at least solve the technical problems existing in the prior art and provide a 3D Gaussian sputtering stylization method, system, device and storage medium.
[0009] In a first aspect, the present invention provides a 3D Gaussian sputtering stylization method, the method comprising: Acquire the style image and the 3D Gaussian scene to be stylized; Perform nonnegative matrix decomposition on the style image to extract the initial color basis matrix and initial coefficient matrix of each pixel in the style image; A color orthogonal decoding model is constructed based on the initial color basis matrix and the initial coefficient matrix, and the color parameters of each Gaussian element in the 3D Gaussian scene are recalculated using the color orthogonal decoding model. Style transfer optimization is performed on Gaussian elements. In each optimization iteration, a camera viewpoint is randomly selected, and a rendered image is generated based on the geometric and color attributes of all current Gaussian elements. The style image and the rendered image are input into a pre-trained VGG network. The loss function is calculated based on the output of the pre-trained VGG network, and the trainable parameters are updated based on the loss function until the iteration termination condition is met. The trainable parameters include the geometric and color attribute parameters of the Gaussian elements. The stylized 3D Gaussian scene is determined based on the updated geometric and color attribute parameters.
[0010] Optionally, the loss function includes style loss, brightness consistency loss, and edge gradient loss. Style loss is used to represent the degree of difference between the elements of the Gram matrix of the rendered image and the style image; brightness consistency loss is used to represent the difference between the rendered image and the original image of the 3D Gaussian scene corresponding to the camera viewpoint under the action of the brightness extraction operator; and edge gradient loss is used to represent the difference between the rendered image and the original image of the 3D Gaussian scene corresponding to the camera viewpoint in terms of object contours.
[0011] Optionally, the style image and the rendered image are input into a pre-trained VGG network, and the Gram matrix difference between the style image and the rendered image is calculated in multiple feature layers of the pre-trained VGG network to obtain the style loss. The original and rendered images of the 3D Gaussian scene corresponding to the camera's viewpoint are converted to the luminance channel, and the luminance consistency loss is determined based on the L1 norm difference between the luminance maps of the original image and the rendered image. The first gradient map is obtained by extracting the first gradient of the brightness map of the original image, and the second gradient map is obtained by extracting the first gradient of the brightness map of the rendered image. The edge gradient loss is determined based on the L1 norm difference between the first gradient map and the second gradient map.
[0012] Optionally, an adaptive gradient modulation mechanism can be used to dynamically adjust the weights of style loss, brightness consistency loss, and edge gradient loss.
[0013] Optionally, the initial color basis matrix includes at least two color basis vectors, and the initial coefficient matrix is the weights corresponding to each color basis vector in the color basis matrix.
[0014] Optionally, the step of performing nonnegative matrix decomposition on the style image to extract the initial color basis matrix and initial coefficient matrix for each pixel in the style image includes: Reshape the style image into a pixel matrix; The pixel matrix is decomposed using a nonnegative matrix factorization algorithm to obtain the basis matrix and the initial coefficient matrix. The row vectors of the basis matrix are the color basis vectors.
[0015] Alternatively, the color orthogonal decoding model can be represented as: ; Gausky element Color parameters, For a Gaussian scene, the Gaussian meta-index, This is the initial color basis matrix; For the first Gausky's unique low-dimensional latent code Sets the global color bias.
[0016] Secondly, the present invention provides a 3D Gaussian sputtering stylization system, the system comprising: The acquisition module acquires the style image and the 3D Gaussian scene to be stylized. The decomposition module is used to perform non-negative matrix decomposition on the style image, extracting the initial color basis matrix and initial coefficient matrix for each pixel in the style image; the initial color basis matrix includes at least two color basis vectors, and the initial coefficient matrix is the weight corresponding to each color basis vector in the color basis matrix; The remapping module is used to construct a color orthogonal decoding model based on the initial color basis matrix and the initial coefficient matrix, and to recalculate the color parameters of each Gaussian primitive in the 3D Gaussian scene using the color orthogonal decoding model; The style optimization module is used to optimize the style transfer of Gaussian elements. In each optimization iteration, a camera viewpoint is randomly selected, and a rendered image is generated based on the geometric and color attributes of all current Gaussian elements. The style image and the rendered image are input into a pre-trained VGG network. The loss function is calculated based on the output of the pre-trained VGG network, and the trainable parameters are updated based on the loss function until the iteration termination condition is met. The trainable parameters include the geometric and color attribute parameters of the Gaussian elements. The output module is used to determine the stylized 3D Gaussian scene based on the updated geometric and color attribute parameters.
[0017] Thirdly, the present invention provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the 3D Gaussian sputtering stylization method described above.
[0018] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the 3D Gaussian sputtering stylization method described above.
[0019] In summary, this application includes the following beneficial technical effects: By introducing nonnegative matrix factorization to extract the color basis of style images, a color orthogonal decoding model is constructed to reparameterize the color of Gaussian elements. The color of each Gaussian element is represented as a linear combination of color basis vectors to achieve color consistency across the entire scene, solving the problems of color inconsistency and dirty colors. In addition, by updating geometric and color attributes simultaneously in iterative optimization and combining the multi-scale feature loss of the VGG network, style transfer and structure preservation are effectively balanced, reducing the possibility of geometric degradation in 3D Gaussian sputtering stylization, and achieving high-fidelity and high-efficiency 3D scene art style transfer. Attached Figure Description
[0020] Figure 1 This is a flowchart illustrating a 3D Gaussian sputtering stylization method provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an electronic device that implements the 3D Gaussian sputtering stylization method according to an embodiment of the present invention.
[0021] Reference numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface.
[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0023] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0024] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0025] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.
[0026] Reference Figure 1 The diagram shown is a flowchart illustrating a 3D Gaussian sputtering stylization method according to an embodiment of the present invention. In this embodiment, the 3D Gaussian sputtering stylization method includes: S1. Obtain the style image and the 3D Gaussian scene to be stylized.
[0027] In the 3D Gaussian Splatting (3DGS) stylization method, a style image refers to a two-dimensional image that serves as an artistic style reference. It defines the visual style that the user wants to transfer to a 3D scene, such as color, brushstrokes, texture, or overall atmosphere.
[0028] S2. Perform non-negative matrix decomposition on the style image to extract the initial color basis matrix and initial coefficient matrix of each pixel in the style image.
[0029] Specifically, the initial color basis matrix includes at least two color basis vectors, and the initial coefficient matrix consists of the weights corresponding to each color basis vector in the color basis matrix. This invention proposes a seeding strategy based on Non-negative Matrix Factorization (NFF) to extract physically meaningful color basis from style images.
[0030] Specifically, nonnegative matrix decomposition is performed on the style image to extract the initial color basis matrix and initial coefficient matrix for each pixel in the style image, including: S21. Reshape the style image into a pixel matrix; First, the system receives the target art style image. To perform algebraic analysis, it is removed from the image space. Reshape into a pixel matrix ,in, The height of the representative style image; The width of the style image is represented by 3, and 3 represents the three RGB color channels of the style image. This represents the total number of pixels.
[0031] S22. The pixel matrix is decomposed using a non-negative matrix decomposition algorithm to obtain the basis matrix and the initial coefficient matrix.
[0032] Unlike principal component analysis (PCA), which allows negative values, this invention uses nonnegative matrix factorization (NMF) algorithm to process the pixel matrix. Decompose the color mixture. Because color mixing is physically additive (non-negative), NMF (Non-Multiple Mixing) is better able to capture the essence of color. The decomposition formula is: ; in, The basis matrix, i.e., the row vectors of the basis matrix are the color basis vectors; the extracted... Each row vector represents the core of this art style. The "basic colors" are the "palette" that makes up the painting.
[0033] This is a coefficient matrix, representing how each pixel is composed of these... It is formed by mixing base colors. The resulting decomposition... Matrix Each row vector is projected into the spherical harmonic (SH) space and used as the initial color base in subsequent optimizations of this invention. Simultaneously, the mean color of the style image is calculated as an initial global bias. This step provides a strong inductive bias for the subsequent optimization process, ensuring that the optimization starting point is directly at the center of the target style's color gamut, thereby significantly accelerating convergence and ensuring tonal accuracy.
[0034] S3. Construct a color orthogonal decoding model based on the initial color basis matrix and the initial coefficient matrix, and recalculate the color parameters of each Gaussian primitive in the 3D Gaussian scene using the color orthogonal decoding model.
[0035] The purpose of constructing a color orthogonal decoding model is to constrain the colors of all Gaussian primitives to a low-dimensional color manifold, thereby achieving global color consistency.
[0036] This invention abandons the existing approach of directly optimizing high-dimensional RGB / SH parameters, and instead adopts a low-rank manifold constraint model.
[0037] Existing techniques directly optimize each Gaussian sphere (i.e., Gaussian elements). Color coefficient (dimension is) ), This represents the 0th order spherical harmonic (SH) color coefficient of the i-th Gaussian sphere. This invention no longer directly optimizes... Instead, it is modeled as a linear combination of a globally shared basis. In this embodiment, the color orthogonal decoding model is represented as: ; Gausky element Color parameters, For a Gaussian scene, the Gaussian meta-index, This is the initial color basis matrix; For the first Gausky's unique low-dimensional latent code Sets the global color bias.
[0038] This embodiment will Set to a minimum value (e.g.) The number of Gaussian spheres is much smaller than the number of Gaussian spheres. This effectively forces all Gaussian sphere colors to fall within the base. In Zhang Cheng's low-dimensional affine subspace, this hard constraint fundamentally eliminates the generation of "dirty colors." Because the substrate... It is initialized from the style map and constrained by style loss, so it ensures that the colors of the entire scene always remain within the color palette of the target art style, thereby forcibly achieving global color coherence across the entire scene.
[0039] In this embodiment, the color base dimension is set to K=3. In alternatives, K can be set to other low-dimensional values such as 2 or 4, depending on the complexity of the target style. Smaller K results in stronger constraints, while larger K provides richer color expression but increases the risk of inconsistency. Color space alternatives: This embodiment reparameterizes the DC component (DC, 0th order) of the spherical harmonic coefficients (SH). Alternatives can apply this constraint to RGB, CIELAB, HSV, or YUV spaces. Alternatively, non-linear decoding can replace linear decoding, and constraints on higher-order spherical harmonic coefficients (SH) can be extended.
[0040] S4. Perform style transfer optimization on Gaussian elements. In each optimization iteration, randomly select a camera viewpoint and generate a rendered image based on the geometric and color attributes of all current Gaussian elements. Input the style image and the rendered image into a pre-trained VGG network, calculate the loss function based on the output of the pre-trained VGG network, and update the trainable parameters based on the loss function until the iteration termination condition is met. The trainable parameters include the geometric and color attribute parameters of the Gaussian elements.
[0041] VGG network is a classic deep convolutional neural network (CNN) architecture, primarily used for image recognition and classification tasks.
[0042] The steps of iterative optimization include: 1. Forward rendering: Renders the 3DGS scene from a random perspective to obtain the current rendered image.
[0043] 2. Loss Calculation: Calculate the loss function.
[0044] 3. Backpropagation and parameter update: Calculate the total gradient and use an optimizer (such as Adam) to update the geometric and color attribute parameters of the Gaussian elements.
[0045] When the set maximum number of iterations or the loss threshold is reached, the iteration stopping condition is met, and the iteration stops in an optimized manner.
[0046] Specifically, in order to solve the problem of stylization destroying geometric structure, this invention proposes a color orthogonal decoding model, which achieves structural anchoring by decoupling luminance and chrominance.
[0047] Based on the characteristics of the human visual system, which is sensitive to luminance for structure and chrominance for style, this invention strictly limits structural protection constraints to the luminance channel without interfering with the stylization of the chrominance channel. To achieve this, the loss function in this embodiment includes style loss, luminance consistency loss, and edge gradient loss. Style loss represents the degree of difference between the elements of the Gram matrix of the rendered image and the style image; luminance consistency loss represents the difference between the rendered image and the original image of the 3D Gaussian scene corresponding to the camera viewpoint under the action of the luminance extraction operator, ensuring that the overall lighting structure does not shift drastically; edge gradient loss represents the difference between the rendered image and the original image of the 3D Gaussian scene corresponding to the camera viewpoint in terms of object contours, using the Sobel operator. Extract the first-order gradient of the luminance channel and calculate Distance. This forces the rendering result to retain high-frequency edge information of the original scene (such as object outlines and texture boundaries).
[0048] The style image and the rendered image are input into a pre-trained VGG network. The Gram matrix difference between the style image and the rendered image is calculated in multiple feature layers of the pre-trained VGG network to obtain the style loss. The original and rendered images of the 3D Gaussian scene corresponding to the camera's viewpoint are converted to the luminance channel, and the luminance consistency loss is determined based on the L1 norm difference between the luminance maps of the original image and the rendered image. The first gradient map is obtained by extracting the first gradient of the brightness map of the original image, and the second gradient map is obtained by extracting the first gradient of the brightness map of the rendered image. The edge gradient loss is determined based on the L1 norm difference between the first gradient map and the second gradient map.
[0049] S5. Determine the stylized 3D Gaussian scene based on the updated geometric and color attribute parameters.
[0050] In a preferred embodiment of this example, an adaptive gradient modulation mechanism is used to dynamically adjust the weights of style loss, brightness consistency loss, and edge gradient loss.
[0051] Calculate the total gradient during backpropagation. In this case, instead of using static weighting, the style gradient is dynamically balanced. and structural gradient ( The formula for the adaptive gradient modulation mechanism is: ; This represents the total gradient during backpropagation, used to update model parameters; This represents the style gradient derived from the style loss (VGG Loss), attempting to change the color to match the art style; This represents a brightness uniformity gradient, used to maintain the overall lighting structure; Represents the edge gradient, derived from the edge differences extracted by the Sobel operator, used to preserve the object's outline; This represents the user-defined luminance loss weighting coefficient; As an adaptive weight, it dynamically adjusts according to the strength of the style gradient. When the stylization intensity is too strong and attempts to destroy the edges, it will automatically increase to protect the structure.
[0052] Among them, adaptive weights Defined as: ; in, for ; for , for .
[0053] Through adaptive weights This formula monitors the modulus of the style gradient in real time. When the stylization is too strong When a large amount of material attempts to break the edge, It will automatically increase to enhance resistance to edge gradients; conversely, it will decrease.
[0054] Adaptive gradient modulation constructs a dynamic luminance scaffold, allowing colors to freely shift styles under CHORD constraints, while the geometric contours are firmly anchored, achieving a balance between style and structure.
[0055] Based on the same inventive concept, an embodiment of the present invention provides a 3D Gaussian sputtering stylization system.
[0056] The 3D Gaussian sputtering stylization system described in this invention can be installed in an electronic device. Depending on the functions implemented, the 3D Gaussian sputtering stylization system includes: The acquisition module can acquire style images and the 3D Gaussian scene to be stylized; The decomposition module can perform non-negative matrix decomposition on the style image, extracting the initial color basis matrix and initial coefficient matrix for each pixel in the style image; the initial color basis matrix includes at least two color basis vectors, and the initial coefficient matrix is the weight corresponding to each color basis vector in the color basis matrix; The remapping module can construct a color orthogonal decoding model based on the initial color basis matrix and the initial coefficient matrix, and use the color orthogonal decoding model to recalculate the color parameters of each Gaussian element in the 3D Gaussian scene; The style optimization module can perform style transfer optimization on Gaussian elements. In each optimization iteration, a camera viewpoint is randomly selected, and a rendered image is generated based on the geometric and color attributes of all current Gaussian elements. The style image and the rendered image are input into a pre-trained VGG network. The loss function is calculated based on the output of the pre-trained VGG network, and the trainable parameters are updated based on the loss function until the iteration termination condition is met. The trainable parameters include the geometric and color attribute parameters of the Gaussian elements. The output module can determine the stylized 3D Gaussian scene based on the updated geometric and color attribute parameters.
[0057] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.
[0058] The various variations and specific examples of the 3D Gaussian sputtering stylization method provided in the above embodiments are also applicable to the 3D Gaussian sputtering stylization system of this embodiment. Through the foregoing detailed description of the 3D Gaussian sputtering stylization method, those skilled in the art can clearly understand the implementation method of the 3D Gaussian sputtering stylization system in this embodiment. For the sake of brevity, it will not be described in detail here.
[0059] This application also discloses an electronic device, such as Figure 2 The diagram shown is a schematic representation of an electronic device for a 3D Gaussian sputtering stylization method according to an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 communicatively connected to the at least one processor, a communication bus 12, and a communication interface 13. It may also include a computer program, such as a 3D Gaussian sputtering stylization method program, stored in the memory 11 and executable on the processor 10.
[0060] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing 3D Gaussian sputtering stylization methods) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.
[0061] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of the 3D Gaussian sputtering stylization method, but also to temporarily store data that has been output or will be output.
[0062] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.
[0063] Communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), and optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.
[0064] Figure 2 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 2 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0065] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.
[0066] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0067] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.
[0068] This application provides a computer-readable storage medium, including, for example, any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM). The computer-readable storage medium stores a computer program capable of being loaded by a processor and executing the 3D Gaussian sputtering stylization method of the above embodiments.
[0069] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0070] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims
1. A 3D Gaussian sputtering stylization method, characterized in that, The method includes: Acquire the style image and the 3D Gaussian scene to be stylized; Perform nonnegative matrix decomposition on the style image to extract the initial color basis matrix and initial coefficient matrix of each pixel in the style image; A color orthogonal decoding model is constructed based on the initial color basis matrix and the initial coefficient matrix, and the color parameters of each Gaussian element in the 3D Gaussian scene are recalculated using the color orthogonal decoding model. Style transfer optimization is performed on Gaussian elements. In each optimization iteration, a camera viewpoint is randomly selected, and a rendered image is generated based on the geometric and color attributes of all current Gaussian elements. The style image and the rendered image are input into a pre-trained VGG network. The loss function is calculated based on the output of the pre-trained VGG network, and the trainable parameters are updated based on the loss function until the iteration termination condition is met. The trainable parameters include the geometric and color attribute parameters of the Gaussian elements. The stylized 3D Gaussian scene is determined based on the updated geometric and color attribute parameters.
2. The 3D Gaussian sputtering stylization method as described in claim 1, characterized in that, The loss function includes style loss, brightness consistency loss, and edge gradient loss. Style loss is used to represent the degree of difference between the elements of the Gram matrix of the rendered image and the style image; brightness consistency loss is used to represent the difference between the rendered image and the original image of the 3D Gaussian scene corresponding to the camera viewpoint under the action of the brightness extraction operator; edge gradient loss is used to represent the difference between the rendered image and the original image of the 3D Gaussian scene corresponding to the camera viewpoint in terms of object contours.
3. The 3D Gaussian sputtering stylization method as described in claim 2, characterized in that, The style image and the rendered image are input into a pre-trained VGG network. The Gram matrix difference between the style image and the rendered image is calculated in multiple feature layers of the pre-trained VGG network to obtain the style loss. The original and rendered images of the 3D Gaussian scene corresponding to the camera's viewpoint are converted to the luminance channel, and the luminance consistency loss is determined based on the L1 norm difference between the luminance maps of the original image and the rendered image. The first gradient map is obtained by extracting the first gradient of the brightness map of the original image, and the second gradient map is obtained by extracting the first gradient of the brightness map of the rendered image. The edge gradient loss is determined based on the L1 norm difference between the first gradient map and the second gradient map.
4. The 3D Gaussian sputtering stylization method as described in claims 1 to 3, characterized in that, An adaptive gradient modulation mechanism is adopted to dynamically adjust the weights of style loss, brightness consistency loss, and edge gradient loss.
5. The 3D Gaussian sputtering stylization method as described in claim 1, characterized in that, The initial color basis matrix includes at least two color basis vectors, and the initial coefficient matrix is the weight of each color basis vector in the color basis matrix.
6. The 3D Gaussian sputtering stylization method as described in claim 5, characterized in that, The step of performing nonnegative matrix decomposition on the style image to extract the initial color basis matrix and initial coefficient matrix for each pixel in the style image includes: Reshape the style image into a pixel matrix; The pixel matrix is decomposed using a nonnegative matrix factorization algorithm to obtain the basis matrix and the initial coefficient matrix. The row vectors of the basis matrix are the color basis vectors.
7. The 3D Gaussian sputtering stylization method as described in claim 4, 5, or 6, characterized in that, The color orthogonal decoding model is represented as: ； Gausky element Color parameters, For a Gaussian scene, the Gaussian meta-index, The initial color basis matrix, This represents the transpose of the initial color basis matrix; For the first Gausky's unique low-dimensional latent code Sets the global color bias.
8. A 3D Gaussian sputtering stylization system for implementing the 3D Gaussian sputtering stylization method according to any one of claims 1 to 7, characterized in that, include: The acquisition module acquires the style image and the 3D Gaussian scene to be stylized. The decomposition module is used to perform non-negative matrix decomposition on the style image, extracting the initial color basis matrix and initial coefficient matrix of each pixel in the style image; The initial color basis matrix includes at least two color basis vectors, and the initial coefficient matrix is the weight of each color basis vector in the color basis matrix. The remapping module is used to construct a color orthogonal decoding model based on the initial color basis matrix and the initial coefficient matrix, and to recalculate the color parameters of each Gaussian primitive in the 3D Gaussian scene using the color orthogonal decoding model; The style optimization module is used to perform style transfer optimization on Gaussian elements. In each optimization iteration, a camera view is randomly selected, and a rendered image is generated based on the geometric and color attributes of all current Gaussian elements. The style image and the rendered image are input into the pre-trained VGG network. The loss function is calculated based on the output of the pre-trained VGG network, and the trainable parameters are updated based on the loss function until the iteration termination condition is reached. The trainable parameters include the geometric attribute parameters and color attribute parameters of the Gaussian elements. The output module is used to determine the stylized 3D Gaussian scene based on the updated geometric and color attribute parameters.
9. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10) to enable the at least one processor (10) to perform the 3D Gaussian sputtering stylization method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the 3D Gaussian sputtering stylization method as described in any one of claims 1 to 7.