Zero-sample stylized three-dimensional graphic content generation method and system based on three-dimensional Gaussian sputtering

Abstract

The invention discloses a zero-sample stylized three-dimensional graphic content generation method and system based on three-dimensional Gaussian sputtering. The method comprises the following steps: down-sampling an input three-dimensional Gaussian sputtering scene through an encoder in an auto-encoder and extracting three-dimensional scene features; after the three-dimensional scene features are projected to a feature space aligned with the two-dimensional style image features through a multi-layer perceptron, style features from the two-dimensional style image are injected into the three-dimensional scene features through adaptive instance normalization, and features after stylization conversion are obtained; and performing up-sampling on the stylized and converted features through a decoder in the auto-encoder, and reconstructing a stylized and converted three-dimensional Gaussian sputtering scene. According to the method, real-time and high-quality style migration of any three-dimensional scene and any style image can be realized, time-consuming optimization for each new scene is not needed, and the efficiency and the flexibility of three-dimensional content creation are greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of 3D scene style transfer technology, specifically involving a zero-sample stylized 3D graphic content generation method and system based on 3D Gaussian sputtering. Background Technology

[0002] High-quality 3D scene representation is fundamental for various 3D vision tasks. Recent research has seen rapid advancements in implicit 3D scene representation, primarily driven by improvements in rendering quality, speed, and scene manipulability. Neural Radiation Field (NeRF) is a significant advancement in this field, utilizing volumetric rendering techniques to reconstruct 3D scenes with detailed geometric details and high realism. Therefore, it has been widely applied in tasks such as novel perspective synthesis, generative modeling, and surface reconstruction. However, despite the remarkable visual results achieved by NeRF methods, their computational overhead is significant. Because NeRF requires intensive volumetric sampling, it often results in high computational costs and low reconstruction speeds. To address this issue, researchers have proposed various variants to improve efficiency by optimizing the computational process, reducing computational costs, or modifying the underlying representation, such as the 3D Gaussian Sputtering (3DGS) method, to ensure both rendering quality and real-time rendering efficiency.

[0003] Building upon this foundation, 3D scene style transfer, as an important application direction of 3D reconstruction technology, relies heavily on the evolution of underlying scene representation technologies. In recent years, the field of 3D style transfer has made rapid progress, primarily driven by the growing demand for more complex and artistic 3D content. Early work on 3D style transfer mainly focused on applying image-based stylization methods to mesh or point cloud representations. While these methods achieved impressive results in certain scenarios, they typically faced challenges in reconstruction accuracy and computational complexity, making it difficult to achieve high-quality 3D reconstruction. Addressing these limitations, implicit representation-based techniques have become powerful tools for high-quality 3D style transfer. These methods can generate visually appealing stylized effects while simultaneously reconstructing detailed and accurate 3D scenes with high precision. Existing stylization methods typically fine-tune the NeRF-reconstructed scene through style loss or train a separate style transfer module. Although these methods have achieved good visual results, they are often limited by the time-consuming optimization process and excessive rendering computational overhead, making them difficult to promote in real-time and large-scale applications.

[0004] To improve the training and rendering efficiency of 3D style transfer, 3DGS has become a compelling alternative. Researchers have proposed various gradient-optimized style transfer methods. For example, StylizedGS fine-tunes the color and geometric parameters of Gaussian primitives using filters and nearest-neighbor style loss, while G-Style proposes a three-step fine-tuning strategy: removing unwanted Gaussian primitives, applying joint loss for stylization, and performing geometric fine-tuning to improve detail. Furthermore, ReGS, by introducing a texture-guided control mechanism, successfully achieves reference image-based style transfer, adaptively adjusting geometry and appearance, and enabling real-time free-view rendering. However, these methods still face the problem of excessively long optimization processes for each individual scene and specific style, limiting their application in large-scale scenes. StyleGaussian proposes a different 3DGS style transfer method. First, it embeds image features extracted by a VGG network into the reconstructed 3DGS. The embedded features are transformed according to the style image and finally decoded into a stylized RGB image. This method can achieve instantaneous style transfer of arbitrary images at a rate of 10 frames per second. However, for specific scenarios, StyleGaussian still requires several hours of training and has high memory consumption, making it unsuitable for style transfer in large-scale scenarios. Summary of the Invention

[0005] In view of the above, the purpose of this invention is to provide a method and system for generating zero-shot stylized 3D graphics content based on 3D Gaussian sputtering. By constructing a general 3D graph convolutional autoencoder network, it can directly match any input 3DGS scene with any style reference image, achieving instant style transfer. Its core innovation lies in using a specially designed 3D graph convolutional network to extract the spatial features of unstructured Gaussian primitives, and through an improved adaptive instance normalization module, effectively solving the problem of modal alignment between 3D content features and 2D artistic styles. This invention can generate high-quality, diverse, and visually consistent artistic 3D scenes while maintaining the original scene geometry, and significantly improves the efficiency of style transfer.

[0006] To achieve the above-mentioned objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a zero-sample stylized 3D graphics content generation method based on 3D Gaussian sputtering, comprising the following steps: The input 3D Gaussian sputtering scene is downsampled and 3D scene features are extracted by the encoder in the autoencoder. In the style transfer module, the 3D scene features are projected onto the feature space aligned with the 2D style image features through a multilayer perceptron. Then, the style features from the 2D style image are injected into the 3D scene features through adaptive instance normalization to obtain the stylized features. The stylized 3D Gaussian sputtering scene is reconstructed by upsampling the stylized features using the decoder in the autoencoder.

[0007] Preferably, the autoencoder adopts a symmetrical encoder-decoder architecture, wherein the encoder consists of at least one 3D graph convolutional layer and a channel max pooling layer for feature extraction and spatial downsampling; the decoder consists of at least one 3D graph convolutional layer and performs upsampling through an inverse distance weighted interpolation algorithm to recover the number of Gaussian primitives in the original scene.

[0008] Preferably, the 3D graph convolutional layer operates through a 3D convolutional kernel with learnable shape and weight. For each Gaussian primitive, local neighborhood features are aggregated within the receptive field defined by its spatial nearest neighbor through inner product operation and direction vector normalization.

[0009] Preferably, the step of projecting the three-dimensional scene features onto a feature space aligned with the two-dimensional style image features via a first multilayer perceptron includes: Two-dimensional style image features are extracted from style images by pre-training a VGG network. Then, a first multilayer perceptron is used to project the three-dimensional scene features into an intermediate feature space that matches the distribution of two-dimensional style image features through feature nonlinear mapping.

[0010] Preferably, the step of injecting style features from a two-dimensional style image into three-dimensional scene features through adaptive instance normalization to obtain stylized features, and then projecting them back to the original feature space through a second multilayer perceptron, includes: In the intermediate feature space, style transfer is achieved by using the mean and variance of the two-dimensional style image features based on the adaptive instance normalization method. Then, the stylized features are remapped back to the original feature space of the three-dimensional scene features based on the second multilayer perceptron to eliminate the modal differences between the three-dimensional content and the two-dimensional style.

[0011] Preferably, a two-stage training strategy is used to train the entire network: In the first stage, mean square error is used as the color reconstruction loss, and an autoencoder is trained separately to learn the ability to reconstruct color features of a 3D Gaussian sputtering scene. In the second stage, the autoencoder parameters are frozen, a style transfer module is introduced, and the style transfer capability of the model is trained using a joint loss function that combines content loss and style loss.

[0012] Preferably, a dataset containing multiple 3D Gaussian sputtering scenes is constructed for training the entire network. The dataset construction process includes: Select 3D models from the 3D model library that cover common objects and have a variety of materials; The rendering engine generates multi-view RGB images for each 3D model; RGB images from multiple perspectives are input into the TriplaneGaussian model and transformed into a 3D Gaussian sputtering scene representation, resulting in a dataset containing several 3D Gaussian sputtering scenes.

[0013] Secondly, embodiments of the present invention also provide a zero-sample stylized 3D graphic content generation system based on 3D Gaussian sputtering, which is implemented using the above-mentioned zero-sample stylized 3D graphic content generation method based on 3D Gaussian sputtering, including: a 3D feature extraction module, a style feature fusion module, and a 3D scene reconstruction module. The three-dimensional feature extraction module is used to downsample the input three-dimensional Gaussian sputtering scene and extract its three-dimensional scene features through the encoder in the autoencoder. The style feature fusion module is used in the style transfer module to project the three-dimensional scene features to the feature space aligned with the two-dimensional style image features through the first multilayer perceptron, and then inject the style features from the two-dimensional style image into the three-dimensional scene features based on adaptive instance normalization to obtain the stylized features, and then project them to the original feature space through the second multilayer perceptron. The 3D scene reconstruction module is used to upsample the stylized features through the decoder in the autoencoder to reconstruct the stylized 3D Gaussian sputtering scene.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described method for generating zero-sample stylized 3D graphic content based on 3D Gaussian sputtering when executing the computer program.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for generating zero-sample stylized 3D graphic content based on 3D Gaussian sputtering.

[0016] Compared with the prior art, the beneficial effects of the present invention include at least the following: This invention proposes an innovative zero-shot stylization model capable of online artistic stylization of any 3DGS scene. By leveraging the local color and geometric features of 3DGS, it overcomes the limitations of optimization-based stylization methods, achieving real-time stylization without time-consuming training or optimization processes. By constructing an autoencoder architecture based on 3D graph convolution, 3D features are effectively aggregated into a latent space. Furthermore, an improved Adaptive Instance Normalization (AdaIN) based on Multilayer Perceptron (MLP) significantly mitigates the modal differences between 3D content and 2D artistic styles, generating stylized results that maintain structural consistency while possessing rich artistic expression, achieving seamless style adaptation within the latent space. After two-stage training, this zero-shot stylization model can transform any 3DGS scene into diverse artistic styles, providing a robust solution for 3DGS stylization. Experimental results demonstrate that the method can generate high-quality, consistent, and realistic stylized effects in various 3D scenes, achieving a two-order-of-magnitude acceleration, opening new avenues for real-time 3D stylization applications in mixed reality, gaming, and other 3D content creation fields. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the zero-sample stylized 3D graphics content generation method based on 3D Gaussian sputtering provided in an embodiment of the present invention. Figure 2 This is a technical roadmap diagram of the zero-sample stylized 3D graphics content generation method based on 3D Gaussian sputtering provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the qualitative results of object-level style transfer provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the scene-level style transfer qualitative results provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the change in style transfer time with the number of Gaussian primitives provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the zero-sample stylized 3D graphics content generation system based on 3D Gaussian sputtering provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0020] Style transfer techniques for 3D scenes hold significant research value in computer vision and computer graphics. The core objective is to efficiently transfer the style of a reference image to a 3D scene while maintaining the consistency of its content structure and spatial dimensions. Most existing methods are based on explicit representations such as point clouds or meshes, which are limited by reconstruction accuracy and computational complexity, making it difficult to achieve high-quality and universal style transfer. Neural implicit representations offer new possibilities for 3D scene style transfer. However, existing 3DGS-based style transfer methods still rely on computationally intensive optimization processes, which are not only time-consuming but also difficult to scale to large-scale scenes, limiting their practicality.

[0021] In view of this, the inventive concept of this invention is as follows: Addressing the problems of high computational overhead, long optimization processes, and the inability to achieve zero-shot transfer in existing 3D scene style transfer methods, this invention provides a zero-shot stylization method and system for generating 3D graphics content based on 3D Gaussian sputtering. By introducing an autoencoder based on graph neural networks (GCN), it can efficiently aggregate local geometric and texture information of 3DGS scenes, and combine this with improved adaptive instance normalization (AdaIN) to achieve seamless injection of style features, thereby completing efficient and flexible style transfer while maintaining the content structure. This invention not only improves the real-time performance and versatility of 3D scene stylization, providing new technical solutions for fields such as real-time rendering, virtual reality, and game design, but also promotes the further application of neural implicit representation in 3D content generation and editing. Through this efficient feedforward 3DGS style transfer framework, it is possible to achieve rapid stylization of any image style to any 3D scene without additional optimization processes.

[0022] Given an image of any style and 3DGS scene collection The objective of this invention is to use a reference image. The style attributes are transferred to the 3DGS scene, generating a stylized 3DGS scene while maintaining the original content structure. .like Figure 1 and Figure 2 As shown, the embodiment provides a method for generating zero-sample stylized 3D graphics content based on 3D Gaussian sputtering, including the following steps: S1 uses the encoder in the autoencoder to downsample the input 3D Gaussian sputtering scene and extract 3D scene features.

[0023] The 3D Gaussian Sputtering (3DGS) method, through explicitly parameterized 3D Gaussian primitives, enables rapid 3D reconstruction and real-time rendering while maintaining rendering quality. Compared to traditional implicit neural network methods, 3DGS exhibits significant advantages in efficiency and real-time performance, thus finding widespread application in various fields, including 3D scene stylization. Furthermore, compared to implicit scene representations, explicit Gaussian primitives provide a more direct geometric representation, facilitating manipulation and modification. These primitives can be considered an enhanced form of point clouds, with their spatial distribution and related parameters possessing immense potential and flexibility. Therefore, in this embodiment of the invention, 3DGS was chosen as the scene representation method to fully explore the potential of explicit Gaussian primitives in efficient zero-style transfer and to explore their application in scene stylization.

[0024] While gradient descent-based style transfer methods have achieved significant results in stylization quality and visual effects when applying 3DGS, the long training time required for style transfer in each new scene is a major bottleneck hindering practical applications. Given that 3DGS is essentially composed of a large number of Gaussian primitives, exhibiting unstructured point cloud features, introducing 3D graph convolutional networks (GCNNs) capable of processing unordered data is a natural choice. GCNNs can effectively capture the spatial relationships and feature associations between Gaussian primitives, providing a structural advantage for the expression and integration of style transfer features in 3D scenes. However, ablation experiments show that directly using the most general-purpose GCNNs for feature extraction is difficult to achieve ideal style transfer results. This phenomenon is mainly attributed to the fact that such networks fail to fully consider the point cloud-like spatial structure characteristics of Gaussian primitives in their design. Based on these observations, this invention adopts a 3D graph convolutional network architecture specifically designed for extracting local 3D features from point cloud data. This structure is optimized for local 3D feature extraction from point cloud data. By introducing learnable convolutional kernel shapes and weights, this network can more effectively adapt to the feature extraction needs in irregular 3D structures. In the specific implementation, for those containing Gaussian primitives 3DGS scenes First, define a size of for each primitive. The feeling of wild To capture its local contextual information: , in, Represents spatial distance The set of nearest neighbor primitives. Each primitive is associated with... 3D feature vector The set of feature vectors for the entire scene is denoted as . .

[0025] To adapt to the characteristics of unstructured 3D data, a novel 3D convolutional kernel structure with learnable geometry and weight parameters is introduced. : , in, Represents the center position of the kernel, and trainable parameters. to Indicates the associated support points, each core point Related Power Vector .

[0026] Based on the above definition, at point The feeling of wild The formula for involution is as follows: , in, This represents the inner product operation. Indicates from point to The direction vector is used to obtain the feature vector after convolution. .

[0027] To improve computational efficiency and enhance feature representation capabilities, this invention designs a hierarchical processing structure combining channel max pooling and point cloud downsampling. The channel max pooling operation first aggregates the feature vectors of each primitive within the receptive field, and then, using a sampling rate... Spatial downsampling of point clouds.

[0028] The two key operations of the above 3D graph convolutional layer and channel max pooling layer can be formally represented as: , , in, This represents the coordinate vector of the downsampled Gaussian primitive. This indicates the number of Gaussian primitives after downsampling. These represent the three-dimensional graph convolutional layers respectively. and channel maximum pooling layer The extracted feature vector set is designed according to the network architecture. It can be used as input to the next layer of the network or as the feature output of the entire encoder-decoder. The 3D graph convolutional layer operates through 3D convolutional kernels with learnable shapes and weights. For each Gaussian primitive, it aggregates local neighborhood features within the receptive field defined by its spatial nearest neighbor through inner product operations and orientation vector normalization.

[0029] In this embodiment, an autoencoder architecture comprising a feature encoder and a color decoder is employed. The encoder consists of at least one of the aforementioned 3D graph convolutional layers and channel max-pooling layers. Through multi-level feature extraction and downsampling, a high-level feature representation is progressively abstracted from the original Gaussian primitives. During this process, the encoder uses a random sampling rate... Spatial downsampling of the input 3DGS effectively reduces computational complexity. The decoder consists of at least one 3D graph convolutional layer, responsible for decoding the color attributes of each Gaussian primitive from the latent space features. To recover the number of Gaussian primitives in the original 3DGS, the decoder performs accurate upsampling of the downsampled Gaussian primitives using an inverse distance-weighted interpolation algorithm.

[0030] For including Scene of Gaussian primitives While keeping its geometric properties and opacity parameters fixed, only the color attributes are subjected to feature extraction and stylization. Therefore, each Gaussian primitive can be represented as a color vector. The encoding process can be formally represented as: , in, This represents the primitives after downsampling. for of The 2D feature vector contains structural and color information of local Gaussian graph primitive clusters. Combined with the style transfer module in step S2 below, the artistic style of the 2D style image is injected into these features, generating a feature representation that both preserves the original scene structure and incorporates the new style. .

[0031] S2, in the style transfer module, the 3D scene features are projected onto the feature space aligned with the 2D style image features through the first multilayer perceptron. Then, the style features from the 2D style image are injected into the 3D scene features through adaptive instance normalization to obtain the stylized features. Subsequently, the 3D scene features are projected onto the original feature space through the second multilayer perceptron.

[0032] In this embodiment, deep features containing local Gaussian cluster structure and color information are obtained during the intermediate feature extraction stage of the autoencoder. Subsequently, this invention introduces the Adaptive Instance Normalization (AdaIN) technique to normalize two-dimensional style images. The artistic features are effectively injected into these 3D scene features to obtain stylized features. These features are then propagated through the decoder to all Gaussian primitives, achieving stylized rendering of the entire scene.

[0033] To achieve efficient zero-shot style transfer, this invention improves upon AdaIN. AdaIN aligns content features... With style characteristics The mean and variance of the affine parameters are used to achieve style transfer in the feature space. However, there is a misalignment in the distribution between the content feature space of a 3D scene and the style feature space of a 2D image. Directly using traditional AdaIN for affine parameter alignment often leads to distortion or inconsistency in style transfer results.

[0034] To address this cross-modal feature alignment problem, this invention introduces a multilayer perceptron (MLP) to learn the nonlinear mapping relationship between different feature spaces. The stylization process can be formally expressed as: , in, and These represent the feature space mapping functions implemented by the first and second multilayer perceptrons, respectively: The function is responsible for extracting the original 3D content features. Projected onto a space with the same style characteristics as the two-dimensional image; This indicates that style images are obtained through a pre-trained VGG network. Features extracted from [the source]. After performing AdaIN style transfer in the intermediate feature space, The function remaps stylized features back to the feature space of the 3D scene representation using a second multilayer perceptron. This bidirectional feature space mapping mechanism effectively solves the modal difference problem between 3D content and 2D style, significantly improving the quality and stability of style transfer.

[0035] S3 uses the decoder in the autoencoder to upsample the stylized features and reconstruct the stylized 3D Gaussian sputtering scene.

[0036] In this embodiment, the styled color vector is finally obtained through the decoder: , in, Represents a 3DGS scene set. This represents a stylized color vector, which, with the aid of NVIDIA's CUDA-accelerated differentiable Gaussian renderer, generates stylized multi-view 3D image outputs. Through this encoding-stylization-decoding process, this invention achieves high-quality stylization conversion of scene appearance while maintaining the geometric structure of the 3D scene.

[0037] During the training phase, a two-stage training strategy is employed to train the entire network: the first stage is autoencoder training, which learns to extract and reconstruct features from unstructured 3DGS; the second stage is stylization training, which learns 3DGS style transfer capabilities. This phased training strategy ensures that the model first masters basic 3DGS encoding and decoding capabilities, and then gradually learns complex style transfer capabilities, thereby ensuring the model's robustness and generalization ability.

[0038] (1) Autoencoder training phase: In the initial stage of model training, the autoencoder is trained independently first, before the style transfer module is introduced into the network architecture. The core objective of this training phase is to enable the autoencoder to effectively perform the spatial aggregation of 3DGS features and the decoding of color features. During training, the original color data from the 3DGS objects is directly used as the supervision signal. Through an end-to-end training strategy, the model is guided to learn the compact encoding and high-precision decoding capabilities of color features. To ensure that the color features output by the model are highly consistent with the original scene, mean squared error is used as the loss function for color reconstruction. Its mathematical definition is: , in, This indicates the number of Gaussian primitives involved in the computation in the current training batch. Indicates the first in a 3DGS scene The original color values ​​of each Gaussian primitive (represented as RGB channel vectors). This represents the output color prediction value of the autoencoder network for this primitive.

[0039] (2) Stylized training stage: After the autoencoder completes its initial training and achieves stable color reconstruction performance, its network parameters are frozen to preserve the learned encoding and decoding capabilities. A style transfer module is then introduced for the second stage of training. The main goal of this stage is to enable the model to inject artistic features from reference style images into 3DGS objects while maintaining consistency with the original input in terms of geometry and content.

[0040] In each training iteration, a random sampling strategy is employed: a 3DGS scene is randomly selected from the training set as the content input, and a style image is randomly chosen as the transfer target. To improve the model's generalization ability under new perspectives, the camera pose is uniformly sampled on a unit sphere, and the resulting new perspective image is rendered as the output target of the style transfer module. The style transfer effect is optimized through multi-scale feature matching, and the loss function consists of two parts: , in, This represents the content loss, used to constrain the consistency between the output image and the input scene in high-level features; This represents style loss and ensures that the output image is similar to the reference style image in artistic characteristics such as texture and brushstrokes. This is an adjustable hyperparameter used to balance the intensity of content preservation and style transfer. Specifically, content loss... Defined as: , in, This represents the input content image. For the generated stylized image, This represents a specific layer in the VGG-19 network used for feature extraction. Example selection. Four levels of features are used, with equal weights. Considering that the stylizer achieves style transfer by transferring the mean and standard deviation of style features, a corresponding style loss function is designed. Specifically, the style loss... The expression is: , in, Indicates a reference style image. and These represent the calculation of the mean and standard deviation of the feature, respectively. This represents the specific number of layers used for feature extraction in the VGG-19 network. This loss function achieves style transfer by matching the statistical properties of deep features.

[0041] To meet the requirements of zero-shot generalization and diverse style transfer, this invention constructs a large-scale 3DGS dataset. Constructing a high-quality 3DGS dataset is a crucial prerequisite for training zero-shot style transfer models; however, existing publicly available datasets are significantly insufficient in terms of data diversity and scale. Traditional 3D reconstruction methods, while achieving high accuracy, face two main challenges: the reconstruction process based on multi-view images is computationally intensive, often requiring hours of GPU computation for a single scene, making it difficult to construct large-scale datasets; and the 3DGS scenes obtained by reconstruction methods are too large for training feedforward neural network models. To address these issues, this invention designs an efficient dataset construction scheme, with the overall process comprising the following three steps: First, 40,000 representative 3D models are selected from the Objaverse dataset, covering common objects and diverse materials; RGB images from different perspectives are generated for each model using the Blender rendering engine; these images are then input into the TriplaneGaussian model to generate the corresponding 3DGS representation. The final constructed dataset contains 40,000 3DGS objects, each containing 16,384 Gaussian points. To verify data quality, 1000 scenes were randomly sampled for manual inspection. The results showed that most generated objects met usable standards in terms of geometric plausibility. For style reference, approximately 90,000 paintings from the WikiArt dataset were used as style source images, covering various major stylistic movements from Baroque to Modern art. This data combination ensured diversity in style transfer while providing a solid foundation for subsequent quantitative evaluation. During training, content and style were paired and randomly sampled, and the camera pose was uniformly sampled on a unit sphere to improve the multi-view consistency of the generated results.

[0042] In the inference and application phases, the method of this invention has been built and verified on a high-performance computing platform equipped with NVIDIA RTX A6000 48GB VRAM. While maintaining the geometric structure and spatial layout of the 3D scene, it efficiently transfers the artistic style of any 2D reference image to the target 3DGS scene, achieving stylized rendering in seconds and real-time interactive preview. Results show that style transfer can be completed within 8 to 12 seconds in scenes with millions of Gaussian primitives, and it supports ultra-large-scale scenes through adaptive spatial clustering and batch pipeline rendering. In qualitative and quantitative experiments, compared with existing methods such as StyleGaussian and StyleSplat, this invention shows significant advantages in style consistency, geometric structure preservation, and visual naturalness, and achieves leading levels in multi-view consistency evaluation (RMSE, LPIPS) and computational speed. User subjective ratings have verified that the results generated by this invention achieve high scores in artistic expression and structural fidelity, demonstrating its broad application potential in virtual reality, game design, digital artifact restoration, and metaverse scene construction.

[0043] The following are some experimental results and analysis.

[0044] (1) Display and comparison of style transfer results: like Figure 3 As shown, the qualitative results of style transfer at the object level are presented.

[0045] like Figure 4 As shown, this paper presents a comparative analysis of the system visual quality of this invention with representative methods in the field of 3D scene style transfer, StyleSplat and StyleGaussian. Experimental results show that, compared with existing methods, this method achieves a more accurate and globally consistent artistic style transfer effect while maintaining the integrity of the scene's geometric structure and the fidelity of texture details. Specifically, although StyleSplat can achieve basic stylization transformation, its generated results show significant inconsistencies with the target style image in local areas. For example... Figure 4As shown in the lower left corner of the second and third rows of the second column, this method fails to accurately capture the color distribution features and brushstroke texture patterns in the style reference image, resulting in local distortion of the stylization effect. StyleGaussian exhibits two main limitations. In terms of geometric structure preservation, this method is insufficient in preserving key edge features such as building outlines, causing degradation of important structural details; in terms of style adaptation accuracy, its generated results show a systematic deviation from the reference image. This paper speculates that this deficiency stems from the inherent limitations of the VGG feature extraction network used in this method in 3D spatial structure modeling, making it difficult to effectively establish an accurate mapping relationship between 2D style features and 3D scenes. In contrast, our method significantly improves the accuracy of style transfer and the ability to preserve 3D structure by introducing an autoencoder based on a graph convolutional network to efficiently aggregate local Gaussian primitives, demonstrating superior performance in all comparative experiments.

[0046] (2) Speed ​​assessment: This method maintains high-quality style transfer while significantly improving the processing efficiency of large-scale 3DGS scenes thanks to its zero-shot learning capability and feedforward network architecture. To quantitatively evaluate the computational performance of different methods, this paper conducts rigorous comparative experiments on the TNT dataset, such as... Figure 5 As shown, the average processing time of each method under different scale scenarios is recorded in detail.

[0047] Existing methods suffer from significant efficiency bottlenecks. While StyleGaussian can achieve real-time style-changing rendering of trained scenes (around 30fps), it requires 3-5 hours of preprocessing training even for new scenes containing only 50,000-100,000 Gaussian primitives. When the scene complexity exceeds 300,000 Gaussian primitives, the method fails to run due to insufficient GPU memory. Although StyleSplat, based on an optimized method, has a relatively stable processing time (average 300±15 seconds), it suffers severe memory overflow issues when processing scenes with more than 4.8 million Gaussian primitives.

[0048] This method can complete style transfer in only 8-12 seconds in typical scenarios with millions of Gaussian primitives, and the algorithm complexity is linearly related to the scene size. For extremely large-scale scenarios, an adaptive block processing mechanism is implemented. First, Gaussian primitives are dynamically clustered based on spatial continuity (cluster size is automatically adjusted according to the video memory capacity), and then a pipelined approach is used to process each cluster. Experimental data show that a single batch can efficiently process 1.2-1.5 million Gaussian primitives, and theoretically, it can support scenes of any size through multi-batch parallel processing.

[0049] In summary, this method can generate high-quality, consistent, and realistic stylized effects in various 3D scenes, achieving a two-order-of-magnitude speedup. Although this method has demonstrated superior real-time style transfer performance and artistic expressiveness in various 3D scenes, there are still many directions worthy of further exploration. Current work mainly focuses on color transfer while maintaining geometric structure integrity; future research can further explore the possibility of style transfer modifications to 3DGS geometric features.

[0050] Based on the same inventive concept, such as Figure 6 As shown, this embodiment of the invention also provides a zero-sample stylized 3D graphics content generation system 600 based on 3D Gaussian sputtering, including: a 3D feature extraction module 610, a style feature fusion module 620, and a 3D scene reconstruction module 630.

[0051] The 3D feature extraction module 610 is used to downsample the input 3D Gaussian sputtering scene and extract its 3D scene features through the encoder in the autoencoder.

[0052] The style feature fusion module 620 is used in the style transfer module to project the 3D scene features into the feature space aligned with the 2D style image features through the first multilayer perceptron, and then inject the style features from the 2D style image into the 3D scene features based on adaptive instance normalization to obtain the stylized features. Subsequently, it is projected back into the original feature space through the second multilayer perceptron.

[0053] The 3D scene reconstruction module 630 is used to upsample the stylized features through the decoder in the autoencoder to reconstruct the stylized 3D Gaussian sputtering scene.

[0054] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described method for generating zero-sample stylized 3D graphic content based on 3D Gaussian sputtering when executing the computer program.

[0055] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for generating zero-sample stylized 3D graphic content based on 3D Gaussian sputtering.

[0056] It should be noted that the zero-sample stylized 3D graphics content generation system, electronic device, and computer-readable storage medium based on 3D Gaussian sputtering provided in the above embodiments all belong to the same inventive concept as the zero-sample stylized 3D graphics content generation method based on 3D Gaussian sputtering. For details of the specific implementation process, please refer to the embodiments of the zero-sample stylized 3D graphics content generation method based on 3D Gaussian sputtering, which will not be repeated here.

[0057] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A zero-shot stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering, characterized in that, The method comprises the following steps: down-sampling and extracting three-dimensional scene features of the input three-dimensional Gaussian sputtering scene through an encoder in the auto-encoder; in the style transfer module, projecting the three-dimensional scene features to a feature space aligned with the two-dimensional style image features through a first multi-layer perceptron, injecting style features from the two-dimensional style image into the three-dimensional scene features through adaptive instance normalization, obtaining style-transferred features, and then projecting the style-transferred features to the original feature space through a second multi-layer perceptron; up-sampling the style-transferred features through a decoder in the auto-encoder and reconstructing the style-transferred three-dimensional Gaussian sputtering scene.

2. The zero sample stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering of claim 1, wherein, The auto-encoder adopts a symmetrical encoder-decoder architecture, wherein the encoder is composed of at least one three-dimensional graph convolutional layer and a channel max-pooling layer, and is used for feature extraction and spatial down-sampling; the decoder is composed of at least one three-dimensional graph convolutional layer and is up-sampled through an inverse distance weighted interpolation algorithm to restore the Gaussian primitive quantity of the original scene.

3. The zero sample stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering of claim 2, wherein, The three-dimensional graph convolutional layer operates through a three-dimensional convolution kernel with a learnable shape and weight, and for each Gaussian primitive, the local neighborhood features are aggregated through inner product operation and directional vector normalization within the receptive field defined by the spatial nearest neighbor.

4. The zero sample stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering of claim 1, wherein, The projecting of the three-dimensional scene features to the feature space aligned with the two-dimensional style image features through the first multi-layer perceptron comprises: extracting two-dimensional style image features from the style image through a pre-trained VGG network, and projecting the three-dimensional scene features to an intermediate feature space matching the distribution of the two-dimensional style image features through feature nonlinear mapping by the first multi-layer perceptron.

5. The zero sample stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering of claim 4, wherein, The injecting of the style features from the two-dimensional style image into the three-dimensional scene features through the adaptive instance normalization to obtain the style-transferred features, and then the projecting of the style-transferred features to the original feature space through the second multi-layer perceptron comprises: in the intermediate feature space, performing style transfer on the projected three-dimensional scene features based on an adaptive instance normalization method using the mean and variance of the two-dimensional style image features, and then remapping the style-transferred features back to the original feature space of the three-dimensional scene features based on the second multi-layer perceptron to eliminate the modal difference between the three-dimensional content and the two-dimensional style.

6. The zero sample, three-dimensional Gaussian-sputter-based stylized three- dimensional graphics content generation method of claim 1, wherein, A two-stage training strategy is adopted to train the entire network: in the first stage, using mean square error as the color reconstruction loss, training the auto-encoder alone to learn the color feature reconstruction capability of the three-dimensional Gaussian sputtering scene; in the second stage, freezing the parameters of the auto-encoder, introducing the style transfer module, and training the style transfer capability of the model using a combined loss function combining the content loss and the style loss.

7. The zero sample stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering of claim 1 or 6, wherein, A dataset containing multiple three-dimensional Gaussian sputtering scenes is constructed for training the entire network, and the construction process of the dataset comprises: selecting three-dimensional models covering common objects and having diverse materials from a three-dimensional model library; generating multi-view RGB images for each three-dimensional model using a rendering engine; inputting the multi-view RGB images into a TriplaneGaussian model to convert them into three-dimensional Gaussian sputtering scene representations, thereby obtaining a dataset containing several three-dimensional Gaussian sputtering scenes.

8. A zero-shot stylized three-dimensional graphics content generation system based on three-dimensional Gaussian sputtering, implemented by the zero-shot stylized three-dimensional graphics content generation method based on three-dimensional Gaussian sputtering in any one of claims 1-7. The method comprises the following steps: The three-dimensional feature extraction module, the style feature fusion module, and the three-dimensional scene reconstruction module; The three-dimensional feature extraction module is configured to down-sample and extract three-dimensional scene features of an input three-dimensional Gaussian sputtering scene by an encoder in a self-encoder; The style feature fusion module is configured to, in a style transfer module, project the three-dimensional scene features to a feature space aligned with two-dimensional style image features by a first multi-layer perception, inject style features from the two-dimensional style image into the three-dimensional scene features based on adaptive instance normalization to obtain features after style conversion, and then project the features after style conversion to the original feature space by a second multi-layer perception; The three-dimensional scene reconstruction module is configured to up-sample the features after style conversion by a decoder in the self-encoder to reconstruct a three-dimensional Gaussian sputtering scene after style conversion.

9. An electronic device comprising a memory and one or more processors, the memory to store a computer program, characterized in that, The processor is configured to, when executing the computer program, implement the three-dimensional Gaussian sputtering based zero-shot stylized three-dimensional graphical content generation method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is configured to, when executed by a computer, implement the three-dimensional Gaussian sputtering based zero-shot stylized three-dimensional graphical content generation method according to any one of claims 1-7.

Images