3D Model Stylization via 2D Projection and Spatial Feature Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods cannot effectively stylize three-dimensional models by transferring the artistic style of a two-dimensional image to a three-dimensional model due to the complexities of three-dimensional convolution and deformation issues during projection.
Innovation Solution
A method and apparatus that utilize a predetermined network to render a three-dimensional model into a two-dimensional image, extracting spatial feature parameters to stylize texture features, ensuring continuity and conformity by processing multiple viewing angles and using a deep neural network with an encoder-decoder structure for style transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If three-dimensional convolution is used for stylization, then the three-dimensional structure can be processed, but the computational complexity increases and deformation issues occur during projection
Solution Approach 1:
The patent segments the three-dimensional model processing into multiple two-dimensional projection views. Instead of applying complex 3D convolution directly to the volumetric data, the method projects the 3D model onto multiple 2D planes, applies style transfer independently to each projection, and then synthesizes the results. This segmentation reduces computational complexity while maintaining stylization effectiveness.
Solution Approach 2:
The patent transforms the problem from three-dimensional space to two-dimensional space by projecting the 3D model onto 2D planes. This dimensionality reduction allows the use of standard 2D convolutional neural networks for style transfer, avoiding the computational burden of 3D convolution while preserving the essential visual features through multi-view projection and synthesis.
2Ease of manufacture
If projection is used to render three-dimensional model, then the model can be processed in two-dimensional space, but deformation issues occur and texture continuity is lost
Solution Approach 1:
The patent merges multiple two-dimensional projections into a unified three-dimensional stylized model. By processing the model from multiple viewing angles and synthesizing the results, the method maintains texture continuity across the 3D surface while benefiting from the simplicity of 2D processing. The synthesis step combines information from different projections to restore and maintain continuous textures.
Solution Approach 2:
The patent employs feedback mechanisms to maintain texture continuity during projection and synthesis. The spatial feature parameters extracted from multiple projections are used to guide the synthesis process, ensuring that the final stylized model maintains consistent textures across different views. The feedback from multiple projections helps correct deformation issues and preserve texture coherence.
3Manufacturing precision
If multiple viewing angles are processed, then texture continuity is improved, but the processing time increases
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing spatial feature parameters from multiple projections before the actual style transfer synthesis. The spatial feature parameters, including depth information and viewing angles, are extracted and organized in advance, allowing the synthesis process to proceed more efficiently. This preliminary processing reduces the computational burden during the final synthesis stage.
Data Source
AI summary
The disclosure discloses a method and an apparatus for stylizing a three-dimensional model, an electronic device, and a storage medium. The method for stylizing the three-dimensional model includes: acquiring a to-be-stylized three-dimensional model and a stylized target image; and rendering the three-dimensional model by using a predetermined network to acquire a two-dimensional rendered image and spatial feature parameters of a pixel, and stylizing a texture feature in the two-dimensional rendered image based on the spatial feature parameters and the stylized target image to acquire a stylized three-dimensional model.