Multimodal Style-Transfer Network for High-Resolution Texture Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional style-transfer networks are limited in their ability to accurately transfer small, intricate textures and artistic styles from style exemplars to high-resolution input images, as they are typically trained on specific resolutions and fail to handle varying image resolutions effectively.
Innovation Solution
A multimodal style-transfer network is employed, comprising a low-resolution-based stylization subnet and a high-resolution stylization subnet, trained with style exemplars of different resolutions to apply multiple style features to input images, allowing for the combination of coarse and intricate texture patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a conventional style-transfer network is trained on a specific resolution (e.g., 256×256), then the training process is simple and fast, but the texture scale in the output image becomes smaller than the artistic style when applied to higher-resolution images
Solution Approach 1:
The patent divides the style transfer task into multiple resolution levels. Instead of using a single resolution network, it creates a hierarchical structure where style features are extracted and applied at different scales (e.g., 256×256, 512×512, 1024×1024). This segmentation allows each subnet to specialize in capturing style features at its appropriate scale, resolving the contradiction between training efficiency and texture accuracy.
Solution Approach 2:
The patent introduces the resolution dimension as an additional parameter for style transfer. By training multiple subnets at different resolutions rather than a single resolution, the system captures style features across multiple scales. This dimensional approach allows the network to preserve both coarse artistic styles and fine texture details simultaneously, solving the texture scale mismatch problem.
2Manufacturing precision
If a conventional style-transfer network is trained on high resolution, then small intricate textures can be captured, but the training complexity and computational cost increase significantly
Solution Approach 1:
The patent segments the high-resolution style transfer task into multiple lower-resolution subtasks. Each subnet is trained at a specific resolution level, dividing the complex high-resolution problem into manageable pieces. This reduces the training complexity of individual subnets while collectively achieving high-resolution texture accuracy through their hierarchical combination.
Solution Approach 2:
The patent applies partial style transfer at each resolution level rather than attempting complete high-resolution transfer in one step. Each subnet performs style transfer partially at its designated resolution, and the results are combined to achieve the complete high-resolution effect. This incremental approach reduces training complexity while maintaining texture accuracy.
3Ease of manufacture
If a single resolution style exemplar is used, then the training process is straightforward, but the system fails to accurately transfer both coarse and fine texture patterns simultaneously
Solution Approach 1:
The patent segments the style exemplar into multiple resolution versions (e.g., low-resolution for coarse styles, high-resolution for fine textures). Each resolution version is used to train a specialized subnet that captures style features at its appropriate scale. This segmentation enables the system to transfer both coarse and fine texture patterns accurately while keeping each training subtask relatively simple.
Solution Approach 2:
The patent changes the resolution parameter of the style exemplar across different subnets. By training subnets with style exemplars at different resolutions rather than a single resolution, the system captures style features across multiple scales. This parameter variation allows accurate transfer of both coarse artistic styles and fine texture patterns simultaneously.
Data Source
AI summary
A computing system transforms an input image into a stylized output image by applying first and second style features from a style exemplar. The input image is provided to a multimodal style-transfer network having a low-resolution-based stylization subnet and a high-resolution stylization subnet. The low-resolution-based stylization subnet is trained with low-resolution style exemplars to apply the first style feature. The high-resolution stylization subnet is trained with high-resolution style exemplars to apply the second style feature. The low-resolution-based stylization subnet generates an intermediate image by applying the first style feature from a low-resolution version of the style exemplar to first image data obtained from the input image. Second image data from the intermediate image is provided to the high-resolution stylization subnet. The high-resolution stylization subnet generates the stylized output image by applying the second style feature from a high-resolution version of the style exemplar to the second image data.