Image Synthesis Using CNN Texture Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for image synthesis fail to effectively transfer local and global texture features from a source image to a target image, relying on fixed assumptions and not capturing the full scope of natural textures, while also preserving the identity of objects in the target image.
Innovation Solution
A method using non-linear transformations, specifically a convolutional neural network, to extract relevant features and represent textures, allowing for the separation and manipulation of content and style information, enabling the generation of high-quality images that combine the content of one image with the style of another, without hard-coding image features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If fixed assumptions and handcrafted summary statistics are used for texture synthesis, then the method is simple and computationally efficient, but it fails to capture the full scope of natural textures and cannot effectively transfer local and global texture features
Solution Approach 1:
The patent replaces handcrafted summary statistics and fixed assumptions with a convolutional neural network that automatically learns texture features from training data. The CNN substitutes manual feature engineering with automated feature extraction, enabling the system to capture both local and global texture characteristics without relying on predefined statistical models.
Solution Approach 2:
The patent transforms the approach from using fixed, handcrafted parameters to using learned parameters from training data. The CNN adapts its internal parameters (weights and biases) during training to optimally represent natural textures, allowing the system to capture the full scope of texture variations in natural images.
2Manufacturing precision
If complex features and non-linear transformations are extracted using neural networks, then the texture representation becomes more accurate and flexible, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs feature extraction and texture representation learning in advance during a training phase using training data. The CNN is pre-trained to learn optimal feature representations, so that during actual image synthesis, the system can efficiently apply these pre-learned features without performing complex computations in real-time.
3Productivity
If handcrafted summary statistics are used, then the method is computationally efficient, but it cannot adapt to different types of natural textures and lacks flexibility
Solution Approach 1:
The patent replaces handcrafted summary statistics with a convolutional neural network that automatically learns texture features from training data. The CNN substitutes manual feature engineering with automated feature extraction, enabling the system to capture both local and global texture characteristics without relying on predefined statistical models.
Solution Approach 2:
The patent transforms the approach from using fixed, handcrafted parameters to using learned parameters from training data. The CNN adapts its internal parameters (weights and biases) during training to optimally represent natural textures, allowing the system to capture the full scope of texture variations in natural images.
4Manufacturing precision
If the full scope of natural textures is captured using learned features, then the texture transfer quality improves, but the processing time and computational resources increase
Solution Approach 1:
The patent performs feature extraction and texture representation learning in advance during a training phase using training data. The CNN is pre-trained to learn optimal feature representations, so that during actual image synthesis, the system can efficiently apply these pre-learned features without performing complex computations in real-time.
Data Source
AI summary
Computer-implemented method for transferring style features from at least one source image to a target image, comprising the steps of generating a result image, based on the source and the target image, wherein one or more spatially-variant features of the result image correspond to one or more spatially variant features of the target image; and wherein a texture of the result image corresponds to a texture of the source image; and outputting the result image, and a corresponding device. According to the invention, the texture corresponds to a summary statistic of spatially variant features of the source image.