Structured Texture Image Generation With Reduced Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning techniques for generating high-quality artwork, such as neural style transfer and pix2pix, require large training datasets that are time-consuming and expensive to create, making it difficult to meet production deadlines in art-based products like video games.
Innovation Solution
A multiple-phase process using a pix2pix-based ML model for initial texture generation followed by neural style transfer to enhance detail, reducing the need for extensive training data and processing power, while maintaining high-quality output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural style transfer or pix2pix techniques are used to generate high-quality artwork, then the quality of generated images is improved, but the time and cost to create training datasets increases significantly
Solution Approach 1:
The patent segments the texture generation process into two distinct phases: (1) a pix2pix-based ML model generates initial texture images from input images, and (2) a neural style transfer model enhances the generated textures by applying style features from reference images. This segmentation allows each phase to be optimized independently, reducing the overall time and computational resources required while maintaining high image quality
Solution Approach 2:
The patent performs preliminary actions by pre-processing input images to extract structural features and pre-training the pix2pix model on a small dataset of image pairs. This preliminary preparation enables the model to generate reasonable initial textures quickly, reducing the need for extensive training data and accelerating the overall generation process
2Manufacturing precision
If pix2pix technique is used for image-to-image translation, then the detail and quality of output images is improved, but the requirement for large training datasets increases
Solution Approach 1:
The patent divides the texture generation task into two phases: first, a pix2pix-based model generates initial texture images from input images using a small training dataset; second, a neural style transfer model enhances the generated textures by applying style features from reference images. This segmentation allows the system to achieve high image detail quality while requiring minimal training data volume
Solution Approach 2:
The patent introduces an intermediary neural style transfer model that acts as a bridge between the pix2pix-generated textures and the final high-quality output. This intermediary model transfers style features from reference images to the generated textures, enhancing detail quality without requiring the pix2pix model to be trained on large datasets
3Manufacturing precision
If large quantities of high-quality artwork are produced manually by skilled artists, then the quality standard is maintained, but the production time and cost increases
Solution Approach 1:
The patent implements a self-service system where the ML models automatically generate and enhance texture images without requiring manual intervention at each step. The system takes input images, automatically generates textured versions through the two-phase process, and produces final high-quality artwork that maintains consistent quality standards while dramatically increasing production volume and reducing costs
4Manufacturing precision
If neural style transfer is applied to generate structured textures, then the stylistic quality is improved, but the computational processing power required increases
Solution Approach 1:
The patent segments the texture generation into two phases where the computationally intensive neural style transfer operation is applied only to enhance already-generated textures rather than creating textures from scratch. This segmentation reduces overall computational processing power requirements while maintaining high texture detail quality
Solution Approach 2:
The patent performs preliminary texture generation using the pix2pix model before applying neural style transfer. This preliminary action creates a reasonable base texture that requires less computational refinement, reducing the processing power needed for the subsequent style transfer enhancement while maintaining output quality
Data Source
AI summary
Techniques are described for a multiple-phase process that uses machine learning (ML) models to produce a texturized version of an input image. During a first phase, using a pix2pix-based ML model, an automatically-generated image that depicts structured texture is generated based on an input image that visually identifies a plurality of image areas for the structured texture. During a second phase, a neural style transfer-based ML model is used to apply the style of a style image (e.g., a target image from the training dataset of the pix2pix-based ML model) to the texture image generated at the first phase (the content image) to produce a modified texture image. According to an embodiment, during a third phase, the generated texture image produced at the first phase and the modified texture image produced at the second phase are combined to produce a structured texture image with a moderated amount of detail.