Image-to-Image Defect Generation With Style-Content Disentanglement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Generative Adversarial Networks (GANs) require large amounts of data for stable training and struggle to generate realistic images of defects on products with limited data, especially when defective samples are scarce, leading to imbalanced datasets.
Innovation Solution
A method for training a generator to disentangle style and content using specific loss functions, allowing the generation of synthetic defective images by transferring defects from one image to another while preserving the background, enabling weakly-supervised learning and enhancing image variance through diversity losses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If Generative Adversarial Networks are used to generate synthetic defective images, then the training dataset can be enriched, but the method requires large amounts of data for stable training
Solution Approach 1:
The patent segments the image into foreground (defect) and background components, and further divides the generator into multiple sub-generators that independently handle different aspects: one sub-generator generates defect regions while another preserves the background. This segmentation allows the system to learn from limited defective samples more effectively by focusing on defect-specific features rather than requiring comprehensive data coverage of all possible defect scenarios.
2Adaptability or versatility
If the generator transforms the entire image to generate defective samples, then diverse defect patterns can be created, but the background style is modified along with the defect
Solution Approach 1:
The patent segments the image transformation process into separate operations: one sub-generator (first sub-generator) specifically generates defect regions by modifying only the foreground portion of the image, while another sub-generator (second sub-generator) preserves the background by generating only the defect-free regions. This segmentation enables the system to create diverse defect patterns without altering the background style, as each sub-generator operates on its designated image portion independently.
Solution Approach 2:
The patent extracts the defect region from the image and handles it separately from the background. By using a first sub-generator that operates specifically on the foreground/defect portion and a second sub-generator that handles the background, the system extracts and processes the defect information independently. This extraction allows defect diversity to be maximized while the background remains unchanged, preventing loss of background style information.
3Ease of manufacture
If defects are generated on normal surface images, then defective samples can be created for training, but the generated defects may not capture the full variance of real defect patterns
Solution Approach 1:
The patent applies preliminary action by using a Style Encoder to extract and encode the style information from reference defective images before the generation process. This style encoding is then integrated into the sub-generators, allowing them to incorporate authentic defect characteristics and stylistic features from real defective samples into the generated defects. This preliminary extraction and encoding of style information enables the system to generate defects with higher variance and realism, capturing the full range of defect patterns without requiring extensive manual defect creation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method of training a generator (1) for transforming a given image (x) according to a given target foreground domain is disclosed. Similar to the STARGAN v2 approach, a generator (1), discriminator (2), Mapping network (3) and a Style-Content encoder (4) are trained. Furthermore, a method of image to image translation according to a given target foreground domain (y) by the trained generator (1) is disclosed. In one embodiment of the invention, the generator is trained to generate images depicting defects.