Generative Neural Network Training via Distorted Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative neural networks struggle to generate synthetic images with detailed local structures and artifacts for semantic segmentation, lacking the necessary fine-grained details and perceptual realism.
Innovation Solution
A method is introduced to train a generative neural network using distorted segmentation images, applying a two-dimensional distortion such as thin-plate spline transformation, and combining it with loss functions to enhance the generation of synthetic images with improved structural details and realism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a generative neural network is trained to generate synthetic images from segmentation images, then the quantity of training images is increased, but the manufacturing precision of local structures and details deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-processing the segmentation image through distortion operations (such as thin-plate spline transformation) before feeding it to the generative neural network. This pre-distortion ensures that the network learns to generate images with proper structural details and geometric consistency, rather than learning from already-perfect segmented images that lack spatial accuracy information.
2Quantity of substance
If a generative neural network generates synthetic images from segmentation images, then the quantity of training data is increased, but the quality of generated images deteriorates due to artifacts
Solution Approach 1:
The patent implements feedback by using a discriminator network that evaluates the generated synthetic images and provides feedback to the generator network. The discriminator analyzes geometric consistency, structural合理性, and visual quality of the generated images, and this feedback is used to iteratively improve the generator's output, thereby reducing artifacts and enhancing image quality.
Solution Approach 2:
The patent applies preliminary action by pre-processing the segmentation image through distortion operations (such as thin-plate spline transformation) before feeding it to the generative neural network. This pre-distortion ensures that the network learns to generate images with proper structural details and geometric consistency, rather than learning from already-perfect segmented images that lack spatial accuracy information.
3Device complexity
If standard training methods are used for generative neural networks, then the training process is simple, but the generated images lack perceptual realism
Solution Approach 1:
The patent implements feedback by using a discriminator network that evaluates the generated synthetic images and provides feedback to the generator network. The discriminator analyzes geometric consistency, structural合理性, and visual quality of the generated images, and this feedback is used to iteratively improve the generator's output, thereby reducing artifacts and enhancing image quality.
Solution Approach 2:
The patent applies parameter changes by introducing multiple loss functions with different weights (such as cycle-consistency loss, identity loss, and perceptual loss) to balance the training objectives. By adjusting these loss parameters, the model achieves both structural accuracy and perceptual realism in the generated images.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A device and a method of training a generative neural network are disclosed, wherein the method of training a generative neural network includes: generating an edge image using an edge detection applied to a digital image, the edge image comprising a plurality of edge pixels determined as representing edges of one or more digital objects in the digital image; selecting edge-pixels from the plurality of edge pixels; providing a segmentation image using the digital image, the segmentation image comprising a plurality of first pixels, the positions of the first pixels corresponding to the positions of the selected edge-pixels; selecting one or more second pixels for each first pixel in the segmentation image; generating a distorted segmentation image using a two-dimensional distortion applied to the segmentation image; and training the generative neural network using the distorted segmentation image as input image to estimate the digital image.