Deep Learning Face Synthesis with Shape Preservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face conversion technologies struggle to naturally synthesize a converted face with a background from another image without distorting the face or background, particularly when extracting only the converted face from a composite image.
Innovation Solution
A method and device using a deep learning network that receives an original image and a converted face image, removes the central part containing the original face from the original image and the background from the converted face image, and performs image synthesis by extracting feature vectors from both images, while considering the face shape and using techniques like color correction and skip connection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If only the converted face is extracted from a composite image for face conversion, then the face conversion can be performed, but the face and background become distorted
Solution Approach 1:
The patent segments the image processing into distinct components: face region extraction, background region extraction, and separate processing streams for each. The face extraction network isolates the face region from the composite image, while the background extraction network preserves the background region, allowing independent processing that maintains both face conversion accuracy and background integrity
Solution Approach 2:
The patent extracts only the necessary components (face region and background region) from the composite image using dedicated extraction networks. By taking out only the face region for conversion and preserving the background region separately, the method avoids distorting the overall image composition while achieving accurate face conversion
2Ease of manufacture
If the converted face is synthesized with the original background directly, then the synthesis process is simple, but the face shape is distorted
Solution Approach 1:
The patent introduces an intermediary face shape extraction network that processes the converted face to preserve and restore the original face shape characteristics. This intermediary step acts as a mediator between the face conversion process and the final synthesis, ensuring that the face shape accuracy is maintained while still allowing relatively simple synthesis with the original background
3Manufacturing precision
If additional networks like face shape extraction networks are used to preserve face shape, then the face shape accuracy is improved, but the device complexity increases
Solution Approach 1:
The patent merges multiple functions into integrated network architectures: the face extraction network simultaneously performs face region extraction and face shape preservation, while the background extraction network handles background preservation. This merging reduces the need for separate dedicated networks for each function, thereby reducing overall device complexity while maintaining face shape preservation accuracy
Data Source
AI summary
A method and a device for synthesizing a background and a face by considering a face shape and using a deep learning network are proposed. The method and the device are characterized to receive an input of an original image and a converted face image, remove a central part from the original image, remove edges so that a central part remains in the converted face image, and then extract a feature vector from each image to perform image synthesis.