3D Face GAN Inversion for Selfie Perspective Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selfie images captured at short camera-to-face distances often exhibit significant distortion, particularly affecting facial features like the nose, cheek bones, jaw line, chin, lips, eyes, eyebrows, ears, hair, and neck, and may not include the ears due to the limited arm's length distance, leading to an abnormal face shape.
Innovation Solution
A pre-trained 3D face GAN is used to invert and process selfie images, fine-tune generator and neural renderer parameters, and manipulate camera settings to generate photorealistic images with reduced distortion, simulating a longer camera-to-face distance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a camera is held at arm's length to take a selfie, then the user can easily capture the self-portrait, but the image appears distorted with abnormal face shapes and missing features
Solution Approach 1:
The system changes the camera parameters (focal length, distance) virtually through GAN manipulation. The generator network transforms the short-distance selfie parameters into long-distance equivalent parameters, effectively changing the optical parameters without physically moving the camera or subject
Solution Approach 2:
The patent replaces the mechanical solution of physically moving the camera to a longer distance with a computational approach using Generative Adversarial Networks. The GAN system substitutes the mechanical camera positioning system with an intelligent algorithm that synthesizes what a long-distance image would look like based on the short-distance input
2Ease of operation
If a short camera-to-face distance is used for selfies, then the capture is convenient, but perspective distortion occurs causing abnormal face shapes and missing features like ears
Solution Approach 1:
The system modifies the perceived geometric parameters of the face by transforming the short-distance perspective into a long-distance perspective. The GAN adjusts the spatial relationships and proportions of facial features to match what would be observed from a greater distance, correcting the distortion without requiring physical repositioning
Solution Approach 2:
The patent creates a synthetic copy of what the face would look like from a long distance based on the short-distance input. The GAN generates a new image that copies the essential facial characteristics but presents them with the correct geometric proportions as if captured from a longer distance
Data Source
AI summary
A method of correcting perspective distortion of a selfie image captured with a short camera-to-face distance by processing the selfie image and generating an undistorted selfie image appearing to be taken with a longer camera-to-face distance. A pre-trained 3D face GAN processes the selfie image, inverts the 3D face GAN to obtain improved face latent code and camera parameters, fine tunes a 3D face GAN generator, and manipulates camera parameters to render a photorealistic face selfie image. The processed selfie image has less distortion in the forehead, nose, cheek bones, jaw line, chin, lips, eyes, eyebrows, ears, hair, and neck of the face.