Neural Network Face Swapping via Texture and Position Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for changing facial identities in video frames and images often produce unrealistic results due to reliance on imperfect 3D geometry and lack of accounting for 3D geometry in neural network models.
Innovation Solution
A computer-implemented method using a machine learning model to generate texture and position maps from an input image, allowing for the rendering of an image with a different facial identity without requiring perfect 3D meshes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing 3D morphable model techniques are used to change facial identities, then facial identity replacement can be achieved, but unrealistic artifacts appear due to imperfect 3D geometry
Solution Approach 1:
The patent introduces a neural network model as an intermediary between the input image and the final facial identity change output. This neural network learns to correct geometric imperfections and generate realistic facial transformations without requiring perfect 3D geometry, thereby resolving the contradiction between achieving reliable facial identity replacement and dealing with imprecise 3D geometric data
Solution Approach 2:
The patent transforms the approach from relying on precise 3D geometric parameters to using learned parameters from a neural network. The model learns optimal transformation parameters directly from image data, bypassing the need for accurate 3D geometry while maintaining realistic facial identity changes
2Device complexity
If neural network techniques that do not account for 3D geometry are used, then processing simplicity is achieved, but unrealistic facial changes occur in profile views and extreme poses
Solution Approach 1:
The patent changes the processing approach by incorporating 3D geometric awareness into the neural network model. The model learns to account for 3D geometry implicitly through training on diverse facial data including profile views and extreme poses, achieving both simplicity and realism without requiring complex explicit 3D processing
Solution Approach 2:
The patent effectively adds a 3D geometric dimension to the neural network's processing capability. By training the model to understand 3D facial structures through learned representations, it can handle extreme poses and profile views realistically while maintaining the simplicity of a neural network approach
3Manufacturing precision
If 3D meshes with perfect geometry are used, then accurate facial representation is achieved, but specialized equipment and difficult image capture are required
Solution Approach 1:
The patent creates a learned representation (a kind of digital copy) of 3D facial geometry from ordinary 2D images using a neural network. This learned 3D model copy achieves accuracy comparable to perfect 3D meshes without requiring specialized scanning equipment, thereby resolving the contradiction between achieving accurate 3D representation and ease of data acquisition
Solution Approach 2:
The patent replaces the mechanical 3D scanning system with a neural network-based system. Instead of using specialized equipment to capture 3D geometry directly, the model learns to infer 3D structures from 2D images, substituting a computational approach for a physical measurement approach and achieving both accuracy and ease of manufacture
Data Source
AI summary
Various embodiments set forth systems and techniques for changing a face within an image. The techniques include receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map and a first position map based on the first image; rendering a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity.