3D Attribute Fusion for Face Swapping Across Poses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face swapping technologies struggle with unnatural face deformation when faces in different poses, leading to low accuracy due to the reliance on simple shape fitting, which fails to maintain similarity between the original and swapped faces.
Innovation Solution
An image processing method that determines a target attribute parameter based on the image and target face attributes, fusing these parameters with face features to generate a refined pixel-level feature alignment, enhancing the similarity and accuracy of the face swapping process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simple shape fitting is used for face swapping, then the process is simple and fast, but the accuracy deteriorates when faces have different poses, leading to unnatural deformation
Solution Approach 1:
The patent transforms 2D face images into 3D representations by extracting and matching 3D landmark points, pose parameters (rotation, translation, scale), and shape coefficients. This parameter transformation from 2D to 3D space enables accurate face alignment and deformation even when poses differ significantly, resolving the accuracy issue while maintaining computational efficiency through parameterized modeling
Solution Approach 2:
The patent introduces a third dimension by converting 2D face images into 3D face models using 3DMM (3D Morphable Models). By operating in 3D space rather than 2D, the system can accurately represent and transform faces with different poses, expressions, and orientations, thereby improving face swapping accuracy without sacrificing processing speed
2Manufacturing precision
If 3D attribute parameters and pixel-level feature alignment are implemented, then face swapping accuracy improves, but the device complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-training 3DMM models on large datasets to capture statistical variations in face shapes, expressions, and poses. These pre-computed 3D models and landmark point configurations are stored and reused during face swapping operations, eliminating the need to perform complex 3D reconstruction from scratch for each image, thus reducing real-time computational complexity while maintaining high accuracy
Solution Approach 2:
The patent introduces 3DMM models and 3D landmark points as intermediary representations between the input 2D face images and the final swapped output. These intermediaries serve as a bridge that simplifies the transformation process by providing a standardized 3D framework for alignment, deformation, and fusion, thereby reducing overall system complexity compared to direct pixel-level manipulation
Data Source
AI summary
This application provides an image processing method performed by a computer device. The image processing method includes: receiving a face swapping request; acquiring an attribute parameter of the image, an attribute parameter of the target face, and a face feature of the target face, the attribute parameter of the image indicating a three-dimensional attribute of the face in the image; determining a target attribute parameter based on the attribute parameter of the image and the attribute parameter of the target face; determining a target comprehensive feature based on the target attribute parameter and the face feature of the target face; encoding the image to obtain an image encoding feature of the image; migrating the target comprehensive feature to the image encoding feature of the image by normalization to obtain a fusion encoding feature; and decoding the fusion encoding feature to obtain a target face-swapped image including a fusion face.