Face Transformation Video Processing With Triangular Patch Deformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face transformation algorithms face issues with significant changes in feature point locations due to posture differences, leading to overlap, misalignment, and jitter in synthesized videos, negatively impacting user experience.
Innovation Solution
The method involves cropping and identifying feature points in faces, performing triangular patch deformation and similarity transformation on image sequences to stabilize the face transformation process, ensuring high-quality video output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If geometric transformation is applied to faces with great posture differences, then face transformation can be achieved, but feature point location changes greatly causing overlap, misalignment, and instability
Solution Approach 1:
The patent divides the face into multiple triangular patches and processes each patch independently through deformation and similarity transformation. This segmentation allows precise control over feature point locations in each patch while maintaining overall face transformation capability, resolving the contradiction between adaptability and precision.
Solution Approach 2:
The patent applies similarity transformation (including scaling, rotation, and translation parameters) to each triangular patch separately. By adjusting these parameters independently for each patch, the system can maintain feature point location precision while achieving flexible face transformation across different postures.
2Adaptability or versatility
If geometric transformation is applied to faces with great posture differences, then face transformation can be achieved, but synthesis video exhibits subtle jitters impacting user experience
Solution Approach 1:
By segmenting the face into triangular patches and processing them independently with controlled deformation, the system reduces cumulative errors that cause jitters in the synthesized video. Each patch's transformation is stabilized individually, improving overall video reliability.
Solution Approach 2:
The patent employs iterative optimization where feature point locations are adjusted based on detected correspondences between source and target faces. This feedback mechanism stabilizes the transformation process and eliminates jitters in the final video synthesis.
3Reliability
If triangular patch deformation is performed on image sequences, then face transformation stability is improved, but processing complexity increases
Solution Approach 1:
The patent segments the face into triangular patches, which simplifies the deformation process compared to handling the entire face as one unit. Each patch can be processed independently with standard image processing techniques, reducing overall algorithmic complexity while maintaining stability.
Solution Approach 2:
The patent uses similarity transformation matrices that can be pre-computed or copied from template data, reducing the computational burden during real-time processing. This approach maintains transformation stability without significantly increasing processing complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
At least two images are acquired (S11). At least two crop images are acquired (S12) by cropping the at least two images for face-containing images. Triangular patch deformation is performed (S13) on two neighbour images, generating a first triangular patch deformation frame image sequence and a second triangular patch deformation frame image sequence. Similarity transformation is performed (S14) on each image sequence of the first triangular patch deformation frame image sequence, acquiring a first transform frame image sequence. Similarity transformation is performed (S14) on each image sequence of the second triangular patch deformation frame image sequence, acquiring a second transform frame image sequence. The first transform frame image sequence and the second transform frame image sequence are fused (S15), acquiring a video frame sequence corresponding to the two neighbour images. A video frame sequence generated by all neighbour images is coded (S16), acquiring a destined video.