Facial Manipulation Processing with Cropped Landmark Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in accurately and efficiently detecting faces and facial landmarks, especially in high-resolution media, leading to inaccuracies and time-consuming processes.
Innovation Solution
A system and method that involves acquiring input frames, identifying and bounding faces, cropping and aligning target faces, and manipulating facial features through a series of preprocessing steps including resolution reduction, padding, and facial landmark detection to achieve precise facial manipulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If facial detection is performed on high-resolution input frames, then detection accuracy is improved, but processing time increases significantly
Solution Approach 1:
The patent segments the processing workflow into distinct stages: initial low-resolution facial detection to locate candidate faces, followed by cropping those regions, and then performing high-resolution landmark detection only on the cropped face regions. This segmentation allows the system to maintain high detection accuracy while significantly reducing overall processing time by applying computationally intensive operations only where necessary.
Solution Approach 2:
The patent performs preliminary facial detection on downsampled/low-resolution frames before conducting detailed landmark detection. By pre-identifying face locations and cropping them, the system prepares the data in advance for the more accurate but time-consuming landmark detection stage, thereby optimizing the balance between speed and precision.
2Measurement precision
If facial landmark detection is performed on full-resolution cropped faces, then landmark precision is improved, but computational resources increase
Solution Approach 1:
The patent applies local quality by performing high-resolution landmark detection only on cropped face regions rather than entire frames. This localized approach maintains high landmark detection precision where it is most needed (on the face) while significantly reducing the overall computational resources required, as the expensive high-precision operations are applied only to small portions of the original image.
3Manufacturing precision
If multiple preprocessing steps are added to align and crop faces, then facial manipulation accuracy is improved, but system complexity increases
Solution Approach 1:
The patent performs preliminary alignment and cropping operations using detected facial landmarks before the actual facial manipulation. By pre-aligning faces to a standard orientation and cropping to consistent dimensions, the system improves manipulation accuracy while organizing the complexity into manageable preprocessing steps that simplify the subsequent manipulation operations.
Data Source
AI summary
A system and method of processing media for facial manipulation. Input frames are downscaled to a lower resolution and facial detection identifies a target face in each input frame. The location of the target face is determined in the downscaled frames and the locations are projected to the input frames based on the pixel differences between the frames. Facial landmark detection is then performed on a cropped image from the original input frame. The facial landmarks are used to adjust the orientation of the input frames so that the target face in each input frame is in a standard orientation. Facial landmark detection is again performed on the target face in each frame while in the standard orientation to produce more accurate landmarks. Facial manipulation can then be executed based on the landmarks in the aligned input images. The orientation of the images with the manipulated faces are then reverted.