Facial Tracking via Selective Frame Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial registration point tracking algorithms face challenges in ensuring both high accuracy and low complexity, with complex anti-jitter rules and high costs due to the need for massive annotated training samples, which affects real-time performance and stability.
Innovation Solution
A facial tracking method that selects adjustment frames from prior frames based on facial feature information to determine an adjustment parameter, reducing errors and eliminating the need for complex anti-jitter rules, allowing for accurate tracking without delayed performance or extensive training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If averaging registration results of multiple previous and subsequent frames is used to improve stability, then stability of registration point is improved, but accuracy of registration points in video sequences with significant variation is severely reduced and real-time performance deteriorates due to tracking delay
Solution Approach 1:
The patent segments the video sequence into multiple frame groups, where each group contains a fixed number of consecutive frames. Instead of using all previous and subsequent frames for averaging, the method selectively processes frames within each group independently, reducing the temporal span used for stabilization while maintaining local accuracy.
Solution Approach 2:
The patent performs preliminary processing by pre-grouping frames and calculating reference values within each group before final registration. This preliminary action allows the system to establish stable reference points locally without waiting for all subsequent frames, thereby improving real-time performance while maintaining stability through structured preprocessing.
2Stability of the object's composition
If complex anti-jitter rules are applied to control jitter amplitudes of registration results, then stability of registration point is improved, but device complexity increases and both accuracy and stability are reduced in particular cases
Solution Approach 1:
The patent changes the parameter representation by using frame group indices and reference frame selections instead of complex jitter amplitude parameters. The method transforms the problem from controlling jitter amplitudes through complex rules to selecting appropriate reference frames from pre-defined groups, significantly simplifying the system while maintaining stability.
Solution Approach 2:
The patent extracts and removes the complex anti-jitter rule component entirely, replacing it with a simpler frame grouping and reference selection mechanism. By taking out the problematic complex rule and substituting it with straightforward frame group processing, the system achieves stability without the associated complexity and its negative effects on accuracy.
3Measurement precision
If deep learning networks such as 3DCNN or LSTM are used to directly train registration points from video, then accuracy of facial tracking is improved, but device complexity increases due to massive training sample requirements and higher time and memory overheads
Solution Approach 1:
The patent replaces expensive, resource-intensive deep learning networks with a simpler, lightweight algorithm that processes frames locally without requiring massive training datasets. The method uses disposable frame groups that can be processed independently with minimal computational resources, achieving good accuracy without the high costs of deep learning deployment.
Solution Approach 2:
The patent substitutes the complex mechanical system of deep learning networks (3DCNN, LSTM) with a simpler algorithmic approach based on frame grouping and reference frame selection. This substitution replaces the need for massive training samples and high computational overhead with a lightweight method that achieves comparable accuracy with significantly reduced device complexity.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
The present disclosure discloses a facial tracking method and apparatus, a storage medium, and an electronic device. The method includes: obtaining facial feature information of a current frame of image in a sequential image and facial tracking information of prior frames of images in the sequential image, the prior frames of images being frames of images located previous to the current frame of image in the sequential image, the facial feature information recording information of facial feature points in the current frame of image, and the facial tracking information being used for recording facial tracking results of the prior frames of images; selecting adjustment frames of images from the prior frames of images based on the facial feature information of the current frame of image and the facial tracking information of the prior frames of images; fitting an adjustment parameter of the current frame of image based on facial tracking information of the adjustment frame of image and the facial feature information of the current frame of image; and determining facial tracking information of the current frame of image based on the adjustment parameter and the facial tracking information of the prior frames of images. According to the present disclosure, a technical problem that it is difficult to ensure both high accuracy and low complexity by using an existing facial registration point tracking algorithm is resolved.