Face Tracking Dual Keyframe Buffer Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time face tracking in video processing is hindered by the inability to add keyframes during facial identity optimization, leading to slow convergence and inaccurate facial identity reconstruction due to omitted keyframes and increased invocations of the optimization thread.
Innovation Solution
A face tracking method that maintains two keyframe data sets, allowing keyframes to be added and updated regardless of the optimization thread's status, optimizing facial identity by updating the first keyframe data set based on the second when necessary, and invoking the optimization thread only when necessary, thereby reducing the number of invocations and improving convergence speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the optimization thread is invoked frequently to ensure accurate facial identity reconstruction, then the accuracy of facial identity optimization is improved, but the convergence speed decreases and resource consumption increases
Solution Approach 1:
The patent implements dynamic thread management where the optimization thread can be paused and resumed based on keyframe availability. The system dynamically switches between tracking thread operations and optimization thread operations, allowing the optimization to continue accumulating accuracy while adapting to real-time processing requirements. This dynamic approach resolves the contradiction by making the system flexible rather than static in its thread invocation strategy.
Solution Approach 2:
The patent uses preliminary action by pre-identifying and storing keyframes in a buffer during the tracking phase before invoking the optimization thread. This allows the optimization thread to process accumulated keyframes in batches rather than waiting for each individual frame, thereby improving convergence speed without sacrificing the accuracy that would result from processing all available keyframes.
2Stability of the object's composition
If keyframes are omitted during optimization thread runtime to maintain system stability, then system stability is preserved, but keyframe extraction accuracy decreases due to slow convergence
Solution Approach 1:
The patent introduces a keyframe buffer as an intermediary data structure that decouples the tracking thread from the optimization thread. This buffer allows keyframes to be accumulated and stored during optimization thread runtime without causing system instability. The buffer acts as a mediator that holds data temporarily, enabling both threads to operate independently and concurrently, thus maintaining system stability while ensuring no keyframes are omitted.
Solution Approach 2:
The patent segments the face tracking process into two independent threads: a tracking thread that continuously identifies and buffers keyframes, and an optimization thread that processes these keyframes. This segmentation allows each thread to perform its specific function without interfering with the other, maintaining system stability while ensuring complete keyframe extraction accuracy by preventing omissions during optimization runtime.
3Speed
If the optimization thread is invoked every time a keyframe is added to ensure real-time processing, then real-time processing capability is improved, but the number of invocations increases leading to higher resource consumption
Solution Approach 1:
The patent implements periodic action by having the optimization thread invoke itself at regular intervals or when a threshold number of keyframes are accumulated, rather than being invoked continuously with every keyframe addition. This periodic invocation strategy maintains real-time processing capability by ensuring the optimization thread processes keyframes frequently enough, while significantly reducing the total number of invocations and associated resource consumption compared to continuous invocation.
Solution Approach 2:
The patent applies partial action by allowing the tracking thread to continue operating and identifying keyframes without always triggering the optimization thread. The optimization thread is invoked selectively based on accumulated keyframe count or time intervals, performing sufficient optimization actions to maintain real-time performance while avoiding excessive invocations that would waste computational resources.
4Loss of time
If two keyframe data sets are maintained to allow continuous keyframe updates, then the instantaneity of facial identity reconstruction is improved, but the device complexity increases
Solution Approach 1:
The patent segments the keyframe data storage into two distinct data sets: a first keyframe data set for tracking thread operations and a second keyframe data set for optimization thread operations. This segmentation allows each thread to access and update its designated data set independently without locking or synchronization overhead, thereby maintaining real-time instantaneity while managing complexity through clear separation of concerns. Each data set has a specific purpose, making the management logic simpler and more predictable.
Solution Approach 2:
The patent uses copying by creating a second keyframe data set that mirrors the structure and content of the first keyframe data set. This copying approach allows the system to maintain identical data structures for both threads, simplifying the management logic since the same data structures and access patterns can be used in both contexts. The copy enables parallel processing without requiring complex synchronization mechanisms, thus improving instantaneity while keeping device complexity manageable.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed are a face tracking method and apparatus, and an electronic device and a storage medium. The face tracking method comprises: during the process of performing face tracking on a video frame, determining whether an optimization thread is run; if an optimization thread is run, when the video frame is a key frame, updating a second key frame data set according to the video frame; when receiving a clearing instruction, which is sent by the optimization thread and is for clearing video frames in the second key frame data set, clearing the video frames in the second key frame data set, and updating the second key frame data set to a first key frame data set; if an optimization thread is not run, when the video frame is a key frame, updating the first key frame data set according to the video frame and the second key frame data set; and after the first key frame data set is updated, calling the optimization thread, such that the optimization thread optimizes a facial identity on the basis of the first key frame data set.