Key Point Detection Model With Inter-Frame Residual Branch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing human face key point detection technologies face instability in key point position detection between adjacent frames, particularly in continuous frame types like videos, leading to jittery results.
Innovation Solution
A method that preprocesses images into multi-channel images based on their type, using a pre-trained key point detection model with a regression branch and an inter-frame residual branch to accurately determine key points in both discrete and continuous frame types, stabilizing key point positions across frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing key point detection technology is used on continuous frame images, then key point detection can be performed on each frame, but the key point positions jitter between adjacent frames resulting in unstable detection results
Solution Approach 1:
The system performs preliminary classification of the image sequence type (continuous frame vs. discrete frame) before executing the key point detection algorithm. This preliminary action allows the system to select the appropriate detection strategy in advance, preventing jitter in continuous frame sequences by applying inter-frame residual detection only when needed, thus resolving the stability-accuracy contradiction.
Solution Approach 2:
The system dynamically adjusts the key point detection strategy based on the detected image sequence type. For continuous frame sequences, it employs inter-frame residual detection that leverages temporal coherence to stabilize key point positions. For discrete frames, it uses standard detection methods. This dynamic adaptation resolves the contradiction by optimizing the detection approach for each specific scenario.
2Device complexity
If a unified key point detection method is used for both discrete and continuous frame images, then the system structure is simplified, but the detection stability for continuous frames deteriorates
Solution Approach 1:
The system segments the detection process into two distinct pathways: one for continuous frame sequences and another for discrete frames. By introducing a sequence type classifier that divides the input stream, the system can apply specialized detection methods to each segment. This segmentation resolves the contradiction by maintaining simple overall structure while enabling specialized stable detection for continuous frames through the inter-frame residual approach.
3Stability of the object's composition
If inter-frame residual detection is always used for continuous frames, then key point position stability improves, but the computational complexity increases
Solution Approach 1:
The system performs preliminary classification of the image sequence type before executing the detection algorithm. By detecting whether the input is a continuous frame sequence or discrete frames in advance, the system only activates the computationally heavier inter-frame residual detection pathway when necessary. This preliminary action resolves the contradiction by maintaining high stability for continuous frames while avoiding unnecessary computational complexity for discrete frame inputs.
Data Source
AI summary
A method and device for detecting the key point of an image. The method includes acquiring the image, performing preprocessing to generate a multi-channel image with a specific type, and inputting the image into a detection model. The detection model includes a regression branch and an inter-frame residual branch. If the image type is the discrete frame, the system determines coordinates of multiple key points in the image based on a regression prediction value outputted by the regression branch. If the image type is continuous frame, the system determines coordinates of multiple key points in the image based on a regression prediction value outputted by the regression branch and a residual prediction value outputted by the inter-frame residual branch. The method can be used in the detection of discrete frame and discrete frame type images and can improve the stability of key point detection of images.