Keypoint Pose Acquisition With Temporal Heat Map Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pose acquisition methods in artificial intelligence face inaccuracies due to varying sizes and poses of target objects in different pictures, leading to inaccurate predictions of key points and poses.
Innovation Solution
A pose acquisition method that utilizes a key point coordinate positioning model, incorporating a heat map encoding model and a temporal multi-scale model, to determine three-dimensional coordinates of key points by processing heat maps from both the target and adjacent image frames, improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If pose acquisition is based on predicted two-dimensional coordinates of key points, then the process is simple, but the accuracy of predicted key point positions deteriorates due to varying sizes and poses of target objects
Solution Approach 1:
The patent transitions from two-dimensional coordinate prediction to three-dimensional coordinate prediction by incorporating temporal information and heat map encoding. The model processes heat maps from multiple frames (current and adjacent frames) to predict 3D key point coordinates, adding the temporal dimension to overcome the inaccuracies of 2D prediction caused by varying object sizes and poses.
Solution Approach 2:
The patent performs preliminary actions by detecting and tracking the target object in advance, and by pre-processing heat maps from adjacent frames before predicting the current frame's key points. This preliminary detection and heat map preparation enables more accurate 3D coordinate prediction by leveraging temporal context.
2Productivity
If pose acquisition uses only single-frame images, then the processing speed is fast, but the accuracy of key point predictions deteriorates due to insufficient contextual information
Solution Approach 1:
The patent merges information from multiple frames by processing heat maps from both the current frame and adjacent frames through the heat map encoding model. This combination of temporal information allows the model to maintain high prediction accuracy while processing videos frame-by-frame, balancing speed and precision.
Solution Approach 2:
The patent maintains continuous useful action by sequentially processing frames in a video stream, using the output of adjacent frames to improve the current frame's prediction. This continuous temporal context extraction enables accurate key point prediction without requiring complete video analysis before producing results.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
A posture acquisition method and device, and a key point coordinate positioning model training method and device. The posture acquisition method comprises: acquiring a target video (201); with regard to an ith image frame in the target video, acquiring a heat map of the ith image frame, i being a positive integer (202); according to the heat map of the ith image frame and the heat map of the adjacent image frame of the ith image frame, obtaining the three-dimensional coordinates of a key point in the ith image frame (203); and according to the three-dimensional coordinates of the key point in the ith image frame, acquiring a target posture of a target object in the ith image frame (204). Said method effectively overcomes the problem of inaccurate prediction caused by different sizes and postures of a target object in different pictures, thereby improving the accuracy of the predicted key point position, and further improving the accuracy of the acquired posture.