Skeleton Action Detection via Sparse Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing action detection methods based on human skeleton features face challenges in multi-person scenarios due to low accuracy caused by motion noise interference, especially in real-time monitoring of elderly safety where timely detection of falls is critical.
Innovation Solution
An action detection method that extracts keypoints from video streams, calculates a center point and approximate area of rigid motion, uses spatial features, and employs a graph convolutional neural network to enhance matching accuracy and enrich feature input for the action detection model, incorporating a Kalman filter for estimation and the Hungarian algorithm for matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If human skeleton feature is extracted using human pose estimation, then computational cost is reduced and real-time performance is enhanced, but the expression ability is weak and action detection accuracy deteriorates
Solution Approach 1:
The patent merges skeleton feature extraction with optical flow computation by computing optical flow at skeleton keypoints rather than throughout the entire image. This combines the efficiency of skeleton-based methods with the rich motion information of optical flow, achieving both real-time performance and high accuracy in action detection.
Solution Approach 2:
The patent applies local quality by computing optical flow only at specific skeleton keypoints rather than processing the entire image. This selective approach maintains computational efficiency while capturing essential motion information at critical body locations, resolving the contradiction between speed and accuracy.
2Measurement precision
If optical flow features are employed to represent object motion information, then action detection accuracy is improved, but computing and storage resources increase significantly
Solution Approach 1:
The patent extracts optical flow computation to only the necessary skeleton keypoints, removing the computationally expensive full-image optical flow calculation. This extraction maintains the essential motion information needed for accurate action detection while dramatically reducing computing and storage resource requirements.
3Device complexity
If person matching is performed directly based on human skeleton feature, then matching process is simplified, but motion noise affects matching result and action detection accuracy deteriorates
Solution Approach 1:
The patent introduces optical flow as an intermediary component that enhances skeleton feature matching. By computing optical flow at skeleton keypoints and using it as an additional matching criterion, the system filters out motion noise while maintaining simplified matching architecture, thereby improving matching accuracy without significantly increasing complexity.
Data Source
AI summary
An action detection method based on a human skeleton feature and a storage medium belong to the field of computer vision, and the method includes: for each person, extracting a series of body keypoints in every frame of the video as the human skeleton feature; calculating a body structure center point and approximating rigid motion area by using the human skeleton feature as a calculated value from the skeleton feature state, and predicting an estimated value in the next frame; performing target matching according to the estimated and calculated value, correlating the human skeleton feature belonging to the same target to obtain a skeleton feature sequence, and then correlating features of each keypoint in the temporal domain to obtain a spatial-temporal domain skeleton feature; inputting the skeleton feature into an action detection model to obtain an action category. In the disclosure, the accuracy of action detection is improved.