Video Human Behavior Recognition via Multi-Frame Key Point Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video-based human behavior recognition methods are inaccurate due to their reliance on single-frame analysis, which fails to account for the relationship between frames and is prone to deviations in interpretation, leading to low recognition accuracy and recall rates.
Innovation Solution
A method that constructs a feature matrix from multiple video frames, incorporating key points and their relationships across frames, using convolutional results from both temporal and spatial dimensions to enhance the analysis of human behavior, with increased key point features and confidence adjustments for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single-frame analysis is used for human behavior recognition, then the processing speed is fast and the method is simple, but the recognition accuracy and recall rate are low due to interpretation deviations
Solution Approach 1:
The patent transitions from single-frame spatial analysis to multi-frame temporal-spatial analysis by constructing a feature matrix that incorporates both spatial key point relationships and temporal frame sequences. This dimensional expansion allows the system to capture behavior evolution over time while maintaining structured feature relationships, thereby improving recognition accuracy without excessive complexity increase.
Solution Approach 2:
The patent segments the video into multiple frames and extracts key point features from each frame independently before combining them into a feature matrix. This segmentation allows for detailed local feature analysis while enabling temporal relationship modeling, resolving the contradiction between simple processing and accurate recognition.
2Measurement precision
If single-frame analysis is used for human behavior recognition, then the computational resources required are minimal, but the recognition accuracy and recall rate are low
Solution Approach 1:
The patent extracts only the most discriminative key point features from each frame and constructs a compact feature matrix containing essential temporal-spatial relationships. By selecting and extracting only relevant features rather than processing all pixel data, the system achieves improved recognition accuracy while controlling computational resource consumption.
3Reliability
If multiple video frames are analyzed with key point features and convolution operations, then the recognition accuracy and recall rate improve, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary feature extraction and organizes key point features into a structured feature matrix before applying convolution operations. This preliminary organization of temporal-spatial features reduces the complexity of subsequent processing steps and enables more efficient computation, balancing improved reliability with maintained processing speed.
Data Source
AI summary
The present application discloses a video-based human behavior recognition method, apparatus, device and storage medium, and relates to the technical field of human recognitions. The specific implementation scheme lies in: acquiring a human rectangle of each video frame of the video to be recognized, where each human rectangle includes a plurality of human key points, and each of the human key points has a key point feature; constructing a feature matrix according to the human rectangle of the each video frame; convolving the feature matrix with respect to a video frame quantity dimension to obtain a first convolution result and convolving the feature matrix with respect to a key point quantity dimension to obtain a second convolution result; inputting the first convolution result and the second convolution result into a preset classification model to obtain a human behavior category of the video to be recognized.