Depth Video Linkage Feature Behavior Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing behavior recognition algorithms using depth videos fail to fully utilize three-dimensional information and feature relationships across dimensions, leading to inaccuracies due to bone joint detection issues and occlusions.
Innovation Solution
A depth video linkage feature-based behavior recognition method that projects depth videos onto multiple sides, extracts dynamic images, and uses a multi-projection linkage feature extraction module to connect features across different projections, feeding them into an average pooling and fully connected layer for behavior recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bone joint-based behavior recognition is used, then behavior recognition can be performed, but accuracy deteriorates due to bone joint detection errors and occlusions
Solution Approach 1:
The patent extracts three-dimensional spatial information from depth videos by projecting onto multiple planes (front, back, left, right, top, bottom), then uses this extracted spatial data to construct dynamic images that capture motion patterns without relying on bone joint detection, thereby eliminating the measurement precision problem
Solution Approach 2:
The patent introduces an intermediary representation (dynamic image constructed from multi-plane projections) that mediates between the raw depth video data and the final behavior recognition, allowing the system to infer behavior from spatial motion patterns rather than directly from potentially erroneous bone joint detections
2Device complexity
If depth video is directly input into network, then processing is simplified, but three-dimensional information and feature relationships across dimensions are not fully utilized
Solution Approach 1:
The patent segments the three-dimensional depth video data into multiple two-dimensional projections (front, back, left, right, top, bottom views), processes each projection independently through dynamic image construction, then integrates the results. This segmentation allows thorough extraction of spatial information while maintaining manageable processing complexity
Solution Approach 2:
The patent transforms the three-dimensional depth video into multiple two-dimensional projections, then reconstructs dynamic images that capture temporal-spatial relationships. This dimensional transformation enables the network to process and utilize three-dimensional information more effectively by representing it through coordinated two-dimensional views across time
Data Source
AI summary
Provided relates to a depth video linkage feature-based behavior recognition method, comprising projecting a depth video of each behavior sample onto a front-side, a right-side, a left-side and a top-side; obtaining a dynamic image of each behavior sample by calculating a dynamic image of each projection sequence; inputting the dynamic image of each behavior sample into a respective feature extraction module and extracting features; inputting the extracted features into a multi-projection linkage feature extraction module and extracting a linkage feature of each projection combination; connecting all the extracted linkage features by channel, and inputting the connected features into an average pooling layer and a fully connected layer; constructing a depth video linkage feature-based behavior recognition network; inputting a depth video of each training behavior sample into the depth video linkage feature-based behavior recognition network, and training the network till convergence; and inputting a depth video of each behavior sample.