Hand Signal Recognition Using RT-GCN Bone-Vector Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for recognizing hand signals and gestures in various conditions, such as different hand positions and occlusions, lead to inaccurate recognition, which can cause confusion and accidents, particularly in traffic scenarios.
Innovation Solution
A behavior recognition method utilizing a robust temporal graph convolution network (RT-GCN) model that incorporates x, y coordinates, confidence, and bone vector length and angle as features for enhanced pose estimation, combined with a spatial feature extraction process to generate new features for improved recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only x, y coordinates and confidence are used as features for pose extraction, then the system complexity is low, but the gesture recognition accuracy deteriorates in various positions and occlusion situations
Solution Approach 1:
The patent changes the feature parameters from simple x, y coordinates and confidence to include bone vector lengths and angles. This parameter expansion enables the system to capture spatial relationships and geometric structures of gestures, significantly improving recognition accuracy in various positions and occlusion situations while maintaining reasonable system complexity.
2Measurement precision
If bone vector length and angle features are added to improve gesture recognition, then recognition accuracy improves, but the feature extraction complexity increases
Solution Approach 1:
The patent segments the pose estimation process into distinct modules: key point detection, skeleton construction, and bone vector calculation. Each module handles a specific aspect of feature extraction, making the complex process more manageable and efficient. The segmentation allows the system to compute bone vectors from detected key points in a structured manner, reducing overall computational complexity while maintaining high recognition accuracy.
3Reliability
If a robust temporal graph convolution network is used to handle misrecognitions and time-series characteristics, then the reliability of gesture recognition improves, but the device complexity increases
Solution Approach 1:
The patent introduces a robust temporal graph convolution network as an intermediary layer between raw pose features and final gesture classification. This intermediary model processes time-series pose data and handles misrecognitions through its graph convolution and temporal modeling capabilities, thereby improving reliability. The intermediary structure allows complex temporal relationships to be captured without directly complicating the overall system architecture.
Data Source
AI summary
There are provided a behavior recognition method for efficient recognition of hand signals and gesture and a behavior recognition AI network system using the same. The behavior recognition method for efficient recognition of hand signals and gestures according to an embodiment includes: an input feature extraction step of extracting, by a behavior recognition AI network system, key points F1 from bounding box data of an object which makes hand signals to be inputted by sequence, and generating skeleton data of the object; and a spatial feature extraction step of calculating, by the behavior recognition AI network system, a length F2 and an angle F3 of a bone vector based on the key points F1 and the skeleton data, and extracting a spatial feature. Accordingly, performance of recognition of hand signals and gestures may be enhanced.