Temporal Gesture Data Alignment for Precise Recognition Timing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition systems fail to optimize the speed of correctly recognizing hand gestures, leading to time discrepancies between the user finishing a gesture and the system recognizing it, which negatively impacts user experience.
Innovation Solution
A temporal sequence alignment method for gesture training data that includes compressing frames, calculating an alignment model through Dynamic Time Warping (DTW), and aligning soft labels to generate aligned training data, which is used to train an AI network to minimize time discrepancies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Deep Learning methods (CNN or LSTM) are used to train the gesture recognition system, then the accuracy of gesture recognition is improved, but the speed of correctly recognizing hand gestures deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-aligning the temporal sequences of gesture training data using Dynamic Time Warping (DTW) before feeding it to the neural network. The alignment process calculates optimal warping paths in advance to synchronize gesture progress across different samples, so the network receives pre-processed, temporally-aligned data that accelerates convergence and improves recognition speed without sacrificing accuracy
2Reliability
If cross entropy training methods are used, then the gesture recognition system can recognize hand gestures, but the speed of correctly recognizing hand gestures is not optimized
Solution Approach 1:
The patent performs preliminary temporal alignment of gesture sequences using DTW before training, pre-processing the data to establish consistent temporal relationships. This preliminary action ensures that the network learns from properly synchronized data, improving both reliability and training speed without requiring additional computational overhead during inference
3Reliability
If the trigger point is set earlier or later than the actual moment a user finishes performing a gesture, then the gesture recognition system can detect the gesture, but the user experience deteriorates due to time discrepancy
Solution Approach 1:
The patent implements feedback by using the aligned training data to train the network to predict the precise trigger point (gesture completion moment). The temporal alignment provides the network with feedback about the correct temporal relationships in gesture data, enabling it to learn when to trigger recognition to minimize time discrepancy and improve user experience
Data Source
AI summary
A temporal sequence alignment method includes the following steps: receiving gesture training data and gesture sample data; wherein the gesture training data includes multiple training frames and multiple training soft labels, and the gesture sample data includes multiple sample frames and multiple sample soft labels; compressing the training frames to generate a compressed training frame; compressing the sample frames to generate a compressed sample frame; calculating an alignment model of the compressed training frame and the compressed sample frame; aligning the sample soft labels to multiple aligned soft labels according to the alignment model; generating an aligned training data according to the gesture sample data and the aligned soft labels. The present invention uses the aforementioned steps to calibrate the sample soft labels of the gesture sample data, allowing a gesture recognition system to minimize time discrepancy for recognizing a gesture.