VR Hand Gesture Recognition with Unified Pose-Command Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition systems in VR scenarios face challenges with low accuracy and inefficient resource utilization due to the need for separate models for pose and command gesture recognition, leading to overfitting and excessive resource consumption.
Innovation Solution
Integrate pose and gesture command information recognition through multiple frames of images with different time sequences, using a two-stage training method to prevent overfitting and enhance accuracy, reducing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If separate models are used for pose recognition and command gesture recognition, then the system can handle different recognition tasks, but the resource consumption increases and overfitting occurs
Solution Approach 1:
The patent combines pose recognition and command gesture recognition into a single unified model. The model simultaneously processes multiple frames of images to output both pose information (hand keypoint coordinates) and command gesture recognition results, eliminating the need for separate models and reducing resource consumption while preventing overfitting.
2Adaptability or versatility
If separate models are used for pose recognition and command gesture recognition, then the system can handle different recognition tasks, but the device complexity increases
Solution Approach 1:
The patent merges pose recognition and command gesture recognition into a single unified model structure. The model takes multiple frames of images as input and simultaneously outputs pose information and command gesture recognition results, simplifying the overall system architecture while maintaining the ability to perform both recognition tasks.
3Use of energy by moving object
If a single model is used for both pose and command gesture recognition, then resource consumption is reduced, but the model may fail to capture temporal dynamics effectively
Solution Approach 1:
The patent applies preliminary action by performing feature extraction on multiple frames of images before inputting them to the unified model. The model then processes these pre-extracted features along with temporal sequence information to accurately capture hand pose dynamics and gesture commands, ensuring both resource efficiency and recognition accuracy.
Solution Approach 2:
The patent incorporates dynamics by designing the unified model to process temporal sequences of multiple frames. The model captures the dynamic changes in hand pose across time, using the temporal relationships between frames to improve gesture recognition accuracy while maintaining resource efficiency through a single integrated architecture.
Data Source
AI summary
Embodiments of the present disclosure provide a method for recognizing a gesture, a device, and a storage medium; and the method includes: acquiring a plurality of frames of images including a hand object within a first preset duration before a current time; respectively performing feature extraction on the plurality of frames of images to obtain a feature vector corresponding to each of the plurality of frames of images; determining at least one frame of target image before the current frame of image, and determining pose information corresponding to the hand object according to a feature vector corresponding to the current frame of image, a feature vector corresponding to each frame of target image and a preset deep learning model; and determining gesture command information corresponding to the hand object according to the feature vector corresponding to each of the frames of images and the preset deep learning model.