Gesture Recognition via Skeleton Sequences for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning methods for dynamic gesture recognition require high computing power, making them unsuitable for use in mobile devices with limited processing capabilities.
Innovation Solution
A gesture recognition method that integrates static and dynamic gesture recognition by obtaining a video stream, performing hand recognition using neural networks, calculating hand centroid positions, and encoding static gesture information to enable dynamic gesture recognition through a sliding window detection, reducing computational complexity and making it suitable for low-power mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning methods are used for dynamic gesture recognition, then recognition accuracy is improved, but computing power requirements increase
Solution Approach 1:
The patent segments the gesture recognition task into two independent parts: static gesture recognition (hand pose estimation) and dynamic gesture recognition (action classification). By dividing the deep learning model into a pose estimation network and an action recognition network that processes skeleton sequences, the computational load is distributed and optimized for mobile devices while maintaining recognition accuracy.
Solution Approach 2:
The patent performs preliminary action by first estimating hand pose and extracting skeleton information from video frames before performing dynamic gesture recognition. This preprocessing step converts complex image data into simplified skeleton sequences, reducing the computational complexity of the subsequent action recognition task and making it feasible for mobile devices with limited computing power.
2Adaptability or versatility
If deep learning methods are used for dynamic gesture recognition, then recognition capability is improved, but device complexity increases
Solution Approach 1:
The system is segmented into modular components: video preprocessing module, pose estimation module, skeleton extraction module, and action recognition module. Each module performs a specific function independently, making the overall system more manageable and adaptable to different mobile device configurations while maintaining comprehensive gesture recognition capability.
Solution Approach 2:
The patent introduces skeleton sequences as an intermediary representation between raw video frames and final gesture recognition results. This intermediate skeleton data structure simplifies the information that needs to be processed by the action recognition network, reducing system complexity while preserving essential gesture dynamics for accurate recognition.
3Use of energy by moving object
If traditional gesture recognition methods are used, then computing power consumption is reduced, but dynamic gesture recognition capability is lost
Solution Approach 1:
By performing preliminary pose estimation and skeleton extraction, the system transforms dynamic video data into simplified temporal sequences that can be processed efficiently. This preliminary processing enables dynamic gesture recognition capability while keeping the subsequent action classification computationally lightweight enough for mobile devices with limited power consumption.
Solution Approach 2:
The patent changes the parameter representation from raw pixel data to skeleton joint coordinates and temporal sequences. This parameter transformation reduces the dimensionality and complexity of the data, enabling dynamic gesture recognition with lower computational requirements and reduced power consumption on mobile devices.
Data Source
AI summary
The present disclosure provides a gesture recognition method as well as a terminal device and a computer-readable storage medium using the same. The method includes: obtaining a video stream collected by an image recording device in real time; performing a hand recognition on the video stream to determine static gesture information of a recognized hand in each video frame of the video stream; encoding the static gesture information in the video frames of the video stream in sequence to obtain an encoded information sequence of the recognized hands; and performing a slide detection on the encoded information sequence using a preset sliding window to determine a dynamic gesture category of each recognized hand. In this manner, static gesture recognition and dynamic gesture recognition are effectively integrated in the same process. The dynamic gesture recognition is realized through the slide detection of the sliding window without complex network calculations.