Hand Gesture Recognition With Ring Buffers on Low-Memory Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition techniques on resource-constrained devices face challenges due to high memory requirements and processing latency, making it difficult to deploy computationally intensive models like RNNs for real-time applications.
Innovation Solution
Implement a buffer-based noise suppression algorithm and a lightweight neural network model optimized for low memory footprint, using a ring buffer to store consecutive image frames and discard noise based on frequency thresholds, enabling accurate gesture recognition on resource-constrained devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RNN models (GRU or LSTM) are used for gesture recognition, then recognition accuracy is improved, but memory requirements and processing time increase significantly
Solution Approach 1:
The patent extracts and removes the recurrent temporal processing component from the RNN architecture, keeping only the spatial feature extraction capability. This allows the model to function as a lightweight CNN that processes individual frames independently, eliminating the need for storing and processing multiple frames through recurrent layers, thus dramatically reducing memory requirements while maintaining essential gesture recognition functionality
Solution Approach 2:
The patent replaces the expensive, memory-intensive RNN model with a lightweight CNN model that has significantly fewer parameters. This simpler model can be deployed on resource-constrained devices with limited memory, sacrificing the ability to perform complex temporal analysis but gaining the ability to run in real-time on mobile devices with minimal memory footprint
2Reliability
If RNN models are deployed on resource-constrained devices, then gesture recognition capability is achieved, but processing latency becomes prohibitively high
Solution Approach 1:
The patent removes the computationally intensive recurrent processing stages from the model architecture. By eliminating the need to process and store sequences of frames through multiple RNN layers, the model achieves significantly faster inference times that are suitable for real-time gesture recognition on mobile devices with limited processing power
Solution Approach 2:
The patent employs a simplified CNN model with far fewer computational operations compared to RNN models. This lightweight architecture reduces the computational burden and processing time, enabling the model to execute gesture recognition tasks in real-time on resource-constrained devices that cannot handle the computational demands of full RNN models
3Reliability
If full RNN models are used for processing multiple video frames, then temporal gesture analysis is improved, but device complexity increases
Solution Approach 1:
The patent extracts only the essential spatial feature extraction functionality from the RNN model, discarding the complex temporal processing mechanisms. This results in a simplified model that maintains the core capability of recognizing hand signs and gestures while eliminating the architectural complexity of recurrent connections, gating mechanisms, and long-term memory structures
Solution Approach 2:
The patent replaces the complex RNN architecture with a simple CNN model that has a straightforward structure suitable for deployment on mobile devices. This simpler model reduces the complexity of the computational graph, training procedures, and inference processes, making the system more accessible to resource-constrained devices with limited memory and processing capabilities
Data Source
AI summary
An electronic device for gesture recognition on resource-constrained devices is provided. The electronic device controls storage of a plurality of first consecutive image frames in a first buffer of a first length. The plurality of first consecutive image frames corresponds to the first length. The electronic device recognizes a first hand sign of a plurality of hand signs in a first subset of image frames of the plurality of first consecutive image frames. The electronic device controls storage of the recognized first hand sign in a second buffer of a second length based on the determination that a ratio of a number of the first subset of image frames and the first length is one of equal to or greater than the threshold. The electronic device determines a gesture corresponding to one or more hand signs of the plurality of hand signs stored in the second buffer.