Gesture Recognition via Motion-Cropped Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gesture recognition systems require complex and costly sensor suites, leading to impractical and slow gesture detection, which hampers user experience and practicality.
Innovation Solution
A computer-based pipeline that detects motion in video frames, performs body pose estimation, and predicts user gestures using cropped images of hands to control electronic devices efficiently and accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex sensor suites (time-of-flight sensors, structured light sensors) are used for gesture detection, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent segments the gesture recognition process into distinct stages: motion detection in video frames, cropping to isolate the user, body pose estimation, and hand gesture prediction. This segmentation allows each stage to be optimized independently, achieving accurate gesture recognition using standard video cameras rather than complex sensor suites.
Solution Approach 2:
The patent introduces body pose estimation as an intermediary step between motion detection and hand gesture recognition. By first determining the user's body pose and then predicting hand locations based on that pose, the system achieves accurate gesture detection using simple video cameras, avoiding the need for complex time-of-flight or structured light sensors.
2Measurement precision
If complex sensor suites and algorithms are used for gesture recognition, then measurement precision is improved, but processing speed deteriorates
Solution Approach 1:
The patent performs preliminary actions by detecting motion in video frames first, then cropping the image to isolate the user before performing pose estimation. This preliminary cropping reduces the computational burden of subsequent processing steps, enabling fast and accurate gesture detection without requiring complex sensor suites.
Solution Approach 2:
By segmenting the processing pipeline into motion detection, cropping, pose estimation, and gesture prediction stages, the patent optimizes processing speed at each stage. The cropping step significantly reduces the computational complexity of pose estimation, enabling real-time gesture recognition with standard video cameras.
3Measurement precision
If full image frames are processed for pose estimation, then measurement precision is improved, but processing time increases
Solution Approach 1:
The patent extracts and isolates the user from the full video frame by detecting motion and cropping to the user's location. This extraction allows pose estimation to be performed on a smaller, focused image region, significantly reducing processing time while maintaining pose estimation accuracy.
Solution Approach 2:
By applying pose estimation only to the cropped region containing the user rather than the entire video frame, the patent optimizes processing efficiency. The local quality of the cropped region is sufficient for accurate pose estimation, eliminating the need to process irrelevant portions of the full frame.
Data Source
AI summary
A method and system may be used to determine gestures of one or more users from a video. Motion may be detected in an image frame of a video, and the image frame may be cropped around the motion. Body pose estimation may be performed on the cropped image frame. The location of the user's hands may be determined from the body pose. Additional processing may be performed to identify hand gestures.