Inverse Kinematics Model for Depth Sensor Gesture Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition systems for computer video games introduce inherent lag due to noisy depth data and restricted field of view, affecting the accuracy and timeliness of user feedback and scoring mechanisms.
Innovation Solution
The use of depth sensors with Inverse Kinematics (IK) Solver models to filter and smooth data, allowing for the calculation of joint positions beyond the camera's field of view and handling occlusions, combined with a gesture classification method that tracks gesture performance frame by frame, providing real-time feedback and scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data collection is done directly on depth data with binary mask sampling, then the system can identify gestures within field of view, but the depth noise reduces measurement precision and limits data points to only visible regions
Solution Approach 1:
The patent introduces an intermediary mathematical model (articulated figure with kinematic chains) between the noisy depth data and the gesture recognition process. This model acts as a mediator that filters noise by constraining joint positions to physically plausible locations, thereby improving measurement precision while handling occlusions naturally.
Solution Approach 2:
The patent replaces the mechanical sampling approach (directly sampling depth data points within field of view) with a mathematical modeling approach (inverse kinematics solver). This substitution eliminates the limitation of being restricted to visible regions by using mathematical constraints to infer positions of occluded body parts.
2Reliability
If explicit start and end times are used for gesture classification, then the system can define clear gesture boundaries, but inherent lag is introduced between gesture completion and recognition
Solution Approach 1:
The patent performs preliminary action by continuously maintaining the articulated figure model updated with current depth data during gesture execution. This allows the system to have the gesture recognition structure ready in advance, enabling immediate classification when a gesture is completed without waiting for explicit end time signals, thereby reducing recognition delay while maintaining accuracy.
3Ease of manufacture
If sampling is restricted to depth data within field of view, then data collection is simplified, but body parts outside camera view cannot be tracked
Solution Approach 1:
The patent makes the articulated figure model universal by enabling it to track body parts both within and outside the camera's field of view. The mathematical model serves multiple functions: it processes visible depth data and simultaneously infers positions of occluded body parts, thereby extending gesture tracking coverage without complicating the data collection process.
4Speed
If frame-by-frame gesture tracking is implemented, then real-time feedback is provided, but computational complexity increases
Solution Approach 1:
The patent applies dynamics by making the articulated figure model adaptive and updateable in real-time. The model dynamically adjusts joint positions based on incoming depth data from each frame, enabling responsive real-time feedback. The mathematical structure allows efficient updates without requiring complete reprocessing, balancing speed and complexity.
Data Source
AI summary
A method of image acquisition and data pre-processing includes obtaining from a sensor an image of a subject making a movement. The sensor may be a depth camera. The method also includes selecting a plurality of features of interest from the image, sampling a plurality of depth values corresponding to the plurality of features of interest, projecting the plurality of features of interest onto a model utilizing the plurality of depth values, and constraining the projecting of the plurality of features of interest onto the model utilizing a constraint system. The constraint system may comprise an inverse kinematics solver.