Depth Camera Target Tracking via Skeletal Model Rasterization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision applications face challenges in accurately tracking targets, especially when targets move into complex poses or occlude each other, requiring efficient and robust systems to interpret user movements as game controls or application inputs.
Innovation Solution
A method utilizing depth cameras to acquire depth information, model targets with skeletal representations, and adjust poses based on differences between observed and synthesized depth images, allowing for real-time tracking and control of game characters or application elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional control systems are used to manipulate game characters, then control precision can be maintained, but ease of operation deteriorates due to complicated controls that are difficult to learn and different from actual game actions
Solution Approach 1:
The system creates a virtual model (avatar) that copies the user's physical movements in real-time. The depth camera captures the user's skeletal pose, and the virtual avatar replicates these movements, allowing natural motion transfer without complex control mappings. This eliminates the need for learned control schemes while maintaining precise character manipulation.
Solution Approach 2:
The patent replaces traditional mechanical control interfaces (controllers, keyboards) with a vision-based system using depth cameras and computer vision algorithms. This substitution enables natural motion capture and automatic translation to game controls, eliminating the learning curve associated with conventional control mechanisms.
2Ease of operation
If depth-based visual tracking is implemented to enable natural movement control, then ease of operation improves, but measurement precision deteriorates when targets move into complex poses or occlude each other
Solution Approach 1:
The system segments the user's body into individual skeletal components (joints, limbs, torso) and tracks each segment independently. This segmentation allows the system to maintain accurate tracking even when parts of the body are occluded, as it can infer the positions of hidden segments based on the positions of visible segments and anatomical constraints.
Solution Approach 2:
The patent introduces a virtual skeletal model as an intermediary between the depth camera data and the final tracking results. This model serves as a mediator that fills in missing information from occluded or ambiguous depth data by using anatomical priors and motion constraints, thereby maintaining tracking accuracy in challenging scenarios.
3Speed
If real-time tracking is achieved through continuous depth image processing, then speed of response improves, but use of energy increases due to continuous computation and model adjustment
Solution Approach 1:
The system performs tracking and model adjustment at discrete time intervals (frames) rather than continuously. By processing depth images periodically at the frame rate of the camera and only updating the virtual model when necessary, the system maintains real-time responsiveness while significantly reducing computational energy consumption compared to continuous processing.
Solution Approach 2:
The patent pre-establishes the virtual skeletal model and its parameters before tracking begins. This preliminary setup allows the system to use the pre-configured model for rapid real-time matching against depth images without needing to create or extensively process complex models during operation, thereby reducing ongoing computational energy requirements.
Data Source
Figure 1A
Figure 1B
Figure 2~3
AI summary
A method of tracking a target includes receiving an observed depth image of the target from a source and obtaining a posed model of the target. The model is rasterized into a synthesized depth image, and the pose of the model is adjusted based, at least in part, on differences between the observed depth image and the synthesized depth image.