Kinect Multi-Human Tracking via Shadow Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-user tracking systems in mobile virtual reality struggle to accurately track users who are severely or fully occluded, leading to disruptions in the immersive experience due to the lack of effective methods for handling occlusion.
Innovation Solution
A multi-human tracking system utilizing a single Kinect device that incorporates multi-sensing data cues, including user shadow analysis, to determine the position of occluded users by converting the calculation of their movement into shadow movement, integrating Kinect skeletal data, color image information, and gyroscope data to ensure accurate tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If detection model is used to detect and track users, then tracking capability is improved, but ability to track severely occluded users deteriorates
Solution Approach 1:
The patent introduces shadow as an intermediary element to track occluded users. When a user is occluded from direct view, their shadow remains visible and can be detected. The system uses shadow detection and tracking algorithms to follow the shadow's movement, which correlates with the occluded user's movement, thereby maintaining tracking continuity without requiring direct visual contact with the user.
Solution Approach 2:
The patent changes the detection parameter from direct user visual features to shadow features. By detecting shadow position, shape, and movement patterns instead of directly detecting user appearance, the system can track users even when they are completely occluded from the camera's direct view, as long as their shadow remains visible.
2Measurement precision
If multiple sensors are used to improve tracking accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent makes the single Kinect sensor perform multiple functions: depth mapping for 3D position, color imaging for shadow detection, and skeletal tracking for user identification. By utilizing different data streams from one device, the system achieves multi-sensor level tracking accuracy without the physical complexity of multiple sensors.
Solution Approach 2:
The patent merges depth information, color image information, and skeletal data from the Kinect into a unified tracking framework. This integration allows the system to leverage complementary strengths of different data types from a single sensor, achieving robust occluded user tracking through shadow detection combined with traditional tracking methods.
3Device complexity
If traditional detection methods are used, then system simplicity is maintained, but ability to handle occlusion deteriorates
Solution Approach 1:
The patent implements dynamic switching between tracking modes based on occlusion detection. The system continuously monitors tracking confidence and automatically switches between direct user tracking and shadow-based tracking depending on which method provides reliable data, allowing the simple system to adapt to varying occlusion conditions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for real-time tracking of users in various occlusion states, enhancing the immersive experience by accurately calculating the position of occluded users and maintaining spatial consistency between the physical and virtual environments, even in complex occlusion scenarios.
Implementation Method 1
It uses the TOF (time-of-flight) calculation method to obtain the phase difference from the light emitted by a sensor after the reflection of the object
Implementation Method 2
the terminal sensor information capturing module acquires the rotation information of the mobile phone gyroscope in order to obtain the user's orientation information
Implementation Method 3
The method utilizes the principle that the user's shadow is not occluded when the user is occluded under certain lighting conditions, and converts the calculation of the motion of the occluded user into a problem of solving the movement of the user's shadow
Data Source
AI summary
The invention discloses a multi-human tracking system and method with single Kinect for supporting mobile virtual reality applications. The system can complete the real-time tracking of users occluded in different degrees with a single Kinect capture device to ensure smooth and immersive experience of players. The method utilizes the principle that the user's shadow is not occluded when the user is occluded under certain lighting conditions, and converts the calculation of the motion of the occluded user into a problem of solving the movement of the user's shadow, and can accurately detect the position of each user, rather than just predicting the user's position, thereby actually realizing tracking.