Depth Image Processing for User Tracking Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional skeletal tracking techniques are unreliable for detecting and displaying user representations when the user is laying or sitting on or near the floor, as they fail to accurately interpret depth images in these scenarios.
Innovation Solution
The use of depth images to detect and fill holes, produce low-resolution user representations, and identify and remove pixels corresponding to the floor, while adjusting for camera tilt, to improve the accuracy of user tracking and representation in various positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If skeletal tracking techniques are used to detect user representations, then user motion can be detected in certain situations, but the system becomes unreliable when the user is laying or sitting on or near the floor
Solution Approach 1:
The system transforms depth image data from standard coordinate space to a tilted coordinate system that accounts for camera angle. By changing the reference frame and applying geometric transformations to depth values, the system adapts to different camera orientations while maintaining reliable user detection across various positions including floor-level activities.
2Measurement precision
If depth images are processed to fill holes and remove floor pixels, then accuracy of user representation is improved, but processing complexity increases
Solution Approach 1:
The depth image processing is divided into distinct segmentation steps: hole detection identifies missing user pixels, hole filling reconstructs them, floor pixel identification separates ground from user, and floor removal extracts only the user portion. This segmented approach improves accuracy while managing complexity through modular processing stages.
Solution Approach 2:
The system performs preliminary actions by detecting and filling holes before final user representation is generated. By preemptively reconstructing missing depth data and removing floor pixels in advance, the system ensures accurate user representation without adding complexity to the main tracking pipeline.
3Measurement precision
If the depth image is adjusted to reduce sensitivity to camera tilt, then tracking accuracy is improved, but computational requirements increase
Solution Approach 1:
The system replaces complex mechanical adjustment of the camera or multiple camera angles with computational geometry transformations. By applying mathematical transformations to the depth image data in software, the system achieves tilt compensation without the energy cost of physical repositioning or additional hardware.
Data Source
AI summary
Embodiments described herein can be used to detect holes in a subset of pixels of a depth image that has been specified as corresponding to a user, and to fill such detected holes. Additionally, embodiments described herein can be used to produce a low resolution version of a subset of pixels that has been specified as corresponding to a user, so that when an image including a representation of the user is displayed, the image respects the shape of the user, yet is not a mirror image of the user. Further, embodiments described herein can be used to identify pixels, of a subset of pixels specified as corresponding to the user, that likely correspond to a floor supporting the user. This enables the removal of the pixels, identified as likely corresponding to the floor, from the subset of pixels specified as corresponding to the user.