Foot Tracking Using Depth Map Skeleton Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing techniques are not suitable for real-time application in augmented reality systems for tracking a user's feet, making it difficult to efficiently locate and track feet within a captured scene for virtual try-on applications.
Innovation Solution
A system using a depth sensor to generate a depth map and a processor to identify candidate objects for foot tracking, determining 3D central skeleton representations, toe, and ankle locations, and calculating foot orientation, with the ability to overlay virtual footwear onto real-time video feed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional image processing techniques are used for foot tracking, then the system can process images, but the tracking is not suitable for real-time application and lacks efficiency
Solution Approach 1:
The patent segments the foot tracking problem into distinct processing stages: depth map generation from depth sensor data, binary mask creation through thresholding, distance transform application, and skeleton extraction. This segmentation allows each stage to be optimized independently for real-time performance while maintaining tracking accuracy through specialized algorithms at each step.
Solution Approach 2:
The patent replaces conventional 2D image processing with a 3D depth-based processing system. By using depth map data and 3D skeleton representation instead of traditional image processing, the system achieves both real-time performance and accurate foot tracking, substituting the mechanical image processing approach with a more efficient depth-based methodology.
2Measurement precision
If detailed foot tracking analysis is performed to improve accuracy, then tracking precision increases, but computational complexity increases
Solution Approach 1:
The patent extracts only the essential features needed for foot tracking by generating a skeleton representation from the depth map. This skeleton extraction process removes unnecessary detailed information while retaining the critical structural elements (toe, heel, and midfoot positions) required for accurate foot location and orientation measurement, thereby reducing computational complexity.
Solution Approach 2:
The patent transitions from 2D image processing to 3D depth-based processing, adding a dimensional aspect that simplifies the tracking problem. By working in 3D space with depth maps and 3D skeletons, the system achieves higher measurement precision for foot location and orientation without proportionally increasing complexity, as the third dimension provides direct spatial information.
3Reliability
If the system processes the entire depth map to ensure complete foot detection, then detection reliability improves, but processing time increases
Solution Approach 1:
The patent applies preliminary actions by first generating a binary mask from the depth map through thresholding before performing the distance transform and skeleton extraction. This preliminary binarization step simplifies subsequent processing by converting continuous depth data into discrete foreground/background regions, reducing the computational load for full depth map processing while maintaining reliable foot detection.
Solution Approach 2:
The patent applies local quality by focusing processing efforts on regions containing feet. The distance transform and skeleton extraction are applied specifically to the binary mask regions identified as containing foot structures, rather than uniformly processing the entire depth map. This localized approach maintains detection reliability for feet while reducing overall processing time.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
A system and method are described for tracking the location and orientation of a user's feet for an augmented reality display. A depth sensor generates a depth map of a scene within the field of view of the depth sensor, the field of view containing at least a portion of the user. A processor is configured to identify a portion of the depth map as a candidate object for foot tracking, determine a central skeleton representation of the identified portion, determine a location of a toe of the user's foot and a location of an ankle of the user's foot, based on the central skeleton representation, and calculate an orientation of the user's foot based on the determine locations of the toe and the ankle.