Depth-Based Extremity Detection via Image Slicing and Graph Traversal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for identifying object extremities, such as fingertips, in images are limited by requiring specific sensor orientations and are computationally expensive, failing when fingers point towards the sensor or are close together, and often require large databases or complex models.
Innovation Solution
A method using depth data to slice images into ranges of distances, building a graph of connected pixels, and determining extremities based on graph traversal, which is robust and can handle various sensor orientations, and is optimized for fast computation on embedded hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If contour based image processing methods are used on segmented hand or body shapes, then extremities can be identified, but the method only works when extremities or fingers are stretched and parallel to the sensor
Solution Approach 1:
The patent transitions from 2D contour-based analysis to 3D depth-based analysis by introducing a depth dimension. Depth images provide three-dimensional information about the object's geometry, allowing extremity detection to work regardless of the object's orientation relative to the sensor. The depth data enables calculation of true spatial distances and positions, making the method invariant to rotation and orientation changes that plague 2D contour methods.
2Measurement precision
If Hough-transform with particle filter and mean-shift procedure is used for fingertip detection, then detection accuracy is improved, but computational cost increases significantly
Solution Approach 1:
The patent extracts and uses only the essential depth information needed for extremity detection, discarding unnecessary computational steps. Instead of applying full Hough-transform, particle filters, and mean-shift procedures, the method directly leverages depth image properties and simple geometric calculations to identify extremities, achieving adequate precision with minimal computation.
Solution Approach 2:
The patent replaces expensive, complex computational models with simple, lightweight algorithms that can be executed rapidly. The solution uses basic depth data analysis and straightforward geometric operations that require minimal processing power, enabling real-time performance on embedded hardware while maintaining sufficient detection accuracy for practical applications.
3Measurement precision
If particle diffusion approach is used to propagate particles from palm center to contour, then fingertip positioning is achieved, but the method only works for stretched fingers that do not point towards the sensor
Solution Approach 1:
By incorporating the depth dimension, the patent enables accurate fingertip positioning for any finger pose. The depth information provides true spatial relationships, allowing the algorithm to correctly identify extremities whether fingers are stretched, curled, or pointing toward the sensor. The depth data compensates for perspective distortions and orientation variations that limit 2D contour-based methods.
4Measurement precision
If geodesic distance method is used to identify extremities from centroid, then extremity localization is achieved, but the method assumes geodesic distance is independent of pose which is not always true
Solution Approach 1:
The patent modifies the approach by using depth-based Euclidean distance calculations instead of assuming pose-independent geodesic distance. The depth information provides accurate spatial measurements that account for actual object geometry and orientation. This parameter change allows the method to adapt to different poses while maintaining localization precision, eliminating the need for the unrealistic assumption that distance metrics remain constant across all orientations.
5Measurement precision
If large database of motion capture data is used to build body part classifier, then pose recognition accuracy is improved, but system complexity and processing time increase significantly
Solution Approach 1:
The patent replaces the expensive, complex solution of building classifiers from large motion capture databases with simple, lightweight depth-based geometric analysis. The method uses straightforward calculations on depth image data to identify extremities and infer pose, achieving sufficient accuracy without requiring extensive training data or complex machine learning models. This approach dramatically reduces system complexity and enables deployment on resource-constrained embedded devices.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Described herein are a system and method for identifying extremities of an object within a scene. The method comprises operating an image processing system to receive image data from a sensor. The image data represents an image of the scene with the object. The image data comprises a two-dimensional array of pixels and each pixel contains a depth value indicating distance from the sensor. The image processing system slices the image into slices. Each respective slice comprises those pixels with depth values that lie within a respective range of distances defined relative to a reference. For each of the slices, the method identifies one or more connected regions of pixels that are neighbors in the two-dimensional array of pixels. The method builds, based on the connected region of pixels that have been identified for the slices and depth information inherent to the respective slices, a graph consisting of interconnected nodes. The connected regions form the nodes of the graph and the nodes are interconnected in the graph based on their relative distance to the reference. Extremities of the object are determined based on the graph.