AR Skin Detection Using Color Thresholds for Hand Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems face challenges in accurately determining the position of virtual entities relative to the user's hand, leading to inconsistent and confusing interactions due to issues like shadow interference and incorrect hand positioning.
Innovation Solution
The system employs a method where a known background color, such as green or blue, is used to differentiate between the user's skin and the background, allowing for aggressive skin tone detection and masking out shadows, while using multiple color spaces to enhance skin tone classification and reduce computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional augmented reality systems use standard video capture and overlay methods, then virtual entities can be displayed on screen, but the precision of relating virtual entity position with user hand position in-camera is insufficient leading to inconsistent interaction
Solution Approach 1:
The patent applies color-based skin detection by classifying pixels as skin or non-skin based on color characteristics. This allows the system to identify user hand positions through skin tone detection in the video feed, enabling precise tracking of hand movements relative to virtual entities. The color change principle is used to differentiate skin pixels from background and virtual entity pixels, solving the position precision problem.
Solution Approach 2:
The patent segments the video image into different pixel categories (skin pixels, non-skin pixels) to separately process and track user hand positions. By creating a skin mask that isolates skin-colored regions, the system can independently analyze hand position without interference from background elements or virtual entities, thereby improving measurement precision and interaction reliability.
2Measurement precision
If skin tone detection is performed to identify user hand position, then interaction precision improves, but computational load increases due to pixel-by-pixel classification
Solution Approach 1:
The patent applies skin tone classification selectively to specific regions of the video image rather than uniformly processing the entire image. By focusing computational resources on areas where skin pixels are likely to appear (based on spatial context and skin mask generation), the system maintains high hand position precision while reducing overall computational energy requirements.
3Adaptability or versatility
If virtual entities are superposed on captured video image, then augmented reality effect is achieved, but shadow interference and background color similarity cause incorrect hand positioning
Solution Approach 1:
The patent uses color-based differentiation to distinguish skin pixels from virtual entity pixels and background pixels. By classifying pixels based on skin tone characteristics, the system can accurately identify hand positions even when virtual entities are superposed on the video image or when shadows are present. This color-based approach allows the system to maintain hand position accuracy despite the complexity of augmented reality compositing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of user interaction in augmented reality comprises the steps of capturing a video image of a scene, and for each pixel in at least a sub-region of the captured video, classifying the pixel as either a skin or non-skin pixel responsive to whether the colour of the pixel exceeds a predetermined threshold purity of red; and generating a mask based upon the classification of the pixels of the captured video, generating an augmentation image layer to superpose on the captured video image, and limiting a mode of combination of the captured video and the augmentation image layer, responsive to the mask.