On-Skin AR Touch Detection via RF Waveguide Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR/VR systems face challenges in precise fine-grained interactions due to limitations in detecting touch versus hover ambiguity, especially with camera-based systems that lack sufficient resolution or field of view, leading to cumbersome user interactions.
Innovation Solution
A system utilizing the human body as a radio frequency (RF) waveguide, combining RF signal detection with computer vision for precise touch segmentation, enabling high-frames per second (FPS) touch detection and robust touch segmentation using a wristband and head-worn receiver, allowing for touchscreen-like interactions on the skin.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If camera-based systems are used for touch detection, then spatial tracking accuracy is improved, but the ability to segment true touches from hovering fingers deteriorates due to insufficient resolution or field of view
Solution Approach 1:
The system segments the detection task into two independent components: optical cameras handle spatial tracking of finger position, while RF sensors handle touch contact detection. This segmentation allows each sensor type to operate in its optimal range, with cameras providing high-resolution spatial data and RF sensors providing unambiguous contact detection regardless of distance.
Solution Approach 2:
The system merges optical sensing and RF sensing into a unified touch interface. The optical component tracks finger position in 3D space with high precision, while the RF component detects actual contact events. By combining these complementary modalities, the system achieves both accurate spatial tracking and reliable touch segmentation that neither system could achieve alone.
2Ease of operation
If handheld controllers or in-air bare hand gestures are used, then fluid coarse-grained input is improved, but fine-grained interaction precision deteriorates
Solution Approach 1:
The system dynamically adapts the interaction modality based on the task requirements. For coarse-grained navigation, in-air gestures provide fluid control. For fine-grained interactions like typing, the system transitions to on-skin touch detection, which offers precise feedback and accurate target acquisition. This dynamic switching allows users to leverage the strengths of each interaction mode.
Solution Approach 2:
The skin surface acts as an intermediary between the user and the virtual interface. By detecting touches directly on the skin, the system provides a tactile feedback medium that bridges the gap between physical gesture and digital interaction, enabling fine-grained control without requiring complex hand coordination in air.
3Measurement precision
If body-worn cameras and computer vision are used, then input precision and user comfort are improved, but the ability to detect touch versus hover deteriorates due to resolution limitations
Solution Approach 1:
The system replaces the mechanical/optical measurement approach with an electromagnetic field-based approach for touch detection. Instead of relying on camera resolution to distinguish contact from hover, the system uses RF sensors that detect changes in electromagnetic fields caused by actual skin contact, providing reliable touch detection independent of visual resolution.
Solution Approach 2:
The system changes the detection parameter from optical distance measurement to RF signal characteristics. By monitoring changes in RF signal strength, phase, or impedance when a finger contacts the skin, the system can reliably distinguish touch from hover without being constrained by camera resolution or field of view.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system achieves high accuracy and precision in touch detection, reducing hover ambiguity and enabling intuitive, comfortable interactions with a high touch segmentation accuracy of 93.8% and mean distance error of 5.3 mm, supporting fine-grained interactions without the need for expensive hardware.
Implementation Method 1
enables precise, high-frames per second (FPS) touch segmentation by using the body as a radio frequency (RF) waveguide
Implementation Method 2
A system utilizing the human body as a radio frequency (RF) waveguide, combining RF signal detection with computer vision for precise touch segmentation
Data Source
AI summary
A system and method for detecting when a user contacts one arm to an opposing arm. The system comprises an emitter worn by the user on one arm and a receiver located on an opposing arm, on the user's head, or on a different part of the user's body. Contact, or touch, is indicated by comparing the value of an electrical signal flowing along a path establish, in part, through the user's skin to a threshold. When touch is initiated, multiple electrical paths are created, including one that generates airborne signal emanating from the user's body.