Mid-Air Haptic Texture Mapping for Visual-Tactile Roughness Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mid-air haptic technology has not adequately addressed the challenge of accurately rendering texture information, leading to potential discrepancies between visual and haptic feedback, which can hinder user immersion in augmented and virtual reality experiences.
Innovation Solution
A method is developed to translate texture information from visual images into mid-air haptic sensations using modulated focused ultrasound, combining perceptual studies and machine learning to predict the visually perceived roughness of an image texture and generate congruent haptic stimuli, utilizing a haptic mapping function to project tactile sensations onto a user's hands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If mid-air haptic technology is used to render texture information, then user immersion in AR/VR experiences is enhanced, but accurate rendering of texture information is not achieved
Solution Approach 1:
The patent transforms visual texture parameters (RGB values, luminance, contrast) into corresponding haptic parameters (vibration frequency, amplitude, waveform) through a mapping function. This parameter transformation enables accurate translation of visual texture information into haptic sensations, resolving the contradiction between enhancing user immersion and achieving accurate texture rendering.
Solution Approach 2:
The patent introduces an intermediary mapping function that acts as a bridge between visual and haptic modalities. This mapping function processes visual texture data and converts it into appropriate haptic stimulation parameters, enabling accurate texture rendering in mid-air haptic displays without compromising user immersion.
2Reliability
If visual and haptic modalities convey discrepant information, then user immersion is threatened, but translating visual texture to haptic sensation is challenging
Solution Approach 1:
The patent establishes consistent parameter relationships between visual and haptic modalities by mapping visual texture parameters (luminance, contrast, color) to corresponding haptic parameters (amplitude, frequency, waveform). This systematic parameter mapping ensures information consistency across modalities while providing a structured approach that manages the complexity of the translation process.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The approach effectively matches visual and tactile perceptions of texture, enhancing user experience by ensuring consistent and immersive haptic feedback in augmented and virtual reality environments.
Implementation Method 1
use ultrasonic phased arrays that electronically focus waves onto a user's hands and fingertips to create a vibro-tactile effect
Implementation Method 2
ultrasound could be used to induce haptic sensation
Data Source
AI summary
Described is a method for instilling the haptic dimension of texture to virtual and holographic objects using mid-air ultrasonic technology. A set of features is extracted from imported images using their associated displacement maps. Textural qualities such as the micro and macro roughness are then computed and fed to a haptic mapping function together with information about the dynamic motion of the user's hands during holographic touch. Mid-air haptic textures are then synthesized and projected onto the user's bare hands. Further, mid-air haptic technology enables tactile exploration of virtual objects in digital environments. When a user's prior and current expectations and rendered tactile texture differ, user immersion can break. A study aims at mitigating this by integrating user expectations into the rendering algorithm of mid-air haptic textures and establishes a relationship between visual and mid-air haptic roughness.