Ultrasound Haptic Feedback for AR Object Shape Recognition
Solution Overview
Problem
Existing augmented reality, virtual reality, and mixed reality technologies face challenges in effectively presenting tactile information to users, making it difficult for them to recognize object shapes or manipulate objects based solely on tactile sensations.
Innovation Solution
An information processing apparatus that acquires viewpoint, motion, and object information to generate both visual and tactile feedback, using ultrasound to create a synchronized experience by processing object images and calculating positions for superimposition with background images, and generating tactile information for an ultrasound generator to provide corresponding tactile sensations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If tactile information is presented simply to confirm object existence, then the system is easy to operate, but the user cannot recognize object shapes or manipulate objects from tactile sensation
Solution Approach 1:
The patent combines visual information and tactile information into a unified presentation system. The tactile information generation unit creates haptic feedback that corresponds to the visual object representation, merging two sensory modalities to provide comprehensive object information that enables both simple operation and shape recognition.
Solution Approach 2:
The patent introduces an information processing apparatus as an intermediary between the user and the objects. This apparatus processes and transforms raw sensory data into enhanced tactile information that mediates the user's perception, enabling shape recognition and manipulation capabilities while maintaining ease of operation.
2Reliability
If ultrasound is used to present tactile information, then the user can sense object existence, but the user still cannot recognize object shapes or manipulate objects
Solution Approach 1:
The patent applies local quality by varying the tactile information characteristics across different spatial locations. The ultrasound-based tactile presentation is localized to specific regions corresponding to different parts of the object, with each location providing differentiated tactile feedback that reveals local geometric features, enabling shape recognition through systematic exploration.
3Device complexity
If visual information and tactile information are presented separately, then the system complexity is reduced, but the user cannot achieve accurate shape recognition from tactile sensation
Solution Approach 1:
The patent implements feedback by using the position information from visual tracking to dynamically adjust and update the tactile information presentation. As the user's hand position is tracked visually, the tactile feedback is continuously updated to correspond to the current contact point on the virtual object, creating a closed-loop system that enhances measurement precision without excessive complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables users to accurately recognize object shapes and interact with virtual objects through enhanced tactile feedback, improving the overall immersive experience by correlating visual and tactile cues.
Implementation Method 1
an ultrasound generator configured to irradiate a user with ultrasound
Data Source
AI summary
An information processing apparatus comprises a processor configured to execute a program so as to: acquire viewpoint information, motion information and object information; process an object image wherein: the object image is an image representing a virtual substance and is updated based on the viewpoint information and the motion information; calculate a position when superimposing a background image and the object image, wherein: the background image is an image according to the field of view of the user; generate visual information superimposing the background image and the object image based on the position calculated, and output the generated visual information to a display device; and generate tactile information for an ultrasound generator irradiating the user with ultrasound corresponding to the object based on the position calculated, the motion information and the object information, and output the generated tactile information to the ultrasound generator.