Haptic Display System Synchronizing Tactile Feedback with Visual Representation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality technologies lack the ability to enable direct, intuitive touch-based interaction with virtual objects using bare hands, relying on force feedback or displacement feedback systems which are limited in their ability to convey realistic haptic information.
Innovation Solution
A haptic display system that synchronizes tactile haptic properties with visual representation, utilizing a deformable tactile display, positioning device, and deformation mechanism to accurately depict surface textures, contours, and other properties of virtual objects in real-time, allowing for dynamic adjustment and movement within three-dimensional space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If force feedback or displacement feedback systems are used for haptic interaction, then some tactile information can be conveyed, but the systems are limited in their ability to enable direct touch-based interaction with virtual objects using bare hands
Solution Approach 1:
The patent creates a haptic copy of the virtual object's surface properties by mapping 3D object data to tactile display elements. The system generates tactile information that replicates the visual appearance and surface characteristics of virtual objects, allowing users to touch and explore virtual surfaces through a tactile display that copies the visual data into haptic form.
Solution Approach 2:
The tactile display serves as an intermediary device between the user's hand and the virtual object. It translates visual 3D data into tactile sensations, mediating the interaction so that users can touch virtual surfaces through the tactile display surface rather than directly interacting with complex feedback systems.
2Measurement precision
If commercially available tactile displays are used, then some haptic information can be conveyed, but their performance is very limited and they focus on special market segments
Solution Approach 1:
The patent creates a universal haptic interaction system that can represent various types of virtual objects and surfaces through a single tactile display interface. The system can adapt to different virtual environments and object types by processing their visual data and converting it into appropriate tactile representations, making it applicable across multiple VR/AR scenarios rather than being limited to specialized uses.
Solution Approach 2:
The system dynamically adjusts tactile display parameters such as element activation patterns, vibration frequencies, and surface deformation characteristics to accurately represent different virtual surface properties. By changing these parameters based on the virtual object's visual data, the system achieves high haptic information accuracy while maintaining versatility across different application scenarios.
3Weight of moving object
If electrocutaneous displays are used to stimulate skin with electrical vibrations, then compact structure is achieved, but skin irritation and pain occur due to fluctuations in skin resistance
Solution Approach 1:
The patent replaces electrical stimulation methods with mechanical or piezoelectric actuation mechanisms. Instead of using electrodes that cause skin irritation through electrical currents and resistance fluctuations, the system employs mechanical elements that can be actuated by piezoelectric or other non-invasive mechanisms to create tactile sensations without harmful electrical effects.
4Loss of information
If vibrotactile displays are used to generate mechanical vibrations, then texture information can be conveyed, but it is very difficult for humans to localize vibrations
Solution Approach 1:
The tactile display is divided into multiple independently controllable elements or pixels arranged in a spatial pattern. Each element can be activated or deactivated individually, allowing the system to create localized tactile patterns that correspond to specific regions of the virtual object. This segmentation enables both texture information transmission and precise spatial localization by controlling which segments are active.
5Measurement precision
If static or displacement displays are used to generate tactile image information through actuator height changes, then all tactile information can be conveyed and excellent localization is achieved, but the ability to convey textures is essentially limited by miniaturization and integration limits
Solution Approach 1:
The patent employs dynamic actuation mechanisms that can rapidly change the state of tactile elements in response to virtual object interactions. Instead of relying solely on static height differences, the system uses dynamic vibration, deformation, or activation patterns to encode texture information. This dynamic approach allows texture representation without requiring extremely fine static miniaturization, as the texture is conveyed through temporal and spatial patterns of actuation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables a more immersive and realistic interaction with virtual objects by providing accurate tactile feedback, allowing users to perceive and explore virtual objects with high local resolution and precision, overcoming the limitations of existing systems.
Implementation Method 1
piezoelectric actuators (JR Summers, CM Chanter, A broadband tactile array on the fingertip J Acoust Soc America 112 (2002), 2118-2126)
Implementation Method 2
ultrasound (T. Watanabe, S. Fukui, A method for controlling tactile sensation of surface roughness using ultrasonic vibration, IEEE Proc. Robot. Automat. 1 (1995), 1134-1139)
Implementation Method 3
shape memory actuators (P.S. Wellman, W.J. Peine, G. Favalora, R.D. Howe, Mechanical design and control of a high-bandwidth shape memory alloy tactile display. Experimental Robotics 5 (1998 ), 56-66)
Implementation Method 4
electroactive polymer actuators (M. Matysek, P. Lotz, K. Flittner, HF Schlaak, Vibrotactile display for mobile applications based on dielectric elastomer stack actuators. Proc. SPIE 7642 (2010), 76420D)
Data Source
Figure 1a~2
Figure 3~4
Figure 4a~4c
AI summary
The invention relates to a system and a method for haptic interaction with virtual objects. The system according to the invention is based on the idea that, for the haptic representation of any virtual three-dimensional object—synchronized with the visual representation—only its perceptible haptic properties in the surface area at the collision point or contact area are important. The haptic display system (morphing) according to the invention, which represents these properties, comprises, in principle, three functional groups that operate synchronously with each other and with a visual subsystem. The operation of the individual subsystems can be static or dynamic. In dynamic operation, for example, the position and/or shape of the tactile display can be changed during touch by the user.Regarding the real-time display of the user's three-dimensional data, at least the body part interacting with the object, e.g., the hand, must be represented in the visual system. This body part can be directly visible, similar to a head-up display. Ideally, all body parts of the user within their field of vision are visually displayed. Through synchronized real-time combination in the visual subsystem, the user and object, whose spatial relationships are actually affected, are virtually displayed in defined spatial proximity. Thus, in this virtual representation, the user sees themselves touching the virtual object. However, at the time and point of collision, they are not actually interacting with the object displayed in the visual system, but rather with a haptic display subsystem, the Morphling, within a tactile subsystem. This subsystem, in the perceptible portion of the virtual object, conveys its haptic properties, e.g.,The surface texture and shape are displayed in real-time synchronization with the visual representation. The interaction of at least one part of the visualized user's body with the visualized object is displayed in the visual subsystem simultaneously with the user's interaction with the tactile display system. When at least one part of the visualized user collides with the visualized object, a collision point is determined. The three-dimensional data of the object at the collision point is mapped in the tactile subsystem. The haptic element has a surface with a structure designed to reflect the three-dimensional structure of the object at the collision point, based on the captured three-dimensional data of the object, at least in the area of the collision point.