Synchronized Haptic and Sonic Feedback for Textured Virtual Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality technologies face challenges in generating realistic haptic and sonic feedback for textured materials in interactive environments, as methods like modal synthesis require offline vibration models and excitation force profiles, making real-time synthesis and consistent feedback unwieldy.
Innovation Solution
A method that synthesizes vibrotactile haptic and sound feedback by modeling sliding contact dynamics at a microscopic level, using Hertz contact theory to estimate contact force profiles, and rendering feedback through actuators, allowing for real-time generation of synchronized auditory and haptic feedback based on virtual object geometry and material descriptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If modal synthesis is used to generate realistic sounds from contact events, then auditory feedback quality is improved, but device complexity increases due to requiring accurate vibration models created offline and excitation force profiles
Solution Approach 1:
The patent creates a virtual copy of the physical object's vibration characteristics through modal synthesis. Instead of requiring complex physical measurement equipment, the system synthesizes a virtual model that replicates the object's acoustic response to contact events, thereby improving auditory feedback quality while avoiding the complexity of physical measurement systems
Solution Approach 2:
The vibration model is created offline in advance before the interactive session. This preliminary action separates the complex model creation process from the real-time interaction, allowing the system to use pre-computed modal parameters during runtime, thus improving auditory feedback quality without adding real-time computational complexity
2Ease of manufacture
If recorded audio and haptics are used for triggered replay, then implementation simplicity is improved, but adaptability deteriorates as it becomes unwieldy for every possible type of interaction
Solution Approach 1:
The system uses material properties (density, elasticity, damping) and geometric parameters to dynamically generate appropriate audio and haptic responses for different interactions. By changing these parameters based on the virtual object's characteristics and the type of contact event, the system achieves both simplicity through parameterized models and versatility through adaptive response generation
Solution Approach 2:
The virtual objects provide their own interaction feedback automatically based on their defined material and geometric properties. When a contact event occurs, the system self-generates appropriate audio and haptic responses using the object's inherent parameters, eliminating the need for manual recording and triggering for each interaction type
3Reliability
If haptic feedback is rendered through actuators based on contact simulation, then haptic realism is improved, but synchronization difficulty with audio increases
Solution Approach 1:
The patent merges the audio synthesis and haptic rendering processes by using the same excitation force profile and vibration model for both modal synthesis (audio) and haptic actuator control. This unified approach ensures that audio and haptic feedback are naturally synchronized since they both respond to the same physical contact events and object properties
Solution Approach 2:
The system maintains continuous real-time simulation of contact dynamics, continuously updating both audio and haptic outputs based on the ongoing interaction. This continuous action ensures that both feedback channels remain synchronized with the current state of the virtual object without interruption or delay
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables realistic and immersive haptic and audio feedback for interactions with textured virtual objects, reducing the need for extensive content authoring and providing consistent sensory experiences in virtual environments.
Implementation Method 1
Some embodiments use Hertz contact theory to estimate a contact force profile (e.g., duration, peak force) of an impact and thereby generates realistic sounds for contact between different materials.
Implementation Method 2
contact simulation to generate haptic feedback, signals for vibrotactile actuators
Data Source
AI summary
A method generates synchronized auditory and haptic feedback for artificial-reality environments. The method includes performing a simulation of a user interaction with a virtual object in an artificial-reality environment. The user interaction (i) traverses a surface of the virtual object (e.g., running a finger over a textured surface), and (ii) includes a set of contact events (e.g., a sparse set of events). The method also includes estimating a trajectory of the user interaction with the virtual object based on the set of contact events. The method also includes determining a surface profile associated with the surface of the virtual object, generating an excitation force profile according to (i) the estimated trajectory and (ii) the surface profile, and rendering, based on the excitation force profile, audio and synchronized haptic feedback for the user interaction.