Multimodal 3D Object Interaction System with Ultrasound Haptics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GUI-based human-computer interaction systems fail to provide users with natural and deep interactions with digital products in e-commerce, lacking the immersive experience offered by physical product interactions in offline stores, due to limitations in display and feedback modalities.
Innovation Solution
A 3D object interaction system incorporating a 3D displayer, eye tracker, motion sensor, and computing device for multimodal feedback, enabling users to interact with 3D digital objects through natural gestures and receiving ultrasound haptic, visual, and audio feedback, simulating real-world interactions without the need for additional eyewear.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional GUI-based interaction systems are used, then device complexity is reduced, but ease of operation deteriorates due to lack of natural interaction
Solution Approach 1:
The patent combines multiple sensing modalities (eye tracking, motion sensing, gesture recognition) with 3D display technology to create an integrated interaction system. This merging of previously separate technologies enables natural interaction while managing complexity through unified system architecture.
Solution Approach 2:
The system implements multi-functional capabilities where a single system supports multiple interaction modes (eye tracking for selection, hand gestures for manipulation, audio for commands). This universal approach allows diverse interaction types without requiring separate dedicated systems for each function.
2Ease of operation
If 3D display technology is implemented, then ease of operation improves through natural gestures, but device complexity increases
Solution Approach 1:
The patent introduces software intermediaries and processing layers that mediate between the complex hardware components (3D display, eye tracker, motion sensors) and the user interactions. These intermediary software layers simplify the control logic and coordinate the multiple sensors, reducing the effective complexity perceived by users.
Solution Approach 2:
The system replaces traditional mechanical interaction interfaces (buttons, keyboards, mice) with optical and acoustic sensing mechanisms. Eye tracking substitutes for manual pointing devices, and gesture recognition replaces physical controls, thereby reducing mechanical complexity while enhancing ease of operation.
3Ease of operation
If multiple feedback modalities are provided, then ease of operation improves through immersive experience, but device complexity increases
Solution Approach 1:
The feedback system is segmented into distinct modalities (visual feedback from 3D display, haptic feedback from ultrasonic actuators, audio feedback from speakers). Each feedback channel operates semi-independently, allowing the system to provide immersive multi-sensory experience while managing complexity through modular architecture where each component can be optimized separately.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system enhances user experience by allowing natural and immersive interactions with digital products, reducing the psychological distance between users and digital goods, and providing a more realistic and efficient way to engage with digital merchandise, similar to offline shopping.
Implementation Method 1
an eye tracker configured to track a position and an angle of a user's eye
Implementation Method 2
a motion sensor configured to track a user's movement in mid-air
Implementation Method 3
a 3D displayer configured to display at least one object in a 3 dimension (3D) view
Implementation Method 4
The feedback includes ultrasound haptic feedback, visual feedback, or audio feedback
Data Source
AI summary
The present disclosure relates to methods and systems for providing a multimodal 3D object interaction to let the user interact with 3D digital object in a natural and realistic way.