Virtual Object Selection Using Real-World Gesture Intermediaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user interfaces for interacting with virtual environments via avatars, especially when using set-top boxes, face challenges in selecting virtual objects due to the small size of representations on television screens, leading to uncertainty in pointing direction and ambiguous object detection.
Innovation Solution
A method and device for selecting a virtual object in a virtual environment by estimating the direction of a pointing gesture using a video camera and depth information, determining a larger associated object in a real or virtual environment, and establishing an association between virtual and real objects to ensure accurate selection, even when virtual objects are small.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture-controlled interface is used to interact with virtual environment, then ease of operation is improved, but measurement precision deteriorates due to uncertainty in determining pointing direction
Solution Approach 1:
The patent introduces a real-world object as an intermediary between the user's pointing gesture and the small virtual object. The camera captures the user's gesture in the real world, maps it to a real object, and uses that real object as a mediator to select the corresponding virtual object. This intermediary approach allows imprecise gestures to be transformed into precise selections.
Solution Approach 2:
The patent transitions from direct 2D screen interaction to 3D real-world gesture recognition. By mapping gestures from the real 3D world to virtual objects on the 2D screen, the system leverages the additional spatial dimension to improve pointing precision. The real object serves as a physical reference point that resolves the ambiguity of pointing direction.
2Adaptability or versatility
If virtual objects are displayed on television screen, then adaptability is improved for living-room usage, but area of moving object deteriorates making object selection difficult
Solution Approach 1:
The patent creates a copy or representation of the real object in the virtual environment. The user interacts with the real object in the physical world, and this interaction is replicated to select the corresponding virtual object. This copying mechanism allows users to interact with familiar real-world objects while controlling virtual objects on the television screen.
Solution Approach 2:
The patent brings the interaction from the 2D screen plane back to the 3D real world. By using real objects with sufficient physical size as selection targets, the system compensates for the small size of virtual objects on the television screen. The real object's larger physical dimensions provide easier and more accurate pointing targets.
3Ease of operation
If pointing gesture is used to select virtual object, then ease of operation is improved, but reliability deteriorates due to ambiguous object detection
Solution Approach 1:
The real object serves as a reliable intermediary that disambiguates the pointing gesture. Instead of directly mapping the gesture to a virtual object (which is ambiguous), the system first identifies the real object being pointed at, then uses that real object as a mediator to determine the intended virtual object. This two-step process significantly improves reliability.
Solution Approach 2:
The system provides feedback by mapping the user's gesture to a real object and then to a virtual object. This feedback loop allows the system to verify the selection and make adjustments if the mapping is incorrect, thereby improving the reliability of object selection while maintaining ease of operation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for selection of a first object in a first virtual environment, the first object being represented in the first environment with a size of value less than a threshold value. In order to make the selection of the first object more convivial, the method comprises steps for: - estimating a direction (120) associated with a pointing gesture of a user, - determining a second object (12) according to said estimated direction (120), said second object having a size of value greater than said threshold value, and - selecting of said first object (112) according to said second determined object (12). The invention also relates to a multimedia terminal (18) adapted for the implementation of the method for selection.