Dynamic Virtual Object Display via User Gaze and Gesture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies for displaying virtual objects fail to naturally present information that a user is interested in, particularly in diverse scenarios like amusement, education, and shopping, as they do not effectively utilize user gestures and gaze information to dynamically control the display of virtual objects.
Innovation Solution
An information processing device and method that includes a movement-information acquisition unit, a gaze-information acquisition unit, and a display control unit to acquire user gestures and gaze points, allowing the display device to dynamically position and modify the display of virtual objects based on user interaction, thereby naturally presenting interested information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual objects are displayed using conventional display devices, then various information can be presented to users, but the information presented does not naturally reflect what the user is actually interested in
Solution Approach 1:
The system continuously monitors user gaze position and hand gesture states, using this feedback to dynamically adjust which virtual objects are displayed and how they are presented. The gaze detection unit tracks where the user is looking, and the gesture recognition unit monitors hand positions, allowing the system to adapt information presentation to actual user interest in real-time
Solution Approach 2:
The display of virtual objects transitions from static to dynamic based on user behavior. Virtual objects are displayed, hidden, or emphasized depending on whether the user is gazing at them and what gestures they are making. This dynamic adaptation allows the system to present information naturally according to user attention and interaction state
2Quantity of substance
If the system displays all virtual objects continuously, then comprehensive information is available, but the user cannot easily identify information they are interested in
Solution Approach 1:
Different display treatments are applied to different virtual objects based on local user attention. Objects in the user's gaze direction receive enhanced visibility or activation, while other objects maintain standard display. This local differentiation helps users quickly identify information of interest without overwhelming them with all available data simultaneously
Solution Approach 2:
The system uses gaze detection feedback to determine which virtual objects warrant special presentation. By monitoring where the user looks and what gestures they make toward specific objects, the system can selectively emphasize or activate only those objects that the user is actually interested in, improving detection accuracy
3Ease of operation
If the system uses gesture recognition to control display, then user interaction is enhanced, but the system complexity increases
Solution Approach 1:
The hand gesture recognition unit serves multiple functions: it detects basic gestures for navigation, identifies specific gestures for object selection, and works in conjunction with gaze detection to determine user interest. This multi-functional approach enhances interaction naturalness while avoiding the need for separate specialized systems for each function
Data Source
AI summary
An information processing device according to an embodiment of the present technology includes a movement-information acquisition unit, a gaze-information acquisition unit, and a display control unit. The movement-information acquisition unit acquires movement information about a gesture by a user. The gaze-information acquisition unit acquires information about a gazing point of the user. The display control unit controls a display device on the basis of the movement information. The display control unit causes the display device to display a first virtual object including information relating to a target object in a first region related to the target object, and to vary, on the basis of a position of the gazing point in duration for which the user is making the gesture, how the first virtual object is displayed.