VR Headset Gesture Recognition for Immersive Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing application processing systems for interacting with virtual characters in VR environments are limited by requiring users to use operation controllers for actions not associated with head movements, which reduces immersion and is cumbersome due to the HMD blocking the view.
Innovation Solution
An application processing system that allows users to perform actions not associated with head movements in VR environments using detection information from a head-mounted display, enabling interactions such as hand movements and tool use by selecting specific objects or gazing points on virtual characters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If operation controllers are used to specify actions not associated with head movements, then the variety of user actions in virtual space is increased, but the sense of immersion is reduced and the operation becomes cumbersome
Solution Approach 1:
The patent combines the functions of the HMD and hand gesture recognition into a single integrated system. The HMD not only displays the virtual space but also captures hand gesture information through its sensors, eliminating the need for separate operation controllers and unifying the interaction interface.
Solution Approach 2:
The system uses the HMD's own sensors to detect hand gestures, allowing the device to serve multiple functions (display and gesture recognition) without requiring additional external controllers. The HMD leverages its inherent sensing capabilities to provide gesture-based control.
2Adaptability or versatility
If operation controllers are used to specify actions, then actions not associated with head movements can be performed, but the HMD blocks the view making it difficult to operate the controller
Solution Approach 1:
The patent extracts the control function from external operation controllers and integrates it directly into the HMD system. By using the HMD's sensors to detect hand gestures, the system removes the physical controller from the user's interaction loop, allowing natural hand movements to directly control virtual actions without the HMD blocking visibility of a separate controller.
3Device complexity
If only head motion detection is used to specify user actions, then the system is simple, but the variety of interaction actions is limited
Solution Approach 1:
The patent makes the HMD multi-functional by enabling it to perform both its primary function of displaying virtual space and its secondary function of detecting hand gestures for action specification. This universal approach allows a single device to handle multiple types of user interactions without adding separate specialized devices.
Solution Approach 2:
The system transitions from detecting only head movements (one dimension of interaction) to also detecting hand gestures (adding another dimension). By utilizing the HMD's sensors to capture hand position and movement data, the system expands the interaction space from head-only control to include hand-based actions, significantly increasing action variety while maintaining system integration.
Data Source
AI summary
An application processing system provides an interaction with a virtual character to a user. The system outputs an image of a virtual space including the virtual character to a head mounted display. The system determines that a menu icon has been selected based on detection information obtained by the head mounted display. The system sets, based on the menu icon being selected, a first selection point and a second selection point on the virtual character, and determines, based on one or more items of detection information obtained by the head mounted display, that the first selection point or the second selection point has been selected. The system specifies, based on the first selection point being selected, a first user action as an action of the user, and specifies, based on the second selection point being selected, a second user action as an action of the user.