VR Interaction Using Hand Tracking and Object Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality technologies fail to provide a convenient and accurate method for users to interact with real objects in a virtual environment, such as reading a virtual book, as they require special instruments and struggle to distinguish hand actions from virtual objects.
Innovation Solution
A virtual reality interaction method that determines the position of a preset interactive object by recognizing a preset identifier and accurately positions the user's hand action relative to it, allowing for precise simulation of interactive actions in a virtual reality environment using a virtual reality helmet with image capturing capabilities and computer vision algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If special electronic instruments or manipulators are used to interact with virtual objects, then interaction accuracy is improved, but device complexity increases
Solution Approach 1:
The system uses the user's own hand as the interaction tool, eliminating the need for external manipulators. The hand tracking system captures images of the user's hand and uses computer vision to recognize gestures and positions, allowing the hand to serve both as the operator and the tool for interaction.
Solution Approach 2:
The patent replaces physical manipulators and electronic instruments with an optical system. Image capturing devices (cameras) and computer vision algorithms substitute for mechanical hand controllers, using light and image processing to achieve hand tracking and gesture recognition without physical contact with virtual objects.
2Ease of operation
If hand position and virtual object position are used to determine interaction, then ease of operation is improved, but measurement precision deteriorates
Solution Approach 1:
The system transitions from 2D image coordinates to 3D spatial coordinates through depth estimation. By analyzing multiple images captured from different angles or using depth-sensing capabilities, the system calculates the actual three-dimensional position of hand features relative to the virtual object, adding a depth dimension to the position information.
Solution Approach 2:
The patent introduces feature points (keypoints) on the hand as intermediaries for position determination. Instead of directly measuring hand position, the system identifies specific feature points on the hand and uses their relative positions and movements to infer the hand's interaction state with the virtual object, improving both accuracy and ease of operation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This method enhances the user experience by providing a more immersive and accurate simulation of interactive actions, allowing users to interact naturally with virtual objects as if they were real, improving the overall virtual reality experience.
Implementation Method 1
recognizing a preset identifier on the preset interactive object by continuously acquiring image information in front of a user; and recognizing a hand action of the user according to the image information
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A virtual reality interaction method comprises: continuously acquiring image information in front of a user; recognizing a preset interactive object by recognizing a preset identifier in the image information, a surface of the preset interactive object having one or more of the preset identifiers; recognizing a hand action of the user according to the image information; and simulating an interactive action of the user on the preset interactive object in a virtual reality environment according to the hand action and a position of the preset interactive object. The virtual reality interaction method provided by the present disclosure can accurately simulate an interactive action of the user on a real object in a virtual reality environment, thereby improving user experience of virtual reality.