Binocular Camera Gesture Recognition for VR Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality systems rely on handle controls for tracker interaction, which are not natural and can be complex, leading to a poor immersive experience, especially in special scenes where handle control is impractical or complicated.
Innovation Solution
A gesture recognition method using a binocular camera to obtain hand images, recognize hand bone points, and determine two-dimensional and three-dimensional positional relations between bone points, allowing for intuitive gesture recognition and control within virtual reality environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If handle controls are used to control the tracker, then the virtual reality system can provide control functionality, but the interaction mode becomes unnatural and complex, reducing immersive experience
Solution Approach 1:
The patent replaces the mechanical handle control system with an optical gesture recognition system. The binocular camera captures hand images, and the recognition model identifies bone points and gestures, substituting physical mechanical interaction with optical field-based gesture detection. This eliminates the need for physical handles while providing more natural hand-based interaction.
Solution Approach 2:
The patent creates a virtual copy of the hand control interface. Instead of requiring physical handles, the system captures and processes images of the user's actual hand gestures, creating a digital representation of hand bone points and gestures that can be used to control virtual reality elements. This copying approach allows direct mapping between physical hand movements and virtual control actions.
2Adaptability or versatility
If handle controls are used in special scenes, then control functionality can be provided, but the control becomes impractical or complicated, degrading user experience
Solution Approach 1:
The patent implements a universal gesture recognition system that can operate across all virtual reality scenes without requiring scene-specific handle configurations. The binocular camera and recognition model provide a unified control mechanism that adapts to different scenes through gesture recognition, eliminating the need for specialized handle controls for each scene type.
Solution Approach 2:
The system allows the user's own hand gestures to serve as the control mechanism. Instead of requiring external handles or controllers, the user's natural hand movements are directly captured and interpreted by the gesture recognition system, enabling self-service control that works universally across all scenes without additional equipment.
3Measurement precision
If traditional handle control methods are used, then basic control functionality is achieved, but gesture recognition accuracy and interaction intuitiveness are insufficient
Solution Approach 1:
The patent segments the hand into multiple bone points for precise recognition. The recognition model identifies specific bone points (fingertips, joint points, palm center) and their spatial relationships, allowing accurate differentiation between various gestures. This segmentation approach enables precise measurement of hand configuration and movement for accurate gesture recognition.
Solution Approach 2:
The patent transitions from two-dimensional image analysis to three-dimensional spatial relationship analysis. By calculating distances and positional relationships between bone points in 3D space, the system achieves more accurate gesture recognition than traditional 2D image-based methods, enabling better differentiation of complex hand gestures.
Data Source
AI summary
The disclosure provides a gesture recognition method and device, a gesture control method and device and a virtual reality apparatus, the gesture recognition method includes: obtaining a hand image, acquired by each lens of a binocular camera, of a user; recognizing, through a pre-constructed recognition model, a first group of hand bone points from the obtained hand image, to obtain a hand bone point image in which the first group of recognized hand bone points is marked on a hand region of the hand image; obtaining, according to the obtained hand bone point image, two-dimensional positional relations and three-dimensional positional relations between various bone points in a second group of hand bone points as hand gesture data of the user; and recognizing a gesture of the user according to the hand gesture data.