Gesture Recognition Interaction Processing for VR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human-machine interaction in virtual reality, augmented reality, and mixed reality is inconvenient due to the need for dedicated devices, which are costly and limit user experience by requiring fixed interaction methods.
Innovation Solution
An interaction processing method and apparatus that utilize a processor and memory to receive dynamic images of user gestures, perform gesture recognition, object detection, and determine corresponding instructions, allowing for dynamic and customizable interactions without the need for specialized hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If dedicated hardware devices (VR headset plus joystick) are used for interaction, then interaction functionality is achieved, but device cost and complexity increase
Solution Approach 1:
The patent replaces the mechanical joystick system with a computer vision-based gesture recognition system. The system uses a camera to capture hand gestures and processes them through gesture recognition algorithms to translate into control commands, eliminating the need for physical joystick hardware while maintaining interaction functionality.
Solution Approach 2:
The patent creates a virtual copy of the interaction interface through software. Instead of requiring physical hardware, the system captures and processes visual information to create a digital representation of user intentions, allowing interaction through gesture recognition rather than direct mechanical input.
2Reliability
If fixed interaction methods are used, then device functionality is maintained, but user experience and adaptability deteriorate
Solution Approach 1:
The patent implements dynamic interaction by continuously tracking hand gestures and adapting to different user inputs in real-time. The system can recognize various hand shapes and movements, allowing the interaction method to change dynamically based on user actions rather than being fixed to a single mechanical button press pattern.
Solution Approach 2:
The system changes interaction parameters by detecting different hand gesture configurations (hand shape, position, movement speed) and translating them into different control commands. This allows a single interaction system to support multiple interaction modes by varying the parameters of hand gestures rather than requiring different hardware devices.
3Extent of automation
If wearable VR devices are used, then immersive experience is achieved, but user comfort and vision clarity deteriorate
Solution Approach 1:
The patent extracts the interaction processing function from the wearable VR headset and relocates it to a separate computing system. The headset only needs to capture hand gestures using its camera, while the complex gesture recognition and command translation are performed by an external device, reducing the computational burden and hardware requirements on the wearable device.
Solution Approach 2:
The patent introduces an intermediary processing system that acts as a mediator between the simple camera-based gesture capture and the complex interaction response. This intermediary system handles the computationally intensive gesture recognition and command mapping, allowing the wearable device to remain simple while still providing sophisticated interaction capabilities.
Data Source
AI summary
An interaction processing method includes: receiving a dynamic image of a gesture move of a user; performing gesture recognition on the dynamic image to obtain gesture recognition result image data of the dynamic image; performing object detection based on the image data, to determine a hand shape change and a gesture motion trajectory of the user; determining, based on the hand shape change and the gesture motion trajectory, a corresponding gesture and an instruction mapped to the gesture; and executing the instruction.