Extended Reality Interaction Mode Switching by Action Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing extended reality (XR) devices require manual selection of interaction modes, limiting efficiency and user experience as they cannot automatically distinguish between handle tracking and gesture recognition.
Innovation Solution
An extended reality interaction method that automatically determines user actions as handle or gesture actions through data acquisition, using reinforcement learning and template matching, to trigger corresponding interaction modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual selection of interaction mode is implemented, then the system can support both handle tracking and gesture recognition, but the interaction efficiency deteriorates due to requiring user configuration
Solution Approach 1:
The system automatically detects user actions and switches interaction modes without requiring manual user configuration. The XR device monitors action data, determines action types, and autonomously triggers the appropriate interaction mode (handle tracking or bare hand tracking), making the system self-adapting to user needs.
Solution Approach 2:
The interaction mode is made dynamic and adaptable rather than static. The system can switch between handle tracking mode and bare hand tracking mode based on real-time action recognition, allowing the interaction mechanism to evolve according to user behavior rather than being fixed in advance.
2Device complexity
If fixed interaction mode is selected, then the system configuration is simplified, but the ease of operation deteriorates as users cannot switch between handle and gesture actions
Solution Approach 1:
The XR device is designed to support multiple interaction modes (handle tracking and bare hand tracking) within a single system. The action recognition system can identify both types of actions and trigger corresponding interaction modes, making the device universal and adaptable to different user preferences and scenarios without requiring separate configurations.
3Productivity
If automatic action recognition is implemented, then interaction efficiency is improved, but the device complexity increases due to reinforcement learning and template matching systems
Solution Approach 1:
The action recognition system is divided into distinct modules: action data acquisition, action type determination (using reinforcement learning and template matching), and interaction mode triggering. This segmentation allows each component to be optimized independently and facilitates easier implementation and maintenance of the complex recognition system.
Data Source
AI summary
The present disclosure provides an extended reality interaction method and apparatus, an electronic device and a storage medium, the extended reality interaction method includes: acquiring action data of a user in an extended reality scene; determining an action type of the user according to the action data; and triggering a corresponding interaction mode according to the action type, the corresponding interaction mode comprising a handle tracking mode and a bare hand tracking mode, according to the method provided by the present disclosure, an action type of the user is recognized and then a target interaction mode according to the action type is triggered for computing processing, without the need to manually set the interaction mode in advance.