VR Camera and Hand Tracking for Intuitive Object Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality (VR) systems lack intuitive and immersive experiences when users attempt to interact with virtual objects, particularly in augmented reality (AR) and mixed reality (MR) environments, as they struggle to accurately select and manipulate objects with precise timing and positioning.
Innovation Solution
An information processing method and system that utilize a head-mounted device (HMD) with a display unit, a detection unit, and an external controller to track the user's head and hand movements, allowing the virtual camera's field of view to adjust dynamically and enabling precise selection and manipulation of virtual objects based on hand object movements, including correction of object directions for intuitive interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a user operates a hand object to interact with virtual objects in VR/AR/MR spaces, then the user can perform basic virtual interactions, but the user cannot acquire a natural and immersive virtual experience that matches real-world object manipulation
Solution Approach 1:
The system provides visual feedback by displaying a selection candidate area around the operation object and highlighting target objects when they satisfy selection conditions. This feedback mechanism allows users to understand the system's recognition of their hand movements and object selection intent, improving both interaction intuitiveness and selection accuracy
Solution Approach 2:
The system changes the display form of target objects (such as changing color or adding highlights) when they satisfy selection conditions. This visual differentiation helps users easily identify which objects are selectable, enhancing the naturalness of interaction while maintaining precise object selection
2Adaptability or versatility
If the virtual camera field of view is fixed, then the system structure is simple, but the user cannot dynamically adjust the view to track hand movements and interact with objects at different positions
Solution Approach 1:
The virtual camera's field of view is made dynamic by adjusting its center position based on the detected position of the operation object. This allows the camera to automatically track and focus on the area where the user is interacting, providing adaptability without requiring complex manual camera control mechanisms
3Measurement precision
If the system requires precise timing and positioning for object selection, then object manipulation accuracy is high, but the user experience becomes less natural and more difficult to operate
Solution Approach 1:
The system performs preliminary action by automatically determining the selection candidate area around the operation object and pre-highlighting potential target objects before the user completes their selection. This reduces the timing precision requirement by guiding the user through the selection process, making interaction more natural while maintaining accuracy
Solution Approach 2:
The selection candidate area acts as an intermediary between the user's hand movement and the final object selection. It provides a transition zone that helps users naturally select objects without requiring precise timing, while the system still maintains accurate object selection through the highlighted target objects
Data Source
AI summary
A method includes defining a virtual space including a virtual camera; an operation object; and a target object. The method includes detecting a position of a head-mounted device and a position of a part of a body other than a head. The method includes moving the virtual camera in accordance with movement of the head-mounted device. The method includes moving the operation object in accordance with movement of the part of the body. The method includes selecting the target object in accordance with movement of the operation object. Selecting the target object includes correcting a direction or a position of the target object based on a positional relationship between the operation object and the target object; and associating the target object, the corrected direction or the corrected position, with the operation object. The method includes moving the target object in accordance with movement of the operation object.