XR Device 3D Interaction via Dual Camera Gesture Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality (AR) devices are limited in that virtual objects can only be moved based on user touch recognized on the screen, and they operate in two dimensions (2D), making it impossible to adjust the depth value of virtual objects, which restricts the creation of three-dimensional (3D) effects and simultaneous selection of multiple virtual objects.
Innovation Solution
An extended reality (XR) device equipped with multiple cameras that capture real objects and users from different directions, allowing for the display and manipulation of virtual objects in 3D space, enabling depth adjustment and simultaneous selection of multiple virtual objects through touch commands and user movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single camera is used for capturing real objects, then the device structure is simple, but it is impossible to create three-dimensional (3D) effects and adjust depth values of virtual objects
Solution Approach 1:
The patent transitions from 2D screen interaction to 3D spatial interaction by introducing a second camera that captures depth information. This dimensional change enables the system to perceive and manipulate virtual objects in three-dimensional space, allowing depth value adjustment and 3D effects while maintaining a relatively simple dual-camera structure.
Solution Approach 2:
The dual-camera system serves multiple functions: the first camera captures real objects for AR display, while the second camera captures user hand movements and gestures. This multi-functionality allows the same hardware configuration to support both traditional AR display and the new 3D interaction capabilities without requiring additional specialized sensors.
2Productivity
If virtual objects are moved only based on screen touch recognition, then the control method is simple, but multiple virtual objects cannot be simultaneously selected and depth adjustment is impossible
Solution Approach 1:
The patent replaces the mechanical 2D touch screen interaction system with an optical gesture recognition system using the second camera. Users can select and manipulate multiple virtual objects simultaneously by making hand gestures in 3D space, which are captured and interpreted by the camera system. This substitution enables multi-object selection and depth adjustment without requiring complex multi-touch screen operations.
Solution Approach 2:
The second camera acts as an intermediary between the user and the virtual objects. Instead of direct screen contact, the camera captures hand gestures and translates them into object manipulation commands. This intermediary system enables more natural and efficient interaction with multiple objects simultaneously while reducing the complexity of direct screen manipulation.
3Adaptability or versatility
If the screen is designed in two dimensions (2D), then the display structure is simple, but it is impossible to adjust depth values and create immersive 3D environments
Solution Approach 1:
The patent adds the depth dimension (Z-axis) to the traditional 2D display system by using the second camera to capture hand gestures that control virtual object depth. This allows users to manipulate objects in three-dimensional space while the display itself remains a 2D screen, creating immersive 3D effects without requiring a complex volumetric display system.
Data Source
AI summary
Disclosed are an extended reality (XR) device and a control method thereof, which are applicable to all of 5G communication technology field, a robot technology field, an autonomous driving technology field, and an AI technology field.