XR Device Controlling Real Objects via Virtual Intermediaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current extended reality (XR) devices fail to control specific real objects in a real space using another real object, limiting user convenience and efficiency in tasks such as cooking, cleaning, and object manipulation.
Innovation Solution
An XR device and method that execute an augmented reality (AR) assistant application to detect the state of a real object and control it using virtual objects, allowing users to manipulate real objects through second real objects, such as robots or household appliances, by displaying virtual guides and transmitting commands for temperature control, cleaning, or object handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If an XR device displays real objects on a screen, then users can view the real space, but the XR device fails to control the specific real object using another real object
Solution Approach 1:
The patent introduces virtual objects as intermediaries between the user and real objects. The XR device captures images of real objects, generates corresponding virtual objects with controllable attributes, and allows users to manipulate real objects indirectly through these virtual representations. This mediator layer enables control functionality that would otherwise be unavailable in pure XR viewing.
Solution Approach 2:
The system creates virtual copies of real objects that replicate their appearance and enable control functions. By generating virtual objects that mirror real objects in the captured image, users can interact with these copies to control the corresponding real objects, such as adjusting temperature of a pot or controlling a robot cleaner.
2Adaptability or versatility
If the XR device provides AR mode and VR mode, then users can experience different reality modes, but the device cannot control real objects through these modes
Solution Approach 1:
The patent makes the XR device's object control capability universal across both AR and VR modes. The same image capture, virtual object generation, and control instruction transmission mechanism works in both augmented reality and virtual reality contexts, allowing users to control real objects regardless of which reality mode they are using.
Data Source
AI summary
The present disclosure relates to an XR device and a method for controlling the same, and more particularly, is applicable to a 5G communication technology field, a robot technology field, an autonomous technology field and an artificial intelligence (AI) technology field. The method for controlling an XR device comprises executing an augmented reality (AR) assistant application in the XR device by a user, displaying a real space, which includes a first real object, on a screen of the XR device, detecting a state of the first real object, displaying at least one virtual object for identifying the state of the first real object on the real space of the screen by overlapping the at least one virtual object on the real space, and controlling the state of the first real object by using one or more second real objects of the real space.