Mixed Reality Viewpoint Separation for Remote Instruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality systems face challenges in providing seamless three-dimensional operation instructions due to restricted instructor viewpoints, limiting the ability to accurately point and manipulate virtual objects in operator mixed reality spaces.
Innovation Solution
A system comprising a first acquisition unit to acquire the viewpoint position, a generation unit to generate virtual space images, and output units to display these images on head-mounted displays, allowing instructors to perceive operator spaces and provide operation instructions independently of the operator's viewpoint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the instructor's viewpoint is restricted to the operator's viewpoint, then the operator and instructor can share the same camera image, but the instructor cannot provide smooth operation instructions from an independent perspective
Solution Approach 1:
The system separates the viewpoint functions into two independent components: the operator's camera viewpoint and the instructor's independent viewpoint. The instructor can set and adjust their own viewpoint position and angle separately from the operator's viewpoint, allowing independent observation and instruction while the operator performs tasks.
Solution Approach 2:
The system adds a new dimension of viewpoint control by allowing the instructor to position their viewpoint in three-dimensional space independently. The instructor can adjust the viewpoint position (x, y, z coordinates) and angle separately from the operator's first-person perspective, creating an additional observational dimension.
2Device complexity
If no coordinates are set in the operator space, then the system is simpler to implement, but virtual objects cannot be positioned at arbitrary locations or interacted with through pointing
Solution Approach 1:
The system pre-establishes a three-dimensional coordinate system in the operator space before any virtual object manipulation occurs. Coordinates are set in advance, enabling subsequent positioning and interaction of virtual objects without requiring complex real-time calculations.
Solution Approach 2:
The system introduces a coordinate system as an intermediary framework that connects the physical operator space with virtual objects. This coordinate mediator enables accurate positioning and pointing interactions by providing a common reference system for both real and virtual elements.
3Ease of operation
If the operator wears an optical see-through HMD with camera, then the operator can see the physical space, but the instructor cannot perceive the operator space seamlessly or point to the camera image in three dimensions
Solution Approach 1:
The system creates a virtual copy of the operator's camera image and transmits it to the instructor's HMD. This copied image allows the instructor to perceive the operator's physical space as if present, enabling accurate spatial understanding and three-dimensional pointing without interfering with the operator's direct view.
Solution Approach 2:
The system uses transmitted image data as an intermediary that bridges the instructor and operator spaces. The instructor receives and displays the operator's camera feed through HMD, creating a seamless perception channel that enables accurate spatial reference for three-dimensional pointing and instruction.
Data Source
AI summary
A system includes a first acquisition unit adapted to acquire a first position/orientation of a first viewpoint of a first observer, a first manipulation unit used by the first observer to manipulate a virtual object, and a second manipulation unit used by a second observer to manipulate the virtual object. A generation unit generates an image of the virtual object viewed from the first viewpoint based on the first position/orientation, and generates an image of the second manipulation unit based on a relative position of the second manipulation unit from a second viewpoint position of the second observer, a second acquisition unit acquires an image of a physical space viewed from the first viewpoint, and a combining unit combines the images generated by the generation unit and the image acquired by the second acquisition unit. In addition, an output unit outputs the combined image to a first head mounted display worn by the first observer and a second head mounted display worn by the second observer, with the manipulation results by the first manipulation unit and the second manipulation unit being reflected in the image of the virtual object.