Remote Manipulator 3D Viewing for HMD Display Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Operators wearing head-mounted displays (HMDs) face difficulties in viewing control reference information such as position and posture data of robots and their surroundings due to limited display size and information overload when trying to switch between robot visual field images and different viewpoint images.
Innovation Solution
A remote control manipulator system that includes right and left cameras mounted on the robot, a stereoscopic display device, and a model image generator to create integrated three-dimensional models of the robot and its environment, allowing operators to view these models from arbitrary viewpoints by moving their heads, thereby simplifying the switching between stereoscopic and model images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the robot visual field image is always displayed on the HMD, then the operator can view the stereoscopic image of the object handled by the manipulator, but the operator cannot view control reference information such as position and posture data
Solution Approach 1:
The patent transitions from a two-dimensional HMD display to a three-dimensional virtual space where the operator can view both the robot visual field image and control reference information simultaneously from different spatial perspectives. By moving the head, the operator can switch between viewing the stereoscopic robot image and the control reference information without manual input, effectively adding a spatial dimension to information presentation.
Solution Approach 2:
The patent creates a universal virtual space that can present multiple types of information (robot visual field image, control reference information, and their combinations) through a single interface. This multi-functional virtual environment eliminates the need for separate display switching by integrating all information types into one adaptable space.
2Loss of information
If the 3D-image and control reference information are superimposed on the HMD, then both types of information are provided, but the display size and information amount are limited and the image is difficult to view
Solution Approach 1:
The patent segments different types of information (robot visual field image and control reference information) into separate viewable areas within the virtual space. Instead of superimposing them in a crowded single view, the operator can position and view each type of information in distinct spatial locations, reducing visual clutter and improving readability.
Solution Approach 2:
The patent utilizes three-dimensional spatial arrangement to separate and organize information that would be crowded in a two-dimensional superimposed display. By distributing information across different positions and orientations in 3D space, the system maintains completeness of information while significantly improving viewability and reducing visual overload.
3Loss of information
If the operator wears the HMD, then the operator can view the stereoscopic image, but the information available to the operator is limited to what is provided through the HMD
Solution Approach 1:
The patent implements a self-service display switching mechanism where the system automatically responds to the operator's head movements to present appropriate information. Instead of requiring manual input to switch between different information displays, the operator's natural head motion triggers the system to automatically adjust the displayed content, eliminating the need for complex manual switching controls.
4Loss of information
If a different viewpoint image is displayed on the HMD, then the operator can view the surrounding environment, but the operator must input a switching instruction which becomes a burden
Solution Approach 1:
The patent implements automatic display switching based on the operator's head movement detection. When the operator moves their head to a specific position or orientation, the system automatically switches between the robot visual field image and the different viewpoint image showing the surrounding environment, eliminating the need for manual switching instructions and reducing operator burden.
Data Source
AI summary
A remote control manipulator system includes: a manipulator being controlled by an operator remotely to handle an object; a stereoscopic display device that displays a right-eye image and a left-eye image so as to be stereoscopically viewed; a model image generator that generates a model image being an image of an integrated three-dimensional model viewed from a viewpoint designated by the operator, the integrated three-dimensional model being a three-dimensional model obtained by integrating the manipulator model and a surrounding environment model being generated by performing image processing on the right-eye image and the left-eye image and being a three-dimensional model of the object and a surrounding environment thereof; and a model image display device that displays the model image and is configured to be able to be viewed by the operator switching between the stereoscopic display device and the model image display device through moving the head of the operator.