Virtual Viewpoint Display for Work Machine Remote Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote control of work machines using two-dimensional images can lead to difficulties in obtaining perspective, making it challenging for operators to accurately gauge distances between the work machine and transport vehicles, thereby reducing work efficiency.
Innovation Solution
A display system that generates virtual viewpoint images of work machines and transport vehicles based on detection data, combining these images to provide a clear, three-dimensional bird's eye view for improved perspective and distance judgment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If two-dimensional images are used for remote control display, then the display apparatus can show the work site, but the operator cannot obtain proper perspective and accurately judge distances
Solution Approach 1:
The patent transforms two-dimensional camera images into three-dimensional virtual viewpoint images by integrating position and attitude detection data. This dimensional transformation enables operators to perceive depth and distance accurately, resolving the limitation of 2D displays while maintaining manageable system complexity through software-based processing.
Solution Approach 2:
The patent introduces virtual viewpoint images as an intermediary representation between the actual work site and the operator. These synthesized images mediate the information gap by combining real camera data with computational geometry to create a perspective that directly aids distance judgment, without requiring the operator to physically presence at the site.
2Productivity
If two-dimensional images are used for remote control, then the system is simple to implement, but work efficiency is reduced due to difficulty in grasping spatial relationships
Solution Approach 1:
The patent merges multiple data sources (camera images, position detection data, attitude detection data) into a single integrated virtual viewpoint image. This combination allows operators to simultaneously perceive location, orientation, and spatial relationships, significantly improving work efficiency despite the increased complexity of the integrated detection and processing system.
3Loss of information
If virtual viewpoint images are generated and combined, then perspective and distance judgment are improved, but the processing complexity increases
Solution Approach 1:
The patent performs preliminary processing of position and attitude detection data before generating virtual viewpoint images. By pre-calculating transformation parameters and preparing detection data in advance, the system reduces real-time processing complexity while ensuring complete spatial information is preserved in the final combined image.
Data Source
AI summary
A display system includes: a work machine image generation unit that generates, on the basis of detection data of a position of a work machine including working equipment and detection data of attitude of the work machine, a work machine image representing a virtual viewpoint image of the work machine viewed from a virtual viewpoint outside the work machine; a transport vehicle image generation unit that generates, on the basis of detection data of a position of a transport vehicle and detection data of attitude of the transport vehicle, a transport vehicle image representing a virtual viewpoint image of the transport vehicle viewed from the virtual viewpoint; a combining unit that generates a combined image in which the work machine image and the transport vehicle image are superimposed; and a display control unit that causes the combined image to be displayed on a display apparatus present outside the work machine.