3D AR Display for Remote Worksite Spatial Awareness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality techniques face challenges in allowing remote assistance personnel to easily grasp the situation of workers at a work site, particularly in understanding distances and spatial relationships, due to the use of two-dimensional images which lack depth perception.
Innovation Solution
An image processing device and method that utilizes a three-dimensional model and distance sensor to generate a display screen showing the work position, work target, and dangerous regions relative to the worker's position, with color coding and placement adjustments to enhance visibility and prevent overlap, allowing remote assistance personnel to visualize the work site's three-dimensional layout.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If two-dimensional photographed images are transmitted to remote assistance personnel, then the system complexity is low and ease of operation is maintained, but the remote assistance personnel cannot easily grasp the worker's situation including distances and spatial relationships
Solution Approach 1:
The patent applies dimensionality change by transforming two-dimensional photographed images into three-dimensional visual representations. The image processing device generates a three-dimensional model of the work site based on the photographed image and adds depth information through pseudo-three-dimensional processing, allowing remote assistance personnel to grasp spatial relationships and distances while maintaining relatively simple system architecture.
2Reliability
If three-dimensional models and distance information are added to show work position and dangerous regions, then situational awareness is improved, but the device complexity and processing requirements increase
Solution Approach 1:
The patent applies local quality by selectively enhancing specific regions of the image rather than processing the entire scene uniformly. The system identifies and highlights critical elements such as the worker's position, work target, and dangerous regions with color coding and depth markers, while leaving other areas in their original two-dimensional form. This reduces processing complexity while improving situational awareness where it matters most.
Solution Approach 2:
The patent introduces an intermediary processing layer that generates a three-dimensional model from the two-dimensional photographed image. This intermediary model serves as a bridge between the simple captured image and the complex spatial information needed by remote assistance personnel, allowing the system to provide accurate depth and distance information without requiring extremely complex processing hardware.
3Loss of information
If multiple information elements (work position, work target, dangerous regions) are displayed on the same screen, then information completeness is improved, but the information becomes difficult to distinguish and visibility decreases
Solution Approach 1:
The patent applies color changes by using distinct color codes to represent different information elements. The worker's position is displayed in one color, the work target in another, and dangerous regions in a third color. This color differentiation allows multiple information elements to be displayed simultaneously on the same screen while maintaining clear visibility and distinguishability of each element.
Solution Approach 2:
The patent uses pseudo-three-dimensional processing to create depth perception in the displayed information. By rendering certain elements with depth effects and shading, the system allows multiple information elements to occupy different visual planes, reducing overlap and improving distinguishability while maintaining information completeness.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An image processing method includes acquiring a three-dimensional model obtained by modeling a plurality of objects included in a work space, acquiring, from a camera which is hold by a user, an image captured by the camera, the user existing in the work space, acquiring, from a sensor which is hold by the user, distance information indicating distances between the sensor and each of the plurality of objects, determining a position of the user in the work space based on the three-dimensional model and the distance information, identifying a predetermined region closest to the position of the user among at least one of predetermined regions defined in the three-dimensional model, generating a display screen displaying the predetermined region and the image, and outputting the display screen to another computer.