Depth-Aware Display Terminal Control for Hidden Operation Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR display systems face issues with operability and realism when virtual objects are positioned such that operation points are hidden behind structural objects, leading to reduced user interaction capabilities.
Innovation Solution
A display terminal equipped with a color image camera, distance image camera, and a display control unit that generates a three-dimensional map of the environment, identifies regions behind structural objects, and corrects display data to show operation points, using additional objects to enhance visibility and realism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hidden surface removal is applied to display virtual objects, then the sense of reality is maintained, but operation points may be hidden behind structural objects reducing operability
Solution Approach 1:
The virtual object is divided into front region and rear region based on spatial relationship with structural objects. The display control unit selectively displays operation points from the rear region even when they are hidden behind structural objects, while maintaining the hidden surface removal effect for the visual appearance of the virtual object itself.
Solution Approach 2:
The display control unit acts as an intermediary that detects the spatial relationship between virtual objects and structural objects using depth information, and mediates the display by superimposing operation points from hidden regions onto the visible display area, allowing users to access hidden operation points without compromising the realistic appearance.
2Ease of operation
If operation points are displayed for all regions including rear regions, then operability is improved, but the sense of reality is compromised when virtual objects are positioned behind structural objects
Solution Approach 1:
Different display strategies are applied to different regions of the virtual object. The front region maintains traditional hidden surface removal behavior to preserve reality, while the rear region has its operation points selectively displayed to ensure operability. This local differentiation allows both realism and functionality to coexist.
Solution Approach 2:
The virtual object is segmented into front and rear regions based on depth information. The display control unit processes these regions differently: the front region follows conventional rendering rules for realism, while the rear region has its operation points extracted and displayed separately to maintain usability.
3Adaptability or versatility
If virtual objects are displayed at various positions including behind structural objects, then adaptability is improved, but operation points become inaccessible reducing ease of operation
Solution Approach 1:
The system dynamically adjusts the display based on the position of virtual objects relative to structural objects. When a virtual object is positioned behind a structural object, the display control unit automatically extracts and displays operation points from the hidden rear region, ensuring accessibility regardless of the virtual object's spatial position.
Solution Approach 2:
The display control unit uses depth information from the depth image to determine the spatial relationship between virtual objects and structural objects. This feedback mechanism allows the system to automatically adjust the display configuration to ensure operation points remain accessible regardless of virtual object positioning.
Data Source
AI summary
A display terminal having a display includes a color image camera and a distance image camera for acquiring a color image and a distance image of a predetermined photographing range, respectively, and a display control unit for displaying a virtual object on the display. The display control unit includes a space recognition unit for using the color image and the distance image to generate a three-dimensional map of a structural object, a display data generation unit for generating display data in which a rear region of the virtual object behind the structural object in a line-of-sight direction is specified, based on the three-dimensional map and real space placement position data of the virtual object, and the display correction unit for correcting the display data to display operation points of the rear region, which accept an operation instruction to the virtual object, and displaying it on the display.