AR Display Control With Depth Mapping for Hidden Operation Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR display technologies face issues with operability and sense of reality when virtual objects are positioned such that operation points are hidden behind structural objects, leading to incomplete visibility and reduced user interaction.
Innovation Solution
A display terminal equipped with color and distance image cameras generates a three-dimensional map of the environment, identifies regions behind structural objects, and corrects display data to show operation points and virtual objects, using additional objects to enhance visibility and maintain a sense of reality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hidden surface removal is applied to display virtual objects in AR, then the sense of reality is maintained, but operation points may be hidden behind structural objects reducing operability
Solution Approach 1:
The virtual object is divided into multiple operation points distributed across its surface. The system identifies which operation points are visible and which are hidden behind structural objects, then selectively displays only the visible operation points while maintaining the virtual object's integration with the real environment. This segmentation allows the system to preserve operability for accessible points while maintaining realism for hidden regions.
Solution Approach 2:
The system introduces an intermediary mechanism that detects the spatial relationship between virtual operation points and real structural objects. By calculating depth information and comparing positions, the system determines which operation points should be displayed based on visibility criteria. This intermediary detection process enables selective display of operation points without compromising the overall realism of the AR display.
2Ease of operation
If virtual objects are displayed without hidden surface removal, then operability is improved, but the sense of reality is compromised
Solution Approach 1:
The system applies different display qualities to different regions of the virtual object. For operation points that are visible (not hidden behind structural objects), the system displays them with full visibility for optimal operability. For operation points that would be hidden, the system either omits them or displays them with reduced visibility, thereby maintaining the sense of reality in those regions while preserving operability where possible.
3Ease of operation
If operation points are always displayed regardless of position, then operability is maximized, but visual realism is reduced due to hidden surfaces
Solution Approach 1:
The system dynamically adjusts the display of operation points based on the spatial relationship between virtual objects and real structural objects. The display configuration changes in real-time according to depth information and positional relationships. When a virtual object is positioned such that certain operation points would be hidden behind real objects, the system adapts by selectively displaying only the visible operation points, thereby maintaining visual realism while preserving operability for accessible points.
Data Source
AI summary
A display terminal having a display includes a color image camera and a distance image camera for acquiring a color image and a distance image of a predetermined photographing range, respectively, and a display control unit for displaying a virtual object on the display. The display control unit includes a space recognition unit for using the color image and the distance image to generate a three-dimensional map of a structural object, a display data generation unit for generating display data in which a rear region of the virtual object behind the structural object in a line-of-sight direction is specified, based on the three-dimensional map and real space placement position data of the virtual object, and the display correction unit for correcting the display data to display operation points of the rear region, which accept an operation instruction to the virtual object, and displaying it on the display.