Blending Real and Virtual Construction Objects in Augmented Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) systems struggle to convincingly blend real and virtual objects in construction sites, leading to perceptible distinctions between the two, which can result in virtual objects appearing to float over real-world views due to lack of spatial accuracy and graphical credibility.
Innovation Solution
A method involving depth sensing and RGB image capture, registration to a common coordinate system, and projection using GIS and CAD data to generate an AR scene, with occlusion algorithms to remove hidden surfaces and ensure accurate depth representation, employing techniques like depth buffering, TOF cameras, and render to texture methods for real-time blending.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If virtual objects are superimposed over real-world views without accurate depth integration, then the AR scene can be generated quickly, but the virtual objects appear to float over real objects rather than blending convincingly
Solution Approach 1:
The patent replaces traditional mechanical image processing with depth buffering technology to achieve accurate depth integration. The depth buffer stores depth information from the real-world scene and uses it to determine which surfaces should be visible and which should be occluded by virtual objects, enabling precise spatial integration without complex mechanical image alignment systems
Solution Approach 2:
The patent changes the parameter representation from traditional 2D image coordinates to 3D depth-based coordinates. By representing surfaces in terms of their depth distances and using depth buffering, the system achieves accurate spatial integration of virtual and real objects while maintaining real-time performance
2Manufacturing precision
If depth sensing and occlusion algorithms are implemented to improve spatial accuracy, then virtual objects blend more convincingly with real objects, but the system complexity increases
Solution Approach 1:
The patent introduces a depth buffer as an intermediary data structure that mediates between the real-world depth sensing and the virtual object rendering. The depth buffer stores depth information and enables the occlusion algorithm to determine visibility without requiring complex direct computation between all object pairs, thus reducing overall system complexity
Solution Approach 2:
The patent segments the rendering process into distinct stages: depth map generation, virtual object projection, and occlusion processing using depth buffering. This segmentation allows each component to be optimized independently and simplifies the overall system architecture by dividing the complex task of realistic blending into manageable processing steps
3Manufacturing precision
If hidden surfaces are removed based on depth distance data, then the graphical credibility of the AR scene improves, but the processing time increases
Solution Approach 1:
The patent replaces complex iterative hidden surface removal algorithms with a more efficient depth buffering approach. Instead of repeatedly checking and removing hidden surfaces, the system uses the depth buffer to pre-determine which surfaces should be visible, enabling faster rendering while maintaining high graphical credibility
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach effectively blends real and virtual objects in AR scenes, enhancing spatial accuracy and graphical credibility, allowing for seamless integration of virtual objects with real-world environments, effective in both indoor and outdoor settings, and capable of real-time rendering.
Implementation Method 1
capturing a depth map image of the construction jobsite and of the at least one real construction jobsite object via a depth sensing device
Data Source
AI summary
A method of blending at least one virtual construction jobsite object and at least one real construction jobsite object in a dynamic augmented reality scene of a construction jobsite includes several steps. One step involves capturing a depth map image via a depth sensing device, and capturing a red, green, and blue (RGB) image via an RGB image-capturing device. Another step involves registering the depth map image and the RGB image to a common coordinate system. Yet another step involves projecting the at least one virtual construction jobsite object in a scene of the construction jobsite with the use of geographical information system (GIS) data, computer-aided design (CAD) data, or both types of data, to generate the augmented reality scene of the construction jobsite. And yet another step involves removing hidden surfaces of the at least one virtual construction jobsite object in the augmented reality scene.