3D Model Window View Synthesis via Color Depth Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital 3D models of architectural spaces often fail to accurately represent views through windows due to limitations in image sensors and depth sensors, particularly when windows are transparent or dirty, leading to incorrect texture correlation.
Innovation Solution
A system comprising an identification component, spatial data component, and rendering component that identifies window views based on color and depth data, determines a surface to render the view, and applies texture data to accurately represent the view within the 3D model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image sensors and depth sensors are used to capture architectural spaces, then 3D models can be generated efficiently, but the accuracy of window views is compromised due to transparency and dirt on windows
Solution Approach 1:
The patent introduces an intermediary processing system that separates window element detection from window view capture. The system uses identified window regions as intermediaries to guide subsequent view synthesis, allowing the transparent window glass to be treated differently from the scene beyond it. This mediator approach enables the system to work around the sensor limitation by processing window areas through multiple specialized steps.
Solution Approach 2:
The patent segments the 3D modeling process into distinct phases: window element identification, window region masking, and view synthesis. By dividing the overall task into separate components, the system can apply different processing strategies to window areas versus non-window areas, thereby maintaining efficiency while improving view accuracy in the segmented window regions.
2Speed
If standard scanning processes are applied to windows, then the overall scanning speed is maintained, but incorrect texture correlation occurs on window surfaces
Solution Approach 1:
The patent performs preliminary identification and masking of window elements before the main texture correlation process. By pre-processing the 3D model to identify window regions and apply appropriate masks, the system prepares the data structure in advance to prevent incorrect texture mapping. This preliminary action ensures that when standard scanning processes are applied, window areas are already protected from erroneous texture correlation while non-window areas proceed at full speed.
3Device complexity
If transparent windows are scanned using conventional methods, then the scanning process is simple, but the view through the window is not accurately captured
Solution Approach 1:
The patent extracts window view information from the overall scanning process by identifying and isolating window regions. The system takes out the window viewing function as a separate capability, using detected window elements to define regions where view synthesis should occur. This extraction allows the system to maintain simple scanning procedures while recovering window view information that would otherwise be lost due to transparency.
Data Source
AI summary
Systems and techniques for determining and/or generating data for an architectural opening area associated with a three-dimensional (3D) model are presented. A portion of an image associated with a 3D model that corresponds to a window view or another architectural opening area is identified based at least in part on color data or depth data. Furthermore, a surface associated with the 3D model and visual data for the window view or the other architectural opening area is determined. The visual data for the window view or the other architectural opening area is applied to the surface associated with the 3D model.