AR Door and Window Detection Using 3D Mesh Hole Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D image projection systems struggle to accurately detect and classify open doors, windows, and holes in 2D and 3D environments, leading to mischaracterization and unreliable feature detection due to unstable classifications that change unexpectedly.
Innovation Solution
A method using a 3D depth scanner and 2D camera to create a 3D mesh, where a 2D virtual picture is taken along a segment of the mesh, and rays are cast to classify holes as doors or windows based on intersection with the vertical picture floor and width criteria.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If 3D images are projected onto 2D images of open windows and doors, then the projection can be displayed, but the detection and classification of these features becomes unreliable and unstable
Solution Approach 1:
The 3D mesh is divided into multiple faces, and rays are cast to detect which faces are occluded by the hole. This segmentation allows the system to identify door and window features by analyzing the spatial relationships between mesh faces and the hole location, improving detection reliability.
Solution Approach 2:
Rays are introduced as an intermediary tool to detect the back sides of mesh faces. By casting rays from the hole location through the 3D mesh and counting how many face back sides are intersected, the system can reliably determine whether a hole represents a door, window, or error, resolving the detection instability.
2Measurement precision
If rays are cast through the 3D mesh to detect face back sides, then hole classification accuracy improves, but the computational complexity increases
Solution Approach 1:
Instead of casting rays in all possible directions, the system casts rays only in the directions needed to detect face back sides that could be occluded by the hole. This partial action approach maintains sufficient classification precision while reducing unnecessary computational complexity.
Solution Approach 2:
The system changes the parameter of ray casting by limiting the number of rays cast based on the hole's position and the mesh structure. By adjusting the ray casting parameters dynamically, the system achieves high classification precision without excessive computational overhead.
3Reliability
If the hole is classified based on intersection with vertical picture floor and width, then door and window distinction improves, but the detection becomes more sensitive to measurement errors
Solution Approach 1:
The system establishes clear classification criteria before detection: holes intersecting the vertical picture floor with width at least as large as a door width are classified as doors, while others are classified as windows. These pre-defined thresholds cushion against measurement variations and maintain classification reliability.
Solution Approach 2:
The classification system uses feedback from the ray casting results to adjust the detection. By counting how many face back sides are intersected and comparing against predefined thresholds, the system provides feedback that stabilizes the classification against measurement errors and produces reliable door/window distinction.
Data Source
AI summary
Various embodiments described herein relate to a method, apparatus, and a non-transitory machine-readable storage medium including one or more of the following: locating a 2D plane correlated with a 3D mesh representing a surface of a room; taking a virtual 2D picture of a 3D mesh along the 2D segment; within the virtual 2D picture, finding a hole; determining the vertical picture floor, and hole width; when the hole intersects the vertical picture floor and is at least as wide as a door width then classifying the hole as a door; else classifying the hole as a window.