Mobile AR Building Modeling with LiDAR and Camera Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision systems lack the capability to rapidly generate accurate building models using three-dimensional sensing and augmented reality techniques, particularly for interior and exterior features of structures, which is essential for industries like insurance, construction, and real estate.
Innovation Solution
A mobile device-based system that captures image frames and three-dimensional data to detect structural features, uses augmented reality icons for user input to align and capture objects, and generates a complete model by processing these inputs through convolutional neural networks or other machine learning algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional computer vision systems are used to generate building models, then the systems can process images, but they lack the capability to rapidly generate accurate three-dimensional building models with interior and exterior features
Solution Approach 1:
The patent combines multiple sensing modalities (camera imaging, LiDAR three-dimensional sensing, and augmented reality display) into a single integrated system. This merging allows the system to simultaneously capture two-dimensional visual data and three-dimensional spatial data, then fuse them to generate accurate building models rapidly, resolving the contradiction between speed and accuracy.
Solution Approach 2:
The system transitions from traditional two-dimensional image-based computer vision to three-dimensional modeling by incorporating LiDAR depth sensing and spatial mapping. This dimensional enhancement enables the system to capture height, width, and depth information simultaneously, achieving both rapid processing and high measurement precision for building features.
2Measurement precision
If detailed building models with interior and exterior features are generated, then the data accuracy improves, but the time required for data collection and processing increases
Solution Approach 1:
The system enables continuous data collection by seamlessly integrating camera imaging and LiDAR scanning as the user moves through the building space. The augmented reality interface provides real-time feedback and guidance, allowing uninterrupted capture of both interior and exterior features. This continuous operation eliminates pauses and repositioning delays, achieving high data accuracy without proportionally increasing total collection time.
Solution Approach 2:
The system incorporates automated feature detection and classification algorithms that independently identify building elements (walls, doors, windows, fixtures) from the captured data. The augmented reality interface automatically overlays detected features and guides the user through the capture process, reducing the need for manual intervention and annotation time while maintaining high measurement precision.
3Adaptability or versatility
If advanced three-dimensional sensing and augmented reality techniques are integrated into mobile devices, then the system capabilities improve, but the device complexity increases
Solution Approach 1:
The patent leverages existing multi-functional mobile device components (camera for imaging, LiDAR sensor for three-dimensional sensing, display for augmented reality output) to perform multiple functions: capturing photos, scanning building geometry, processing spatial data, and providing user feedback. This universal use of existing components enhances system adaptability without significantly increasing device complexity, as the same hardware serves multiple purposes in the building modeling workflow.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables rapid and accurate generation of building models, allowing users to create detailed interior and exterior models of structures using existing mobile device technology, enhancing data accuracy and efficiency for various industries.
Implementation Method 1
three-dimensional sensors such as light detection and ranging (LiDAR) sensors
Data Source
AI summary
Computer vision systems and methods for generating building models using three-dimensional sensing and augmented reality (AR) techniques are provided. Image frames including images of a structure to be modeled are captured by a camera of a mobile device such as a smart phone, as well as three-dimensional data corresponding to the image frames. An object of interest, such as a structural feature of the building, is detected using both the image frames and the three-dimensional data. An AR icon is determined based upon the type of object detected, and is displayed on the mobile device superimposed on the image frames. The user can manipulate the AR icon to better fit or match the object of interest in the image frames, and can capture the object of interest using a capture icon displayed on the display of the mobile device.