Mobile AR Building Model Generation via LiDAR and Camera Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision systems lack the capability to rapidly generate accurate building models using three-dimensional sensing and augmented reality techniques, particularly for structures with complex features like stairs and cabinets.
Innovation Solution
The system employs a mobile device equipped with a camera and three-dimensional sensors to capture image frames and three-dimensional data. It uses computer vision algorithms to detect objects, determines AR icons based on detected features, and allows users to manipulate these icons to match the objects. The system can generate models of complex structures by capturing key data points and extrapolating the remaining structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional computer vision systems are used to generate building models, then model generation is possible, but the process is slow and lacks accuracy for complex structures
Solution Approach 1:
The patent combines multiple sensing technologies (LiDAR, camera, inertial sensors) into an integrated mobile mapping system. The LiDAR sensor captures precise three-dimensional point cloud data while the camera captures visual information, and inertial sensors provide orientation and position data. This merging of multiple data sources enables both rapid data collection and high-precision model generation, resolving the contradiction between productivity and measurement precision.
Solution Approach 2:
The patent replaces traditional manual measurement and modeling methods with automated three-dimensional sensing and computer vision algorithms. The system automatically captures spatial data using LiDAR and camera sensors, then uses algorithms to generate building models without manual intervention. This substitution dramatically increases productivity while maintaining high accuracy through automated feature detection and model generation.
2Measurement precision
If detailed manual measurement is performed for complex structures, then model accuracy improves, but time consumption increases significantly
Solution Approach 1:
The system performs self-service measurement by automatically capturing three-dimensional data, detecting features, and generating models without requiring manual measurement for each element. The computer vision algorithms automatically identify walls, windows, doors, and other building features from the captured data, eliminating the need for time-consuming manual measurement while maintaining high accuracy.
Solution Approach 2:
The system performs preliminary data capture using LiDAR and camera sensors to acquire comprehensive three-dimensional information about the building structure. This preliminary action captures all necessary measurement data in a single pass, eliminating the need for subsequent manual measurement and model refinement, thus reducing total measurement time while ensuring accuracy.
3Loss of information
If comprehensive building features are captured, then model completeness improves, but system complexity increases
Solution Approach 1:
The mobile device is designed with multi-functionality, integrating LiDAR sensing, camera imaging, inertial measurement, and computer vision processing into a single universal platform. This universal system can capture various building features (walls, windows, doors, furniture) using the same hardware and software infrastructure, achieving comprehensive feature capture without proportionally increasing system complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the rapid and accurate generation of building models, including complex structures like stairs and cabinets, by leveraging the powerful three-dimensional sensing and augmented reality capabilities of mobile devices.
Implementation Method 1
three-dimensional sensors such as light detection and ranging (LiDAR) sensors
Data Source
AI summary
Computer vision systems and methods for generating building models using three-dimensional sensing and augmented reality (AR) techniques are provided. Image frames including images of a structure to be modeled are captured by a camera of a mobile device such as a smart phone, as well as three-dimensional data corresponding to the image frames. An object of interest, such as a structural feature of the building, is detected using both the image frames and the three-dimensional data. An AR icon is determined based upon the type of object detected, and is displayed on the mobile device superimposed on the image frames. The user can manipulate the AR icon to better fit or match the object of interest in the image frames, and can capture the object of interest using a capture icon displayed on the display of the mobile device.