Portable structure measurement / differential analysis ai device and method

Abstract

In a construction site or an infrastructure facility, in order to immediately compare and evaluate a three dimensional model of a design and a construction situation, it depends on expensive equipment or cloud processing, and there are problems in real-time property, portability, and offline use.SOLUTION: To provide a portable device for analyzing a construction error and deformation by generating three dimensional information by depth estimation from a site image acquired by a camera and collating the three dimensional information with stored design data. The analysis result is converted into a natural-language report in a generation AI and presented to a field worker in real time by speech synthesis and GUI display, thereby realizing field support without using a cloud.

Description

[Technical Field]

[0001] This invention relates to a technology for comparing 3D design data (BIM, CAD, 3D scan, etc.) of structures with on-site measurement data at construction sites and infrastructure facilities, and analyzing and reporting the differences using AI. In particular, it relates to a site support device and method that uses a portable device (Raspberry Pi 5 + Hailo-8L) to perform depth measurement, difference evaluation, and natural language output on-site in real time. [Background technology]

[0002] In the fields of architecture, civil engineering, and infrastructure, discrepancies can occur between design models (BIM and CAD) and the actual construction status on-site, and early detection of these discrepancies is crucial for quality control. Conventional methods involve using laser scanners or cloud-based BIM software on-site to assess discrepancies, but these methods face challenges in terms of real-time performance and portability. Furthermore, workers require specialized knowledge to understand the analysis results, making them difficult to understand intuitively. Meanwhile, advances in edge AI technology have made it possible to perform depth analysis, object recognition, and voice synthesis on portable devices. However, no device has yet been developed that can integrate these capabilities to automatically analyze discrepancies between design data and the actual site and generate natural language reports. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2019-202330: Name of invention: BIM model interference detection device and method for construction sites (Summary). This technology compares 3D scan data of the site with a BIM model to detect inconsistencies in construction. While related in terms of relevance and comparison of BIM data with actual measurement data, depth estimation and audio output functions using portable devices or edge AI are not disclosed. Patent Publication No. 2021-100115: Name of invention: BIM-linked visualization device for construction sites (Summary). This technology uses a tablet device to display and reference BIM models on-site. While relevance and utilization of BIM data are common, depth measurement using Hailo AI or MonoDepth and generated AI reports are not disclosed. [Non-patent literature]

[0004] [Non-Patent Document 1] Abstract of the bSI buildingSMART International openBIM Standard Specification, which specifies IFC / BCF data formats and open BIM standards. While this document provides the foundation for the BIM data integration of this invention, it does not mention AI or mobile devices. Also, "Real-Time Monocular Depth Estimation on Edge Devices using MiDaS" (arXiv:2104.06493) proposes a technology for real-time monocular depth estimation using the MiDaS model. While directly related to depth estimation, it does not mention integration with BIM or audio output. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention aims to provide a field support device that analyzes the differences between a structure's design data and on-site data in real time on a portable device and provides natural language reports and voice notifications using generative AI. [Means for solving the problem]

[0006] The present invention includes the following means. The site was photographed using a USB camera. Depth analysis method,We run MonoDepth and YOLO-based models on the Hailo-8L accelerator to,generate depth maps. Design data acquisition means: Acquire design information such as BIM, CAD, and 3D scans from API or local storage. Differential analysis means compares depth data with design data to analyze construction deviations, deformations, and progress. The analysis results are generated as Japanese text using natural language generation methods and generation AI such as Gemma 2B. The report is synthesized using Open JTalk, a voice output method, and notified to the worker. GUI display means, 3.5-inch touch display shows depth color map and report. Measurement can be started by voice command via the USB microphone. Powered by a mobile battery for portability. [Effects of the Invention]

[0007] On-site analysis is possible in a wide range of fields, including infrastructure, construction sites, disaster sites, and factory facilities Cloud-independent and real-time operation Natural language reports and voice notifications make it easy for workers to understand Convenience and cost savings with portable devices [Brief explanation of the drawings]

[0008] [Figure 1] Acquiring BIM data, etc.: A process of acquiring coordinates and component information for IFC models through the Catenda Hub API. On-site video acquisition: Capture on-site video using a USB camera. Depth estimation: Run the MonoDepth model using Hailo-8L to generate a depth map. BIM comparison analysis: Compare the depth map with BIM data to extract construction deviations. AI report generation: Create a natural language report using Ollama (Gemma 2B). Audio output and GUI display: Audio guidance using Open JTalk, and visualize the results on a 3.5-inch display. DETAILED DESCRIPTION OF THE INVENTION

[0009] As an example, we will show a configuration in which a Hailo-8L AI accelerator is connected to a Raspberry Pi 5, images acquired from a USB camera are subjected to depth analysis, and compared with IFC data to detect construction misalignment. The results are then converted into a natural language report using Gemma 2B (Ollama), output as audio using Open JTalk, and the results and depth color map are displayed in a Tkinter GUI. [Example]

[0010] 1. Image capture unit: Uses a USB-connected UVC camera to capture images at a resolution of 1280 x 720px and 30 fps. 2. Voice input / output unit: Voice commands are received via a USB microphone and Japanese voice is generated using Open JTalk. 3.4. Computing device: Raspberry Pi 5 is used and runs on the OS. Equipped with the edge AI accelerator: Hailo-8L, it performs real-time inference of depth estimation models (MonoDepth models converted from ONNX to HEF). 5. Audio output 6. Display: Equipped with a 3.5-inch touch display, it forms a GUI that displays a color map of the depth map and an analysis report. [Industrial Applicability]

[0011] The "portable structural measurement and differential analysis AI device and method" of the present invention can be widely used in the following industrial fields: Specifically, it can be applied in the following fields: Construction industry: Construction progress and quality inspections of buildings, interference detection and construction error detection by comparing with BIM and CAD models, and accuracy control of prefabricated buildings and steel frame construction. In civil engineering and infrastructure management, construction of bridges, roads, tunnels, etc., measurement at repair sites, infrastructure deterioration inspections, on-site data collection before seismic reinforcement, and deformation and collapse risk detection of damaged structures at disaster recovery sites. In industrial facilities and plants, on-site confirmation of factory equipment and piping layouts, inspection of plant equipment and 3D model updates, and measurement of differences between design drawings and actual equipment. In disaster prevention and response, safety confirmation of damaged buildings and temporary housing, rapid 3D situation assessment at disaster sites, and use as a drone-mounted system for overall site analysis. The portable configuration of this device (Raspberry Pi 5, Hailo-8L, and mobile battery) allows for on-site measurement and analysis immediately, and inference and generation processing can be completed on the edge side without using the cloud, making it usable in offline environments and at sites where security is required. In addition, natural language reports using generative AI and voice synthesis allow field workers and managers to understand measurement results without specialized knowledge, significantly improving work efficiency and quality control. [Explanation of symbols]

[0012] 1. Imaging unit: USB-connected UVC camera 2. Audio input / output section: USB microphone 3. Computing device: Raspberry Pi 5 4. Equipped with the edge AI accelerator Hailo-8L, 5. Audio output speaker 6. Display: 3.5-inch touch display

Claims

1. A portable on-site analysis AI device that acquires 3D structural data or design data of on-site objects, performs depth analysis using an edge AI accelerator from on-site video captured by a camera, compares the design data with the on-site data, automatically analyzes the differences, and generates a natural language report and voice output using a generative AI.

2. 2. The apparatus according to claim 1, wherein the design data may be BIM data, CAD data, or 3D scan data of a structure.

3. 3. The device according to claim 1, further comprising a USB camera and a microphone, and capable of starting the analysis process by voice command.

4. 4. The device according to claim 1, wherein the edge AI accelerator is a Hailo-8L or equivalent inference device, and the depth analysis model uses MonoDepth or a YOLO-based model.

5. The device according to any one of claims 1 to 4, characterized in that the generation AI converts the results of the on-site difference analysis into natural language text and provides a voice notification using a voice synthesis engine.

6. 6. The device according to any one of claims 1 to 5, characterized in that it has a portable configuration that can be powered by a mobile battery and is capable of performing measurements at outdoor sites and infrastructure facilities.

7. A site analysis method characterized by including a series of processes that compare design data and on-site measurement data of on-site objects, analyze the differences, generate explanations in natural language using generative AI, and output them as voice.

Images