Multi-mode sensing concrete construction quality detection system and method

Abstract

The invention provides a multi-modal sensing concrete construction quality detection system and method.The multi-modal sensing concrete construction quality detection system comprises an integrated sensing module, a time synchronization module, a data processing module and a concrete quality diagnosis module, and the integrated sensing module comprises a plurality of sensors which are rigidly fixed and have calibrated spatial relations; synchronously acquiring data of the acoustic sensor and the thermal infrared imager; the time synchronization module is connected with the integrated sensing module, and when the robot moves to a detection point, the time synchronization module sends out a synchronization pulse signal and triggers a sensor in the integrated sensing module to carry out data acquisition at the same time; the data processing module is used for generating a multi-attribute fused three-dimensional point cloud model from data acquired by all sensors in the integrated sensing module; and the concrete quality diagnosis module maps the multi-modal data to a three-dimensional space, extracts multi-channel characteristic parameters of the same spatial position, and inputs the multi-channel characteristic parameters to a pre-trained cross-modal combined diagnosis deep learning model to obtain the defect type, severity and confidence of the position.

Description

Technical Field

[0001] This invention relates to the field of civil engineering technology, and in particular to a multimodal sensing system and method for detecting the quality of concrete construction. Background Technology

[0002] Concrete is a major component of building structures, and its construction quality directly affects the overall project quality and even the service life of the structure. Currently, concrete quality inspection relies heavily on manual inspection using single-point equipment (such as straightedges, rebound hammers, and crack detectors). This method is inefficient, lacks comprehensive coverage, and its results are susceptible to subjective influence from inspectors, making it difficult to detect internal structural defects. With technological advancements, digital technologies such as image acquisition devices, laser scanners, and ultrasonic radar have been applied to concrete quality inspection. However, currently, the data formats and spatiotemporal accuracy of different testing equipment are inconsistent, limiting the ability to perform single-item inspections and hindering cross-sectional analysis.

[0003] Therefore, how to provide a multimodal sensing system and method for detecting the quality of concrete construction is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this invention proposes a multimodal sensing concrete construction quality inspection system and method. By employing multi-sensor synchronous triggering, data fusion processing, and artificial intelligence diagnosis, it solves the technical challenges currently encountered in concrete inspection, achieving integrated, synchronized, and precise intelligent inspection of concrete surface quality, internal quality, geometric dimensions, and environmental conditions.

[0005] The technical solution of the multimodal sensing concrete construction quality detection system and method of the present invention is as follows:

[0006] A multimodal sensing concrete construction quality detection system, comprising:

[0007] An integrated sensing module includes a rigidly fixed high-definition camera, a multispectral camera, a laser scanner, an infrared thermal imager, and an impact echo sensor, all with their spatial relationships calibrated. The laser scanner and the high-definition camera are arranged around the core of the module. The impact echo sensor incorporates an automated light-impact excitation device and an acoustic sensor. The triggering of the light-impact excitation device is linked to a main synchronization signal. The impact induces a transient thermal response, enhancing defect contrast and simultaneously acquiring data from the acoustic sensor and the infrared thermal imager.

[0008] A time synchronization module is connected to the integrated sensing module. When the robot moves to a detection point, the time synchronization module sends a synchronization pulse signal, which simultaneously triggers all sensors in the integrated sensing module to collect data.

[0009] The data processing module is connected to the integrated sensing module. The data processing module performs unified processing on the data collected by all sensors in the integrated sensing module. Using the laser point cloud as a unified spatial reference, the multi-source 2D image data texture is mapped to the reference to generate a multi-attribute fused 3D point cloud model.

[0010] The concrete quality diagnosis module, which is connected to the data processing module, is used to acquire spatiotemporally synchronized multimodal data, map the multimodal data to three-dimensional space, extract multi-channel feature parameters, multispectral features, and surface temperature at the same spatial location, and input the multi-channel features into a pre-trained cross-modal joint diagnostic deep learning model to obtain the defect type, severity, and confidence level at that location.

[0011] Furthermore, all sensors in the integrated sensing module—high-definition camera, multispectral camera, laser scanner, infrared thermal imager, and shock echo sensor—are rigidly fixed on a precisely calibrated common bracket, ensuring that the optical centers of all sensors are aligned and that they have fixed relative positions and orientations.

[0012] Furthermore, the lens axis of the infrared thermal imager is parallel to the antenna of the millimeter-wave radar, allowing its detection depth and area to be mutually referenced, which is used to distinguish between shallow temperature anomalies and deep structural anomalies.

[0013] Furthermore, the multi-channel features at the same spatial location include RGB texture, multispectral water content index, temperature, three-dimensional curvature, and the intensity of the millimeter-wave echo below.

[0014] This invention also provides a multimodal sensing method for detecting the construction quality of concrete, and provides the multimodal sensing system for detecting the construction quality of concrete, the method comprising the following steps:

[0015] Step S1: Train a cross-modal joint diagnostic deep learning model;

[0016] Step S2: Combine the specific band ratio calculated by the multispectral camera with the subtle temperature difference captured by the infrared thermal imager. When a region simultaneously exhibits a high moisture index and a low temperature anomaly, the model determines with high confidence that it is an early water stain or leakage.

[0017] Step S3: If a surface crack is found, analyze the acoustic and thermal characteristics of the area directly below it using a unified coordinate system. If a crack, a low-frequency peak in the impact echo spectrum, and abnormal diffusion of the thermal spot pattern after impact are found at the same time, it is determined to be a crack caused by structural hollowing, which has a high risk level. If there is only a crack, it is a surface shrinkage crack, which requires secondary finishing.

[0018] Step S4: The depth and outline of the reinforcing bars are provided by millimeter-wave radar; combined with the accurate surface model provided by 3D laser point cloud, the propagation path of the millimeter-wave radar is reversed, and the positioning accuracy of the internal cavity boundary and the size of the reinforcing bars is improved by using the wave velocity tomography algorithm.

[0019] Step S5: Input the multi-channel features into the pre-trained cross-modal joint diagnostic deep learning model to obtain the defect type, severity and confidence level at that location.

[0020] Furthermore, the concrete quality diagnosis module employs a Bayesian network-based dynamic quality assessment and prediction algorithm to evaluate quality development trends, specifically including:

[0021] First, real-time data are collected on ambient temperature and humidity, defect probability output by the fusion diagnostic model, and the age of the components.

[0022] Secondly, a Bayesian network is constructed, using environmental conditions and the current defect state as input nodes, to assess the future quality evolution risk, thereby providing forward-looking protection decision support.

[0023] Furthermore, the data processing method includes the following steps:

[0024] Step 1: Input the collected data, which includes RGB images, multispectral images, infrared thermal images, depth maps, millimeter-wave radar point clouds, and laser point clouds, all collected synchronously with a unified physical space coordinate system as the reference.

[0025] The second step is to establish a unified benchmark, using high-precision laser point clouds as a unified three-dimensional spatial coordinate system.

[0026] The third step is data registration. Based on the camera calibration parameters, the data parameters of the two-dimensional image are accurately textured and mapped to the three-dimensional point cloud model, so that each three-dimensional point not only has coordinates, but also color, multispectral features, and surface temperature properties.

[0027] The fourth step is to generate a model, which is a multi-attribute fused 3D point cloud model. On this model, the appearance, interior and geometric information of any location can be queried.

[0028] This multimodal sensing system and method for detecting concrete construction quality has the following advantages:

[0029] The multimodal sensing concrete construction quality detection system and method of the present invention discovers various quality problems existing in the early stage of concrete construction through multimodal fusion. Based on multi-source data correlation analysis and prediction, it realizes early warning of quality risks, identifies multiple quality problems and their correlations, and replaces multiple and various manual inspections with simultaneous detection by multiple sensors, which greatly improves the detection efficiency. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a multimodal sensing concrete construction quality detection system according to an embodiment of the present invention. Detailed Implementation

[0031] The multimodal sensing concrete construction quality detection system and method of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0032] Example 1

[0033] refer to Figure 1 The structural composition of the multimodal sensing concrete construction quality detection system of the present invention is described in detail.

[0034] Please continue to refer to this. Figure 1 A multimodal sensing concrete construction quality detection system, comprising:

[0035] An integrated sensing module includes a rigidly fixed high-definition camera, a multispectral camera, a laser scanner, an infrared thermal imager, and an impact echo sensor, all with their spatial relationships calibrated. The laser scanner and the high-definition camera are arranged around the core of the module. The impact echo sensor incorporates an automated light-impact excitation device and an acoustic sensor. The triggering of the light-impact excitation device is linked to a main synchronization signal. The impact induces a transient thermal response, enhancing defect contrast and simultaneously acquiring data from the acoustic sensor and the infrared thermal imager.

[0036] A time synchronization module is connected to the integrated sensing module. When the robot moves to a detection point, the time synchronization module sends a synchronization pulse signal, which simultaneously triggers all sensors in the integrated sensing module to collect data.

[0037] The data processing module, connected to the integrated sensing module, performs unified processing on data collected by all sensors in the integrated sensing module. Using the laser point cloud as a unified spatial reference, it maps the texture of multi-source 2D image data onto this reference to generate a multi-attribute fused 3D point cloud model. The specific steps are as follows:

[0038] Step 1: Input the collected data, which includes RGB images, multispectral images, infrared thermal images, depth maps, millimeter-wave radar point clouds, and laser point clouds, all collected synchronously with a unified physical space coordinate system as the reference.

[0039] The second step is to establish a unified benchmark, using high-precision laser point clouds as a unified three-dimensional spatial coordinate system.

[0040] The third step is data registration. Based on the camera calibration parameters, the data carried by the two-dimensional image, such as RGB, multispectral, infrared and other parameters, are accurately textured and mapped to the three-dimensional point cloud model. This makes each three-dimensional point not only have coordinates (X,Y,Z), but also have multiple attributes such as color (R,G,B), multispectral features, and surface temperature (T).

[0041] (4) Generate a model. Generate a three-dimensional point cloud model with multi-attribute fusion. On this model, you can query the appearance, interior and geometric information of any location.

[0042] A concrete quality diagnosis module, connected to a data processing module, is used to acquire spatiotemporally synchronized multimodal data, map the multimodal data to three-dimensional space, extract multi-channel feature parameters, multispectral features, and surface temperature at the same spatial location, and input the multi-channel features into a pre-trained cross-modal joint diagnostic deep learning model to obtain the defect type, severity, and confidence level at that location. In this embodiment, more preferably, the multi-channel features at the same spatial location include RGB texture, multispectral moisture content index, temperature, three-dimensional curvature (to determine whether it is a convexity or a depression), and the intensity of the millimeter-wave echo below.

[0043] In this embodiment, more preferably, the cross-modal joint diagnostic deep learning model can be used to diagnose hidden water stains and uneven composition on concrete surfaces, hollow areas and cracks, and to accurately locate reinforcing bars and internal voids.

[0044] In this embodiment, more preferably, the specific method for identifying hidden water stains and uneven composition in concrete is as follows: combining the specific band ratio calculated by the multispectral camera (such as the normalized moisture index NDWI) with the subtle temperature difference captured by the infrared thermal imager. When a certain area simultaneously exhibits "high moisture index" and "low temperature anomaly" (water evaporation heat absorption), the model can determine with high confidence that it is an early water stain or leakage.

[0045] In this embodiment, more preferably, the specific method for determining the correlation between hollow areas and cracks is as follows: when a surface crack is found, the acoustic characteristics (whether a low-frequency peak appears in the impact echo spectrum) and thermal characteristics (whether abnormal diffusion occurs in the thermal spot pattern after impact) of the area directly below it are analyzed based on a unified coordinate system. If a crack, a low-frequency peak in the impact echo spectrum, and abnormal diffusion in the thermal spot pattern after impact are present at the same time, it is determined to be a crack caused by structural hollow areas, which has a high risk level. If there is only a crack, it is a surface shrinkage crack, which can be treated with secondary plastering.

[0046] In this embodiment, more preferably, the specific method for accurately locating the reinforcing bars and internal cavities is as follows: the depth and contour of the reinforcing bars are provided by millimeter-wave radar; the propagation path of the radar waves is reversed by combining the accurate surface model provided by the three-dimensional laser point cloud; and the positioning accuracy of the internal cavity boundary and the size of the reinforcing bars is improved by using a wave velocity tomography algorithm.

[0047] The concrete quality diagnosis module uses a Bayesian network dynamic quality assessment and prediction algorithm to evaluate quality development trends.

[0048] Its features include, firstly, real-time collection of ambient temperature and humidity, defect probability output by the fusion diagnostic model, and age of the component.

[0049] Secondly, a Bayesian network is constructed, using "environmental conditions" and "current defect state" as input nodes to assess the risk of future quality evolution. For example, "Under the current high-temperature and dry environment, there is an 85% probability that the identified microcracks will expand into macrocracks within 48 hours," thus providing proactive maintenance decision support.

[0050] In this embodiment, more preferably, a millimeter-wave radar is also included. The antenna of the millimeter-wave radar is designed to be parallel to the lens axis of the infrared thermal imager, allowing their detection depth and area to be mutually referenced, thus distinguishing between shallow temperature anomalies and deep structural anomalies. In the integrated sensing module, multiple sensors—a high-definition camera, a multispectral camera, a laser scanner, an infrared thermal imager, and a shock echo sensor—are rigidly fixed on a precisely calibrated common support, ensuring that their optical centers are as close as possible and that they have fixed relative positions and attitude relationships. The integrated sensing module adopts a core-satellite layout, with the laser scanner and high-definition camera at the core, and other sensors arranged around them. The laser scanner provides an absolute coordinate reference and three-dimensional geometric parameters.

[0051] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A multimodal sensing concrete construction quality detection system, characterized in that, include: An integrated sensing module includes a rigidly fixed high-definition camera, a multispectral camera, a laser scanner, an infrared thermal imager, and an impact echo sensor, all with their spatial relationships calibrated. The laser scanner and the high-definition camera are arranged around the core of the module. The impact echo sensor incorporates an automated light-impact excitation device and an acoustic sensor. The triggering of the light-impact excitation device is linked to a main synchronization signal. The impact induces a transient thermal response, enhancing defect contrast and simultaneously acquiring data from the acoustic sensor and the infrared thermal imager. A time synchronization module is connected to the integrated sensing module. When the robot moves to a detection point, the time synchronization module sends a synchronization pulse signal, which simultaneously triggers all sensors in the integrated sensing module to collect data. The data processing module is connected to the integrated sensing module. The data processing module performs unified processing on the data collected by all sensors in the integrated sensing module. Using the laser point cloud as a unified spatial reference, the multi-source 2D image data texture is mapped to the reference to generate a multi-attribute fused 3D point cloud model. The concrete quality diagnosis module, which is connected to the data processing module, is used to acquire spatiotemporally synchronized multimodal data, map the multimodal data to three-dimensional space, extract multi-channel feature parameters, multispectral features, and surface temperature at the same spatial location, and input the multi-channel features into a pre-trained cross-modal joint diagnostic deep learning model to obtain the defect type, severity, and confidence level at that location.

2. The multimodal sensing concrete construction quality detection system according to claim 1, characterized in that, In the integrated sensing module, all sensors—high-definition camera, multispectral camera, laser scanner, infrared thermal imager, and shock echo sensor—are rigidly fixed on a precisely calibrated common bracket, ensuring that the optical centers of all sensors are aligned and that they have fixed relative positions and orientations.

3. The multimodal sensing concrete construction quality detection system according to claim 1, characterized in that, The lens axis of the infrared thermal imager is parallel to the antenna of the millimeter-wave radar, allowing its detection depth and area to be cross-referenced, thus distinguishing between shallow temperature anomalies and deep structural anomalies.

4. The multimodal sensing concrete construction quality detection system according to claim 1, characterized in that, The multi-channel features at the same spatial location include RGB texture, multispectral water content index, temperature, three-dimensional curvature, and the intensity of the millimeter-wave echo below.

5. A multimodal sensing method for detecting the construction quality of concrete, characterized in that, The multimodal sensing concrete construction quality detection system according to any one of claims 1 to 4 is provided, the method comprising the following steps: Step S1: Train a cross-modal joint diagnostic deep learning model; Step S2: Combine the specific band ratio calculated by the multispectral camera with the subtle temperature difference captured by the infrared thermal imager. When a region simultaneously exhibits a high moisture index and a low temperature anomaly, the model determines with high confidence that it is an early water stain or leakage. Step S3: If a surface crack is found, analyze the acoustic and thermal characteristics of the area directly below it using a unified coordinate system. If a crack, a low-frequency peak in the impact echo spectrum, and abnormal diffusion of the thermal spot pattern after impact are found at the same time, it is determined to be a crack caused by structural hollowing, which has a high risk level. If there is only a crack, it is a surface shrinkage crack, which requires secondary finishing. Step S4: The depth and outline of the reinforcing bars are provided by millimeter-wave radar; combined with the accurate surface model provided by 3D laser point cloud, the propagation path of the millimeter-wave radar is reversed, and the positioning accuracy of the internal cavity boundary and the size of the reinforcing bars is improved by using the wave velocity tomography algorithm. Step S5: Input the multi-channel features into the pre-trained cross-modal joint diagnostic deep learning model to obtain the defect type, severity and confidence level at that location.

6. The method according to claim 5, characterized in that, The concrete quality diagnosis module uses a Bayesian network dynamic quality assessment and prediction algorithm to evaluate quality development trends, specifically including: First, real-time data are collected on ambient temperature and humidity, defect probability output by the fusion diagnostic model, and the age of the components. Secondly, a Bayesian network is constructed, using environmental conditions and the current defect state as input nodes, to assess the future quality evolution risk, thereby providing forward-looking protection decision support.

7. The method according to claim 5, characterized in that, The data processing method includes the following steps: Step 1: Input the collected data, which includes RGB images, multispectral images, infrared thermal images, depth maps, millimeter-wave radar point clouds, and laser point clouds, all collected synchronously with a unified physical space coordinate system as the reference. The second step is to establish a unified benchmark, using high-precision laser point clouds as a unified three-dimensional spatial coordinate system. The third step is data registration. Based on the camera calibration parameters, the data parameters of the two-dimensional image are accurately textured and mapped to the three-dimensional point cloud model, so that each three-dimensional point not only has coordinates, but also color, multispectral features, and surface temperature properties. The fourth step is to generate a model, which is a multi-attribute fused 3D point cloud model. On this model, the appearance, interior and geometric information of any location can be queried.

Images