Infrared thermal imaging building facade defect intelligent diagnosis method based on multi-modal fusion

Abstract

The invention provides an infrared thermal imaging building facade defect intelligent diagnosis method based on multi-modal fusion, and relates to the technical field of building detection.The method comprises the steps that infrared thermal imaging, visible light images and three-dimensional point cloud data are synchronously collected to construct a multi-modal data set; segmenting a hot spot region by adopting an improved morphological watershed algorithm and extracting contour and temperature features; recognizing a surface crack and peeling area based on a double-branch attention network to generate a texture defect feature map; curvature distribution and thermal deformation gradient are calculated through space registration constrained by a heat conduction equation; multi-source features are fused to calculate a hot spot form dispersion TSMD and a structure risk quantification factor SRQF; constructing a defect risk decision matrix to output defect types, positions and risk levels; and superposing the diagnosis result to a BIM model to generate a three-dimensional visual report and predicting a thermodynamic evolution trend. The multi-modal data collaborative analysis is realized, the defect risk is accurately quantified, and the problems of poor anti-interference performance, inaccurate segmentation and large registration error of a traditional method are solved.

Description

Technical Field

[0001] The present invention relates to the field of building detection technology, and in particular to an intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion. Background Art

[0002] Building facade defect detection is a core task to ensure the safe operation and maintenance of buildings. Infrared thermal imaging technology has become a mainstream method due to its non-contact and large-scale coverage capabilities, but the existing technology system has fundamental flaws. The sole reliance on infrared thermal imaging data leads to severe limitations in the defect recognition dimension. The separation of visible light texture information and thermodynamic characteristics causes systematic misjudgment. Environmental interference (such as solar reflection and instantaneous temperature changes) forms a large number of pseudo hot spots in thermal images. Traditional methods cannot distinguish between real defects and noise signals, and the false detection rate of hollowing remains high. There are structural defects in the hot spot segmentation link. The adaptive threshold segmentation has poor adaptability to the emissivity differences of building surface materials, and the segmentation boundary drifts significantly under the interference of temperature transients. When the morphological watershed algorithm directly processes the original thermal image, gradient noise causes regional over-segmentation, and the fragmented areas need to be manually merged, which reduces the detection efficiency and makes the results highly subjective.

[0003] Quantitative defect assessment methods are severely outdated. Existing technologies rely solely on simple parameters such as temperature thresholds or contour area, failing to establish a correlation model between hot spot morphological irregularities (e.g., sudden changes in contour curvature) and structural risk parameters (e.g., thermal deformation gradients). This results in an inability to quantitatively assess the risk level of defects such as hollowing and structural cracks, forcing repair decisions to rely on empirical judgment, posing a significant safety hazard. Inadequate multimodal data registration accuracy has become a technical bottleneck. Registration methods based on feature point matching ignore differences in the thermophysical properties of building materials, and the difference in thermal diffusivity between concrete and brick walls is not factored into the calculation model. Registration errors are magnified to unacceptable levels on curved facades, directly leading to distorted thermal deformation gradient calculations.

[0004] There are theoretical flaws in the multi-source feature fusion process, and traditional weighted averaging or feature concatenation methods cannot handle evidence conflict scenarios. When the hot spot contour feature indicates hollowing and the visible light texture feature shows a normal surface, the evidence conflict exceeds the critical value, causing the fusion result to fail and the defect detection rate to drop sharply. The problem of insufficient adaptability of the classification model is particularly prominent. The rule threshold classifier has difficulty dealing with the morphological similarity of cracks and structural cracks. Machine learning models such as support vector machines lack an effective arbitration mechanism in the overlapping areas of decision boundaries, and the high misjudgment rate directly affects the reliability of diagnosis. There is a technical fault in the application layer of the detection results. The two-dimensional diagnostic report cannot be spatially associated with the building information model (BIM). Manual positioning of the defect position is inefficient and has significant errors. The ability to predict thermodynamic evolution trends is lacking, and the risk of defect expansion cannot be dynamically assessed, resulting in a lack of data support for the formulation of preventive maintenance strategies. Summary of the Invention

[0005] In order to solve the technical problems in the existing technology that single infrared data defect recognition dimension is insufficient and has poor anti-interference ability, hot spot segmentation has boundary drift and over-segmentation, the lack of risk quantification model leads to inaccurate assessment, multimodal alignment does not consider the thermal properties of materials and causes errors, feature fusion does not resolve evidence conflicts and causes a sharp drop in detection rate, the classification model decision boundary overlapping area has a high misjudgment rate, the detection result three-dimensional positioning is difficult and the risk prediction ability is lacking, the present invention provides an infrared thermal imaging building facade defect intelligent diagnosis method based on multimodal fusion.

[0006] The technical solutions provided by the present invention are as follows: The present invention provides an intelligent diagnostic method for building facade defects based on infrared thermal imaging and multimodal fusion, comprising: S1. Synchronously collect infrared thermal imaging data, visible light image data, and 3D point cloud data of the target building facade to construct a multimodal dataset. S2. Segment the hot spot area of ​​the infrared thermal imaging data and extract the hot spot contour features and temperature distribution features; S3, based on visible light image data, identifies surface cracks and peeling areas through a deep learning model and generates a texture defect feature map; S4, spatially registering the 3D point cloud data with the thermal imaging data to calculate the curvature distribution and thermal deformation gradient of the facade structure; S5. Fusion of hot spot contour features, texture defect feature maps, and thermal deformation gradients to calculate the hot spot morphological dispersion TSMD and structural risk quantification factor SRQF. S6. Based on the coupling relationship between TSMD and SRQF, a defect risk decision matrix is ​​constructed to output the diagnosis results of facade defect type, location and risk level.

[0007] Furthermore, the S5 further includes: The hot spot morphology dispersion TSMD is calculated by the following formula: ; in, is the number of contour sampling points in the hot spot area, For the The second-order derivative of the profile curvature at the sampling points, is the local window area centered at the sampling point, is the average area of ​​hot spots, is the area standard deviation, 、 is the normalization coefficient, satisfying .

[0008] Furthermore, the S5 further includes: The structural risk quantification factor SRQF is calculated by the following formula: ; in, is the thermal gradient amplitude in °C / m, is the local binary pattern texture variance, is the maximum value of the point cloud curvature, 、 is the dimensional balance coefficient.

[0009] Furthermore, the hot spot area segmentation in S2 adopts an improved morphological watershed algorithm, which specifically includes: S201, applying adaptive threshold segmentation to infrared thermal imaging data to generate an initial hot spot mask; S202. Use the morphological gradient reconstruction formula to eliminate noise artifacts: ; in, Representation based on structural elements The morphological reconstruction operator, is the morphological gradient operator; S203: Perform watershed transformation on the gradient reconstruction image to merge the over-segmented regions.

[0010] Furthermore, the feature fusion in S5 adopts an improved form of DS evidence theory: S501, using the hot spot contour feature, texture defect feature map, and thermal deformation gradient as independent evidence sources; S502. Calculate the conflict degree between evidences using the Josselme distance: ; in, is the focal element correlation matrix; S503: Dynamically adjust the basic probability distribution function according to the conflict degree, and fuse and generate the input feature vector of TSMD and SRQF.

[0011] Furthermore, the spatial registration of S4 adopts a physical constraint registration method based on the heat conduction equation: S401. Establish the partial differential equation of heat conduction on the facade: ; in, is the thermal diffusivity of the material; S402. Using the equation solution result as a constraint condition, optimize the affine transformation matrix of the three-dimensional point cloud and the thermal imaging data.

[0012] Furthermore, the defect risk decision matrix of S6 is constructed using a support vector machine multi-classifier: S601, taking TSMD and SRQF as input feature vectors; S602, mapping to a high-dimensional space using a radial basis kernel function; S603. Solve the optimal classification hyperplane using the structural risk minimization principle.

[0013] Furthermore, the deep learning model in S3 is a dual-branch attention network, including: S301, branch 1 uses U-Net architecture to extract pixel-level crack features; S302, branch 2 uses the ResNet-50 architecture to extract regional-level spalling features; S303: Weighted fusion of dual-branch outputs through the channel attention module.

[0014] Furthermore, S7, the diagnosis results are superimposed on the BIM model to generate a three-dimensional visual defect distribution map.

[0015] Furthermore, S8 generates a maintenance priority report based on the risk level, the report including defect type, location coordinates, risk index and thermodynamic evolution trend prediction.

[0016] The beneficial effects brought about by the technical solution provided by the present invention include at least: (1) In this invention, by synchronously collecting and spatially registering infrared thermal imaging, visible light images, and 3D point cloud data, and combining it with the physically constrained heat conduction equation to optimize registration accuracy, positioning errors caused by differences in material thermal properties are eliminated. An improved morphological watershed algorithm is used to suppress thermal noise artifacts and achieve precise segmentation of hot spot boundaries. This solution completely solves the problems of incomplete defect recognition from a single data source, misjudgment due to environmental interference, and registration distortion, significantly improving the reliability of detecting defects such as hollows and cracks.

[0017] (2) In this paper, a dual-parameter model of hot spot morphological dispersion (TSMD) and structural risk quantification factor (SRQF) is proposed, integrating multidimensional features such as profile curvature mutation, thermal gradient amplitude, and texture variance. Based on an improved DS evidence theory, the conflict of evidence is dynamically adjusted, the conflict degree is quantified using the Jouselme distance, and the probability distribution is reconstructed. This design overcomes the problem of a sharp drop in detection rate caused by the lack of a risk quantification model and the conflict of integration, and achieves an objective assessment of the defect risk level.

[0018] (3) In this invention, the diagnostic results are deeply integrated with the Building Information Model (BIM). The spatial location of defects and risk levels are mapped using the IFC standard to generate a three-dimensional visual defect distribution map. The ARIMA model is combined to predict thermodynamic evolution trends, and a maintenance priority strategy is dynamically generated based on the risk index. This system addresses the pain points of difficult positioning and lack of risk prediction in two-dimensional reports, providing full-cycle decision support for preventive maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of a flow chart of an intelligent diagnostic method for building facade defects using infrared thermal imaging based on multimodal fusion provided by an embodiment of the present invention; Figure 2 A schematic diagram of the process of a dual-branch attention network in the intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion provided by an embodiment of the present invention; Figure 3 A schematic diagram of the process of constructing a defect risk decision matrix in the intelligent diagnosis method for building facade defects using infrared thermal imaging based on multimodal fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0022] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0023] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0024] In the embodiment of the present invention, sometimes the subscript is as follows It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning is the same.

[0025] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0026] Reference Manual Figure 1 , which shows a flow chart of an intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion provided by an embodiment of the present invention.

[0027] The embodiment of the present invention provides an intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion. The processing flow may include the following steps: S1. Synchronously collect infrared thermal imaging data, visible light image data, and 3D point cloud data of the target building facade to construct a multimodal dataset.

[0028] It should be noted that step S1 is first executed to obtain multimodal data of the target building facade through a synchronous acquisition device equipped with an infrared thermal imager, a high-resolution visible light camera, and a lidar. The infrared thermal imager must be a model with a thermal sensitivity of not less than 0.05°C, and the acquisition frequency is set to 5Hz to ensure the ability to capture thermal dynamics. The resolution of the visible light camera is not less than 20 million pixels, and a polarizing filter is used to eliminate reflective interference. The three-dimensional point cloud data is obtained by scanning with an angular resolution of 0.1° by the lidar, and the point cloud density must reach more than 500 points per square meter. The three types of data are time-aligned through a hardware synchronization trigger device. The spatial coordinate system is the east-north-sky coordinate system. Finally, a multimodal data set containing a thermal radiation matrix, an RGB image matrix, and a point cloud coordinate set is constructed.

[0029] S2. Segment the hot spot area of ​​the infrared thermal imaging data and extract the hot spot contour features and temperature distribution features.

[0030] It should be noted that when performing the hot spot area segmentation in step S2, adaptive threshold segmentation is first applied to the infrared thermal imaging data: the temperature threshold is calculated using the maximum inter-class variance method. , the temperature is higher than ( The pixels with the standard deviation of the temperature of the entire facade are marked as candidate hot spots, and an initial binary mask is generated. Then the improved morphological watershed algorithm is performed: In step S201, adaptive threshold segmentation is applied to the infrared thermal imaging data to generate an initial hot spot mask, a morphological opening operation is performed on the initial mask to eliminate noise, and a 3×3 circular template is selected as the structural element.

[0031] Step S202 uses the morphological gradient reconstruction formula to eliminate noise artifacts: First, according to the formula Construct a gradient reconstruction map. The morphological gradient operator The specific implementation method is: Input thermal image Perform expansion and corrosion operations respectively, namely: ; in represents the dilation operation, Represents the erosion operation, the structural element It is a 5×5 rectangle to match the typical size of hot spots on building surfaces. The iterative execution process is as follows: Initialize the gradient map ; Carry out the Iteration calculation: ( Indicates taking the minimum value pixel by pixel); When the adjacent iteration results meet Stop iteration when is the L1 norm), then This is the reconstructed gradient map .

[0032] The iterative process is performed by constrained dilation operation on the original image. Reconstruction is performed within the grayscale range of the image, which can eliminate artifacts caused by thermal noise (such as isolated highlights caused by solar reflection) while retaining the edge structure characteristics of the real hot spot.

[0033] In step S203 , a watershed transform is performed on the reconstructed gradient image, the minimum area of ​​the region is set to 50 pixels, and adjacent similar regions are merged.

[0034] S3. Perform watershed transform on the gradient reconstruction image, merge the over-segmented regions, identify surface cracks and peeling areas through a deep learning model based on visible light image data, and generate a texture defect feature map.

[0035] In one possible implementation, Figure 2 As shown in Figure 2, the deep learning model is a two-branch attention network, specifically: S301, branch 1 uses U-Net architecture to extract pixel-level crack features; S302, branch 2 uses the ResNet-50 architecture to extract regional-level spalling features; S303: Weighted fusion of dual-branch outputs through the channel attention module.

[0036] It should be noted that in step S3, the dual-branch attention network is implemented as follows for surface defect recognition: The U-Net architecture encoder in branch one uses a VGG16 backbone, while the decoder fuses shallow detail features via skip connections to output a pixel-level segmentation map of cracks. The ResNet-50 architecture in branch two removes the global pooling layer, retains the spatial feature map, and then outputs a probability map of spalling areas through 3×3 convolution. The channel attention module is implemented as follows: the two-branch feature maps are concatenated and input into the SE-block. Global average pooling is first performed to generate a channel description vector. Channel weights are then learned through two fully connected layers (with a dimensionality reduction ratio of r=16 in the middle layer). Finally, this weighted fusion generates a texture defect feature map.

[0037] S4. Spatially align the 3D point cloud data with the thermal imaging data to calculate the curvature distribution and thermal deformation gradient of the facade structure.

[0038] It should be noted that the spatial registration in step S4 needs to be implemented in stages: S401. First, establish the partial differential equation of heat conduction , where the thermal diffusivity According to the building material database (concrete , brick wall ).

[0039] The finite difference method is used to solve the equation: the facade is discretized into 1cm×1cm grids, the boundary conditions are set to the measured values ​​of the ambient temperature, and the steady-state temperature field is calculated iteratively.

[0040] Furthermore, during the finite difference method solution, the thermal diffusivity of each 1cm×1cm grid node is Assign values ​​based on building material type: If the LiDAR point cloud reflection intensity is > 0.7 (concrete feature), then ; If the reflection intensity is ≤ 0.7 (brick wall characteristics), then .

[0041] Boundary conditions are applied to the facade edge nodes: ; in is the measured value of ambient temperature (unit: ), is the temperature rise caused by solar radiation (measured by a photovoltaic radiometer, unit: ). The steady-state judgment criterion is the root mean square error of the node temperature change between adjacent iteration steps. .

[0042] S402. Using the equation solution as a constraint, optimize the affine transformation matrix of the three-dimensional point cloud and the thermal imaging data, wherein an affine transformation optimization objective function is constructed: ; in is the point cloud coordinate, is the thermal image pixel coordinate, To simulate temperature, To measure the temperature, the rotation matrix is ​​solved by the Levenberg-Marquardt algorithm and translation vectors .

[0043] S5. Fusion of hot spot contour features, texture defect feature maps, and thermal deformation gradients to calculate the hot spot morphological dispersion TSMD and structural risk quantification factor SRQF.

[0044] Furthermore, the hot spot morphology dispersion TSMD is calculated by the following formula: ; in, is the number of contour sampling points in the hot spot area, For the The second-order derivative of the profile curvature at the sampling points, is the local window area centered at the sampling point, is the average area of ​​hot spots, is the area standard deviation, 、 is the normalization coefficient, satisfying .

[0045] It should be noted that the TSMD calculation process is specifically as follows: sampling at equal intervals on a single hot spot profile points, and the second-order derivative of curvature at each point is calculated using Spline fitting curve, local window area Take a circular area with a radius of 5 cm. Normalization coefficient 、 Adaptive adjustment based on hot spot area: When hot spot area > Time , Otherwise, take , .

[0046] Furthermore, the structural risk quantification factor SRQF is calculated by the following formula: ; in, is the thermal gradient amplitude in °C / m, is the local binary pattern texture variance, is the maximum value of the point cloud curvature, 、 is the dimensional balance coefficient.

[0047] It should be noted that the thermal gradient amplitude in SRQF calculation is The Sobel operator is used for calculation, and the template size is set to 7 × 7. The local binary pattern (LBP) adopts the uniform mode, the neighborhood radius R = 3 pixels, and the sampling points P = 24. Take the mean of the variance of the 8×8 sub-block. Dimensional balance coefficient 、 The value rules are: , ,in The function takes the maximum value of the training set.

[0048] It should be noted that the feature fusion and parameter calculation in step S5 are performed according to the following process: Hot spot contour feature extraction uses the Frechet distance to describe contour similarity, and temperature distribution feature calculation is performed using histogram statistics with a 0.5°C interval. The thermal deformation gradient is calculated using the registered point cloud: the change in the angle between the point cloud normal vectors within adjacent 1m×1m grids is used as the local deformation. The fusion process improved by DS evidence theory is as follows: S501. Use the hot spot contour feature, texture defect feature map and thermal deformation gradient as independent evidence sources. Specifically, normalize the three types of features to the [0,1] interval as evidence sources. 、 、 .

[0049] S502. Calculate the conflict degree between evidences using the Josselme distance: ; in, is the focal element correlation matrix. When calculating the Jousellme distance, the focal element correlation matrix Take the diagonal matrix diag(1,0.5,0.5) and set the conflict threshold to 0.3.

[0050] It should be noted that the focal element correlation matrix The construction rule is: diagonal elements Indicates source of evidence The confidence weight of Corresponding to hot spot profile characteristics (high reliability), and Corresponding to texture defect feature map and thermal deformation gradient (medium reliability); non-diagonal elements Indicates that there is no prior correlation between the evidence. The weight distribution is based on the stability analysis of the training set features: the standard deviation of the temperature sensitivity of the hot spot contour feature is , which is lower than the texture feature ( ) and deformation gradient ( ).

[0051] S503: Dynamically adjust the basic probability distribution function according to the conflict degree, and fuse and generate the input feature vector of TSMD and SRQF.

[0052] when When , dynamically adjust the basic probability distribution: let ; in is the mean of the three evidence sources. The fused feature vector is input into the TSMD and SRQF calculation modules.

[0053] S6. Based on the coupling relationship between TSMD and SRQF, a defect risk decision matrix is ​​constructed to output the diagnosis results of facade defect type, location and risk level.

[0054] In one possible implementation, Figure 3 As shown, the defect risk decision matrix is ​​constructed using support vector machine multi-classifier: S601, taking TSMD and SRQF as input feature vectors; S602, mapping to a high-dimensional space using a radial basis kernel function; S603. Solve the optimal classification hyperplane using the structural risk minimization principle.

[0055] It should be noted that when constructing the defect risk decision matrix in step S6, the implementation parameters of the support vector machine SVM (Support Vector Machine) are: the kernel function uses the radial basis function ; bandwidth Optimize through grid search. The structural risk minimization objective function is ; Penalty Factor Take 5.0, the slack variable A 10% misclassification tolerance is allowed. The output layer adopts a one-to-one multi-classification strategy and defines four types of defect decision boundaries: Hollowing (TSMD>0.6 and SRQF>0.8) Structural cracks (TSMD < 0.3 and SRQF > 0.7) Surface cracking (TSMD>0.4 and SRQF<0.3) Spalling (0.3 < TSMD < 0.6 and SRQF > 0.5).

[0056] Furthermore, when the input feature vector falls into the overlapping area of the multi-class defect decision boundaries, a secondary decision is made: 1. Calculate the Mahalanobis distance to the centroids of various types of defects , where , is the covariance matrix; 2. Select the defect category with the minimum DM as the output; 3. If ( is the distance standard deviation of the training set), then mark it as "unknown defect type" and initiate the manual review process.

[0057] Covariance matrix is calculated through the historical data set: , centroid takes the mean value of the feature of the samples of this class.

[0058] S7. Superimpose the diagnosis result on the BIM model to generate a three-dimensional visual defect distribution map.

[0059] It should be noted that in step S7, during the BIM (Building Information Modeling) model integration stage, the diagnosis result is converted into the IFC standard format: the defect location coordinates are mapped to the GlobalId attribute of the BIM component, and the risk level is written into the Pset_RiskAssessment property set. The three-dimensional visualization adopts a color coding scheme: red (risk level I), orange (II), yellow (III), green (risk-free), and the rendering transparency is set to 70% to ensure the readability of the model.

[0060] S8. Generate a maintenance priority report according to the risk level. The report includes the defect type, location coordinates, risk index, and prediction of the thermodynamic evolution trend.

[0061] It should be noted that when generating the maintenance priority report in step S8, the risk index . The prediction of the thermodynamic evolution trend adopts the ARIMA model: the previous 30-minute thermal imaging sequence is used as the input, the autoregressive order , the differencing order , the moving average order , and the temperature change rate in the next 10 minutes is predicted . The maintenance priority is arranged in descending order of RiskIndex, and when RiskIndex > 0.75, it is marked as an urgent disposal item.

[0062] Furthermore, the preprocessing process of the thermal image sequence input: 1. Extract the centroid temperature of each hot spot area , (Time points are spaced 1 minute apart); 2. Yes Perform first-order differences , eliminating non-stationarity; 3. As the ARIMA input series, the model fit is tested by Ljung-Box test (lag = 10, significance level )verify; 4. If the Q statistic If the value is > 0.05, the residual is accepted as white noise and the model is valid; otherwise, increase the difference order. to Refit.

[0063] Prediction results Get the next 10 minutes The linear regression slope of .

[0064] All algorithm modules are implemented in Python 3.8, using PyTorch 1.10 as the deep learning framework. Point cloud processing utilizes the Open3D 0.15 library, and 3D visualization is developed using the Revit API. The data processing server is configured with a minimum of 32 CPU cores and 128GB of RAM, with computational latency limited to 2 seconds per square meter of wall surface.

[0065] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: (1) In this invention, by synchronously collecting and spatially registering infrared thermal imaging, visible light images, and 3D point cloud data, and combining it with the physically constrained heat conduction equation to optimize registration accuracy, positioning errors caused by differences in material thermal properties are eliminated. An improved morphological watershed algorithm is used to suppress thermal noise artifacts and achieve precise segmentation of hot spot boundaries. This solution completely solves the problems of incomplete defect recognition from a single data source, misjudgment due to environmental interference, and registration distortion, significantly improving the reliability of detecting defects such as hollows and cracks.

[0066] (2) In this paper, a dual-parameter model of hot spot morphological dispersion (TSMD) and structural risk quantification factor (SRQF) is proposed, integrating multidimensional features such as profile curvature mutation, thermal gradient amplitude, and texture variance. Based on an improved DS evidence theory, the conflict of evidence is dynamically adjusted, the conflict degree is quantified using the Jouselme distance, and the probability distribution is reconstructed. This design overcomes the problem of a sharp drop in detection rate caused by the lack of a risk quantification model and the conflict of integration, and achieves an objective assessment of the defect risk level.

[0067] (3) In this invention, the diagnostic results are deeply integrated with the Building Information Model (BIM). The spatial location of defects and risk levels are mapped using the IFC standard to generate a three-dimensional visual defect distribution map. The ARIMA model is combined to predict thermodynamic evolution trends, and a maintenance priority strategy is dynamically generated based on the risk index. This system addresses the pain points of difficult positioning and lack of risk prediction in two-dimensional reports, providing full-cycle decision support for preventive maintenance.

[0068] The above content is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0069] There are a few points to note: (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0070] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present invention are exaggerated or reduced, that is, these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element may be "directly on" or "under" the other element or intervening elements may be present.

[0071] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to form new embodiments.

[0072] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An intelligent diagnostic method for building facade defects based on infrared thermal imaging using multimodal fusion, characterized by: include: S1. Synchronously collect infrared thermal imaging data, visible light image data, and 3D point cloud data of the target building facade to construct a multimodal dataset. S2. Segment the hot spot area of ​​the infrared thermal imaging data and extract the hot spot contour features and temperature distribution features; S3, based on visible light image data, identifies surface cracks and peeling areas through a deep learning model and generates a texture defect feature map; S4, spatially registering the 3D point cloud data with the thermal imaging data to calculate the curvature distribution and thermal deformation gradient of the facade structure; S5. Fusion of hot spot contour features, texture defect feature maps, and thermal deformation gradients to calculate the hot spot morphological dispersion TSMD and structural risk quantification factor SRQF. S6. Based on the coupling relationship between TSMD and SRQF, a defect risk decision matrix is ​​constructed to output the diagnosis results of facade defect type, location and risk level.

2. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: Said S5 further comprises: The hot spot morphology dispersion TSMD is calculated by the following formula: ； in, is the number of contour sampling points in the hot spot area, For the The second-order derivative of the profile curvature at the sampling points, is the local window area centered at the sampling point, is the average area of ​​hot spots, is the area standard deviation, 、 is the normalization coefficient, satisfying .

3. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: Said S5 further comprises: The structural risk quantification factor SRQF is calculated by the following formula: ； in, is the thermal gradient amplitude in °C / m, is the local binary pattern texture variance, is the maximum value of the point cloud curvature, 、 is the dimensional balance coefficient.

4. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: The hot spot area segmentation in S2 adopts an improved morphological watershed algorithm, which specifically includes: S201, applying adaptive threshold segmentation to infrared thermal imaging data to generate an initial hot spot mask; S202. Use the morphological gradient reconstruction formula to eliminate noise artifacts: ； in, Representation based on structural elements The morphological reconstruction operator, is the morphological gradient operator; S203: Perform watershed transformation on the gradient reconstruction image to merge the over-segmented regions.

5. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: The feature fusion in S5 adopts an improved form of DS evidence theory: S501, using the hot spot contour feature, texture defect feature map, and thermal deformation gradient as independent evidence sources; S502. Calculate the conflict degree between evidences using the Josselme distance: ； in, is the focal element correlation matrix; S503: Dynamically adjust the basic probability distribution function according to the conflict degree, and fuse and generate the input feature vector of TSMD and SRQF.

6. The method for intelligent diagnosis of building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: The spatial registration of S4 adopts the physical constraint registration method based on the heat conduction equation: S401. Establish the partial differential equation of heat conduction on the facade: ； in, is the thermal diffusivity of the material; S402. Using the equation solution result as a constraint condition, optimize the affine transformation matrix of the three-dimensional point cloud and the thermal imaging data.

7. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: The defect risk decision matrix of S6 is constructed using support vector machine multi-classifier: S601, taking TSMD and SRQF as input feature vectors; S602, mapping to a high-dimensional space using a radial basis kernel function; S603. Solve the optimal classification hyperplane using the structural risk minimization principle.

8. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: The deep learning model in S3 is a dual-branch attention network, including: S301, branch 1 uses U-Net architecture to extract pixel-level crack features; S302, branch 2 uses the ResNet-50 architecture to extract regional-level spalling features; S303: Weighted fusion of dual-branch outputs through the channel attention module.

9. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: Also includes: S7. Superimpose the diagnosis results on the BIM model to generate a three-dimensional visual defect distribution map.

10. The intelligent diagnosis method for building facade defects based on infrared thermal imaging and multimodal fusion according to claim 1 is characterized in that: Also includes: S8. Generate a maintenance priority report based on the risk level. The report includes defect type, location coordinates, risk index, and thermodynamic evolution trend prediction.

Images