Intelligent Diagnosis Method for Building Facade Defects Based on Multimodal Fusion Infrared Thermal Imaging

Abstract

This invention provides an intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging, belonging to the field of building inspection technology. The method includes: simultaneously acquiring infrared thermal images, visible light images, and 3D point cloud data to construct a multimodal dataset; segmenting hot spot regions using an improved morphological watershed algorithm and extracting contour and temperature features; identifying surface cracks and spalling areas using a bi-branch attention network to generate texture defect feature maps; calculating curvature distribution and thermal deformation gradient through spatial registration constrained by the heat conduction equation; fusing multi-source features to calculate the hot spot morphological dispersion degree (TSMD) and structural risk quantification factor (SRQF); constructing a defect risk decision matrix to output defect type, location, and risk level; and overlaying the diagnostic results onto a BIM model to generate a 3D visualization report and predict thermodynamic evolution trends. This invention achieves collaborative analysis of multimodal data, accurately quantifies defect risks, and solves the problems of poor anti-interference, inaccurate segmentation, and large registration errors in traditional methods.

Description

Technical Field

[0001] This invention relates to the field of building inspection technology, and in particular to an intelligent diagnostic method for building facade defects based on multimodal fusion infrared thermal imaging. Background Technology

[0002] Building facade defect detection is a core task for ensuring safe building operation and maintenance. Infrared thermal imaging technology has become the mainstream method due to its non-contact and wide-area coverage capabilities, but the existing technology system has fundamental flaws. Relying solely on infrared thermal imaging data severely limits the dimensions of defect identification. The separation of visible light texture information from thermodynamic features leads to systematic misjudgments. Environmental interference (such as solar reflection and instantaneous temperature changes) creates numerous false hot spots in thermal images. Traditional methods cannot distinguish between real defects and noise signals, resulting in a persistently high false detection rate for hollow areas. The hot spot segmentation process has structural flaws. Adaptive threshold segmentation has poor adaptability to differences in the emissivity of building surface materials, and the segmentation boundary drifts significantly under temperature transient interference. When the morphological watershed algorithm directly processes the original thermal image, gradient noise causes regional over-segmentation, requiring manual merging of fragmented regions, reducing detection efficiency and increasing the subjectivity of the results.

[0003] Defect quantification assessment methods are severely outdated. Existing technologies rely solely on simple parameters such as temperature thresholds or contour area for judgment, failing to establish a correlation model between irregular hot spot morphology (e.g., abrupt changes in contour curvature) and structural risk parameters (e.g., thermal deformation gradient). This results in the inability to quantify the risk levels of defects such as hollow areas and structural cracks, leading to maintenance decisions that depend on experience and pose significant safety hazards. Insufficient multimodal data registration accuracy has become a technical bottleneck. Registration methods based on feature point matching ignore the differences in the thermophysical properties of building materials; the difference in thermal diffusivity between concrete and brick walls is not included in the calculation model. Registration errors are amplified to unacceptable levels in curved facade areas, directly causing distortion in the calculation of thermal deformation gradient.

[0004] The multi-source feature fusion process suffers from theoretical flaws. Traditional weighted averaging or feature concatenation methods cannot handle scenarios with conflicting evidence. When hot spot contour features indicate hollowness while visible light texture features show a normal surface, the evidence conflict exceeds a critical threshold, causing the fusion result to fail and the defect detection rate to plummet. The problem of insufficient adaptability of classification models is particularly prominent. Rule-based threshold classifiers struggle to handle the morphological similarity of cracks and structural fissures, and machine learning models such as support vector machines lack effective adjudication mechanisms in overlapping decision boundaries, resulting in a high false positive rate that directly impacts diagnostic reliability. There are technological gaps in the application layer of detection results. Two-dimensional diagnostic reports cannot be spatially correlated with Building Information Modeling (BIM), and manual defect location is inefficient and prone to significant errors. The lack of thermodynamic evolution trend prediction capabilities makes it impossible to dynamically assess the risk of defect expansion, resulting in a lack of data support for the formulation of preventive maintenance strategies. Summary of the Invention

[0005] To address the technical problems in existing technologies, such as insufficient dimensions and poor anti-interference ability of single infrared data for defect identification, boundary drift and over-segmentation in hot spot segmentation, inaccurate assessment due to lack of risk quantification model, errors caused by failure to consider material thermal properties in multimodal registration, sharp drop in detection rate due to failure of feature fusion to resolve evidence conflicts, high misjudgment rate in overlapping areas of decision boundary of classification model, difficulty in three-dimensional localization of detection results and lack of risk prediction capability, this invention provides an intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging.

[0006] The technical solution provided by this invention is as follows:

[0007] The present invention provides an intelligent diagnostic method for building facade defects based on multimodal fusion infrared thermal imaging, comprising:

[0008] S1. Simultaneously collect infrared thermal imaging data, visible light image data, and 3D point cloud data of the target building facade to construct a multimodal dataset;

[0009] S2. Perform hot spot region segmentation on infrared thermal imaging data and extract hot spot contour features and temperature distribution features;

[0010] S3. Based on visible light image data, identify surface cracks and peeling areas through a deep learning model to generate texture defect feature maps;

[0011] S4. Spatial registration of 3D point cloud data and thermal imaging data, and calculation of curvature distribution and thermal deformation gradient of the facade structure;

[0012] S5. By integrating hot spot contour features, texture defect feature maps and thermal deformation gradients, the hot spot morphology dispersion TSMD and structural risk quantification factor SRQF are calculated.

[0013] S6. Based on the coupling relationship between TSMD and SRQF, construct a defect risk decision matrix and output the diagnostic results of facade defect type, location and risk level.

[0014] Furthermore, S5 further includes:

[0015] The hot spot morphology dispersion TSMD is calculated using the following formula:

[0016] ;

[0017] in, This represents the number of contour sampling points within the hot spot area. For the first Second derivative of the contour curvature at each sampling point The area of ​​the local window centered on the sampling point. The average area of ​​the hot spot. The standard deviation of the area. , Let be the normalization coefficient, satisfying .

[0018] Furthermore, S5 further includes:

[0019] The structural risk quantification factor (SRQF) is calculated using the following formula:

[0020] ;

[0021] in, This represents the thermal gradient amplitude, in °C / m. For local binary pattern texture variance, This represents the maximum curvature of the point cloud. , This is the dimensional balance coefficient.

[0022] Furthermore, the hot spot region segmentation in S2 employs an improved morphological watershed algorithm, specifically including:

[0023] S201. Apply adaptive threshold segmentation to infrared thermal imaging data to generate an initial hot spot mask;

[0024] S202. Eliminate noise artifacts using morphological gradient reconstruction formula:

[0025] ;

[0026] in, Represents based on structural elements Morphological reconstruction operator, For morphological gradient operators;

[0027] S203. Perform watershed transformation on the gradient reconstruction map and merge oversegmented regions.

[0028] Furthermore, the feature fusion in S5 adopts an improved form of the DS evidence theory:

[0029] S501. Hot spot contour features, texture defect feature maps, and thermal deformation gradients are each used as independent sources of evidence.

[0030] S502. Calculate the degree of conflict between pieces of evidence using Jousselme distance:

[0031] ;

[0032] in, The focal element correlation matrix;

[0033] S503. Dynamically adjust the basic probability allocation function according to the degree of conflict, and fuse them to generate the input feature vectors of TSMD and SRQF.

[0034] Furthermore, the spatial registration of S4 adopts a physical constraint registration method based on the heat conduction equation:

[0035] S401. Establish the partial differential equation for heat conduction on the exterior facade:

[0036] ;

[0037] in, The thermal diffusivity of the material;

[0038] S402. Optimize the affine transformation matrix of 3D point cloud and thermal imaging data using the equation solution results as constraints.

[0039] Furthermore, the defect risk decision matrix of S6 is constructed using a support vector machine multi-classifier:

[0040] S601, using TSMD and SRQF as input feature vectors;

[0041] S602, Mapping to a higher-dimensional space using radial basis kernel functions;

[0042] S603. Solve for the optimal classification hyperplane by minimizing structural risk.

[0043] Furthermore, the deep learning model in S3 is a dual-branch attention network, comprising:

[0044] S301, Branch 1 uses the U-Net architecture to extract pixel-level crack features;

[0045] S302 and Branch 2 use the ResNet-50 architecture to extract region-level peeling features;

[0046] S303, weighted fusion of dual-branch outputs through channel attention module.

[0047] Furthermore, S7, the diagnostic results are overlaid onto the BIM model to generate a three-dimensional visualized defect distribution map.

[0048] Furthermore, S8 generates a maintenance priority report based on the risk level. The report includes the defect type, location coordinates, risk index, and thermodynamic evolution trend prediction.

[0049] The beneficial effects of the technical solution provided by this invention include at least the following:

[0050] (1) In this invention, the registration accuracy is optimized by synchronously acquiring and spatially registering infrared thermal imaging, visible light images, and three-dimensional point cloud data, combined with the heat conduction equation constrained by physical constraints, thus eliminating the positioning error caused by differences in material thermal properties; the improved morphological watershed algorithm is used to suppress thermal noise artifacts and achieve accurate segmentation of hot spot boundaries. This scheme completely solves the problems of incomplete defect identification from a single data source, misjudgment due to environmental interference, and registration distortion, and significantly improves the detection reliability of defects such as hollow areas and cracks.

[0051] (2) In this invention, a two-parameter model of hot spot morphology dispersion (TSMD) and structural risk quantification factor (SRQF) is proposed, which integrates multi-dimensional features such as contour curvature abrupt change, thermal gradient amplitude, and texture variance; based on the improved DS evidence theory, evidence conflict is dynamically adjusted, and the conflict degree is quantified and the probability allocation is reconstructed through Jousselme distance. This design overcomes the problem of a sharp drop in detection rate caused by the lack of risk quantification model and fusion conflict, and realizes the objective assessment of defect risk level.

[0052] (3) In this invention, the diagnostic results are deeply integrated with Building Information Modeling (BIM), and the spatial location and risk level of defects are mapped through the IFC standard to generate a three-dimensional visualized defect distribution map; combined with the ARIMA model to predict the thermodynamic evolution trend, a maintenance priority strategy is dynamically generated based on the risk index. This system ends the pain points of the difficulty in locating two-dimensional reports and the lack of risk prediction, and provides full-cycle decision support for preventive maintenance. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart illustrating the intelligent diagnosis method for building facade defects based on multimodal fusion using infrared thermal imaging, provided in an embodiment of the present invention.

[0055] Figure 2 This is a flowchart illustrating the dual-branch attention network in the intelligent diagnosis method for building facade defects based on multimodal fusion using infrared thermal imaging, as provided in an embodiment of the present invention.

[0056] Figure 3 This is a schematic diagram illustrating the process of constructing a defect risk decision matrix in the intelligent diagnosis method for building facade defects based on multimodal fusion infrared thermal imaging provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0058] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0059] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0060] In embodiments of the present invention, sometimes the subscript is as follows: It may be mistakenly written as a non-subscript form such as W1. When the distinction is not emphasized, the meaning they express is the same.

[0061] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0062] Reference manual attached Figure 1 The diagram illustrates a flowchart of an intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging, as provided in an embodiment of the present invention.

[0063] This invention provides an intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging. The processing flow may include the following steps:

[0064] S1. Simultaneously collect infrared thermal imaging data, visible light image data, and 3D point cloud data of the target building facade to construct a multimodal dataset.

[0065] It should be noted that, firstly, step S1 is executed, where multimodal data of the target building's facade is acquired using a synchronous acquisition device equipped with an infrared thermal imager, a high-resolution visible light camera, and a lidar. The infrared thermal imager must be a model with a thermal sensitivity of at least 0.05°C, and the acquisition frequency must be set to 5Hz to ensure dynamic thermal capture capabilities. The visible light camera must have a resolution of at least 20 megapixels and employ a polarizing filter to eliminate reflective interference. Three-dimensional point cloud data is acquired using lidar scanning at a 0.1° angular resolution, with a point cloud density of at least 500 points per square meter. The three types of data are synchronized using a hardware synchronization triggering device to ensure time alignment. The spatial coordinate system is an East-North-Sky coordinate system, ultimately constructing a multimodal dataset containing a thermal radiation matrix, an RGB image matrix, and a point cloud coordinate set.

[0066] S2. Perform hot spot region segmentation on the infrared thermal imaging data and extract hot spot contour features and temperature distribution features.

[0067] It should be noted that when performing hot spot region segmentation in step S2, adaptive threshold segmentation is first applied to the infrared thermal imaging data: the temperature threshold is calculated using the maximum inter-class variance method. , temperature higher ( Pixels with a temperature standard deviation (for the entire facade) are labeled as candidate hotspots, generating an initial binary mask. An improved morphological watershed algorithm is then executed.

[0068] In step S201, an initial hot spot mask is generated by applying adaptive threshold segmentation to the infrared thermal imaging data. Morphological opening operation is performed on the initial mask to eliminate noise. A 3×3 circular template is selected as the structural element.

[0069] Step S202 uses the morphological gradient reconstruction formula to eliminate noise artifacts: First, according to the formula... Construct a gradient reconstruction graph. This includes the morphological gradient operator. The specific implementation method is as follows:

[0070] For input thermal images Perform expansion and erosion operations separately, i.e.:

[0071] ;

[0072] in This indicates the expansion operation. Represents erosion operation, structuring element A 5×5 rectangle is used to match the typical size of hot spots on building surfaces. Reconstruction operator The iterative execution process is as follows:

[0073] Initialize gradient map ;

[0074] Proceed to the first Next iteration calculation: ( (This indicates taking the minimum value pixel by pixel).

[0075] When the results of adjacent iterations satisfy Stop iteration ( (where L1 norm) This is the reconstructed gradient map. .

[0076] This iterative process uses constrained dilation operations on the original image. Reconstruction within the grayscale range can eliminate artifacts caused by thermal noise (such as isolated bright spots caused by solar reflection) while preserving the edge structure features of real hot spots.

[0077] Step S203 performs a watershed transformation on the reconstructed gradient map, sets the minimum area of ​​the region to 50 pixels, and merges adjacent similar regions.

[0078] S3. Perform watershed transformation on the gradient reconstruction map, merge oversegmented regions, and based on visible light image data, identify surface cracks and peeling areas through a deep learning model to generate a texture defect feature map.

[0079] In one possible implementation, such as Figure 2 As shown, the deep learning model is a dual-branch attention network, specifically:

[0080] S301, Branch 1 uses the U-Net architecture to extract pixel-level crack features;

[0081] S302 and Branch 2 use the ResNet-50 architecture to extract region-level peeling features;

[0082] S303, weighted fusion of dual-branch outputs through channel attention module.

[0083] It should be noted that the specific implementation of the dual-branch attention network in step S3 for surface defect recognition is as follows: Branch 1 uses a U-Net architecture encoder with a VGG16 backbone, and the decoder fuses shallow detail features through skip connections to output a pixel-level crack segmentation map. Branch 2 uses a ResNet-50 architecture that removes the global pooling layer, retains the spatial feature map, and outputs a probability map of the peeling region after 3×3 convolution. The channel attention module is implemented as follows: the feature maps from both branches are concatenated and input into the SE-block. First, global average pooling is used to generate channel description vectors, then two fully connected layers (with an intermediate layer dimensionality compression rate r=16) are used to learn channel weights, and finally, weighted fusion is performed to generate a texture defect feature map.

[0084] S4. Spatial registration of 3D point cloud data and thermal imaging data is performed to calculate the curvature distribution and thermal deformation gradient of the facade structure.

[0085] It should be noted that the spatial registration in step S4 needs to be implemented in stages:

[0086] S401. First, establish the partial differential equation for heat conduction. thermal diffusivity Values ​​were taken from the building materials database (concrete). brick wall ).

[0087] The equations are solved using the finite difference method: the facade is discretized into a 1cm×1cm grid, the boundary conditions are set to the measured ambient temperature, and the steady-state temperature field is calculated iteratively.

[0088] Furthermore, in the finite difference method solution process, the thermal diffusivity of each 1cm×1cm grid node is... Assign values ​​based on building material type:

[0089] If the lidar point cloud reflection intensity > 0.7 (concrete characteristics), then ;

[0090] If the reflection intensity is ≤0.7 (characteristic of brick wall), then .

[0091] Boundary conditions are applied to the edge nodes of the facade: Let

[0092] ;

[0093] in Measured ambient temperature (unit: ), Temperature rise caused by solar radiation (measured by a photovoltaic radiometer, unit: The steady-state criterion is the root mean square error of the node temperature change between adjacent iteration steps. .

[0094] S402. Using the equation solution results as constraints, optimize the affine transformation matrix of the 3D point cloud and thermal imaging data, where the objective function for affine transformation optimization is constructed as follows:

[0095] ;

[0096] in For point cloud coordinates, These are the pixel coordinates of the thermal image. To simulate temperature, To obtain the measured temperature, the rotation matrix was solved using the Levenberg-Marquardt algorithm. Translation vector .

[0097] S5. By integrating hot spot contour features, texture defect feature maps, and thermal deformation gradients, the hot spot morphology dispersion (TSMD) and structural risk quantification factor (SRQF) are calculated.

[0098] Furthermore, the hot spot morphology dispersion TSMD is calculated using the following formula:

[0099] ;

[0100] in, This represents the number of contour sampling points within the hot spot area. For the first Second derivative of the contour curvature at each sampling point The area of ​​the local window centered on the sampling point. The average area of ​​the hot spot. The standard deviation of the area. , Let be the normalization coefficient, satisfying .

[0101] It should be noted that the TSMD calculation process is specifically defined as: sampling at equal intervals on the profile of a single hot spot. When calculating the second derivative of curvature at each point, the following method is used: spline fitting curve, local window area Consider a circular region with a radius of 5cm. Normalization coefficient. , Adaptive adjustment based on hotspot area: when hotspot area > Time to take , Otherwise take , .

[0102] Furthermore, the structural risk quantification factor (SRQF) is calculated using the following formula:

[0103] ;

[0104] in, This represents the thermal gradient amplitude, in °C / m. For local binary pattern texture variance, This represents the maximum curvature of the point cloud. , This is the dimensional balance coefficient.

[0105] It should be noted that the thermal gradient magnitude in SRQF calculation The template size is set to 7×7 and calculated using the Sobel operator. The Local Binary Pattern (LBP) adopts a uniform pattern with a neighborhood radius of R=3 pixels and a sampling point of P=24. Take the mean variance of the 8×8 sub-blocks. Dimensional balance coefficient. , The rules for determining the value are as follows: , ,in The function takes the maximum value from the training set.

[0106] It should be noted that the feature fusion and parameter calculation in step S5 are implemented according to the following process: Hot spot contour feature extraction uses Frechet distance to describe contour similarity; temperature distribution feature calculation uses histogram statistics at 0.5°C intervals. Thermal deformation gradient is calculated using the registered point cloud: the change in the angle between the normal vectors of adjacent 1m×1m grids is used as the local deformation. The DS evidence theory-improved fusion process is as follows:

[0107] S501. Hot spot contour features, texture defect feature maps, and thermal deformation gradients are used as independent sources of evidence. Specifically, the three types of features are normalized to the [0,1] interval as sources of evidence. , , .

[0108] S502. Calculate the degree of conflict between pieces of evidence using Jousselme distance:

[0109] ;

[0110] in, This is the focal element correlation matrix. When calculating the Jousselme distance, the focal element correlation matrix is... Take the diagonal matrix diag(1,0.5,0.5) and set the conflict threshold to 0.3.

[0111] It should be noted that the focal element correlation matrix The construction rule is: diagonal elements Indicate the source of evidence The confidence weights, where Corresponding hot spot contour features (high reliability). and Corresponding to texture defect feature maps and thermal deformation gradients (medium reliability); off-diagonal elements This indicates that there is no prior correlation between the evidence. The weighting is based on the stability analysis of the training set features: the standard deviation of the temperature sensitivity of the hot spot contour features is... Lower than texture features ( ) and deformation gradient ( ).

[0112] S503. Dynamically adjust the basic probability allocation function according to the degree of conflict, and fuse them to generate the input feature vectors of TSMD and SRQF.

[0113] when At that time, dynamically adjust the basic probability allocation: Let

[0114] ;

[0115] in The sum of the three evidence sources is used as the mean. The fused feature vector is then input into the TSMD and SRQF calculation modules.

[0116] S6. Based on the coupling relationship between TSMD and SRQF, construct a defect risk decision matrix and output the diagnostic results of facade defect type, location and risk level.

[0117] In one possible implementation, such as Figure 3 As shown, the defect risk decision matrix is ​​constructed using a support vector machine multi-classifier:

[0118] S601, using TSMD and SRQF as input feature vectors;

[0119] S602, Mapping to a higher-dimensional space using radial basis kernel functions;

[0120] S603. Solve for the optimal classification hyperplane by minimizing structural risk.

[0121] It should be noted that when constructing the defect risk decision matrix in step S6, the implementation parameters of the Support Vector Machine (SVM) are as follows: the kernel function is selected as the radial basis function.

[0122] ;

[0123] bandwidth Optimization is achieved through grid search. The objective function for minimizing structural risk is:

[0124] ;

[0125] Punishment factor Take 5.0, slack variable A 10% misclassification tolerance is allowed. The output layer employs a one-to-one multi-classification strategy, defining four types of defect decision boundaries:

[0126] Hollow drum (TSMD>0.6 and SRQF>0.8)

[0127] Structural cracks (TSMD<0.3 and SRQF>0.7)

[0128] Surface cracking (TSMD > 0.4 and SRQF < 0.3)

[0129] Spalling (0.3 < TSMD < 0.6 and SRQF > 0.5).

[0130] Furthermore, when the input feature vector falls into the overlapping area of multi-class defect decision boundaries, a secondary decision is executed:

[0131] 1. Calculate the Mahalanobis distance to the centroid of each type of defect , where , is the covariance matrix;

[0132] 2. Select the defect class with the minimum DM as the output;

[0133] 3. If ( is the distance standard deviation of the training set), then mark it as "unknown defect type" and initiate the manual review process.

[0134] Covariance matrix is calculated through the historical data set: , centroid takes the mean of the feature of this class of samples.

[0135] S7. Overlay the diagnostic results on the BIM model to generate a three-dimensional visual defect distribution map.

[0136] It should be noted that in step S7 during the BIM (Building Information Modeling) model integration stage, the diagnostic results are converted into the IFC standard format: the defect location coordinates are mapped to the GlobalId attribute of the BIM component, and the risk level is written into the Pset_RiskAssessment property set. The three-dimensional visualization adopts a color coding scheme: red (risk level I), orange (II), yellow (III), green (no risk), and the rendering transparency is set to 70% to ensure the readability of the model.

[0137] S8. Generate a maintenance priority report based on the risk level, and the report includes the defect type, location coordinates, risk index, and prediction of the thermodynamic evolution trend.

[0138] It should be noted that when generating the maintenance priority report in step S8, the risk index . The prediction of the thermodynamic evolution trend adopts the ARIMA model: taking the previous 30-minute thermal imaging sequence as the input, the autoregressive order , the differencing order , the moving average order , predicting the temperature change rate in the next 10 minutes Repair priorities are sorted in descending order of RiskIndex, and items with RiskIndex > 0.75 are marked as emergency items.

[0139] Further, the preprocessing flow for inputting thermal image sequences:

[0140] 1. Extract the centroid temperature of each hot spot region. , (Time interval 1 minute);

[0141] 2. Regarding Perform first-order difference Eliminate nonstationarity;

[0142] 3. with As the input sequence for ARIMA, the model goodness of fit was assessed using the Ljung-Box test (lag=10, significance level). )verify;

[0143] 4. If the Q statistic If the value is >0.05, the residuals are accepted as white noise, and the model is valid; otherwise, the difference order is increased. to Refit.

[0144] Prediction results Get the next 10 minutes The slope of the linear regression.

[0145] All algorithm modules are implemented using Python 3.8, the deep learning framework uses PyTorch 1.10, point cloud processing uses the Open3D 0.15 library, and 3D visualization relies on secondary development based on the Revit API. The data processing server is configured with at least 32 CPU cores / 128GB RAM, and the computation latency is controlled to within 2 seconds per square meter of wall surface.

[0146] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0147] (1) In this invention, the registration accuracy is optimized by synchronously acquiring and spatially registering infrared thermal imaging, visible light images, and three-dimensional point cloud data, combined with the heat conduction equation constrained by physical constraints, thus eliminating the positioning error caused by differences in material thermal properties; the improved morphological watershed algorithm is used to suppress thermal noise artifacts and achieve accurate segmentation of hot spot boundaries. This scheme completely solves the problems of incomplete defect identification from a single data source, misjudgment due to environmental interference, and registration distortion, and significantly improves the detection reliability of defects such as hollow areas and cracks.

[0148] (2) In this invention, a two-parameter model of hot spot morphology dispersion (TSMD) and structural risk quantification factor (SRQF) is proposed, which integrates multi-dimensional features such as contour curvature abrupt change, thermal gradient amplitude, and texture variance; based on the improved DS evidence theory, evidence conflict is dynamically adjusted, and the conflict degree is quantified and the probability allocation is reconstructed through Jousselme distance. This design overcomes the problem of a sharp drop in detection rate caused by the lack of risk quantification model and fusion conflict, and realizes the objective assessment of defect risk level.

[0149] (3) In this invention, the diagnostic results are deeply integrated with Building Information Modeling (BIM), and the spatial location and risk level of defects are mapped through the IFC standard to generate a three-dimensional visualized defect distribution map; combined with the ARIMA model to predict the thermodynamic evolution trend, a maintenance priority strategy is dynamically generated based on the risk index. This system ends the pain points of the difficulty in locating two-dimensional reports and the lack of risk prediction, and provides full-cycle decision support for preventive maintenance.

[0150] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0151] The following points need to be explained:

[0152] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0153] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0154] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0155] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent diagnosis of building facade defects based on multimodal fusion infrared thermal imaging, characterized in that, include: S1. Simultaneously collect infrared thermal imaging data, visible light image data, and 3D point cloud data of the target building facade to construct a multimodal dataset; S2. Perform hot spot region segmentation on infrared thermal imaging data and extract hot spot contour features and temperature distribution features; S3. Based on visible light image data, a deep learning model is used to identify surface cracks and peeling areas, and generate texture defect feature maps. S4. Spatial registration of 3D point cloud data and thermal imaging data, and calculation of curvature distribution and thermal deformation gradient of the facade structure; S5. By integrating hot spot contour features, texture defect feature maps and thermal deformation gradients, the hot spot morphology dispersion (TSMD) and structural risk quantification factor (SRQF) are calculated. S6. Based on the coupling relationship between TSMD and SRQF, construct a defect risk decision matrix and output the diagnostic results of the type, location and risk level of facade defects.

2. The intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging as described in claim 1, characterized in that, S5 further includes: The hot spot morphology dispersion TSMD is calculated using the following formula: in, This represents the number of contour sampling points within the hotspot area. For the first Second derivative of the contour curvature at each sampling point The area of ​​the local window centered on the sampling point. The average area of ​​the hot spot. The standard deviation of the area. Let be the normalization coefficient, satisfying .

3. The intelligent diagnosis method for building facade defects based on multimodal fusion using infrared thermal imaging as described in claim 1, characterized in that, S5 further includes: The structural risk quantification factor (SRQF) is calculated using the following formula: in, This represents the thermal gradient amplitude, in °C / m. For local binary pattern texture variance, This represents the maximum curvature of the point cloud. This is the dimensional balance coefficient.

4. The intelligent diagnosis method for building facade defects based on multimodal fusion using infrared thermal imaging according to claim 1, characterized in that, The hot spot region segmentation in S2 employs an improved morphological watershed algorithm, specifically including: S201. Apply adaptive threshold segmentation to infrared thermal imaging data to generate an initial hot spot mask; S202. Eliminate noise artifacts using morphological gradient reconstruction formula: in, Represents based on structural elements Morphological reconstruction operator, For morphological gradient operators, Input thermal image; S203. Perform watershed transformation on the gradient reconstruction map and merge oversegmented regions.

5. The intelligent diagnosis method for building facade defects based on multimodal fusion using infrared thermal imaging according to claim 1, characterized in that, The feature fusion in S5 adopts an improved form of DS evidence theory: S501. Hot spot contour features, texture defect feature maps, and thermal deformation gradients are each used as independent sources of evidence. S502, Use Jousselme distance to calculate the degree of conflict between pieces of evidence: in, The focal element correlation matrix; S503. Dynamically adjust the basic probability allocation function according to the degree of conflict, and fuse them to generate the input feature vectors of TSMD and SRQF.

6. The intelligent diagnosis method for building facade defects based on multimodal fusion using infrared thermal imaging according to claim 1, characterized in that, The spatial registration of S4 adopts a physical constraint registration method based on the heat conduction equation: S401. Establish the partial differential equation for heat conduction on the exterior facade. in, The thermal diffusivity of the material; S402. Optimize the affine transformation matrix of 3D point cloud and thermal imaging data using the equation solution results as constraints.

7. The intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging according to claim 1, characterized in that, The defect risk decision matrix of S6 is constructed using a support vector machine multi-classifier: S601, using TSMD and SRQF as input feature vectors; S602, Mapping to a higher-dimensional space using radial basis kernel functions; S603. Solve for the optimal classification hyperplane by minimizing structural risk.

8. The intelligent diagnosis method for building facade defects based on multimodal fusion infrared thermal imaging according to claim 1, characterized in that, The deep learning model in S3 is a dual-branch attention network, including: S301, Branch 1 uses the U-Net architecture to extract pixel-level crack features; S302 and Branch 2 use the ResNet-50 architecture to extract region-level peeling features; S303, weighted fusion of dual-branch outputs through channel attention module.

9. The intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging according to claim 1, characterized in that, Also includes: S7. Overlay the diagnostic results onto the BIM model to generate a three-dimensional visualization of the defect distribution map.

10. The intelligent diagnostic method for building facade defects based on multimodal fusion using infrared thermal imaging according to claim 1, characterized in that, Also includes: S8. Generate a maintenance priority report based on the risk level. The report includes the defect type, location coordinates, risk index, and thermodynamic evolution trend prediction.

Images