Building envelope local multi-type damage fine identification system

Abstract

The application relates to the field of building engineering detection, and discloses a building envelope local area multi-type damage fine identification system, which comprises the following steps: constructing standardized test pieces covering different materials, thicknesses and defect types, synchronously collecting spectra by using a hyperspectral camera, a long-wave infrared camera and a controllable heat source to generate an original data set; correcting, bad point repairing and radiation calibrating are performed to form a standardized cross-modal data set; temperature differences, temperature change rates, peak times and frequency domain features are extracted, a defect area is highlighted through background-observation difference, and an interpretable defect feature map and a thermal response parameter are generated; the defect feature map and the thermal response parameter are combined with the standardized cross-modal data set and sent into a three-dimensional Transformer multi-task network to generate a cross-modal model; after performance verification, compression and optimization are carried out based on a deep partition decoupling strategy to generate a compressed cross-modal model. The application has the advantages of improving identification precision and intelligent diagnosis capability.

Description

Technical Field

[0001] This invention relates to the field of building engineering inspection, specifically a refined identification system for multiple types of damage in the local area of ​​building envelope structures. Background Technology

[0002] Building envelopes are subject to long-term exposure to alternating temperatures, humidity fluctuations, and external forces, making them prone to hidden damage such as hollowing, cracking, delamination, and localized moisture absorption. If these damages are not detected in time, they will lead to a decline in thermal insulation performance and risks to durability. Firstly, the detection methods mainly rely on manual tapping, RGB images, and ordinary infrared imaging. However, RGB images can only reflect the surface condition and are difficult to detect internal defects. Traditional infrared detection is affected by the small difference in thermal conductivity of materials and environmental noise, and the temperature difference signal is prone to fluctuation. Small-sized or deep defects often cannot be effectively identified. Secondly, infrared detection typically employs a single heating or cooling method, failing to utilize the key principles of different heating durations and temperature change rates to understand the differences in thermal response between shallow and deep defects. Furthermore, indoor experiments suffer from uneven heating and reflection interference, making it difficult to stably reproduce temperature gradients. Consequently, the method lacks adaptability to different materials and structures of varying thicknesses. Thirdly, in terms of data processing, key time-series information such as temperature change rate, time gradient, and frequency domain features are not fully utilized, and deep learning models are mostly limited to defect area identification, with limited quantitative prediction ability for defect depth and insufficient generalization. In summary, both infrared and image-based detection methods for building envelopes suffer from drawbacks such as high defect invisibility, unstable parameters, difficulty in quantifying depth, weak model generalization, and significant differences between experimental and actual working conditions. These limitations make it difficult to meet the engineering requirements for high-precision, multi-type, and quantifiable damage identification. Therefore, it is essential to design a refined identification system for localized multi-type damage in building envelopes that improves identification accuracy and intelligent diagnostic capabilities. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a refined identification system for multiple types of damage in the local area of ​​building envelope structures, which has the advantages of improved identification accuracy and intelligent diagnostic capabilities, and solves the problems mentioned in the background technology.

[0004] To achieve the aforementioned goals of improving recognition accuracy and intelligent diagnostic capabilities, this invention provides the following technical solution: a refined recognition system for multiple types of damage in a localized area of ​​a building envelope, comprising: Specimen Acquisition Module: Constructs standardized specimens covering different materials, thicknesses, and defect types. Simultaneously acquires spectra using a hyperspectral camera, a long-wave infrared camera, and a controllable heat source. Through multi-rate heating-cooling and natural sunlight, it generates a multi-excitation, multi-modal, labeled raw data set. Data processing module: performs correction, bad pixel repair and radiometric calibration based on the original dataset, adaptively selects key spectral bands, and processes infrared time series data to form a standardized cross-modal dataset; Feature modeling module: Based on standardized cross-modal datasets, extract temperature difference, temperature change rate, peak time and frequency domain features, combine thermal response evolution mode to construct background thermal field inversion, and highlight defect areas through background-observation difference to generate interpretable defect feature maps and thermal response parameters; Multimodal network module: It feeds the defect feature map and thermal response parameters together with the standardized cross-modal dataset into the 3D Transformer multi-task network, and generates a cross-modal model by fusing spectral and thermal field features through cross-modal attention; Compression Deployment Module: After performance verification of the cross-modal model, compression and optimization are performed based on a deep partitioning decoupling strategy, and incremental updates are triggered by backhauling runtime data to generate the compressed cross-modal model.

[0005] Preferably, the process of generating a multi-stimulus, multi-modal, labeled raw dataset is as follows: The acquisition strategy was set according to the spectral characteristics and thermophysical parameters of the specimen material, and a multi-angle, multi-band acquisition scheme was adopted to obtain high-resolution spectral data. Micro-area resolution thermal excitation gradient scanning was performed on infrared images to obtain multi-temporal thermal response data under different heating-cooling rates and natural sunlight conditions. During the acquisition process, the geometric information of the micro-regions of the specimen is recorded simultaneously, forming a raw data set with unified spatial-temporal-modal annotation.

[0006] Preferably, the process of correction, dead pixel repair, and radiation calibration based on the original data is as follows: Based on the original dataset, dark current correction, whiteboard correction and spectral drift correction are performed on the hyperspectral image data, and bad pixels are repaired by combining an adaptive abnormal pixel detection algorithm. Radiometric calibration, dynamic noise suppression, and non-uniformity correction are performed on infrared temporal images; A multi-scale spatial alignment method is used to map single-modal data to a unified micro-grid, generating a single-modal standardized dataset.

[0007] Preferably, based on a single-mode standardized dataset, statistical significance and multidimensional correlation indicators are used to identify the bands most sensitive to local material damage, and spectral noise and interference are weighted and suppressed. By combining the historical defect feature database, the importance of each micro-region band is evaluated with multiple objectives, and a genetic algorithm is used to dynamically select the key spectral bands that best reflect the micro-region defect characteristics. The selection process simultaneously considers band redundancy, spatial consistency, and cross-modal complementarity to form a subset of spectral features.

[0008] Preferably, the process of forming a standardized cross-modal dataset is as follows: The infrared thermal response sequence is time-normalized and micro-area aligned, and multi-scale local weighted filtering and adaptive noise suppression algorithm are used to eliminate sensing noise and environmental interference. By combining a subset of spectral bands, micro-region feature tensor quantization is performed on the spectral data, and the spectral data and infrared thermal response data are precisely registered on the micro-region grid. We use the temporal peak value, slope, and frequency domain characteristics of the thermal response curve to perform weighted fusion with the correlation of the spectral signal; The fused data is compensated and standardized by multimodal interpolation and local regression correction methods to generate a standardized cross-modal dataset containing spectral-thermal time series information, micro-region geometric coordinates, thermal response evolution parameters, and feature weight matrices.

[0009] Preferably, the process of extracting temperature difference, temperature change rate, peak time, and frequency domain features is as follows: Based on a standardized cross-modal dataset, high-order difference features are extracted from infrared thermal response sequences to obtain transient temperature difference and temperature change rate. By combining a subset of spectral features fused across modalities, frequency domain features are extracted using fast Fourier transform and continuous wavelet transform to form a multi-scale time-frequency joint feature vector. By integrating spectral-thermal response information through micro-region weighted fusion, local high-dimensional defect features are generated.

[0010] Preferably, the process for generating interpretable defect feature maps and thermal response parameters is as follows: Based on the local high-dimensional defect characteristics, the background thermal field is fitted on the micro-region grid points using the historical thermal response evolution mode. The observed thermal field and the fitted background thermal field are subjected to micro-region difference to form a local anomaly distribution matrix, and weighted anomaly enhancement is performed by combining high-dimensional defect features. By highlighting potential defect areas through multi-layer adaptive filtering, the difference matrix is ​​mapped to micro-area grid points to generate interpretable defect feature maps, while thermal response parameters are extracted.

[0011] Preferably, the process of generating a cross-modal model is as follows: The defect feature map, thermal response parameters and key spectral band data are tensorized and embedded into the three-dimensional input space; Joint modeling of spatial-temporal-modal features is achieved through a 3D Transformer multi-task network. A cross-modal attention mechanism is used to dynamically adjust the weights of features from different modalities to generate a cross-modal model that represents multiple types of damage in micro-regions.

[0012] Preferably, the compression and optimization process based on the deep partitioning decoupling strategy is as follows: Micro-area accuracy assessment and error analysis were performed on cross-modal models to quantify the contribution of different spatial-temporal-modal features in micro-area damage identification. Based on the deep partitioning decoupling strategy, the model is divided into multi-level partitioning modules. Hierarchical pruning is performed on low-contribution branches and redundant parameter blocks, and parameter compression is performed in combination with quantization and sparsification methods. During the compression process, the integrity of key micro-region features and the absence of loss of multimodal correlation information are ensured through a micro-region sensitivity preservation mechanism and a feature response weighting strategy, resulting in a cross-modal model that is both lightweight and has high micro-region feature fidelity.

[0013] Preferably, the process of generating the compressed cross-modal model is as follows: In actual monitoring, newly acquired multimodal data are input into a cross-modal model that combines lightweight design with high micro-area feature fidelity. Error analysis is performed on the model prediction output and observed spectral-thermal response data for each micro-area grid point to calculate the local residual matrix and identify potential abnormal feature regions. For the compressed model parameters, the key micro-region feature weights are updated by combining micro-region residuals and cross-modal feature mutual information, while the low contribution parameter blocks are constrained to prevent overfitting. During the incremental update process, the modal fusion coefficients and temporal weights are dynamically optimized to generate a compressed cross-modal model that has undergone micro-region-level adaptive adjustment.

[0014] Compared with existing technologies, the present invention provides a refined identification system for multiple types of local damage to building envelope structures, which has the following advantages: This invention achieves simultaneous acquisition of hyperspectral and infrared multimodal data by constructing standardized specimens covering different materials, thicknesses, and defect types. It also combines multi-rate heating-cooling and natural sunlight to obtain rich thermal response information, thereby generating a high-resolution, multi-excitation, labeled raw dataset. During data processing, spectral and infrared data undergo correction, bad pixel repair, radiometric calibration, and adaptive selection of key bands. Simultaneously, a standardized cross-modal dataset is formed, ensuring accurate registration and information consistency of different modal data at the micro-area level. Based on the standardized cross-modal dataset, temperature difference, temperature change rate, peak time, and frequency domain features are extracted. Background thermal field inversion is performed using thermal response evolution models, and potential defect areas are highlighted through background-observation difference analysis, resulting in interpretable defect feature maps and thermal response parameters. This method inputs multimodal features into a 3D Transformer multi-task network and utilizes a cross-modal attention mechanism to achieve deep fusion of spectral and thermal field features, constructing a high-precision cross-modal damage identification model. Finally, the model is compressed and optimized through a deep partitioning decoupling strategy, and incremental updates are triggered by backhauling running data to ensure that the model is efficient, lightweight, preserves micro-area-level features, and has long-term stable adaptability in actual monitoring environments. It integrates the advantages of the entire process of multimodal data acquisition, standardization processing, micro-area feature modeling, deep cross-modal modeling, and lightweight incremental optimization, significantly improving the identification accuracy, quantification capability, and practicality of various types of local damage to building envelope structures. It can effectively solve the problems of invisible defects, difficulty in deep quantification, weak model generalization, and poor environmental adaptability in existing methods. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1: Please refer to Figure 1 As shown in the embodiment of the present invention, the refined identification system for localized multi-type damage to building envelope structures includes: Specimen Acquisition Module: Constructs standardized specimens covering different materials, thicknesses, and defect types. Simultaneously acquires spectra using a hyperspectral camera, a long-wave infrared camera, and a controllable heat source. Through multi-rate heating-cooling and natural sunlight, it generates a multi-excitation, multi-modal, labeled raw data set.

[0018] The process of generating a multi-excitation, multi-modal, labeled raw data set in the specimen acquisition module is as follows: The acquisition strategy was set according to the spectral characteristics and thermophysical parameters of the specimen material, and a multi-angle, multi-band acquisition scheme was adopted to obtain high-resolution spectral data. Based on the material spectral characteristics and thermophysical parameters of the specimen, the spectral acquisition bands and resolutions are preset. The specimen is fixed on a rotatable bracket, and data is acquired in sequence according to the set angle. A high-resolution spectrophotometer is used to record the reflectance or transmittance data of different bands in sequence. The spatial coordinates of each acquisition point are recorded during acquisition, and the acquisition sequence is marked in chronological order to form a preliminary spectral data matrix.

[0019] Micro-area resolution thermal excitation gradient scanning was performed on infrared images to obtain multi-temporal thermal response data under different heating-cooling rates and natural sunlight conditions. The infrared camera is aimed at the surface of the specimen, the micro-area resolution sampling range and pixel spacing are set, and different heating-cooling rate programs are executed under the control of the heating device to scan the surface of the specimen in sections and record the infrared image sequence of temperature change over time. Then the acquisition process is repeated under natural sunlight conditions to record the ambient temperature and light intensity information, and the micro-area coordinates of each pixel are marked simultaneously.

[0020] During the acquisition process, the geometric information of the micro-region of the specimen is recorded simultaneously to form a raw data set with unified spatial-temporal-modal annotation; During the data acquisition process, the geometric information of the micro-regions on the surface of the specimen is measured, including the surface contour, thickness distribution, and local concave and convex features. The coordinates of the micro-regions are obtained using 3D scanning or high-precision displacement sensors, and they are matched one-to-one with the spectral and infrared thermal response data to generate a data structure with unified spatial-temporal-modal annotation. The multi-angle, multi-band spectral data, infrared thermal response sequences, and micro-region geometric annotations are integrated into the original data set according to a unified data format.

[0021] Data processing module: Based on the original dataset, it performs correction, bad pixel repair and radiometric calibration, and adaptively selects key spectral bands. At the same time, it processes infrared time series data to form a standardized cross-modal dataset.

[0022] The data processing module performs correction, dead pixel repair, and radiation calibration based on the raw data as follows: Based on the original dataset, dark current correction, whiteboard correction and spectral drift correction are performed on the hyperspectral image data, and bad pixels are repaired by combining an adaptive abnormal pixel detection algorithm. For the raw hyperspectral image data, dark current correction is performed. Dark field images are acquired in an unexposed state of the camera, the dark current value of each pixel is calculated, and it is subtracted from the original spectral signal of the corresponding pixel. Then, whiteboard correction is performed. Reference images are acquired using a standard whiteboard with known reflectance, and the pixel value of each band is divided by the whiteboard response to obtain the relative reflectance data. Next, spectral drift correction is performed. The center wavelength drift of each band is calculated using pre-acquired spectral calibration lines, and the image data is interpolated and corrected. For any bad or abnormal pixels, an adaptive abnormal pixel detection algorithm is used to scan the spatial neighborhood of each band, and the detected bad pixel pixel value is replaced with the weighted average of the neighboring pixels to complete the bad pixel repair.

[0023] Radiometric calibration, dynamic noise suppression, and non-uniformity correction are performed on infrared temporal images; For infrared time-series images, radiometric calibration is first performed. Reference images are acquired using a calibrated blackbody or standard radiation source, the mapping relationship between pixels and actual radiation intensity is calculated, and applied to each time frame of the infrared image. Then, dynamic noise suppression is performed. Moving average or median filtering is applied to each pixel in the time series to reduce random noise interference in time. Next, non-uniformity correction is performed. Spatial deviation of pixel response is calculated using a calibration board or smoothing template, and the deviation value is subtracted from the image to make the same radiation intensity uniformly displayed across the entire pixel array.

[0024] A multi-scale spatial alignment method is used to map single-modal data to a unified micro-grid, generating a single-modal standardized dataset. A multi-scale spatial alignment method is adopted to map the corrected hyperspectral image and infrared time-series image to a unified micro-region grid. A micro-region grid is defined in each modality. The original image data is resampled to the grid nodes according to the pixel coordinates and geometric calibration information. Spatial registration is performed between the multiple modalities. Consensus feature points or geometric control points are selected. The infrared image is aligned to the grid position of the hyperspectral image through affine or nonlinear transformation. Finally, a single-modality standardized dataset is generated.

[0025] The adaptive selection process for key spectral bands in the data processing module is as follows: Based on a single-mode standardized dataset, statistical significance and multidimensional correlation indicators are used to identify the bands most sensitive to local material damage, and spectral noise and interference are weighted and suppressed. For each microband data in the single-mode standardized dataset, the mean difference, variance, and signal-to-noise ratio between the locally damaged and undamaged regions are calculated to assess the response strength of the band to damage characteristics. Multidimensional correlation indices between each band and other bands are calculated, including Pearson correlation coefficient or mutual information. For bands with high noise or interference, weighting factors are assigned based on noise amplitude and inter-band correlation to reduce the influence of noisy bands in the selection.

[0026] By combining the historical defect feature database, the importance of each micro-region band is evaluated with multiple objectives, and a genetic algorithm is used to dynamically select the key spectral bands that best reflect the micro-region defect characteristics. The response patterns of each micro-region band are compared with those in the historical defect feature database. By calculating the similarity between the band response and known defect features, the sensitivity score of each band in identifying different types of defects is obtained. Combining multiple evaluation indicators, each band is scored for multiple objectives. The scoring results are stored as a band weight table. Using the band weight table as the initial population, each chromosome is represented as a candidate band combination in the genetic algorithm. Through crossover, mutation, and selection operations, the fitness is calculated based on the total score and redundancy of the bands in the combination. The algorithm is iteratively updated for several generations to gradually select the band combination that best reflects the characteristics of micro-region defects and has the least redundancy.

[0027] The selection process simultaneously considers band redundancy, spatial consistency, and cross-modal complementarity to form a subset of spectral features; For the candidate band combinations selected by the genetic algorithm, the consistency of the response of each band in adjacent micro-regions is checked in space, and local abnormal or discontinuous bands are eliminated. At the same time, the band complementarity index, such as the synergistic information gain of spectral and infrared thermal response, is calculated in the multimodal data, and bands with high cross-modal complementarity are retained to form the final spectral feature subset.

[0028] The process of forming a standardized cross-modal dataset in the data processing module is as follows: The infrared thermal response sequence is time-normalized and micro-area aligned, and multi-scale local weighted filtering and adaptive noise suppression algorithm are used to eliminate sensing noise and environmental interference. The acquired infrared thermal response sequence was normalized according to a unified time axis. At the micro-area scale, the infrared image was divided into blocks, and a local weighted filtering algorithm was used to smooth the pixel values ​​within the micro-area. At the same time, adaptive noise suppression was performed based on the local noise amplitude to remove high-frequency noise points caused by sensor drift or environmental changes. Micro-area alignment was achieved by calculating the center coordinates of the micro-area and geometrically mapping the corresponding pixels.

[0029] By combining a subset of spectral bands, micro-region feature tensor quantization is performed on the spectral data, and the spectral data and infrared thermal response data are precisely registered on the micro-region grid. Based on the spectral band subset, the hyperspectral image is subjected to feature tensor quantization on the micro-region grid. That is, the spectral values ​​of each micro-region in each band are arranged in a fixed order to form a tensor structure. Combined with the geometric coordinates of the micro-region, the spectral tensor is precisely registered with the corresponding infrared thermal response micro-region.

[0030] We use the temporal peak value, slope, and frequency domain characteristics of the thermal response curve to perform weighted fusion with the correlation of the spectral signal; For each micro-region, the time-series peak value, slope, and frequency domain characteristics of the infrared thermal response curve are calculated. The correlation between these thermal response characteristics and the spectral signal in each band is calculated. The correlation is used as a weight to perform weighted fusion of the spectral and thermal response characteristics to generate a comprehensive feature vector for each micro-region.

[0031] The fused data is compensated and standardized by multimodal interpolation and local regression correction methods to generate a standardized cross-modal dataset containing spectral-thermal time series information, micro-region geometric coordinates, thermal response evolution parameters and feature weight matrices. Multimodal interpolation is applied to the fused micro-region feature vectors to estimate and complete missing or outlier values. At the same time, local regression correction is used to spatially smooth and adjust the amplitude of micro-region features, so that the numerical dimensions of different micro-regions and different modes are unified. Finally, the spectral-thermal time series information, micro-region geometric coordinates, thermal response evolution parameters and feature weight matrix are integrated to generate a structured and standardized cross-modal dataset.

[0032] Feature modeling module: Based on standardized cross-modal datasets, it extracts temperature difference, temperature change rate, peak time and frequency domain features, constructs background thermal field inversion by combining thermal response evolution mode, and highlights defect areas through background-observation difference, generating interpretable defect feature maps and thermal response parameters.

[0033] The process of extracting temperature difference, temperature change rate, peak time, and frequency domain features in the feature modeling module is as follows: Based on a standardized cross-modal dataset, high-order difference features are extracted from infrared thermal response sequences to obtain transient temperature difference and temperature change rate. Within a standardized cross-modal dataset, the infrared thermal response sequence is processed frame by frame according to micro-regions. The temperature difference between adjacent time frames is calculated to obtain the transient temperature difference matrix. Then, high-order difference operations are performed on the temperature difference matrix to obtain the temperature change rate information of the micro-region at different time steps.

[0034] By combining a subset of spectral features fused across modalities, frequency domain features are extracted using fast Fourier transform and continuous wavelet transform to form a multi-scale time-frequency joint feature vector. Align the micro-region thermal response sequence with the spectral feature subset, first perform a Fast Fourier Transform (FFT) on the thermal response sequence to calculate the frequency domain amplitude spectrum and phase spectrum, record the main frequency components and corresponding amplitudes of each micro-region as frequency domain features, perform a continuous wavelet transform on the same micro-region sequence, select a suitable mother wavelet, calculate the local energy distribution at different scales to form the time-frequency energy matrix of the micro-region, and perform micro-region-level weighted integration of the FFT frequency domain amplitude features and the CWT multi-scale time-frequency energy matrix to generate a multi-scale time-frequency joint feature vector.

[0035] By integrating spectral-thermal response information through micro-region weighted fusion, local high-dimensional defect features are generated. The temperature difference and temperature change rate data are weighted and fused with the multi-scale time-frequency joint feature vector according to the micro-region coordinates. For each micro-region, the spectral features and thermal response features are combined according to the preset weight coefficients. The time series and spatial coordinate information are integrated to form a local high-dimensional defect feature that includes temperature change parameters, frequency domain parameters and spectral response.

[0036] The process of generating interpretable defect feature maps and thermal response parameters in the feature modeling module is as follows: Based on the local high-dimensional defect characteristics, the background thermal field is fitted on the micro-region grid points using the historical thermal response evolution mode. Based on the local high-dimensional defect characteristics of each micro-region, the corresponding historical thermal response evolution data is read, and the background thermal field curve of each micro-region is reconstructed using linear or polynomial fitting methods. The temperature change trend of the micro-region at each time point is fitted, and the fitting results are rearranged according to the grid coordinates of the micro-region to generate a complete micro-region background thermal field matrix.

[0037] The observed thermal field and the fitted background thermal field are subjected to micro-region difference to form a local anomaly distribution matrix, and weighted anomaly enhancement is performed by combining high-dimensional defect features. The observed thermal field data and the background thermal field matrix are subdivided into micro-regions one by one. The temperature deviation value of each micro-region is calculated to obtain a preliminary local anomaly distribution matrix. For each micro-region, the difference value is weighted and integrated with the corresponding local high-dimensional defect feature vector. The high-dimensional features are amplified by the weighting coefficient to display the micro-regions with stronger anomalies, forming an enhanced anomaly distribution matrix.

[0038] By highlighting potential defect areas through multi-layer adaptive filtering, the difference matrix is ​​mapped to micro-area grid points to generate an interpretable defect feature map, while thermal response parameters are extracted. The enhanced anomaly distribution matrix is ​​mapped onto two-dimensional or three-dimensional grid points according to the micro-region coordinates to form an interpretable defect feature map. During the mapping process, a multi-layer local weighted filter is applied to smooth and enhance the outliers of neighboring micro-regions to highlight the local anomaly pattern. At the same time, the thermal response parameters of each micro-region are extracted, including the maximum temperature difference, temperature change rate, peak time and local time series curve.

[0039] Multimodal network module: It feeds the defect feature map and thermal response parameters together with the standardized cross-modal dataset into the 3D Transformer multi-task network, and generates a cross-modal model by fusing spectral and thermal field features through cross-modal attention.

[0040] The process of generating a cross-modal model in the multimodal network module is as follows: The defect feature map, thermal response parameters and key spectral band data are tensorized and embedded into the three-dimensional input space; The micro-region defect feature map is unfolded according to a two-dimensional spatial grid. Each micro-region contains corresponding thermal response parameters, such as maximum temperature difference, temperature change rate, peak time, etc. At the same time, the key spectral band data obtained by previous adaptive screening are arranged in the order of micro-regions. These two-dimensional spatial data and spectral data are superimposed on the modal dimension to form a three-dimensional tensor. The first dimension represents the spatial location, the second dimension represents the time series or thermal response evolution, and the third dimension represents the modal features, generating a unified three-dimensional input data structure.

[0041] Joint modeling of spatial-temporal-modal features is achieved through a 3D Transformer multi-task network. The three-dimensional tensor is input into the three-dimensional Transformer multi-task network. In the network, the spatial dimension captures the local and global dependencies between micro-regions through a self-attention mechanism, the temporal dimension records the evolution of thermal response over time through a sequence encoding module, and the modal dimension encodes different data sources through feature embedding. During the forward propagation process, the network performs multi-head attention calculation and feature fusion on the features of the spatial, temporal and modal dimensions to generate a multimodal representation of each micro-region.

[0042] A cross-modal attention mechanism is used to dynamically adjust the weights of features from different modalities to generate a cross-modal model that represents multiple types of damage in a micro-region. In the cross-modal attention layer of the network, independent attention weights are calculated for different modal features such as thermal response and spectral density. The weights are adjusted according to the contribution of each micro-region feature, and the information from different modalities is weighted and superimposed to generate a fused feature vector. Finally, the fused feature vector of each micro-region is output to form a complete cross-modal model.

[0043] Compression Deployment Module: After performance verification of the cross-modal model, compression and optimization are performed based on a deep partitioning decoupling strategy, and incremental updates are triggered by backhauling runtime data to generate the compressed cross-modal model.

[0044] The compression and optimization process based on the deep partitioning decoupling strategy in the compression deployment module is as follows: Micro-area accuracy assessment and error analysis were performed on cross-modal models to quantify the contribution of different spatial-temporal-modal features in micro-area damage identification. Forward reasoning is performed on the generated cross-modal model in each micro-region to obtain the output feature vector, which is then compared with the original local defect features. For the features in the spatial, temporal and modal dimensions, the contribution index of each micro-region in the damage identification task is calculated, such as feature response amplitude, gradient sensitivity and local error distribution. The micro-region contribution and error information are stored in a grid to form a quantifiable micro-region accuracy evaluation matrix.

[0045] Based on the deep partitioning decoupling strategy, the model is divided into multi-level partitioning modules. Hierarchical pruning is performed on low-contribution branches and redundant parameter blocks, and parameter compression is performed in combination with quantization and sparsification methods. Based on the micro-area accuracy evaluation matrix, the 3D Transformer network is divided into multiple partition modules according to the hierarchy. Each module corresponds to a specific spatial-temporal-modal feature subset. In each partition module, low-contribution branches and parameter redundancy blocks are identified, such as neurons with low contribution in the feature embedding layer, attention head, or feedforward network. Hierarchical pruning is performed on these low-contribution branches, and the corresponding weight matrix is ​​deleted or set to zero. Then, a fixed ratio of quantization is applied to the remaining parameter blocks to convert the floating-point weights into low-bit representations. At the same time, sparsity methods are combined to reset small weights to zero, reducing storage and computation requirements.

[0046] During the compression process, the integrity of key micro-region features and the absence of loss of multimodal correlation information are ensured through the micro-region sensitivity preservation mechanism and feature response weighting strategy, resulting in a cross-modal model that is both lightweight and has high micro-region feature fidelity. During pruning and quantization, the key feature responses of each micro-region are calculated in real time, and the activation amplitude and gradient information of each modality feature are recorded. According to the micro-region sensitivity preservation mechanism, the pruning ratio and quantization level are dynamically adjusted to ensure that the feature integrity of high-contribution micro-regions is not destroyed. Through the feature response weighting strategy, the outputs of different modalities are weighted and superimposed to finally generate a compressed cross-modal model. Each micro-region contains the fused lightweight feature vector and weight information.

[0047] The process of generating the compressed cross-modal model in the compressed deployment module is as follows: In actual monitoring, newly acquired multimodal data are input into a cross-modal model that combines lightweight design with high micro-area feature fidelity. Error analysis is performed on the model prediction output and observed spectral-thermal response data for each micro-area grid point to calculate the local residual matrix and identify potential abnormal feature regions. The newly acquired multimodal data is input into the compressed cross-modal model. The model output, including the predicted spectral and infrared thermal response characteristics, is calculated for each micro-region grid point. The predicted value of each grid point is compared with the actual observed spectral-thermal response data point by point to calculate the micro-region residual. The corresponding timestamp, spatial coordinates and modal type are recorded. All micro-region residuals are summarized and stored in order of micro-region location and time to form a micro-region residual information set.

[0048] For the compressed model parameters, the key micro-region feature weights are updated by combining micro-region residuals and cross-modal feature mutual information, while the low contribution parameter blocks are constrained to prevent overfitting. Based on the micro-region residual information set, the 3D Transformer network is divided into multiple partition modules according to the hierarchy. Each module corresponds to a specific spatial-temporal-modal feature subset. In each partition module, low contribution branches and parameter redundancy blocks are identified, such as neurons with low contribution in feature embedding layers, attention heads, or feedforward networks. Hierarchical pruning operations are performed on these low contribution branches, deleting or zeroing the corresponding weight matrix. Then, a fixed ratio of quantization is applied to the remaining parameter blocks to convert floating-point weights into low-bit representations. At the same time, sparsity methods are combined to reset small weights to zero, reducing storage and computation requirements.

[0049] During the incremental update process, the modal fusion coefficients and temporal weights are dynamically optimized to generate a compressed cross-modal model that has undergone micro-region-level adaptive adjustment. During the pruning and quantization process, the key feature responses of each micro-region are calculated in real time, and the activation amplitude and gradient information of each modality feature are recorded. According to the micro-region sensitivity preservation mechanism, the pruning ratio and quantization level are dynamically adjusted to ensure that the feature integrity of high-contribution micro-regions is not destroyed. Through the feature response weighting strategy, the outputs of different modalities are weighted and superimposed to maintain the multimodal correlation information, and finally a compressed cross-modal model is generated.

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0051] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A refined identification system for multiple types of damage in a localized area of ​​a building envelope, characterized in that, include: Specimen Acquisition Module: Constructs standardized specimens covering different materials, thicknesses, and defect types. Simultaneously acquires spectra using a hyperspectral camera, a long-wave infrared camera, and a controllable heat source. Through multi-rate heating-cooling and natural sunlight, it generates a multi-excitation, multi-modal, labeled raw data set. Data processing module: Based on the original dataset, it performs correction, bad pixel repair and radiometric calibration, and adaptively selects key spectral bands. At the same time, it processes infrared time series data to form a standardized cross-modal dataset. Feature modeling module: Based on standardized cross-modal datasets, it extracts temperature difference, temperature change rate, peak time and frequency domain features, constructs background thermal field inversion by combining thermal response evolution mode, and highlights defect areas through background-observation difference, generating interpretable defect feature maps and thermal response parameters; The process of generating interpretable defect feature maps and thermal response parameters is as follows: Based on the local high-dimensional defect characteristics, the background thermal field is fitted on the micro-region grid points using the historical thermal response evolution mode. The observed thermal field and the fitted background thermal field are subjected to micro-region difference to form a local anomaly distribution matrix, and weighted anomaly enhancement is performed by combining high-dimensional defect features. By highlighting potential defect areas through multi-layer adaptive filtering, the difference matrix is ​​mapped to micro-area grid points to generate an interpretable defect feature map, while thermal response parameters are extracted. Multimodal network module: It feeds the defect feature map and thermal response parameters together with the standardized cross-modal dataset into the 3D Transformer multi-task network, and generates a cross-modal model by fusing spectral and thermal field features through cross-modal attention; Compression Deployment Module: After performance verification of the cross-modal model, compression and optimization are performed based on a deep partitioning decoupling strategy, and incremental updates are triggered by running data feedback to generate the compressed cross-modal model; The compression and optimization process based on the deep partitioning decoupling strategy is as follows: Micro-area accuracy assessment and error analysis were performed on cross-modal models to quantify the contribution of different spatial-temporal-modal features in micro-area damage identification. Based on the deep partitioning decoupling strategy, the model is divided into multi-level partitioning modules. Hierarchical pruning is performed on low-contribution branches and redundant parameter blocks, and parameter compression is performed in combination with quantization and sparsification methods. During the compression process, the integrity of key micro-region features and the absence of loss of multimodal correlation information are ensured through a micro-region sensitivity preservation mechanism and a feature response weighting strategy, resulting in a cross-modal model that is both lightweight and has high micro-region feature fidelity.

2. The refined identification system for localized multi-type damage to building envelope structures according to claim 1, characterized in that, The process of generating a multi-stimulus, multi-modal, labeled raw dataset is as follows: The acquisition strategy was set according to the spectral characteristics and thermophysical parameters of the specimen material, and a multi-angle, multi-band acquisition scheme was adopted to obtain high-resolution spectral data. Micro-area resolution thermal excitation gradient scanning was performed on infrared images to obtain multi-temporal thermal response data under different heating-cooling rates and natural sunlight conditions. During the acquisition process, the geometric information of the micro-regions of the specimen is recorded simultaneously, forming a raw data set with unified spatial-temporal-modal annotation.

3. The refined identification system for localized multi-type damage to building envelope structures according to claim 2, characterized in that, The process of correction, dead pixel repair, and radiation calibration based on raw data is as follows: Based on the original dataset, dark current correction, whiteboard correction and spectral drift correction are performed on the hyperspectral image data, and bad pixels are repaired by combining an adaptive abnormal pixel detection algorithm. Radiometric calibration, dynamic noise suppression, and non-uniformity correction are performed on infrared temporal images; A multi-scale spatial alignment method is used to map single-modal data to a unified micro-grid, generating a single-modal standardized dataset.

4. The refined identification system for localized multi-type damage to building envelope structures according to claim 3, characterized in that, The adaptive screening process for key spectral bands is as follows: Based on a single-mode standardized dataset, statistical significance and multidimensional correlation indicators are used to identify the bands most sensitive to local material damage, and spectral noise and interference are weighted and suppressed. By combining the historical defect feature database, the importance of each micro-region band is evaluated with multiple objectives, and a genetic algorithm is used to dynamically select the key spectral bands that best reflect the micro-region defect characteristics. The selection process simultaneously considers band redundancy, spatial consistency, and cross-modal complementarity to form a subset of spectral features.

5. The refined identification system for localized multi-type damage to building envelope structures according to claim 4, characterized in that, The process of creating a standardized cross-modal dataset is as follows: The infrared thermal response sequence is time-normalized and micro-area aligned, and multi-scale local weighted filtering and adaptive noise suppression algorithm are used to eliminate sensing noise and environmental interference. By combining a subset of spectral bands, micro-region feature tensor quantization is performed on the spectral data, and the spectral data and infrared thermal response data are precisely registered on the micro-region grid. We use the temporal peak value, slope, and frequency domain characteristics of the thermal response curve to perform weighted fusion with the correlation of the spectral signal; The fused data is compensated and standardized by multimodal interpolation and local regression correction methods to generate a standardized cross-modal dataset containing spectral-thermal time series information, micro-region geometric coordinates, thermal response evolution parameters, and feature weight matrices.

6. The refined identification system for localized multi-type damage to building envelope structures according to claim 5, characterized in that, The process of extracting temperature difference, temperature change rate, peak time, and frequency domain features is as follows: Based on a standardized cross-modal dataset, high-order difference features are extracted from infrared thermal response sequences to obtain transient temperature difference and temperature change rate. By combining a subset of spectral features fused across modes, frequency domain features are extracted using fast Fourier transform and continuous wavelet transform to form a multi-scale time-frequency joint feature vector. By integrating spectral-thermal response information through micro-region weighted fusion, local high-dimensional defect features are generated.

7. The refined identification system for localized multi-type damage to building envelope structures according to claim 1, characterized in that, The process of generating a cross-modal model is as follows: The defect feature map, thermal response parameters and key spectral band data are tensorized and embedded into the three-dimensional input space; Joint modeling of spatial-temporal-modal features is achieved through a 3D Transformer multi-task network. A cross-modal attention mechanism is used to dynamically adjust the weights of features from different modalities to generate a cross-modal model that represents multiple types of damage in micro-regions.

8. The refined identification system for localized multi-type damage to building envelope structures according to claim 7, characterized in that, The process of generating the compressed cross-modal model is as follows: In actual monitoring, newly acquired multimodal data are input into a cross-modal model that combines lightweight design with high micro-area feature fidelity. Error analysis is performed on the model prediction output and observed spectral-thermal response data for each micro-area grid point to calculate the local residual matrix and identify potential abnormal feature regions. For the compressed model parameters, the key micro-region feature weights are updated by combining micro-region residuals and cross-modal feature mutual information, while the low contribution parameter blocks are constrained to prevent overfitting. During the incremental update process, the modal fusion coefficients and temporal weights are dynamically optimized to generate a compressed cross-modal model that has undergone micro-region-level adaptive adjustment.

Images