Infrared nondestructive testing method and system based on defect spatio-temporal characteristic information

Abstract

The invention relates to the technical field of signal modulation and processing, and discloses an infrared nondestructive testing method and system based on defect spatio-temporal characteristic information, and the method comprises five steps of infrared data acquisition, thermal imaging signal reconstruction, sampling strategy customization, defect spatio-temporal characteristic extraction and automatic defect segmentation. Through the thermal imaging signal reconstruction method, the influence of noise and uneven heating can be effectively reduced, and the prediction accuracy of the background is effectively improved. And secondly, by reasonably formulating a sampling strategy, the data calculation memory can be effectively reduced, and the detection efficiency and stability of the model are improved. Furthermore, the defect space-time information is fully utilized through space-time feature extraction, and the segmentation precision of the shape and size of the defect is effectively improved. And finally, processing the spatio-temporal data by using a defect segmentation model based on a Transform network, so that high-precision automatic defect segmentation is realized.

Description

Technical Field

[0001] This invention relates to the field of signal modulation and processing technology, specifically to an infrared nondestructive testing method and system based on the spatiotemporal characteristics of defects. Background Technology

[0002] Infrared thermography is a non-destructive testing (NDT) technique that detects defects by detecting the infrared thermal signals radiated from the surface of materials. It offers advantages such as high efficiency, non-contact operation, and full-field detection, providing a visual means of detecting defects in composite material components. However, in actual testing, uneven heating and external noise can lead to low thermal contrast and blurred edges in defective areas on the material surface. Inspectors may find it difficult to effectively identify all defects from the raw infrared thermogram. Therefore, some data preprocessing methods have been proposed to improve the defect contrast of infrared thermograms. However, these methods only enhance image readability; defect identification still requires professional interpretation, inevitably leading to false positives and false negatives. Therefore, it is necessary to develop automated defect detection technology to improve detection efficiency and accuracy.

[0003] The basic principle of existing automatic defect detection technology is to combine deep learning networks to process nonlinear data and use neural networks to learn effective defect features in thermal images, thereby automatically identifying complex defects. This method is currently mainly applied to three aspects of infrared nondestructive testing: defect classification, defect localization, and defect segmentation. Defect segmentation classifies each pixel in the experimentally acquired infrared thermal image, mapping the continuous infrared thermal image into a discrete mask image. In the mask image, binary digits 1 and 0 are used as label values ​​to represent defects and background, respectively, assigned to each pixel. By statistically analyzing the pixels with a value of 1 in the mask image, the location, shape, and area of ​​the defect can be accurately identified, achieving both qualitative and quantitative detection. Currently, defect segmentation methods in the field of infrared nondestructive testing are mainly divided into two categories: One method is a defect segmentation method based on one-dimensional time-temperature signals. This method first collects time-series samples of single pixels, then uses time-series neural networks such as RNNs and LSTMs to process the one-dimensional time-temperature data pixel by pixel to determine whether the pixel belongs to a defect region. Finally, all the determination results are combined into a binary image to achieve automatic defect segmentation. The drawback of this method is that it only considers the changing trend of defect features in the time dimension and ignores the correlation in the two-dimensional spatial dimension, such as the influence of lateral thermal diffusion, thus reducing the prediction accuracy of defect edges.

[0004] The second method is a defect segmentation method based on two-dimensional spatial heatmaps. This method first acquires an infrared heatmap with high thermal contrast, then processes the acquired heatmap using a CNN network, and finally determines the defective and non-defective regions in a single frame image to achieve automatic defect segmentation. The drawback of this method is that, because the temperature of each pixel changes dynamically over time, analyzing only two-dimensional spatial features prevents the model from fully learning the temporal information in the three-dimensional sequence heatmap, thus reducing the model's defect recognition accuracy. Summary of the Invention

[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides an infrared nondestructive testing method and system based on the spatiotemporal features of defects, effectively improving the segmentation accuracy of defect shape and size. Firstly, it solves the problems of noise and non-uniform heating affecting the raw data, increasing the probability of model misjudgment; and the failure to fully extract spatiotemporal features during sample acquisition, leading to lateral thermal diffusion affecting the detection effect on defect edge regions. Secondly, by using a defect segmentation model based on a Transformer network to process the spatiotemporal data, this invention achieves advantages such as high-precision automatic defect segmentation.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: an infrared nondestructive testing method and system based on spatiotemporal feature information of defects, comprising the following steps: S101: Infrared data acquisition; during the cooling phase after thermal excitation, infrared cameras are used to acquire sequential infrared thermal images of the sample. S102: Thermal imaging signal reconstruction; The original infrared sequence heat map is reconstructed into a logarithmic domain first derivative sequence heat map using the TSR method; S103: Customized sampling strategy; Based on the actual situation of the thermal data, a sampling plan is formulated considering the balance of the samples and the rationality of the time series. S104: Defect spatiotemporal feature extraction; according to the sampling strategy, extract the spatiotemporal information of defects pixel by pixel from the heatmap of the first derivative sequence in the logarithmic field to construct spatiotemporal samples; S105: Automatic Defect Segmentation; Construct a defect segmentation model based on Transformer, which effectively learns spatiotemporal information through a multi-head self-attention mechanism to achieve binary classification of defective and non-defective regions.

[0007] Preferably, the infrared data acquisition process is as follows: First, place the infrared thermal imager 1m to 1.5m directly in front of the target, adjust the focus to ensure the image is clear and the target is centered in the field of view; second, place the two halogen lamps of the long-pulse thermal loading excitation device on both sides of the infrared thermal imager to ensure uniform heating of the sample surface; finally, set the excitation time, number of images, and acquisition frequency in the image acquisition software, open the louvers on the lamp cover to begin excitation of the sample surface, and immediately close the louvers after excitation to prevent residual infrared radiation from affecting the sample surface temperature. The acquired infrared sequence images will be displayed in a three-dimensional matrix (N... x ×N y ×N t The N is stored in the computer in the form of ) x N is the width of the infrared image. y N represents the height of the infrared image. t This represents the number of infrared sequence images.

[0008] Preferably, the thermal imaging signal reconstruction reconstructs a first-order derivative sequence image through a first-order derivative step of the logarithmic domain fitted signal. This enhances defect contrast and reduces the impact of non-uniform heating. The formulas for logarithmic domain signal fitting and the first-order derivative of the logarithmic domain fitted signal are as follows: (1) (2) Where T is the material surface temperature, a n denoted as polynomial fitting coefficients, where n is the fitting order and t is the cooling time.

[0009] Preferably, based on the order of the polynomial fitting in the logarithmic domain signal fitting step, the polynomial fitting coefficient set obtained for each pixel and the t sequence of each pixel are substituted into formula (2) pixel by pixel to obtain the dln(T) / dln(t) sequence of each pixel. This step is equivalent to converting the logarithmic sequence image into the first derivative sequence image, which effectively reduces the influence of non-uniform heating and enhances the defect contrast.

[0010] Preferably, in the training data sampling strategy customization, regarding sample balance, a region not exceeding the defect edge is selected at the defect center, and defect data is then sampled within this region. Simultaneously, defect-free samples are collected proportionally in areas far from the defect region to ensure data balance. Regarding the time series rationality, the infrared thermal image of the target before reaching thermal equilibrium during the cooling phase is retained, ensuring high contrast between defective and defect-free areas. The length of the sequence image after removing redundant data is N. lIn customizing the test data sampling strategy, data needs to be sampled from all test areas. Therefore, sample balance is not considered. However, for the rationality of the time series, the strategy can be consistent with that of the training data.

[0011] Preferably, the defect spatiotemporal feature extraction process includes the following steps: S201: Three-dimensional spatiotemporal feature extraction; When constructing training and testing datasets, temperature curve data of all pixels within a 3×3 window of varying size are extracted over time from defective and non-defective regions selected from the first derivative sequence images of the detected target, respectively, as three-dimensional spatiotemporal feature samples. The dimension of a single three-dimensional spatiotemporal feature sample is (3×3)×N. l ; S202: Window Feature Tiling; After acquiring the 3D spatiotemporal feature training and testing datasets, window feature tiling is performed on each 3D spatiotemporal feature sample. This involves tiling the defect feature signals of all pixels contained in each frame of a 3×3 window of each 3D spatiotemporal feature sample into a one-dimensional vector containing 9 elements. Each 3D spatiotemporal feature sample will then form an N... l A one-dimensional vector; S203: Two-dimensional spatiotemporal sample reconstruction; further processing of the tiled one-dimensional vector to convert N of each three-dimensional spatiotemporal feature sample. l One-dimensional vectors at different times are stacked row-wise to reconstruct (N) l , 9) thus generate a two-dimensional sample containing the spatiotemporal information of defects in the pixel and the surrounding area; S204: Sample label assignment; Assign a corresponding classification label to each two-dimensional spatiotemporal sample in the training dataset so that the model can learn the difference between the predicted label and the true label.

[0012] Preferably, the module operation process of the defect segmentation model includes the following steps: Position encoder module 301: The position encoder module first creates a fully connected linear layer, which takes the input dimension as (N) l The two-dimensional spatiotemporal sample data of (9) is mapped as (N) l , 8), and then a learnable vector is generated for each time series position through the position encoder. The dimension of the vector is consistent with the dimension of the feature. The learned position information will be added to the mapped data to enhance the model’s perception of the sequence position. Transformer module 302: The position-encoded feature data will be input into the Transformer module for processing. The Transformer module consists of a multi-head self-attention layer, two residual connection normalization layers, and a feedforward neural network. Classifier module 303: The classifier module determines the category of the output y of the Transformer module. The classifier module first performs mean processing on the data in the time dimension, that is, calculates the mean of each column of the output matrix y of the Transformer module to obtain a vector with shape (1, 8). Then, it performs dimension transformation through three linear layers, so that the feature dimension decreases layer by layer, from the initial 8 dimensions to 2 dimensions. A ReLU activation function is added between each linear layer to increase non-linear changes.

[0013] Preferably, the forward propagation equation of the position encoder module is as follows: (3) Where x is a shape of (N) l Given input samples of size (N, 9), E is a linear layer with input of 9 and output of 8, and Epos is a layer of size (N, 9). l The learnable location encoding matrix of dimension (N, 8), where xʹ is a matrix of dimension (N, 8). l , 8) Position encoding output.

[0014] Preferably, the location-encoded feature data is input into the Transformer module for processing. The Transformer module consists of a multi-head self-attention layer, two residual connection normalization layers, and a feedforward neural network. Taking a 4-layer self-attention mechanism as an example, its calculation formula is as follows: (4) (5) (6) (7) (8) Among them, W Q W K and W V Let Q be the learning matrix, K be the query matrix, K be the key matrix, and V be the value matrix. m , K m V m For a submatrix, α m Let d be the attention score matrix, and b be the feature processing dimension of each attention head. m The output for each attention head; Subsequently, the output b of the multi-head self-attention layer is passed to the first residual connection normalization layer, then through a feedforward neural network and the second residual connection normalization layer, with the final layer being of size (N). lThe output y of , 8) will be used as the final output of the Transformer module. The relevant operational equations of the Transformer module are as follows: (9) (10) (11) (12) Where F is the multi-head self-attention layer, and xʹ is a layer of size (N... l , 8) is the input of the multi-head self-attention layer, b is the output of the multi-head self-attention layer, Z1 is the result of the first residual connection normalization layer, W1 and W2 are the learnable weight matrices of the linear layer of the feedforward neural network, a1 and a2 are the bias terms of the linear layer of the feedforward neural network, Z2 is the output of the feedforward neural network, and y is a matrix of shape (N l The output of the Transformer module (8).

[0015] Preferably, the Softmax function is used to transform the two-dimensional features processed by the linear layer into classification probability values ​​for whether the sample belongs to a defective or non-defective region. The category corresponding to the highest probability value is selected as the classification prediction result of the model for that pixel sample. The classifier module implements the binary classification equation for defects and background as follows: (13) (14) (15) (16) In the formula, l is the length of the time series, and y i Here, H1 is the input at the i-th time step, H2 is the output of the first linear layer, H3 is the output of the second linear layer, P is the binary classification probability distribution of the predicted output of the classifier module, and f is the input at the ith time step. j Let H3 be the j-th element of the two-dimensional vector H3; During the training phase, the training dataset is input into the defect segmentation model to obtain the binary classification probability distribution of each spatiotemporal sample. Then, the binary cross-entropy loss function is used as the loss metric during training to evaluate the difference between the predicted and true labels. The formula for the binary cross-entropy loss function is: (17) Where N is the batch size during model training, Y is the true label, and Ŷ is the probability value of the predicted result being a defect. By substituting the predicted probability value and the true label into formula (17), the parameters of the defect segmentation model are finally updated iteratively by minimizing this loss function during training using the batch gradient descent method. During the testing phase, the test dataset is input into the trained defect segmentation model, which then outputs the spatiotemporal sample classification results pixel by pixel. Pixels identified as defective regions are assigned a value of 1, while pixels in non-defective regions are assigned a value of 0. Finally, the prediction results of all pixels are reconstructed into a binary image, thereby achieving high-precision automatic defect segmentation of the detected target.

[0016] Beneficial effects Compared with existing technologies, this invention provides an infrared nondestructive testing method and system based on the spatiotemporal characteristics of defects, which has the following advantages: This invention effectively reduces the impact of noise and uneven heating through thermal imaging signal reconstruction, significantly improving the accuracy of background prediction. Secondly, by rationally formulating a sampling strategy, this invention effectively reduces data computation memory, improving the model's detection efficiency and stability. Furthermore, this invention fully utilizes spatiotemporal information of defects through spatiotemporal feature extraction, effectively improving the segmentation accuracy of defect shape and size. Finally, by using a defect segmentation model based on a Transformer network to process spatiotemporal data, this invention achieves high-precision automatic defect segmentation. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation of the present invention; Figure 2 This is a flowchart of the spatiotemporal sample acquisition process of the present invention; Figure 3 This is a flowchart of the module operation of the defect segmentation model of the present invention; Figure 4 (a) A true label diagram showing the actual distribution of defects in the laminate sample numbered S1 of this invention; Figure 4 (b) A true label diagram showing the actual distribution of defects in the laminate sample numbered S2 of this invention; Figure 4 (c) A true label diagram showing the actual distribution of defects in the laminate sample numbered S3 of this invention; Figure 5 (a) is the original infrared image of the laminate numbered S2 obtained by the present invention; Figure 5 (b) is the original infrared image of the laminate numbered S3 obtained by the present invention; Figure 6(a) is a binary image of defect segmentation of the polyvinyl chloride laminate S2 of the present invention; Figure 6 (b) is a binary image of defect segmentation of the polyvinyl chloride laminate S3 of the present invention; Figure 7 (a) is a comparison diagram of the difference between the predicted label image and the real label image of the sample S2 of the present invention; Figure 7 (b) is a comparison diagram of the difference between the predicted label image and the actual label image of the sample S3 of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention provides an infrared nondestructive testing method based on the spatiotemporal characteristics of defects. Figure 1 The flowchart of the present invention includes: S101: Infrared data acquisition; during the cooling phase after thermal excitation, infrared cameras are used to acquire sequential infrared thermal images of the sample.

[0020] S102: Thermal imaging signal reconstruction; The original infrared sequence heatmap is reconstructed into a logarithmic domain first derivative sequence heatmap using the TSR method.

[0021] S103: Customized sampling strategy; Based on the actual situation of the thermal data, a sampling plan is formulated considering the balance of the samples and the rationality of the time series.

[0022] S104: Defect spatiotemporal feature extraction; according to the sampling strategy, extract the spatiotemporal information of defects pixel by pixel from the heatmap of the first derivative sequence in the logarithmic field to construct spatiotemporal samples.

[0023] S105: Automatic Defect Segmentation; Construct a defect segmentation model based on Transformer, which effectively learns spatiotemporal information through a multi-head self-attention mechanism to achieve binary classification of defective and non-defective regions.

[0024] It should be noted that this method is only applicable to infrared thermal imaging data, and the model needs to be trained accordingly when dealing with specific industrial tasks.

[0025] The embodiments of the present invention will be analyzed in detail step by step below.

[0026] 1. Infrared data acquisition Infrared data is collected from the target object using an infrared non-destructive testing system. The following section will detail the composition of the infrared non-destructive testing system and the process of infrared data acquisition.

[0027] 1.1 Composition of an Infrared Non-destructive Testing System The infrared nondestructive testing system is an infrared sequence image acquisition system based on thermal excitation. This system is particularly suitable for infrared nondestructive testing of defects with long pulse excitation. The system includes a thermal loading excitation device, an infrared thermal imager, and an acquisition and control system.

[0028] The thermal loading excitation device used is a long-pulse thermal loading excitation device, which can be placed within 0.5m of the sample surface, parallel to the sample surface, and symmetrically placed on both sides of the center of the horizontal line of the sample to ensure uniform and sufficient thermal loading on the sample surface. Preferably, the long-pulse thermal loading excitation device can use two 1000W halogen lamps, which are placed on both sides of the infrared camera to ensure uniform heating of the sample surface; the heating time can be precisely controlled by the program to achieve the long-pulse excitation loading method. Ideally, the halogen lamps are placed in a lamp cover, and a set of louvers is installed at the lamp cover opening. After loading is completed, the louvers are immediately closed to avoid residual infrared radiation affecting the detection results.

[0029] The infrared thermal imager is the core component of this detection system. It can be positioned horizontally at a distance of 1m to 1.5m from the sample surface, depending on the size of the detection area, to acquire sequential images and transmit the acquired temperature distribution sequence images of the sample surface to the acquisition and control system. Preferably, the infrared thermal imager can be a FLIR SC5000 infrared camera, which has the following performance parameters: Wavelength range 2.5-5μm Detector type: Indium antimonide (InSb) cooled detector Resolution 320×256 or 640×512 Detector pixel distance 15μm or 30μm NETD <20mK Standard temperature measurement range: -20℃ to 1500℃ Selectable temperature range: -20°C to 3000°C Operating temperature range: -20℃ to 55℃ Accuracy reading ±1°C or ±1% The emissivity coefficient can detect emissivity between 0.01 and 1. Infrared frame rate up to 380Hz Digital Data Protocol Ethernet (Ethernet version 2.0) Interface Gigabit Ethernet video link Minimum focusing distance: 0.5m This infrared camera has the advantages of clear image acquisition, high detection sensitivity, accurate temperature measurement, and good reliability, making it very suitable for the field of infrared thermal imaging non-destructive testing. Of course, other types or models of infrared thermal imagers can also be used in the detection system of this invention.

[0030] The data acquisition and control system includes a personal computer, microcontroller, or server, and has two main functions: image acquisition control and image processing. For image acquisition control, it can control the operating parameters of the heat-loading excitation device and the infrared thermal imager, i.e., set the excitation time of the heat-loading excitation device, the number of images acquired by the infrared thermal imager, and the acquisition frequency. Image processing mainly relies on the image acquisition software attached to the infrared camera to perform simple data processing on the infrared temperature sequence images. Preferably, the data acquisition and control system includes a personal computer (PC) and image acquisition software installed on the PC. The basic hardware and software requirements of the PC are: 32GB of memory, a 64-bit Windows 10 operating system, and an Intel Pentium eight-core processor. The image acquisition software is the infrared camera's accompanying software programmed in a language supported by the Windows operating system.

[0031] 1.2 Infrared Data Acquisition Process The infrared data acquisition method of this invention is a non-destructive testing method based on thermal excitation. This method heats the surface of the object under test through light radiation and records the temperature change of the object's surface over time using an infrared camera. Preferably, there are various excitation methods for light radiation; this invention selects long-pulse excitation, which is suitable for detecting small defects in composite materials. The specific process of infrared data acquisition is as follows: First, place the infrared thermal imager 1m to 1.5m directly in front of the target, adjusting the focus to ensure a clear image and that the target is centered within the imager's field of view. Second, place the two halogen lamps of the long-pulse thermal loading excitation device on either side of the imager to ensure uniform heating of the sample surface; for example, position the excitation device within 0.5m of the sample surface. Finally, set the excitation time, number of images, and acquisition frequency in the image acquisition software. Open the louvers on the lamp cover to begin excitation of the sample surface. Immediately close the louvers after excitation to prevent residual infrared radiation from affecting the sample surface temperature. The image acquisition software will then acquire infrared thermal images of the sample surface as it cools, according to the previously set acquisition quantity and frequency, and perform basic data processing on the thermal images.

[0032] The acquired infrared sequence images will be displayed in a three-dimensional matrix (N x ×N y ×N t The N is stored in the computer in the form of ) xN is the width of the infrared image. y N represents the height of the infrared image. t This represents the number of infrared sequence images. Since the temperature peak contrast is usually most significant during the initial cooling phase of the sample, the temperature recording time should not be too short to ensure that the camera fully captures the characteristic signals and avoids data distortion.

[0033] The infrared data acquisition is now complete. Next, the infrared data will undergo preprocessing.

[0034] 2. Thermal imaging signal reconstruction The thermal imaging signal reconstruction in this invention consists of two steps: logarithmic domain signal fitting and first-order derivative of the logarithmic domain fitted signal.

[0035] In most cases, infrared sequence images are inevitably affected by noise. The logarithmic domain signal fitting step, which involves fitting the logarithmic domain signal pixel-by-pixel, can eliminate most of the noise in the infrared sequence image. Then, the first-order derivative sequence image is reconstructed through the first-order derivative step of the logarithmic domain fitted signal, which enhances defect contrast and reduces the effects of non-uniform heating. The formulas for logarithmic domain signal fitting and the first-order derivative of the logarithmic domain fitted signal are: (1) (2) Where T is the material surface temperature, a n denoted as polynomial fitting coefficients, where n is the fitting order and t is the cooling time.

[0036] 2.1 Steps for fitting a logarithmic domain signal The polynomial fitting order n selected in this invention is 8. When n is 8, the tT curve data of each pixel in the infrared sequence image is first substituted into Formula 1 pixel by pixel. The polynomial fitting coefficient set of each pixel can be obtained by calculation. Then, polynomial fitting is performed on the logarithmic domain signal. The polynomial fitting coefficient set and the t sequence of each pixel are substituted into Formula 1 pixel by pixel to obtain the ln(T) sequence of each pixel. The logarithmic sequence image composed of all ln(T) sequences can compress the dynamic range of the data, thereby effectively eliminating the influence of noise.

[0037] It is important to note that the choice of fitting order affects the fitting accuracy of the data. For different detection tasks, if the actual fitting effect does not meet the expected fitting accuracy, the fitting order needs to be reselected. The order should generally not exceed 10 and not be lower than 4. By comparing the fitting effect of the logarithmic temperature-logarithmic time curves of the detected target defect and the defect-free area at different orders, the optimal polynomial fitting order for the actual task can be quickly determined.

[0038] 2.2 Steps for First-Order Differentiation of Logarithmic Domain Fitted Signal Based on the polynomial fitting order in the logarithmic domain signal fitting step, the polynomial fitting coefficient set obtained for each pixel and the t sequence of each pixel are substituted into Formula 2 to obtain the dln(T) / dln(t) sequence of each pixel. This step is equivalent to converting the logarithmic sequence image into a first derivative sequence image, which effectively reduces the influence of non-uniform heating and enhances the defect contrast.

[0039] Uneven heating can cause temperature variations on an object's surface. The first derivative signal can eliminate static background temperature differences while preserving dynamic thermal response characteristics. Furthermore, the thermal diffusivity of defective regions differs from that of non-defective regions, and the first derivative signal can amplify this difference, thereby increasing the contrast between defects and the background.

[0040] 3. Customized sampling strategy Before sampling the first derivative sequence image, a corresponding sampling strategy needs to be formulated according to the actual situation of the detection area. The detection area may have defects such as holes, delamination, and debonding. This invention takes holes as an example to formulate its corresponding sampling strategy. Other types of defects can be sampled in the same way as holes.

[0041] The sampling strategy primarily considers two aspects: sample balance and time series rationality. Sample balance focuses on the consistency of the quantity of different types of data. Since the defect-free area of ​​the detection target is much larger than the defective area, it is necessary to ensure that the ratio of defective samples to defect-free samples is balanced during sampling. This helps improve the stability of the model and prevents overfitting or underfitting. Time series rationality needs to ensure the effectiveness and continuity of the samples in the time dimension. By eliminating redundant time series data to retain key temperature change information, the model can effectively learn the temporal characteristics of defects and reduce computational memory usage.

[0042] In view of the above considerations, the sampling strategy of the present invention is mainly divided into two implementation parts: training data sampling and test data sampling.

[0043] 3.1 Training Data Sampling In the training data sampling strategy, to ensure sample balance, this invention selects a region within the defect center that does not extend beyond the defect edge, and then samples defect data within this region. Simultaneously, defect-free samples are collected proportionally from areas far from the defect to ensure data balance. Regarding the time series validity, this invention only retains the infrared thermogram of the target before it reaches thermal equilibrium during the cooling phase, maintaining high contrast between defective and defect-free areas. The length of the sequence image after removing redundant data is N. l .

[0044] 3.2 Test Data Sampling In the test data sampling strategy, since data needs to be sampled from the entire test area, sample balance is not considered. However, for the time series data reasonableness, the strategy is the same as that for training data sampling.

[0045] 4. Defect Spatiotemporal Feature Extraction After the first derivative sequence image is reconstructed from the thermal imaging signal, it needs to undergo defect spatiotemporal feature extraction to form defect spatiotemporal samples, thereby improving the utilization rate of spatiotemporal information and reducing the impact of lateral thermal diffusion. The defect spatiotemporal feature extraction module of this invention combines the sampling strategy in Section 3 to collect spatiotemporal samples from the first derivative sequence image.

[0046] The process of spatiotemporal sample acquisition is as follows: Figure 2 As shown, the specific process is as follows: S201: Three-dimensional spatiotemporal feature extraction; When constructing the training and testing datasets, the sampling strategy from Section 3 is used to extract the temperature curve data of all pixels within a 3×3 window from the defective and non-defective regions selected from the first derivative sequence images of the detected target as three-dimensional spatiotemporal feature samples. The dimension of a single three-dimensional spatiotemporal feature sample is (3×3)×N. l By extracting the first derivative values ​​of all pixels within the grid, the spatial information of the data can be enriched. This allows for more accurate segmentation of defect edge regions by utilizing the spatial correlation between the central pixel and surrounding pixel areas, thus overcoming the influence of lateral thermal diffusion on defect determination.

[0047] The 3×3 window is the defect spatiotemporal feature extraction window, and the center pixel of each window is the corresponding pixel selected in the first derivative sequence image.

[0048] S202: Window feature tiling; After acquiring the 3D spatiotemporal feature training and testing datasets, a window feature tiling process is performed on each 3D spatiotemporal feature sample. This involves tiling the defect feature signals of all pixels within a 3×3 window of each frame of each 3D spatiotemporal feature sample into a one-dimensional vector containing 9 elements. Each 3D spatiotemporal feature sample will then form an N... l A one-dimensional vector.

[0049] S203: Two-dimensional spatiotemporal sample reconstruction; Further processing is performed on the tiled one-dimensional vector to obtain the N of each three-dimensional spatiotemporal feature sample. l One-dimensional vectors at different times are stacked row-wise to reconstruct (N) l The two-dimensional matrix of 9 is used to generate a two-dimensional sample containing the spatiotemporal information of the pixel and the surrounding area defects, thereby improving the utilization rate of spatiotemporal information.

[0050] S204: Sample label assignment; Finally, each two-dimensional spatiotemporal sample is assigned a corresponding classification label, enabling the model to learn the difference between the predicted label and the true label. For the training dataset, two-dimensional spatiotemporal samples from defective regions are labeled as 1, while those from defect-free regions are labeled as 0. For the test dataset, since the testing process does not require sample labels, no labels are assigned to the two-dimensional spatiotemporal samples.

[0051] 5. Automatic defect segmentation Through the above steps, the spatiotemporal feature dataset of defects has been prepared. Next, the training dataset is used to train the defect segmentation model, and then the trained defect segmentation model is used to predict the test dataset, thereby achieving high-precision automatic defect segmentation.

[0052] The defect segmentation model used in this invention is based on the Transformer network. The Transformer is a deep learning architecture that relies on self-attention. Through the multi-head self-attention mechanism, it can explore the relationship between the temperature of pixels and surrounding areas in the infrared thermal image and the cooling time, and focus on important features related to defects, thereby significantly improving the defect segmentation accuracy. Its main modules include a position encoder module, a Transformer module, and a classifier module. Figure 3 The following section details the module computation flow of the defect segmentation model, explaining the specific computation process of each module: Position encoder module 301: The position encoder module first creates a fully connected linear layer, which encodes the input layer with dimension N. l The two-dimensional spatiotemporal sample data of (9) is mapped as (N) l 8). Next, a learnable vector is generated for each time series location using a location encoder. The vector's dimension matches the feature dimension. This learned location information is then added to the mapped data to enhance the model's perception of sequence location. The forward propagation equation for the location encoder module is as follows: (3) Where x is a shape of (N) l Given input samples of size (N, 9), E is a linear layer with input of 9 and output of 8, and Epos is a layer of size (N, 9). l The learnable location encoding matrix of dimension (N, 8), where xʹ is a matrix of dimension (N, 8). l , 8) Position encoding output.

[0053] Transformer module 302: The location-encoded feature data is then input into the Transformer module for processing. The Transformer module consists of a multi-head self-attention layer, two residual connection normalization layers, and a feedforward neural network. In this invention, a 4-head self-attention mechanism is used as an example of the multi-head self-attention layer, and its relevant calculations are as follows: (4) (5) (6) (7) (8) The process first processes the position-encoded feature data xʹ with three learnable matrices W of size (8, 8). Q W K and W V The operations are respectively mapped to a size of (N) l , 8) The query matrix Q, key matrix K, and value matrix V are then split into four submatrices of the same shape Q. m , K m V m (m=0,1,2,3). Then, for Q... m With K m The attention score matrix α is obtained by performing Softmax calculation. m Where d is the feature processing dimension of each attention head, and α is the score matrix. m Then with submatrix V m Dot product yields the output b of each attention head. m The output b after parallel processing of the four attention heads 0 , b 1 , b 2 , b 3 When pieced together, the shape is (N) l The B matrix is ​​obtained by comparing it with the learned matrix W. b The dot product is linearly transformed to obtain the final output b of the multi-head self-attention layer.

[0054] Subsequently, the output *b* of the multi-head self-attention layer is passed to the first residual connection normalization layer. The residual connection normalization layer significantly improves the model's trainability and stability. It first adds the input *xʹ* and output *b* of the previous layer through residual connections, allowing the network to learn the residual features between the input and output. Then, through normalization, each row of the residual layer's output is normalized to data with a mean of 0 and a variance of 1. The normalized result *Z1* is then passed to the feedforward neural network, which consists of two linear transformation layers and an activation function (ReLU). The activation function, located between the two linear transformation layers, is responsible for performing a non-linear transformation on the output, thereby enhancing the model's expressive power. Finally, the input *Z1* and output *Z2* of the feedforward neural network are passed to the second residual connection normalization layer, which has a size of (N... l The output y of step 8) will be the final output of the Transformer module. The relevant operational equations of the Transformer module are as follows: (9) (10) (11) (12) Where F is the multi-head self-attention layer, and xʹ is a layer of size (N... l , 8) is the input of the multi-head self-attention layer, b is the output of the multi-head self-attention layer, Z1 is the result of the first residual connection normalization layer, W1 and W2 are the learnable weight matrices of the linear layer of the feedforward neural network, a1 and a2 are the bias terms of the linear layer of the feedforward neural network, Z2 is the output of the feedforward neural network, and y is a matrix of shape (N l The output of the Transformer module (8).

[0055] Classifier module 303: Finally, the classifier module classifies the output y of the Transformer module. The classifier module first averages the data along the time dimension, calculating the mean of each column of the Transformer module's output matrix y to obtain a vector of shape (1, 8). Then, it performs a dimensionality transformation through three linear layers, progressively reducing the feature dimension from the initial 8 dimensions to 2 dimensions. A ReLU activation function is added between each linear layer to increase non-linearity. Finally, the Softmax function is used to transform the 2-dimensional features processed by the linear layers into classification probabilities of the sample belonging to a defective or non-defective region. The category corresponding to the highest probability value is selected as the model's classification prediction for that pixel. The binary classification equation for defects and background implemented by the classifier module is as follows: (13) (14) (15) (16) In the formula, l is the length of the time series, and y i Here, H1 is the input at the i-th time step, H2 is the output of the first linear layer, H3 is the output of the second linear layer, P is the binary classification probability distribution of the predicted output of the classifier module, and f is the input at the ith time step. j Let J be the j-th element of the two-dimensional vector H3.

[0056] During the training phase, the training dataset is input into the defect segmentation model to obtain the binary classification probability distribution of each spatiotemporal sample. Subsequently, this invention employs the binary cross-entropy loss function as a loss metric during training to evaluate the difference between the predicted and true labels. The formula for the binary cross-entropy loss function is: (17) Where N is the batch size during model training, Y is the true label, and Ŷ is the probability value of the predicted result being a defect. By substituting the predicted probability value and the true label into Equation 17, the parameters of the final defect segmentation model are iteratively updated during training by minimizing this loss function using batch gradient descent.

[0057] During the testing phase, the test dataset is input into the trained defect segmentation model, which then outputs the spatiotemporal sample classification results pixel by pixel. Pixels identified as defective regions are assigned a value of 1, while pixels in non-defective regions are assigned a value of 0. Finally, the prediction results of all pixels are reconstructed into a binary image, thereby achieving high-precision automatic defect segmentation of the detected target.

[0058] It is important to note that the parameters of the defect segmentation model trained in this invention are configured as follows: input dimension of 9, output dimension of 2, number of self-attention heads of 4, hidden layer dimension of 8, number of multi-head self-attention layers of 1, learning rate of 0.001, and training period of 20 epochs. Furthermore, the undetermined parameters of the sampling strategy in this invention are set as follows: the length Nl of the sequence image after removing redundant data is 686, the number of defect samples collected from sample S1 is 665, and the number of defect-free samples collected is 672. Thus, through five major steps—infrared data acquisition, thermal imaging signal reconstruction, customized sampling strategy, extraction of spatiotemporal features of defects, and automatic defect segmentation—automatic detection and high-precision defect segmentation of subsurface defects in composite materials are achieved.

[0059] The applicant will now disclose relevant experimental implementation examples to further demonstrate the superiority of the technical solution of this invention.

[0060] The detection method of the present invention is used to detect two polyvinyl chloride laminates with hole-type defects on the back, so as to realize the automated detection of hole defects and quantitatively measure the area, shape and location of hole defects.

[0061] This invention first selects three polyvinyl chloride (PVC) laminates as experimental subjects, labeled S1, S2, and S3. S1 represents the sample used for collecting training data, while S2 and S3 represent the samples used for collecting test data. Each laminate measures 100 mm × 100 mm × 5 mm, and each laminate has a hole-type defect with a diameter of 2–10 mm and a depth of 2.5–4.5 mm at its bottom. The actual label images drawn based on the defect distribution of samples S1, S2, and S3 are shown below. Figure 4 As shown in a, 4b, and 4c, the specific imaging process of infrared thermal imaging is as follows: The sample is fixed on the experimental platform, with the test surface of the sample directly in front of the infrared camera. Ensure that the sample and the infrared camera are at the same horizontal level. The halogen lamps are located on both sides of the infrared camera at the same height. The infrared camera is 1.5m away from the sample surface, and the halogen lamps are 0.5m away from the sample surface to ensure the best test results.

[0062] Open the infrared image acquisition software, set the excitation time to 1.2s, the number of sequential images to 1810, the acquisition frequency to 10Hz, and the infrared camera resolution to 320×256. Then, start exciting the sample surface and simultaneously record the temperature data of the sample surface. The final acquired raw infrared images of S2 and S3 are as follows: Figure 5 As shown in a and 5b.

[0063] At this point, the infrared data acquisition is complete. Subsequently, steps S102-S105 of the infrared non-destructive testing method proposed in this invention are executed, including noise reduction and other processing on the acquired infrared sequence images. Defect segmentation of the sample is then achieved using a defect segmentation model, ultimately yielding binary defect-segmented images of the polyvinyl chloride laminate S2 and S3, as shown below. Figure 6 As shown in a and 6b.

[0064] Comparison of the differences between the predicted label images and the actual label images of samples S2 and S3: Figure 7 As shown in images a and 7b, dark yellow represents areas that are actually defects and predicted to be defects, purple represents areas that are actually defects but predicted to be defect-free, gray represents areas that are actually defect-free but predicted to be defects, and dark blue represents areas that are actually defect-free and predicted to be defect-free. Observation reveals that this invention successfully identified all defects in the two PVC laminates, and the identified defect shapes and sizes are essentially consistent with the actual defects, indicating that this invention is applicable to non-destructive testing of PVC materials.

[0065] To better evaluate the defect segmentation performance, this example uses two defect segmentation evaluation metrics: pixel accuracy (PA) and intersection-over-union (IOU). (18) (19) Where TP represents the number of pixels that are actually defects and predicted as defects, TN represents the number of pixels that are actually defect-free and predicted as defect-free, FP represents the number of pixels that are actually defect-free but predicted as defects, and FN represents the number of pixels that are actually defects but predicted as defect-free. Both of these evaluation metrics are positive indicators; a higher value indicates better defect segmentation performance.

[0066] Table 1 Quantitative evaluation results of defect segmentation images

[0067] Table 1 shows the quantitative evaluation results of defect segmentation of test samples S2 and S3. It can be seen that the detection results of the present invention have a small error compared with the actual situation, and the proposed infrared non-destructive testing method can effectively detect internal defects in polyvinyl chloride laminates.

[0068] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An infrared nondestructive testing method and system based on spatiotemporal feature information of defects, characterized in that, Includes the following steps: S101: Infrared data acquisition; during the cooling phase after thermal excitation, infrared cameras are used to acquire sequential infrared thermal images of the sample. S102: Thermal imaging signal reconstruction; The original infrared sequence heat map is reconstructed into a logarithmic domain first derivative sequence heat map using the TSR method; S103: Customized sampling strategy; Based on the actual situation of the thermal data, a sampling plan is formulated considering the balance of the samples and the rationality of the time series. S104: Defect spatiotemporal feature extraction; According to the sampling strategy, spatiotemporal information of defects is extracted pixel by pixel from the heatmap of the first derivative sequence in the logarithmic field to construct spatiotemporal samples; S105: Automatic Defect Segmentation; Construct a defect segmentation model based on Transformer, which effectively learns spatiotemporal information through a multi-head self-attention mechanism to achieve binary classification of defective and non-defective regions.

2. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 1, characterized in that, The infrared data acquisition process is as follows: First, place the infrared thermal imager 1m to 1.5m directly in front of the target, adjust the focus to ensure the image is clear and the target is centered in the imager's field of view; second, place the two halogen lamps of the long-pulse thermal loading excitation device on either side of the imager to ensure uniform heating of the sample surface; finally, set the excitation time, number of images, and acquisition frequency in the image acquisition software, open the louvers on the lamp cover to begin excitation of the sample surface, and immediately close the louvers after excitation to prevent residual infrared radiation from affecting the sample surface temperature. The acquired infrared sequence images will be displayed in a three-dimensional matrix (N... x ×N y ×N t The N is stored in the computer in the form of ) x N is the width of the infrared image. y N represents the height of the infrared image. t The number of infrared sequence images.

3. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 1, characterized in that, Thermal imaging signal reconstruction reconstructs a first-order derivative sequence image by taking the first-order derivative of the logarithmic domain fitted signal. This enhances defect contrast and reduces the effects of non-uniform heating. The formulas for logarithmic domain signal fitting and the first-order derivative of the logarithmic domain fitted signal are as follows: (1) (2) Where T is the material surface temperature, a n denoted as polynomial fitting coefficients, where n is the fitting order and t is the cooling time.

4. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 3, characterized in that, Based on the polynomial fitting order in the logarithmic domain signal fitting step, the polynomial fitting coefficient set obtained for each pixel and the t sequence of each pixel are substituted into formula (2) to obtain the dln(T) / dln(t) sequence of each pixel. This step is equivalent to converting the logarithmic sequence image into the first derivative sequence image, which effectively reduces the influence of non-uniform heating and enhances the defect contrast.

5. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 1, characterized in that, In customizing the training data sampling strategy, to ensure sample balance, a region within the defect center that does not extend beyond the defect edge is selected, and defect data is then sampled within this region. Simultaneously, defect-free samples are collected proportionally away from the defect area to maintain data balance. Regarding the time series' rationality, the infrared thermogram of the target before reaching thermal equilibrium during the cooling phase is retained, ensuring high contrast between defective and defect-free areas. The length of the sequence image after removing redundant data is N. l In customizing the test data sampling strategy, data needs to be sampled from all test areas. Therefore, sample balance is not considered. However, for the rationality of the time series, the strategy can be consistent with that of the training data.

6. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 4, characterized in that, The process of extracting spatiotemporal features of defects includes the following steps: S201: Three-dimensional spatiotemporal feature extraction; When constructing training and testing datasets, temperature curve data of all pixels within a 3×3 window of varying size are extracted over time from defective and non-defective regions selected from the first derivative sequence images of the detected target, respectively, as three-dimensional spatiotemporal feature samples. The dimension of a single three-dimensional spatiotemporal feature sample is (3×3)×N. l ; S202: Window Feature Tiling; After acquiring the 3D spatiotemporal feature training and testing datasets, window feature tiling is performed on each 3D spatiotemporal feature sample. This involves tiling the defect feature signals of all pixels contained in each frame of a 3×3 window of each 3D spatiotemporal feature sample into a one-dimensional vector containing 9 elements. Each 3D spatiotemporal feature sample will then form an N... l A one-dimensional vector; S203: Two-dimensional spatiotemporal sample reconstruction; further processing of the tiled one-dimensional vector to convert N of each three-dimensional spatiotemporal feature sample. l One-dimensional vectors at different times are stacked row-wise to reconstruct (N) l , 9) thus generate a two-dimensional sample containing the spatiotemporal information of defects in the pixel and the surrounding area; S204: Sample label assignment; Assign a corresponding classification label to each two-dimensional spatiotemporal sample in the training dataset so that the model can learn the difference between the predicted label and the true label.

7. The infrared nondestructive testing method and system based on the spatiotemporal feature information of defects according to claim 1, characterized in that, The module operation process of the defect segmentation model includes the following steps: Position encoder module 301: The position encoder module first creates a fully connected linear layer, which takes the input dimension as (N) l The two-dimensional spatiotemporal sample data of (9) is mapped as (N) l , 8), and then a learnable vector is generated for each time series position through the position encoder. The dimension of the vector is consistent with the dimension of the feature. The learned position information will be added to the mapped data to enhance the model’s perception of the sequence position. Transformer module 302: The position-encoded feature data will be input into the Transformer module for processing. The Transformer module consists of a multi-head self-attention layer, two residual connection normalization layers, and a feedforward neural network. Classifier module 303: The classifier module determines the category of the output y of the Transformer module. The classifier module first performs mean processing on the data in the time dimension, that is, calculates the mean of each column of the output matrix y of the Transformer module to obtain a vector with shape (1, 8). Then, it performs dimension transformation through three linear layers, so that the feature dimension decreases layer by layer, from the initial 8 dimensions to 2 dimensions. A ReLU activation function is added between each linear layer to increase non-linear changes.

8. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 7, characterized in that, The forward propagation equation of the position encoder module is as follows: (3) Where x is a shape of (N) l Given input samples of size (N, 9), E is a linear layer with input of 9 and output of 8, and Epos is a layer of size (N, 9). l The learnable positional encoding matrix of dimension (N, 8), where xʹ is a matrix of dimension (N, 8). l , 8) Position encoding output.

9. The infrared nondestructive testing method and system based on defect spatiotemporal feature information according to claim 8, characterized in that, The location-encoded feature data will be input into the Transformer module for processing. The Transformer module consists of a multi-head self-attention layer, two residual connection normalization layers, and a feedforward neural network. Taking a 4-layer self-attention mechanism as an example, the calculation formula is as follows: (4) (5) (6) (7) (8) Among them, W Q W K and W V Let Q be the learning matrix, K be the query matrix, K be the key matrix, and V be the value matrix. m , K m V m For a submatrix, α m Let d be the attention score matrix, and b be the feature processing dimension of each attention head. m The output for each attention head; Subsequently, the output b of the multi-head self-attention layer is passed to the first residual connection normalization layer, then through a feedforward neural network and the second residual connection normalization layer, with the final layer being of size (N). l The output y of , 8) will be used as the final output of the Transformer module. The relevant operational equations of the Transformer module are as follows: (9) (10) (11) (12) Where F is the multi-head self-attention layer, and xʹ is a layer of size (N... l , 8) is the input of the multi-head self-attention layer, b is the output of the multi-head self-attention layer, Z1 is the result of the first residual connection normalization layer, W1 and W2 are the learnable weight matrices of the linear layer of the feedforward neural network, a1 and a2 are the bias terms of the linear layer of the feedforward neural network, Z2 is the output of the feedforward neural network, and y is a matrix of shape (N l The output of the Transformer module (8).

10. The infrared nondestructive testing method and system based on the spatiotemporal feature information of defects according to claim 9, characterized in that, The Softmax function is used to transform the two-dimensional features processed by the linear layer into classification probability values ​​for whether the sample belongs to a defective or non-defective region. The category corresponding to the highest probability value is selected as the model's classification prediction result for that pixel sample. The classifier module implements the binary classification equation for defects and background as follows: (13) (14) (15) (16) In the formula, l is the length of the time series, and y i Here, H1 is the input at the i-th time step, H2 is the output of the first linear layer, H3 is the output of the second linear layer, P is the binary classification probability distribution of the predicted output of the classifier module, and f is the input at the ith time step. j Let H3 be the j-th element of the two-dimensional vector H3; During the training phase, the training dataset is input into the defect segmentation model to obtain the binary classification probability distribution of each spatiotemporal sample. Then, the binary cross-entropy loss function is used as the loss metric during training to evaluate the difference between the predicted and true labels. The formula for the binary cross-entropy loss function is: (17) Where N is the batch size during model training, Y is the true label, and Ŷ is the probability value of the predicted result being a defect. By substituting the predicted probability value and the true label into formula (17), the parameters of the defect segmentation model are finally updated iteratively by minimizing this loss function during training using the batch gradient descent method. During the testing phase, the test dataset is input into the trained defect segmentation model, which then outputs the spatiotemporal sample classification results pixel by pixel. Pixels identified as defective regions are assigned a value of 1, while pixels in non-defective regions are assigned a value of 0. Finally, the prediction results of all pixels are reconstructed into a binary image, thereby achieving high-precision automatic defect segmentation of the detected target.

Images