Three-dimensional convolution data enhancement method for infrared thermal imaging detection of composite material defects

Abstract

The invention discloses a three-dimensional convolution enhancement method for infrared thermal imaging detection of composite material defects, and belongs to the field of defect detection in an industrial process. The method comprises the following steps: (1) acquiring an infrared thermal imaging defect image data set; (2) image preprocessing; and (3) constructing a 3DCAT data enhancement model. According to the method, the defect image of the composite material is processed by adopting the three-dimensional convolution encoder, the space-time information is considered in the three-dimensional convolution operation, and the feature extraction capability of the encoder is combined, so that the model can capture potential change and defect information at different time points, and the defect and background information can be well separated. And subsequently, the most representative main component is extracted by using a dimensionality reduction method, and the defect detection is more effectively carried out.

Description

Technical Field

[0001] The present invention belongs to the field of defect detection in industrial processes, and in particular relates to a three-dimensional convolution data enhancement method for infrared thermal imaging detection of composite material defects. Background Art

[0002] With the rapid development of materials science, the application of composite materials in various fields is becoming increasingly popular. Taking carbon fiber reinforced polymer (CFRP) as an example, it has the advantages of high specific strength, high specific modulus, low density, high temperature resistance and corrosion resistance. It has been widely used to replace traditional metal materials and applied in aircraft, automobile manufacturing and energy fields. However, the complex structure makes CFRP prone to defects such as debonding, porosity, delamination, looseness and inclusions during manufacturing and use, which significantly reduce its residual stiffness and load-bearing capacity. Therefore, regular defect detection is particularly important.

[0003] At present, internal defect detection is mainly divided into two categories: destructive testing and non-destructive testing. Destructive testing is usually only for sampling inspection and may lead to product waste. In contrast, non-destructive testing has been widely used because it does not damage the inspected material, has a wide detection range and high efficiency. Current non-destructive testing methods for composite materials include ultrasonic testing, infrared thermal imaging, X-ray testing and eddy current testing. Among them, infrared thermal imaging (IRT) is widely welcomed because of its low cost, simple operation and wide range of application.

[0004] Infrared thermal imaging is a non-destructive testing method based on the principle of infrared radiation. It analyzes the surface and internal defects of the workpiece being tested by scanning and recording the temperature changes of the tested surface. Depending on whether external heat source excitation is required, infrared thermal imaging can be divided into passive and active infrared thermal imaging. Since composite materials usually do not contain internal heat sources, active infrared thermal imaging technology is generally used for defect detection. Common thermal excitation methods include pulse excitation, halogen lamp excitation, ultrasonic excitation and electromagnetic excitation. Among them, pulse excitation is widely used in industrial inspection because it is suitable for large-area surface detection.

[0005] Although IRT technology has many advantages, the thermal images obtained are often blurred and noisy. In addition, due to the high pulse excitation power and short heating and cooling time, the temperature distribution may be uneven, making the contrast between defects and background information unclear. Therefore, using data enhancement technology to reduce the uneven background and noise in the image is crucial to improve the effectiveness of infrared thermal imaging non-destructive testing.

[0006] In recent years, with the development of computer technology and information technology, deep learning algorithms have been widely used in target recognition, speech recognition, image classification and other fields. However, there are relatively few studies on using deep learning to enhance the defect information of infrared thermal imaging. Summary of the invention

[0007] In view of the above problems existing in the prior art, the present invention proposes a three-dimensional convolution data enhancement method for infrared thermal imaging detection of composite material defects, named as three-dimensional convolution autoencoder thermal imaging (3D-Convolution Autoencoder Thermography, 3DCAT) method.

[0008] The method first uses three-dimensional convolution operations to consider spatiotemporal information, combined with the feature extraction capabilities of the encoder, so that the model can capture potential changes and defect information at different time points. This method that comprehensively considers time and space information has played a significant role in infrared thermal imaging data processing, providing a more comprehensive and accurate basis for defect detection. Subsequently, the principal component analysis (PCA) method is used to reduce the dimensionality of the encoded thermal imaging data to extract the most representative main components. This method not only effectively reduces the data dimension, but also enhances the performance of defect detection. By analyzing the main components, defects in composite materials can be more clearly identified and located, thereby improving the accuracy and reliability of detection.

[0009] The technical solution adopted by the present invention to solve its technical problem is:

[0010] A three-dimensional convolution data enhancement method for infrared thermal imaging detection of composite material defects, the method comprising the following steps:

[0011] (1) Acquisition of infrared thermal imaging defect image dataset

[0012] The IRT system is used to collect data. First, the sample is transiently heated, and then the infrared camera is used to record the surface temperature change process during the cooling stage, and finally the received thermal image data is stored in the computer.

[0013] (2) Image preprocessing

[0014] In order to eliminate the impact of dimension and speed up model training, the raw thermal image data needs to be preprocessed. This process includes data expansion and standardization to ensure the consistency and comparability of the input data.

[0015] (3) Building a 3DCAT data enhancement model

[0016] The present invention constructs an autoencoder data enhancement model based on three-dimensional convolution operation, fully considers the characteristics of spatiotemporal information, and combines the feature extraction capability of the encoder to effectively capture potential changes and defect information at different time points. Through the in-depth extraction of spatiotemporal features, defect information can be effectively identified. To further verify the effectiveness of the present invention, the dimensionality reduction method is subsequently used to reduce the number of images for easy comparative analysis.

[0017] Further, the process of step (1) is:

[0018] Step 1.1: IRT system composition and data collection

[0019] The IRT system is mainly composed of a high-power excitation heat source, an infrared camera and a computer equipped with infrared thermal imaging software. During the data acquisition process, the sample is instantaneously heated, and then the infrared camera is used to record its surface temperature changes during the cooling stage. The computer is responsible for receiving and storing the recorded data.

[0020] Step 1.2: Preparation and collection of defect samples

[0021] The defect sample used in the present invention is made of carbon fiber sheets and epoxy resin through a resin transfer molding instrument. Then, it is collected through an IRT system, and finally n t thermal images, each of size n x ×n y Pixels. Through computer collection, we can get n t ×n x ×n y A three-dimensional matrix.

[0022] Further, the process of step (2) is:

[0023] Step 2.1: Unfolding of the 3D data matrix

[0024] Thermal image data collected by an IRT system is essentially a three-dimensional matrix with a dimension of n t ×n x ×n y , indicating that there are a total of n t thermal images, each of size n x ×n y Since the three-dimensional matrix cannot be trained directly, it is necessary to expand the three-dimensional matrix into a two-dimensional matrix n t ×n x n y .

[0025] Step 2.2: Normalization of the two-dimensional matrix

[0026] In order to eliminate uneven background and speed up model training, the two-dimensional matrix is ​​row-normalized. The specific method is to subtract the mean value of each row of data and divide it by the standard deviation of the row to achieve data standardization.

[0027] Further, the process of step (3) is:

[0028] Step 3.1: Build 3DCAE network

[0029] 3D-Convolution Autoencoder (3DCAE) is a deep learning model for processing three-dimensional data (such as videos, medical images, volumetric data, etc.). Unlike traditional two-dimensional convolutional autoencoders, 3D convolutional autoencoders use three-dimensional convolution kernels to extract spatial and temporal features. 3DCAE introduces an additional dimension for convolution calculation on the basis of two-dimensional convolution. The convolution kernel of 3DCAE is designed as a three-dimensional sliding window, which moves in three dimensions of the data, multiplies the data element by element and sums them to generate an output feature map. Assume that the input feature map of the c'th channel of the l-1th layer is A c’ l-1 , the value of the cth channel at position (d, h, w) in the lth layer output feature map It is expressed as:

[0030]

[0031] In the formula, Represents the convolution kernel K of the lth layer l The weight between the output channel c and the input channel c', the value of the convolution kernel at position (i, j, k), c represents the index of the output channel, h, w and d represent the position index of height, width and depth respectively, C l-1 represents the total number of channels of the input feature map of the l-1 layer, D k , H k and W k Respectively represent the size of the three-dimensional convolution kernel in depth, height and width, represents the bias term of the output channel c of the lth layer, and f(·) represents the activation function.

[0032] The structure of the 3DCAE model usually includes two parts: encoder and decoder. Generally, the autoencoder consists of an encoder and a decoder. The former compresses the input data to obtain the essential characteristics of the data, and the latter is responsible for reconstructing the input data with the minimum error. Assume X∈R M×N×T represents a set of infrared thermal imaging data, M, N and T represent its length, width and number of sheets respectively, then the process of extracting infrared thermal imaging data features and reconstructing using 3DCAE is expressed as:

[0033] XL = f(W en ·X + b en ) (2)

[0034]

[0035] where X L ∈ R M×N×t (t << T) represents the features extracted by the encoder, t is the number of feature maps, W en and b en represent the weight values and biases of the encoder; represents the infrared thermal imaging data reconstructed by the decoder, W de and b de represent the weight values and biases of the decoder, and f(·) represents the activation function.

[0036] Designing multi-layer three-dimensional convolutional layers and three-dimensional deconvolutional layers in the encoder and decoder can construct a three-dimensional convolutional autoencoder. In a neural network model, generally, the more layers (mainly hidden layers), the higher the level of abstraction of the input features, and the more capable of fitting complex sample features. The input of the neurons in the latter layer is the weighted sum of the output of the previous layer, that is, the features of the previous layer are abstracted in the latter layer. The process of model learning is the process of adjusting and optimizing the connection weights and thresholds. Similarly, by adding convolutional layers and deconvolutional layers within a reasonable range to obtain a three-dimensional convolutional autoencoder model, the convolutional autoencoder can learn features with a high level of data abstraction, that is, extract essential defect features and remove uneven backgrounds and noises.

[0037] Step 3.2: Training the 3DCAT model

[0038] Apply the three-dimensional convolutional autoencoder method to infrared thermal imaging image processing, and name the model 3DCAT. Use the standardized two-dimensional matrix as a sample for model training, set the number of network layers, the number of neurons, the activation function, and the number of training epochs, and finally save the trained model.

[0039] Step 3.3: Data dimensionality reduction

[0040] Use the encoder part of the model obtained in Step 3.2 to extract features layer by layer from the thermal image data, and obtain high-level abstract features of the image (which has achieved the effect of removing noise and uneven backgrounds) in the intermediate convolutional layer. Since the number is large, use the common PCA for dimensionality reduction.

[0041] Step 3.4: Data augmentation and visualization

[0042] Use the dimensionality-reduced feature maps obtained in Step 3.3 and visualize them for defect detection.

[0043] The beneficial effects of the present invention are mainly manifested in: the present invention proposes a three-dimensional convolution data enhancement method for composite material defect infrared thermal imaging detection. The three-dimensional convolution operation takes into account the spatiotemporal information and combines the feature extraction capability of the encoder to enable the model to capture potential changes and defect information at different time points, and can better separate defects from background information. Subsequently, the most representative main components are extracted using the dimensionality reduction method to more effectively detect defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 Schematic diagram of the structure of the three-dimensional convolutional autoencoder of the present invention;

[0046] Figure 3 is a defect location distribution map of the thermal image data set of the present invention;

[0047] Figure 4 It is a schematic diagram of the data enhancement result of the method of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings.

[0049] Reference Figure 1 to Figure 4 A three-dimensional convolution data enhancement method for composite material defect infrared thermal imaging detection, the method comprising the following steps:

[0050] (1) Obtaining infrared thermal imaging defect image dataset

[0051] The IRT system is used to collect data. First, the sample is transiently heated, then the surface temperature change process is recorded with an infrared camera during the cooling stage, and finally the received thermal image is stored in a computer. The details are as follows:

[0052] Step 1.1: IRT system composition and data collection

[0053] The thermal imaging data used in the experiment is obtained through the IRT system and collected and processed by a computer.

[0054] Step 1.2: Preparation of defective samples

[0055] The six-defect sample used in this embodiment is made of carbon fiber sheets and epoxy resin through a resin transfer molding instrument, and three different sizes of square Teflon strips are inserted into the carbon fiber plate as defects, and the sizes from large to small are: 1.6cm×1.6cm, 0.8cm×0.8cm and 0.4cm×0.4cm. The three Teflon strips on the left are covered by a layer of fiber sheets, and the Teflon strips on the right are covered by two layers of fiber sheets, and the thickness of each fiber sheet is 0.26cm. The acquisition process of the IRT system is only 2 seconds. After removing the lossy data, 59 thermal images are obtained, and the size of each thermal image is 90×100. A three-dimensional matrix of 59×90×100 is collected by a computer.

[0056] (2) Image preprocessing

[0057] In order to eliminate uneven background and speed up model training to a certain extent, the original thermal image data needs to be preprocessed, which consists of two parts: data expansion and standardization. The details are as follows:

[0058] Step 2.1: Unfolding of the 3D matrix

[0059] The six-defect thermal image data collected by the IRT system is essentially a three-dimensional matrix (59×90×100). Since the three-dimensional matrix cannot be directly used for model training, it needs to be expanded into a two-dimensional matrix (59×9000).

[0060] Step 2.2: Normalization of the two-dimensional matrix

[0061] In order to eliminate uneven background and speed up model training to a certain extent, it is necessary to perform row normalization on the two-dimensional matrix by subtracting the mean value μ of each row and dividing it by the standard deviation σ of each row to center it. The specific formula is as follows:

[0062]

[0063] Among them, x′ is the standardized data and x is the original data.

[0064] (3) Building a 3DCAT data enhancement model

[0065] Step 3.1: Build 3DCAE network

[0066] The structure of the 3DCAE model usually includes two parts: encoder and decoder. Generally, the autoencoder consists of an encoder and a decoder. The former compresses the input data to obtain the essential characteristics of the data, and the latter is responsible for reconstructing the input data with the minimum error. Assume X∈R M×N×T represents a set of infrared thermal imaging data, M, N and T represent its length, width and number of sheets respectively, then the process of extracting infrared thermal imaging data features and reconstructing using 3DCAE is expressed as:

[0067] X L = f(W en · X + b en ) (5)

[0068]

[0069] where X L ∈ R M×N×t (t << T) represents the features extracted by the encoder, t is the number of feature maps, W en and b en represent the weight values and biases of the encoder; represents the infrared thermal imaging data reconstructed by the decoder, W de and b de represent the weight values and biases of the decoder, and f(·) represents the activation function.

[0070] Step 3.2: Train the 3DCAT model

[0071] Apply the three-dimensional convolutional autoencoder method to infrared thermal imaging image processing, and the model is named 3DCAT. Use the standardized two-dimensional matrix as a sample for model training. Set 3 convolutional layers and 3 deconvolutional layers in the encoder and decoder respectively. The number of neurons in each convolutional layer is 8, 16, and 64 respectively, and the number of neurons in each deconvolutional layer is 16, 8, and 1 respectively. An activation function is set for each layer. The Tanh function is used in the last convolutional layer, and the LeakyReLU activation function is used in the remaining layers. Train the training samples in batches, then perform backpropagation to update the model parameters, select the Adam optimizer to optimize the parameters, and finally save the trained model.

[0072] Step 3.3: Data dimensionality reduction

[0073] Input the data into the model trained in Step 3.2 and extract the encoder. Subsequently, use the common PCA for dimensionality reduction.

[0074] Step 3.4: Visualization of data augmentation

[0075] Utilize the dimensionality-reduced feature maps obtained in Step 3.3 and visualize them for defect detection.

[0076] To evaluate the effectiveness of the proposed method, we need an index to quantitatively evaluate the quality of the results. In this embodiment, the Contrast-to-Noise Ratio (CNR) is used, and this index is usually used for quantitative evaluation of thermal image defects. Its calculation formula is as follows:

[0077]

[0078] Among them, m def and m in are the average values ​​of the pixel values ​​in the defective area and the non-defective area in the thermal image, σ in is the standard deviation of the pixel values ​​in the non-defective area. CNR reflects the contrast between the defective area and the non-defective area. The larger the CNR value, the stronger the defect recognition ability of the method. Table 1 compares the CNR values ​​of the original thermal image and the different defects after data processing by the 3DCAT method. From the results, it can be seen that the method of this embodiment has a strong defect recognition ability.

[0079] Table 1 Comparison of CNR values ​​of different defects before and after data processing

[0080]

[0081] From the above comparison results, it can be seen that the defect detection performance of the present invention is much higher than that of the traditional principal component model. The present invention proposes a three-dimensional convolution data enhancement method for composite material defect infrared thermal imaging detection, which improves the accuracy of defect recognition and positioning.

[0082] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A three-dimensional convolution data enhancement method for infrared thermal imaging detection of composite material defects, characterized in that: The method comprises the following steps: (1) Acquisition of infrared thermal imaging defect image dataset Infrared thermal imaging technology (IRT) is used to collect data on the samples. First, the samples are heat treated by transient heating, and then in the cooling stage, the surface temperature change process is recorded by an infrared camera, and finally the received thermal image data is stored in the computer system. (2) Image preprocessing In order to improve the efficiency of model training and reduce the impact of background inhomogeneity on the results, the raw thermal image data is preprocessed, including data expansion and standardization, to optimize subsequent analysis; (3) Building a data enhancement model Construct an autoencoder data enhancement model based on three-dimensional convolution operations. The model takes into account the characteristics of spatiotemporal information and combines the feature extraction capabilities of the encoder to effectively capture potential changes and defect information at different time points. The model can effectively identify defect information by extracting spatiotemporal features. (4) To further verify the effectiveness of the method, the dimensionality reduction method is used to reduce the number of images to facilitate comparative analysis.

2. The three-dimensional convolution data enhancement method for composite material defect infrared thermal imaging detection as claimed in claim 1, wherein in step (1), the IRT system includes a high-power excitation heat source, an infrared camera and a computer equipped with infrared thermal imaging software; during the data acquisition process, the sample is instantaneously heated, and then the infrared camera is used to record its surface temperature change during the cooling stage, and the computer is responsible for receiving and storing the recorded data; the specific defect image data set acquisition process is: Carbon fiber sheets and epoxy resin were used to make defect samples through resin transfer molding equipment; defect samples were collected through IRT system to obtain n t thermal images, each of size n x ×n y Pixels (where n t Represents the number of images, n x and n y Represents the number of pixels in the width direction and the number of pixels in the height direction of the image respectively); collected by computer, we get n t ×n x ×n y A three-dimensional matrix.

3. The three-dimensional convolution data enhancement method for composite material defect infrared thermal imaging detection according to claim 1, wherein the process of step (2) is: Step 2.1: Unfolding of the 3D matrix The thermal image data collected by the IRT system essentially forms a three-dimensional matrix, which is unfolded into a two-dimensional matrix; Step 2.2: Normalization of the two-dimensional matrix For a two-dimensional matrix n t ×n x n y Perform row standardization. The specific method is: subtract the mean value of each row of data and divide it by the standard deviation of the row to achieve data standardization.

4. The three-dimensional convolution data enhancement method for composite material defect infrared thermal imaging detection according to claim 1, wherein the process of step (3) is: Step 3.1: Build a 3D Convolutional Autoencoder 3DCAE network The three-dimensional convolutional autoencoder 3DCAE model is constructed by designing multiple layers of three-dimensional convolutional layers and three-dimensional deconvolutional layers in the encoder and decoder. The three-dimensional convolutional autoencoder 3DCAE uses a three-dimensional convolutional kernel to extract spatial and temporal features, which introduces an additional dimension for convolution calculation on the basis of 2D convolution; in the three-dimensional convolutional autoencoder 3DCAE model, the convolutional kernel is designed as a three-dimensional sliding window, which moves in three dimensions of the data, and multiplies and sums the data element by element to generate an output feature map; Step 3.2: Training the 3DCAT model The three-dimensional convolutional autoencoder method is applied to infrared thermal imaging image processing to construct a 3DCAT model. During the model training process, both the encoder and the decoder set n convolutional layers and deconvolutional layers and their number of neurons, set an appropriate activation function after each convolutional layer, and select an appropriate optimizer for parameter optimization. Finally, the trained model is saved. Step 3.3: Data Dimensionality Reduction Use the encoder of the model obtained in step 3.2 to extract features of the thermal image data layer by layer, obtain high-level abstract features of the image from the third convolutional layer, and use principal component analysis PCA for dimensionality reduction; Step 3.4: Data augmentation visualization By using the reduced-dimensional feature map obtained in step 3.3 and visualizing it, defects can be detected.

Images