A dual-model defect detection method, training method, and device based on Bi-LSTM neural network.

Abstract

This invention provides a dual-model defect detection method, training method, and apparatus based on a Bi-LSTM neural network. The dual-model defect detection method includes: acquiring N frames of infrared thermal images during the cooling process; acquiring a pulsed thermal imaging data sequence of each pixel on the test surface of the workpiece as a function of time from the N frames of infrared thermal images; inputting the pulsed thermal imaging data sequence into a trained Bi-LSTM neural network classification model to obtain a binary classification map representing whether the pixel location is a defect or not; inputting the pulsed thermal imaging data sequence into a trained Bi-LSTM neural network regression model to obtain a defect depth prediction map representing the defect depth; and obtaining the defect distribution of the workpiece from the binary classification map and the defect depth prediction map. This invention does not require prior information such as material thermal properties, reference areas, or characteristic times, and is more convenient to apply.

Description

Technical Field

[0001] This invention relates to the field of nondestructive testing technology for workpiece defects, and in particular to a dual-model defect detection method, training method, and apparatus based on a Bi-LSTM neural network. Background Technology

[0002] Pulse thermal imaging nondestructive testing technology can be used for non-contact, large-area, qualitative and quantitative detection of internal defects in materials. With increasingly stringent requirements for material quality, particularly regarding the safety and economic needs of in-service equipment, greater attention is being paid to the three-dimensional structural information of internal material defects. For the quantitative measurement of the three-dimensional structure of internal material defects, researchers have proposed methods such as thermal contrast analysis (defect depth prediction), logarithmic second derivative peak value method (defect depth prediction), full width at half maximum (FWHM) method (defect size prediction), and deconvolution method (defect size prediction).

[0003] However, while the above methods can achieve three-dimensional reconstruction of defects, they require prior information such as the thermal properties of the material, characteristic time, or reference area. In many detection scenarios, this information is not known, or even if it is known, the original prior information cannot guarantee the accuracy of the detection due to differences in parameters such as material and shape. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] The present invention aims to solve at least one of the above-mentioned technical problems in at least part.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, according to one aspect of the present invention, a dual-model defect detection method based on a Bi-LSTM neural network is provided.

[0008] This dual-model defect detection method includes:

[0009] Step A': Obtain N frames of infrared thermal images showing the change of the test surface of the workpiece under test over time during the cooling process;

[0010] Step B': Obtain the pulsed thermal imaging data sequence SEQ' of the pixel points on the test surface of the workpiece under test changing over time from the N frames of infrared thermal images;

[0011] Step F: Input the pulsed thermal imaging data sequence SEQ' of each pixel into the trained Bi-LSTM neural network classification model to obtain a binary classification map that represents whether the pixel position of the workpiece under test is a defect or not. When the corresponding position is a defect, the value of the corresponding pixel in the binary classification map is 1; otherwise, the value of the corresponding pixel in the binary classification map is 0.

[0012] Step G: Input the pulsed thermal imaging data sequence SEQ' of each pixel into the trained Bi-LSTM neural network regression model to obtain a defect depth prediction map that represents the defect depth at the pixel corresponding position of the workpiece under test. The value of the corresponding pixel in the defect depth prediction map is the defect depth prediction value.

[0013] Step H: The defect distribution of the workpiece under test is obtained from the binary classification map and the defect depth prediction map.

[0014] In some embodiments of the present invention, step B' further includes: dividing the 1×N pulsed thermal imaging data sequence SEQ' into W segments of 1×N / W pulsed thermal imaging data subsequences in chronological order; in step F, inputting the pixel pulsed thermal imaging data sequence SEQ' into the trained Bi-LSTM neural network classification model includes: independently inputting the W segments of pulsed thermal imaging data subsequences into the trained Bi-LSTM neural network classification model; in step G, inputting the pixel pulsed thermal imaging data sequence SEQ' into the trained Bi-LSTM neural network regression model includes: independently inputting the W segments of pulsed thermal imaging data subsequences into the trained Bi-LSTM neural network regression model; wherein, W is the same as the segment value during the training process of the Bi-LSTM neural network classification model and the Bi-LSTM neural network regression model, and 2≤W≤5.

[0015] In some embodiments of the present invention, in step B', a pulsed thermal imaging data sequence of all pixels on the test surface of the workpiece under test changing over time is obtained; after step H, the method further includes: step I, using the mesh function in Matlab software to reconstruct the three-dimensional defect structure of the workpiece under test from the defect distribution.

[0016] In some embodiments of the present invention, step A' includes: acquiring a background infrared thermal image of the test surface of the workpiece under test before heating using an infrared thermal imager; heating the workpiece under test using a high-energy flash lamp, and acquiring N frames of original infrared thermal images of the test surface of the workpiece under test changing over time during the cooling process using an infrared thermal imager; subtracting the background infrared thermal image before heating from the N frames of original infrared thermal images during the cooling process to obtain N frames of infrared thermal images of the test surface of the workpiece under test changing over time during the cooling process.

[0017] To achieve the above objectives, according to a second aspect of the present invention, a training method based on a Bi-LSTM neural network is also provided.

[0018] The training method includes:

[0019] Step A: Obtain N frames of infrared thermal images showing the change of the test surface of the training workpiece over time during the cooling process. The training workpiece and the workpiece to be tested have the same thermal properties. The training workpiece has M defect regions with different depths. The depth of each defect region is known, and M≥2.

[0020] Step B1: Obtain the pulsed thermal imaging data sequence of each pixel point of the defect area on the test surface of the training workpiece as a function of time from the N frames of infrared thermal images: SEQ D,1 , ..., SEQ D,s , ..., SEQ D,S Where S is the total number of pixels in the defect area, SEQ D,s The pulsed thermal imaging data sequence of the s-th pixel within the defect area as a function of time;

[0021] Step B2: Obtain the pulsed thermal imaging data sequence of each pixel point on the non-defect area of ​​the test surface of the training workpiece as a function of time from the N frames of infrared thermal images: SEQ U,1 , ..., SEQ U,t , ..., SEQ U,T Where T is the total number of pixels in the non-defect region, SEQ U,t The pulsed thermal imaging data sequence of the t-th pixel within the non-defect region as a function of time;

[0022] Step C1: Construct a regression model training dataset from the pulsed thermal imaging data sequence of each pixel in the defect area over time and the corresponding defect depth;

[0023] Step C2: Use the data from the regression model training dataset to train the Bi-LSTM neural network to obtain the Bi-LSTM neural network regression model;

[0024] Step D1: Construct a classification model training dataset from the pulsed thermal imaging data sequence of each pixel point in the defective and non-defective regions as a function of time and the corresponding defect categories.

[0025] Step D2: Use the data from the training dataset of the classification model to train the Bi-LSTM neural network to obtain the Bi-LSTM neural network classification model.

[0026] In some embodiments of the present invention, the regression model training dataset is: in, Let be the normalized defect depth at the known position corresponding to the s-th pixel, where s = 1, 2, ..., S.

[0027] In some embodiments of the present invention, in the classification model training dataset, it is defined that if the pixel corresponds to a defect, the defect category is 1; otherwise, the defect category is 0. The classification model training dataset is: (SEQ D,1 ,1),...,(SEQ D,s ,1),...,(SEQ D,S ,1) (SEQ U,1 ,0),...,(SEQ U,t ,0),...,(SEQ U,T ,0).

[0028] In some embodiments of the present invention, the non-defect region is defined as an annular region outside the defect region, and the number of pixels T in the non-defect region satisfies: 0.5S≤T≤1.5S.

[0029] In some embodiments of the present invention, steps B1 and B2 further include: dividing the 1×N pulsed thermal imaging data sequence into W segments of 1×N / W pulsed thermal imaging data subsequences in chronological order; in each training step C2 and D2, the W segments of 1×N / W pulsed thermal imaging data subsequences are independently input into the Bi-LSTM neural network for training, where 2≤W≤5.

[0030] In some embodiments of the present invention, step B1 further includes: multiplying each frame of infrared thermal image with a digital mask image of the defect area to filter out pixels in the defect area, wherein the pixel value of the defect area in the digital mask image of the defect area is 1, and the pixel value of other areas is 0; step B2 further includes: multiplying each frame of infrared thermal image with a digital mask image of the non-defect area to filter out pixels in the non-defect area, wherein the pixel value of the non-defect area in the digital mask image of the non-defect area is 1, and the pixel value of other areas is 0; before steps B1 and B2, the present invention further includes: acquiring a visible light digital photograph of the test surface of the training workpiece; performing image segmentation on the visible light digital photograph to obtain a binary digital photograph of the defect area of ​​the test surface of the training workpiece; registering the binary digital photograph of the defect area with the photograph of the infrared thermal image to obtain the digital mask image of the defect area; selecting an annular region around the defect area of ​​the registered two-dimensional digital photograph of the defect area as the non-defect area to obtain the digital mask image of the non-defect area.

[0031] In some embodiments of the present invention, the normalized defect temperature in step C1 is obtained by the following method: Among them, D D,s D represents the physical depth of the defect at the pixel location. mean D max D minThese represent the average, maximum, and minimum values ​​of the defect depth in the training workpiece, respectively.

[0032] Steps C2 and D2 include: feeding the training data from the classification model training dataset into the Bi-LSTM neural network, and inputting the feature vector O extracted by the forward LSTM. f and the feature vector O extracted by inverse LSTM b The feature vector O is obtained by concatenating the features.

[0033] Step C2 further includes: multiplying the feature vector O with the learning matrix V to obtain the deep regression value y. gp , the deep regression value y gp Inversely mapped to its true scale depth y gm y gm =y gp ×(D max -D min )+D mean During training, the L1 distance loss function F is used. L1 :F L1 =|y gp -y g |;

[0034] Step D2 further includes: multiplying the feature vector O with the learning matrix U to obtain the probability vector f. c : The normalized probability vector y with dimension 1×C is obtained by applying the softmax function. cp : Among them, y cp The values ​​in dimension C are summed to 1. The values ​​in dimension C represent the probability of predicting a defect or not. We choose y. cp The index value with the highest probability in dimension C is the predicted category, and the following loss function is used: Among them, y c This represents the ground truth value for the defect category in the training data.

[0035] To achieve the above objectives, according to a second aspect of the present invention, an apparatus is also provided.

[0036] The device includes: a digital memory; and a processor electrically coupled to the digital memory, configured to execute the dual-model defect detection method as described above, or the training method as described above, based on instructions stored in the digital memory.

[0037] (III) Beneficial Effects

[0038] As can be seen from the above technical solution, the present invention has at least one of the following beneficial effects:

[0039] (1) The Bi-LSTM neural network is trained using the training dataset of the training workpiece with the same thermal properties as the workpiece to be tested. It does not require prior information such as material thermal properties, reference area or feature time. This facilitates detection while ensuring the accuracy of each defect detection.

[0040] Bi-LSTM neural networks are trained using pulsed thermal imaging data sequences during the cooling process. These pulsed thermal imaging data sequences do not require any algorithmic processing (such as Fourier transform algorithms, principal component analysis algorithms, etc.) or manual extraction of feature data for network training. Bi-LSTM neural networks can achieve fully automated data analysis, making them more convenient to use.

[0041] Furthermore, compared to a single LSTM network, the Bi-LSTM neural network can avoid the loss of previous signal features, infer the long-term and short-term dependencies between signal values ​​at each time step and global signal values, and ultimately extract rich global data features, which can improve the accuracy of classification tasks or regression tasks.

[0042] (2) The pulsed thermal imaging data sequence is divided into different sub-sequences, and the length dependence between the signals in each interval is extracted using a bidirectional LSTM network. Experiments show that the length setting of the interval is sensitive, and this method of dividing the interval can help improve the accuracy of defect inversion.

[0043] (3) In the process of predicting the size of defects, the influence of lateral thermal diffusion is considered. The cooling sequence data of the pixels corresponding to the non-defect area adjacent to the defect is used as the non-defect dataset, so that the network can better learn the difference between the pulse thermal imaging data signal corresponding to the defect area and the pulse thermal imaging data signal corresponding to the non-defect area, especially the pulse thermal imaging data signal corresponding to the defect edge area, and effectively improve the accuracy of predicting the size of defects.

[0044] (4) In the process of predicting the depth of defects, non-defect data was not used as training data, which can effectively improve the accuracy of the depth prediction of defect edges.

[0045] (5) The network hyperparameters are the same for classification and regression tasks, and manual operation only requires adjusting the network hyperparameters once, making it more convenient to reconstruct the three-dimensional structure of defects.

[0046] (6) After obtaining the defect depth data of all pixels on the test surface of the workpiece to be tested, the three-dimensional structure of the internal defects of the workpiece to be tested can be reconstructed more intelligently by combining the mesh function of Matlab, which facilitates observation and use. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating the working principle of Bi-LSTM.

[0048] Figure 2 LSTM network architecture diagram

[0049] Figure 3A This is a flowchart of the training method based on the Bi-LSTM neural network of the present invention.

[0050] Figure 3B This is a flowchart of the dual-model defect detection method based on Bi-LSTM neural network of the present invention.

[0051] Figure 4 This is a schematic diagram of a pulsed infrared thermal imaging experiment for training workpieces and workpieces to be tested.

[0052] Figure 5 This is a schematic diagram of the normalization preprocessing of pulsed thermal imaging data sequences in a Bi-LSTM network during the cooling process of this invention.

[0053] Figure 6 This is a schematic diagram illustrating how W segments of pulsed thermal imaging data are independently input into the Bi-LSTM neural network for training during one training process.

[0054] Figure 7A A top view of the carbon fiber flat-bottom hole training workpiece and a cross-sectional schematic diagram of the defect.

[0055] Figure 7B This is a three-dimensional structural diagram of the carbon fiber flat-bottom hole workpiece to be tested.

[0056] Figure 8 This is a sequence of cooling data corresponding to regions with depths of 1.1 mm, 1.64 mm, 2.19 mm, 2.81 mm, and 10 mm.

[0057] Figure 9 Thermal image of a carbon fiber flat-bottom hole training workpiece 8 seconds after flash.

[0058] Figure 10 The thermal image of the carbon fiber flat-bottom hole workpiece 8 seconds after the flash.

[0059] Figure 11 A digital mask image A for the defect area of ​​the training workpiece.

[0060] Figure 12 Digital mask B for the non-defect area of ​​the training workpiece.

[0061] Figure 13 This is a binary classification diagram of defects and non-defects in the workpiece under test.

[0062] Figure 14C is the mask image of the workpiece to be tested.

[0063] Figure 15 This is a depth prediction map of the workpiece to be measured.

[0064] Figure 16 This is a diagram of the defect structure of the workpiece to be tested.

[0065] Figure 17 Three-dimensional structural diagram of the flat-bottom hole defect in the workpiece under test. Detailed Implementation

[0066] This invention applies the Bi-LSTM neural network algorithm to pulsed thermal imaging technology, automatically learning the characteristics of pulsed thermal imaging data sequences. The pulsed thermal imaging data sequences of each pixel on the test surface of the test workpiece are input into the trained Bi-LSTM neural network to automatically reconstruct the three-dimensional structure of the defects built into the test workpiece.

[0067] First, before introducing the specific embodiments of the present invention, a brief explanation of the Bi-LSTM neural network will be given.

[0068] Bi-LSTM neural networks are a type of recurrent neural network (RNN) that has the advantage of describing the dependencies between signals, making them very effective for processing sequential information. A Bi-LSTM neural network trains two LSTMs simultaneously, with independent parameters and opposite directions. Finally, the feature vectors extracted by the two LSTMs are concatenated to capture both past and future features. The working principle of Bi-LSTM is as follows: Figure 1 O f and O b These represent the feature vectors extracted by the forward LSTM and the backward LSTM, respectively. The LSTM employs a gating mechanism, including a forget gate, an input gate, and an output gate, and its structure is as follows: Figure 2 As shown, gating is used to control the signal transmission state, thereby selectively remembering features of important information. In the diagram, ⊙ represents the multiplication of corresponding elements in the feature matrix. This represents the summation of corresponding elements in the feature matrix, where σ and tanh represent the sigmoid and tanh nonlinear activation functions, respectively. The main calculations involved are shown in formulas (1) to (6). The forgetting gate uses the function z... f Selectively forget the output c of the previous node t-1 W f With b f The forget gate needs to learn the weights and biases. The input gate is then passed through the function z. i Perform control and retain the current time c. t The state information that needs to be stored, W i With b i The input gate needs to learn the weights and biases. The output gate passes through the function z.o Determine the current neural node c t The output needs to be sent to the hidden state h. t Information, W o With b o The input gate needs to learn the weights and biases, and the network parameters W also need to be learned. c and b c In summary, this unique gating mechanism is well-suited for processing sequential data. It can not only learn long-term dependencies between data points but also, to some extent, address the vanishing and exploding gradient problems encountered by standard recurrent neural networks during feature extraction.

[0069]

[0070] z f =σ(W f ·x+b f (2)

[0071] z i =σ(W i ·x+b i (3)

[0072] z o =σ(W o ·x+b o (4)

[0073] c t =z f ⊙c t-1 +z i ⊙tanh(W c ·x+b c (5)

[0074] h t =z o ⊙tanh(c t (6)

[0075] Those skilled in the art will understand that a Bi-LSTM neural network simultaneously trains a forward LSTM network and a backward LSTM network. The hidden layer units of these two LSTM networks can influence and promote each other. Then, by concatenating the feature vectors extracted by the forward and backward LSTM networks, the global features of the sequence data can be captured. Furthermore, compared to a single LSTM network, the Bi-LSTM neural network can avoid the loss of previous signal features, infer the long-term and short-term dependencies between signal values ​​at each time step and the global signal values, and ultimately extract rich global data features, thereby improving the accuracy of classification or regression tasks.

[0076] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0077] Secondly, the training and detection parts involved in this invention will be described as follows.

[0078] The technical solution of this invention is divided into two parts: a training part and a detection part. Those skilled in the art should understand that after the Bi-LSTM neural network is trained, it can be directly used for testing. Therefore, the detection part can be implemented independently of the training part. Similarly, the training part can also be implemented independently of the detection part. In summary, the implementation of this invention, whether it involves implementing the detection part alone, implementing the training part alone, or implementing both the detection and training parts together, is all within the scope of protection of this invention. In the detailed description of the embodiments below, there are similar technical contents between the training and detection parts; the description will mainly focus on their differences. Identical content will not be elaborated further, and those skilled in the art can refer to the preceding content.

[0079] Figure 3A This is a flowchart of the training method based on the Bi-LSTM neural network of the present invention. Figure 3B This is a flowchart of the dual-model defect detection method based on Bi-LSTM neural network of the present invention. The following is in conjunction with... Figure 3A and Figure 3B The present invention will be described.

[0080] I. Training Section

[0081] Please refer to Figure 3A The training method for the Bi-LSTM neural network of this invention includes:

[0082] Step A: Obtain N frames of infrared thermal images showing the change of the test surface of the training workpiece over time during the cooling process. The training workpiece and the workpiece to be tested have the same thermal properties. The training workpiece has M defect regions with different depths. The depth of each defect region is known, and M≥2.

[0083] First, prepare a training workpiece with the same thermal properties as the workpiece to be tested. Design multiple defects of a certain size and different depths within the training workpiece. It is important to note that the defect depth range of the training workpiece includes the defect depth range of the workpiece to be tested.

[0084] Figure 4 This is a schematic diagram of a pulsed infrared thermal imaging experiment for training and testing workpieces. Figure 4As shown, the flash lamp and infrared thermal imager are placed on the same side of the specimen. Before and after the flash heating of the training workpiece and the test surface, the thermal imager monitors and records the thermal wave time sequence signal of the specimen surface changing over time in real time.

[0085] Please combine Figure 3A and Figure 4 Step A of the training section further includes:

[0086] Sub-step A1: Use an infrared thermal imager to acquire the background infrared thermal image of the test surface of the training workpiece before heating;

[0087] Sub-step A2 involves heating the training workpiece using a high-energy flash lamp and acquiring N frames of original infrared thermal images of the test surface of the training workpiece changing over time during the cooling process using an infrared thermal imager.

[0088] Sub-step A3: Subtract the background infrared thermal image before heating from the N frames of original infrared thermal images during the cooling process to obtain N frames of infrared thermal images showing the change of the test surface of the training workpiece over time during the cooling process.

[0089] In practice, an infrared thermal imager is used to acquire 10 frames of thermal images of the training workpiece or the workpiece under test before heating at a preset acquisition frequency. These unheated thermal image sequences are stored in a general-purpose memory. While the training workpiece or the workpiece under test is pulse-heated using a flash lamp heating device, the infrared thermal imager is used at a preset acquisition frequency and a preset acquisition time (for shallower defects, the preset acquisition frequency can be appropriately increased, and the acquisition time can be appropriately decreased, and vice versa; however, the total amount of data acquired must not exceed the thermal imager's memory) to obtain an infrared thermal image sequence of the cooling process of the object's surface. This sequence is also stored in the general-purpose memory. Subtracting the average unheated thermal image from each frame of the pulse-heated thermal image of the training workpiece or the workpiece under test yields N frames of infrared thermal images showing the change of the test surface over time during the cooling process.

[0090] After step A is completed, proceed to steps B1 and B2. Steps B1 and B2 can be performed sequentially and in an adjustable order, or they can be performed simultaneously.

[0091] In steps B1 and B2, all thermal imaging data sequences of pixels belonging to defect areas in the training workpiece are categorized into a defect category dataset, while thermal imaging data sequences of pixels belonging to non-defect areas (which are adjacent to defect areas) are categorized into a non-defect category dataset. It is important to note that the non-defect category dataset and the defect category dataset should be similar and not significantly different. The non-defect category dataset and the defect category dataset are then input into a Bi-LSTM network for classification learning, training the Bi-LSTM network's classification model.

[0092] Step B1: Obtain the pulsed thermal imaging data sequence of each pixel point of the defect area on the test surface of the training workpiece as a function of time from N frames of infrared thermal images: SEQ D,1 , ..., SEQ D,s , ..., SEQ D,S S is the total number of pixels in the defect area, SEQ D,s The pulsed thermal imaging data sequence of the s-th pixel within the defect area as a function of time;

[0093] Step B2: Obtain the pulsed thermal imaging data sequence of each pixel point on the non-defect area of ​​the test surface of the training workpiece as a function of time from N frames of infrared thermal images: SEQ U,1 , ..., SEQ U,t , ..., SEQ U,T Where T is the total number of pixels in the non-defect area, SEQ U,t This is a pulsed thermal imaging data sequence showing the time-varying changes of the t-th pixel within a non-defect region.

[0094] Because pulsed thermal imaging data has time-series properties, and pulsed thermal imaging data signals at different times contain feature information about the depth and size of defects, this invention utilizes the advantages of Bi-LSTM neural networks for processing time-series data. Classification and regression tasks are designed using Bi-LSTM neural networks. The pulsed thermal imaging data sequence during the cooling process is used as input, and the Bi-LSTM neural network is used to extract features to predict the size and depth of defects, thereby achieving three-dimensional structural reconstruction of the defects.

[0095] Regarding steps B1 and B2, the following two points need to be explained.

[0096] 1. Standardization of pulsed thermal imaging data sequences

[0097] In this invention, the pulsed thermal imaging data sequence of each pixel in the training set during the cooling process is standardized. Figure 5 This is a schematic diagram illustrating the normalization and preprocessing of pulsed thermal imaging data sequences in a Bi-LSTM network during the cooling process of this invention. Please refer to... Figure 5 In steps B1 and B2, the method further includes: dividing the 1×N pulsed thermal imaging data sequence into W segments of 1×N / W pulsed thermal imaging data subsequences in chronological order; in each training step C2 and D1, the W segments of 1×N / W pulsed thermal imaging data subsequences are independently input into the Bi-LSTM neural network for training. Where 2 ≤ W ≤ 5. Experiments show that W = 2 yields particularly good results.

[0098] It should be noted that the present invention uses a method commonly used in the art to represent vectors or matrices. For example, the thermal imaging data sequence above has a dimension of 1×N, meaning the thermal imaging data sequence is a vector with 1 row and N columns; then, the pulsed thermal imaging data sequence is divided into W segments of 1×N / W pulsed thermal imaging subsequences, meaning the pulsed thermal imaging subsequences are vectors with 1 row and N / W columns. The representation of vectors and matrices is the same in subsequent training and detection processes.

[0099] Experiments demonstrate that dividing pulsed thermal imaging data sequences into different sub-sequences and using a bidirectional LSTM network to extract the length dependencies between signals in each interval is effective. The experiments also show that the interval length setting is sensitive, and this method of interval division can help improve the accuracy of defect inversion.

[0100] 2. Acquisition of pixel data in defective and non-defective areas

[0101] In the task of classifying defects, before training the Bi-LSTM network, it is necessary to accurately obtain the dataset of flat-bottom hole defects and non-defect datasets in the training workpiece as the training dataset.

[0102] Step B1 further includes: multiplying each frame of infrared thermal image with the digital mask image of the defect area to filter out the pixels of the defect area. In the digital mask image of the defect area, the pixel value of the defect area is 1, and the pixel value of other areas is 0.

[0103] Step B2 further includes: multiplying each frame of infrared thermal image with the digital mask of non-defective areas to filter out the pixels of non-defective areas. In the digital mask of non-defective areas, the pixel value of non-defective areas is 1, and the pixel value of other areas is 0.

[0104] Before step B1, the process further includes: acquiring a visible light digital photograph of the test surface of the training workpiece; performing image segmentation on the visible light digital photograph to obtain a binary digital photograph of the defect region on the test surface of the training workpiece; registering the binary digital photograph of the defect region with an infrared thermal image to obtain a digital mask of the defect region; selecting a region outside the defect region in the registered two-dimensional digital photograph of the defect region as a non-defect region to obtain a digital mask of the non-defect region. The non-defect region is defined as a ring-shaped region outside the defect region, and the number of pixels T in the non-defect region satisfies: 0.5S ≤ T ≤ 1.5S.

[0105] It is worth noting that, in the process of acquiring the non-defect dataset, the influence of lateral thermal diffusion on the predicted defect size is considered. The cooling data sequence of the corresponding pixels in the annular non-defect region adjacent to the flat-bottom hole defect is used as the non-defect training dataset, so that the network can better learn the difference between the pulsed thermal imaging data signal corresponding to the defect region and the pulsed thermal imaging data signal corresponding to the non-defect region (especially the pulsed thermal imaging data signal corresponding to the defect edge region).

[0106] Please refer to Figure 3A Step C1 is executed after step B1.

[0107] Step C1: Construct a regression model training dataset from the pulsed thermal imaging data sequence of each pixel in the defect area over time and the corresponding defect depth. in, Let s be the normalized defect depth at the known position corresponding to the s-th pixel, where s = 1, 2, ..., S;

[0108] The normalized defect temperature is the recorded physical depth of the defect mapped to a value between 0 and 1. Specifically, it is obtained in the following way: Among them, D D,s D represents the physical depth of the defect region at the corresponding pixel location. mean D max D min These represent the average, maximum, and minimum values ​​of the defect depth in the training workpiece, respectively.

[0109] It is important to note that non-defect data was not used as training data in the process of predicting defect depth, which effectively improves the accuracy of defect edge depth prediction. The network hyperparameters are set identically for both the classification and regression tasks, requiring only one adjustment by the operator, making the reconstruction of the 3D defect structure more convenient.

[0110] Please refer to Figure 3A Step D1 is executed after steps B1 and B2.

[0111] Step D1: Define the defect category y if the pixel corresponds to a defect. c If it is 1, otherwise the defect category y c The value is 0, representing the pulsed thermal imaging data sequence of each pixel in the defective and non-defective regions over time, and the corresponding defect category y. c Construct a training dataset for the classification model: (SEQ D,1 ,1),...,(SEQ D,s ,1),...,(SEQ D,S ,1) (SEQ U,1 ,0),...,(SEQ U,t,0),...,(SEQ U,T ,0);

[0112] By following the steps above, we have obtained the training datasets for the regression model and the classification model. With these two training datasets, we can train the regression model and the classification model.

[0113] Step C2: Use the data from the regression model training dataset to train the Bi-LSTM neural network to obtain the Bi-LSTM neural network regression model;

[0114] Step D2: Use the data from the training dataset of the classification model to train the Bi-LSTM neural network to obtain the Bi-LSTM neural network classification model.

[0115] By using a Bi-LSTM neural network to extract features from the pulsed thermal imaging data subsequence of segment W, the characteristics of each segment of the signal in the single-point time series signal are fully learned. Figure 6 This is a schematic diagram illustrating how W segments of pulsed thermal imaging data are independently input into the Bi-LSTM neural network for training during one training process.

[0116] Please refer to Figure 6 The feature vector O extracted by the forward LSTM f (Dimension: 1×H) and the feature vector O extracted by the inverse LSTM b (Dimension: 1×H) are concatenated to obtain the feature vector O (Dimension: 1×2H).

[0117] For the training and testing process of the classification model that determines whether something is a defect, the feature vector O (dimension: 1×2H) is multiplied by the learning matrix U (dimension: 2H×C) to obtain the probability vector f. c (Dimension: 1×C), as shown in Formula 7, its normalized probability vector y is obtained after passing through the softmax function. cp (Dimension: 1×C), as shown in Formula 8, y cp The values ​​in dimension C are summed to 1, and the values ​​in dimension C represent the probability of predicting a defect or not a defect.

[0118] Specifically, during training, cross-entropy loss F is used. CE Measure y cp With truth value y c The difference between them is shown in Equation (9), which is used to optimize the Bi-LSTM neural network for defect identification. During the test, y is selected. cp The index value with the highest probability in dimension C is the predicted category.

[0119]

[0120] For the training and testing process of the regression model for determining defect depth, the feature vector O (dimension: 1×2H) is multiplied by the learning matrix V (dimension: 2H×1) to obtain the depth regression value y. gp (Dimension: 1×1). Note that during training, this paper uses L1 distance loss Fi. L1 (As shown in Formula 11) Measure the deep regression value y gp With true depth y g The difference between the two is used to optimize the Bi-LSTM neural network for defect depth prediction. During the test, due to the difference between the true depth y and the actual depth y, the Bi-LSTM neural network is optimized for defect depth prediction. g After normalization, the deep regression vector y needs to be... gp Inversely mapped to its true scale depth y gm y gm The calculation process is shown in Equation 10.

[0121] y gm =y gp ×(D max -D min )+D mean (10)

[0122] F L1 =|y gp -y g | (11)

[0123] In the two training processes described above, the learning matrix U (2H×C, where C is the number of categories) and the learning matrix V (2H×1) are matrices that need to be continuously updated during the training of the deep learning network in classification and regression tasks. Specifically, the learning matrices are initialized at the beginning of training. During each iteration, the loss function calculates the loss value. During backpropagation, the parameters of U and V are automatically learned using gradient descent during optimization. Gradient descent is currently the default method for updating network parameters in deep neural networks. Simply put, each training iteration calculates the loss and updates the network parameters. The updating process and method of the U and V matrices are the same as those used in the main feature extraction network of LSTM. These are known to those skilled in the art and will not be elaborated further.

[0124] At this point, the Bi-LSTM neural network regression model and the Bi-LSTM neural network classification model are obtained, and the training process is complete.

[0125] II. Testing Section

[0126] It should be noted that some aspects of the detection section are the same as those of the training section. These aspects will not be described in detail here; the focus will be on the unique features of the detection section.

[0127] Please refer to Figure 3B The dual-model defect detection method of the Bi-LSTM neural network of the present invention includes:

[0128] Step A': Obtain N frames of infrared thermal images showing the change of the test surface of the workpiece under test over time during the cooling process;

[0129] Similar to the training section, step A' includes: acquiring a background infrared thermal image of the test surface of the workpiece before heating using an infrared thermal imager; heating the workpiece using a high-energy flash lamp, and acquiring N frames of original infrared thermal images of the test surface of the workpiece changing over time during the cooling process using an infrared thermal imager; subtracting the background infrared thermal image before heating from the N frames of original infrared thermal images during the cooling process to obtain N frames of infrared thermal images of the test surface of the workpiece changing over time during the cooling process. For more detailed information, please refer to the relevant instructions in the training section, which will not be repeated here.

[0130] Step B': Obtain the pulsed thermal imaging data sequence SEQ' of the pixel points on the test surface of the workpiece under test changing over time from the N frames of infrared thermal images;

[0131] It is important to note that, similar to the training section, the pulsed thermal imaging data sequence SEQ' is also standardized in the detection section. Furthermore, the value of W should be the same as the value of W during training.

[0132] Specifically, the pulsed thermal imaging data sequence SEQ' with dimension 1×N is divided into W segments of pulsed thermal imaging data subsequence with dimension 1×N / W according to the time order.

[0133] Step F: Input the pulsed thermal imaging data sequence SEQ' of each pixel into the trained Bi-LSTM neural network classification model to obtain a binary classification map that represents whether the pixel position of the workpiece under test is a defect or not. When the corresponding position is a defect, the value of the corresponding pixel in the binary classification map is 1; otherwise, the value of the corresponding pixel in the binary classification map is 0.

[0134] Adapted to the above-mentioned pulsed thermal imaging data subsequence, in step F, the W-segment pulsed thermal imaging data subsequence is independently input into the trained Bi-LSTM neural network classification model to obtain a binary classification map that characterizes whether the pixel position of the workpiece under test is a defect or not.

[0135] Step G: Input the pulsed thermal imaging data sequence SEQ' of each pixel into the trained Bi-LSTM neural network regression model to obtain a defect depth prediction map that represents the defect depth at the pixel corresponding position of the workpiece under test. The value of the corresponding pixel in the defect depth prediction map is the defect depth prediction value.

[0136] Similarly, adapting to the above-mentioned pulsed thermal imaging data subsequence, step G, which involves inputting the pulsed thermal imaging data sequence SEQ' of the pixels into the trained Bi-LSTM neural network regression model, includes: independently inputting the W-segment pulsed thermal imaging data subsequence into the trained Bi-LSTM neural network regression model to obtain a defect depth prediction map that characterizes the defect depth at the pixel-corresponding position of the workpiece under test.

[0137] Step H: The defect distribution of the workpiece under test is obtained from the binary classification map and the defect depth prediction map;

[0138] Specifically, a defect structure map is obtained by multiplying the binary classification map of defects and non-defects with the defect depth prediction map. The numerical value of each pixel in the map represents the depth, and the shape distribution of the defects can be directly seen in the map.

[0139] Step 1: Using the mesh function in Matlab software, the three-dimensional structure of the defects in the workpiece under test is reconstructed from the defect distribution.

[0140] In this invention, after obtaining the defect depth data of all pixels on the test surface of the workpiece, the three-dimensional structure of the internal defects of the workpiece is reconstructed more intelligently by combining the mesh function of Matlab, which facilitates observation and use. However, those skilled in the art should understand that obtaining the three-dimensional defect structure is only more intuitive in terms of representation. If it is only used for subsequent data processing, obtaining the defect distribution of the workpiece in step H is sufficient.

[0141] As can be seen from the above introduction of the dual-model defect detection method of this invention, this invention uses a training dataset of a training workpiece with the same thermal properties as the workpiece under test to train the Bi-LSTM neural network. It does not require prior information such as material thermal properties, reference areas, or feature times, thus facilitating detection while ensuring the accuracy of each defect detection. The Bi-LSTM neural network is trained using pulsed thermal imaging data sequences during the cooling process. These pulsed thermal imaging data sequences do not require any algorithmic processing (such as Fourier transform algorithms, principal component analysis algorithms, etc.) or manual extraction of feature data for network training. The Bi-LSTM neural network can achieve fully automated data analysis, making it more convenient to apply. Furthermore, compared to a single LSTM network, the Bi-LSTM neural network can avoid the loss of previous signal features, infer the long-term and short-term dependencies between signal values ​​at each time point and the global signal value, and ultimately extract rich global data features, which can improve the accuracy of classification tasks or regression tasks.

[0142] At this point, the three-dimensional structure of the defect in the workpiece under test is obtained, and the testing process is complete.

[0143] III. Computer Devices

[0144] According to another aspect of the present invention, a computer device is also provided. The computer device includes: a digital memory; and a processor electrically coupled to the digital memory, configured to execute the training method described above based on instructions stored in the digital memory.

[0145] According to another aspect of the present invention, a computer device is also provided. The computer device includes: a digital memory; and a processor electrically coupled to the digital memory, configured to execute the dual-model defect detection method as described above based on instructions stored in the digital memory.

[0146] IV. Examples of Training and Defect Detection

[0147] Carbon fiber flat-bottom hole training workpieces such as Figure 7A As shown, the three-dimensional dimensions of the carbon fiber plate are 200mm in length, 160mm in width, and 10mm in height. Eight artificial defects with different aspect ratios are pre-embedded on the training workpiece plate. The width of each defect is 18mm, and the depths of the defects are 1.11mm, 1.35mm, 1.64mm, 1.97mm, 2.19mm, 2.55mm, 2.81mm, and 2.97mm, respectively.

[0148] The carbon fiber flat-bottom hole test piece is shown in Figure 7. The test piece plate and the training piece plate have the same three-dimensional dimensions. Six artificial defects with different width-to-depth ratios are pre-embedded on the test piece plate. The width of the defects is 20 mm, and the depth distribution of the defects is 1.1 mm, 1.35 mm, 1.67 mm, 1.94 mm, 2.22 mm, and 2.5 mm respectively.

[0149] During the experiment, a high-energy flash lamp (flash duration 2ms, output energy 9.6KJ) and a FLIR SC3000 infrared thermal imager (wavelength range 8um-9um, pixel 320×240, temperature sensitivity 0.03℃) were placed on the same side of the specimen. The thermal imager's acquisition frequency was set to 60Hz and the acquisition time to 30s. Before and after the flash excitation of the specimen surface, the thermal imager monitored and recorded the cooling data sequence of the specimen surface over time in real time. The cooling data sequences corresponding to the depth distributions of 1.1mm, 1.64mm, 2.19mm, 2.81mm, and 10mm are shown below. Figure 8 As shown. Thermal images of the carbon fiber flat-bottomed hole training workpiece and the carbon fiber flat-bottomed hole test workpiece 8 seconds after flash, as shown. Figure 9 and Figure 10 As shown.

[0150] In the following steps, it is necessary to select the region of interest from the infrared thermal image. The following describes the preparation of the digital mask image of the defective region and the digital mask image of the non-defective region used in the training process.

[0151] The production details are as follows: In acquiring the dataset of flat-bottomed hole defects in the training workpiece, firstly, a visible light photograph is taken of the carbon fiber flat-bottomed hole training workpiece. Then, image segmentation is performed on the visible light photograph to obtain a binary image. This binary image is then registered with an infrared thermal image to obtain a digital mask image A of the defect region (the pixel value of the flat-bottomed hole defect region is 1, and the pixel value of the non-flat-bottomed hole defect region is 0). Figure 11 As shown, the flat-bottom hole defect dataset is extracted by multiplying each frame of the infrared thermal image of the carbon fiber flat-bottom hole training workpiece with the digital mask image A of the defect region.

[0152] Furthermore, a five-pixel-sized ring was defined outside the defect region to ensure that the number of pixels in the ring region was close to the number of pixels in the defect region. This prevented the decision boundary of the Bi-LSTM network from being biased towards either the defect or non-defect category. The non-defect region digital mask B is shown below. Figure 12 As shown, the pixel value of the circular area is 1, and the pixel value of other areas is 0. The non-defect training dataset is extracted by multiplying the heat map of each frame of the carbon fiber flat-bottom hole training workpiece with the digital mask image B of the non-defect area. Finally, the flat-bottom hole defect dataset and the non-defect dataset from the training workpiece are input into a Bi-LSTM network for classification learning.

[0153] In both classification and regression tasks, the data input parameters were set as follows: the number of frames for the pulse time-series signal N = 1800, and the number of segments for each group of pulse thermal imaging data signals W = 2. The hyperparameters of the Bi-LSTM network were set as follows: the feature dimension of the hidden layer nodes was 128, the number of Bi-LSTM network layers was 2, the random loss ratio was 0.5, the learning rate was 0.001, and the batch size was 64.

[0154] During the testing process, the cooling data sequence of the workpiece under test is input into a pre-trained Bi-LSTM network model to generate a binary classification map of defects and non-defects in the workpiece, such as... Figure 13 As shown. To evaluate the accuracy of defect classification, the F1-score is introduced. The closer the F1-score is to 1, the more accurate the classification. Its calculation method is shown in (12) to (14):

[0155]

[0156] in:

[0157]

[0158] TP represents the number of pixels that were originally defective and were also determined to be defective; FP represents the number of pixels that were originally non-defective but were determined to be defective; and FN represents the number of pixels that were originally defective but were determined to be non-defective.

[0159] To determine the true number of defects in the workpiece under test, the mask image C of the workpiece under test is used as the standard. Its fabrication process is the same as that of the training workpiece mask image A. Figure 14 As shown in Table 1, the classification and evaluation of six flat-bottomed hole defects in the workpiece under test, and the lateral dimensional errors of the flat-bottomed hole defects are presented.

[0160] Table 1

[0161]

[0162] Table 1 shows that the F1-scores of the six flat-bottomed hole defects in the workpiece under test are all above 0.85, indicating good classification performance. The predicted diameter of the flat-bottomed hole defect relative to the actual diameter is within 2 pixels, with a maximum lateral dimension prediction error of 8.3%. If the resolution of the infrared thermal imager can be improved, the lateral dimension prediction error can be effectively reduced.

[0163] In the regression task, the mask image A is multiplied with the heat map of each frame of the carbon fiber flat-bottom hole training workpiece to obtain eight flat-bottom hole defect datasets at different depths. From these datasets, datasets with depths of 1.11mm, 1.35mm, 1.64mm, 1.97mm, 2.19mm, 2.55mm, 2.81mm, and 2.97mm are classified. These eight datasets are then used as the training dataset and input into the Bi-LSTM network for regression learning.

[0164] During the testing process, the cooling data sequence of the workpiece under test is input into a pre-trained Bi-LSTM network model to generate a depth prediction map of the workpiece, such as... Figure 15 As shown in Table 2, the evaluation table for the defect depth of six flat-bottomed holes in the test piece is as follows: the defect depth of each flat-bottomed hole is calculated by multiplying the binary classification map of defects and non-defects with the depth prediction map to obtain a defect structure map (the pixel values ​​in the map represent the depth, and the shape and distribution of the defects can be directly seen in the map). Figure 16 As shown.

[0165] Table 2

[0166]

[0167] Because there was no training data included in the non-defective regions, from Figure 15 The results show that the depth prediction for non-defect areas is inaccurate, failing to effectively distinguish between defective and non-defective areas. Table 2 shows that the prediction error for the defect depth of the six flat-bottomed holes is within 7.7%, indicating good accuracy.

[0168] Will Figure 16 The three-dimensional structure of the flat-bottomed hole defect in the workpiece under test was reconstructed using the mesh function in Matlab software, such as... Figure 17 As shown. From Figure 17 As can be seen, in the edge region of deeper flat-bottomed hole defects, the predicted depth is not uniform, but it does not affect the average depth of the flat-bottomed hole defects as a whole. In engineering applications, depth averaging can be used before reconstructing the three-dimensional structure of the flat-bottomed hole defects.

[0169] This concludes the description of the embodiments of the present invention.

[0170] The embodiments of the present invention have now been described in detail with reference to the accompanying drawings. Based on the above description, those skilled in the art should have a clear understanding of the present invention.

[0171] In summary, this invention provides a dual-model defect detection method and training method based on Bi-LSTM neural network. It fully utilizes the advantages of Bi-LSTM neural network and proposes innovative methods such as data sequence segmentation, local defect region, and partial data training regression model in combination with the practical application of defect detection. It has many advantages such as convenience and high accuracy in practical use and has strong practical value.

[0172] It should be noted that for some implementation methods, if they are not the key content of this invention and are well known to those skilled in the art, they are not described in detail in the accompanying drawings or the main text of the specification. In this case, they can be understood by referring to the relevant prior art.

[0173] Furthermore, the above embodiments are provided merely to enable the invention to meet legal requirements, and the invention can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. In addition, the definitions of the elements and methods described above are not limited to the various specific structures, shapes, or methods mentioned in the embodiments, and those skilled in the art can easily modify or substitute them.

[0174] Unless explicitly stated otherwise, the numerical parameters in the specification and claims of this invention may be approximate values ​​and can be changed according to the content of this invention. Specifically, all figures used in the specification and claims to indicate the content of composition, reaction conditions, etc., should be understood to be modified by the term "about" in all cases, which means that there may be variations of ±10% in some embodiments, ±5% in some embodiments, ±1% in some embodiments, and ±0.5% in some embodiments.

[0175] Furthermore, the word "comprising" does not exclude the presence of elements or steps not listed in the claims. The ordinal numbers used in the specification and claims, such as "first," "second," "third," "primary," "secondary," as well as Arabic numerals and letters, to modify the corresponding elements or steps, are intended only to clearly distinguish one element (or step) with a certain name from another element (or step) with the same name; they do not imply that the element (or step) has any ordinal number, nor do they represent the order of one element (or step) with another.

[0176] Furthermore, unless specifically described or required to occur in a specific order, the order of the above steps is not limited to those listed above and can be varied or rearranged according to the desired design. Moreover, the above embodiments can be used in combination with each other or with other embodiments based on design and reliability considerations; that is, technical features from different embodiments can be freely combined to form more embodiments.

[0177] The algorithms and displays provided herein are not related to any particular computer, virtual system, or other inherent device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of this invention can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this invention.

[0178] This invention can be implemented using hardware comprising several different elements and a suitably programmed computer. Various component embodiments of the invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the related devices according to embodiments of the invention. The invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the invention can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0179] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features of the invention in this specification (including the claims, abstract, and drawings) and all processes or units of any method or device so invented. Unless expressly stated otherwise, each feature of the invention in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose. Furthermore, in the unit claims enumerating several means, several of these means may be embodied by the same hardware item.

[0180] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the claims, the various inventive aspects consist of fewer than all the features of the preceding single embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0181] The specific embodiments described above have provided a detailed explanation of the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dual-model defect detection method based on Bi-LSTM neural network, characterized in that, include: Step A': Obtain N frames of infrared thermal images showing the change of the test surface of the workpiece under test over time during the cooling process; Step B': Obtain the pulsed thermal imaging data sequence SEQ' of the pixel points on the test surface of the workpiece under test changing over time from the N frames of infrared thermal images; Step F: Input the pulsed thermal imaging data sequence SEQ' of each pixel into the trained Bi-LSTM neural network classification model to obtain a binary classification map that represents whether the pixel position of the workpiece under test is a defect or not. When the corresponding position is a defect, the value of the corresponding pixel in the binary classification map is 1; otherwise, the value of the corresponding pixel in the binary classification map is 0. Step G: Input the pulsed thermal imaging data sequence SEQ' of each pixel into the trained Bi-LSTM neural network regression model to obtain a defect depth prediction map that represents the defect depth at the pixel corresponding position of the workpiece under test. The value of the corresponding pixel in the defect depth prediction map is the defect depth prediction value. Step H: The defect distribution of the workpiece under test is obtained from the binary classification map and the defect depth prediction map.

2. The dual-model defect detection method according to claim 1, characterized in that, Step B' further includes: dividing the 1×N pulsed thermal imaging data sequence SEQ' into W segments of 1×N / W pulsed thermal imaging data subsequences in chronological order; In step F, inputting the pulsed thermal imaging data sequence SEQ' of the pixels into the trained Bi-LSTM neural network classification model includes: independently inputting the W segments of pulsed thermal imaging data subsequence into the trained Bi-LSTM neural network classification model; In step G, inputting the pulsed thermal imaging data sequence SEQ' of the pixels into the trained Bi-LSTM neural network regression model includes: independently inputting the W segments of pulsed thermal imaging data subsequence into the trained Bi-LSTM neural network regression model; Where W is the same as the segment value during the training of the Bi-LSTM neural network classification model and the Bi-LSTM neural network regression model, and 2≤W≤5.

3. The dual-model defect detection method according to claim 1, characterized in that: In step B', a pulsed thermal imaging data sequence of all pixels on the test surface of the workpiece under test as a function of time is obtained. The step H is followed by: Step I, which uses the mesh function in Matlab software to reconstruct the three-dimensional structure of the defects in the workpiece under test from the defect distribution.

4. The dual-model defect detection method according to claim 1, characterized in that, Step A' includes: Use an infrared thermal imager to acquire the background infrared thermal image of the test surface of the workpiece before heating; A high-energy flash lamp is used to heat the workpiece under test, and an infrared thermal imager is used to acquire N frames of original infrared thermal images of the test surface of the workpiece under test changing over time during the cooling process. By subtracting the background infrared thermal image before heating from the N original infrared thermal images during the cooling process, N infrared thermal images of the test surface of the workpiece under test changing over time are obtained.

5. A training method based on a Bi-LSTM neural network, characterized in that, include: Step A: Obtain N frames of infrared thermal images showing the change of the test surface of the training workpiece over time during the cooling process. The training workpiece and the workpiece to be tested have the same thermal properties. The training workpiece has M defect regions with different depths. The depth of each defect region is known, and M≥2. Step B1: Obtain the pulsed thermal imaging data sequence of each pixel point of the defect area on the test surface of the training workpiece as a function of time from the N frames of infrared thermal images: SEQ D,1 , ..., SEQ D,s , ..., SEQ D,S Where S is the total number of pixels in the defect area, SEQ D,s The pulsed thermal imaging data sequence of the s-th pixel within the defect area as a function of time; Step B2: Obtain the pulsed thermal imaging data sequence of each pixel point on the non-defect area of ​​the test surface of the training workpiece as a function of time from the N frames of infrared thermal images: SEQ U,1 , ..., SEQ U,t , ..., SEQ U,T Where T is the total number of pixels in the non-defect region, SEQ U,t The pulsed thermal imaging data sequence of the t-th pixel within the non-defect region as a function of time; Step C1: Construct a regression model training dataset from the pulsed thermal imaging data sequence of each pixel in the defect area over time and the corresponding defect depth; Step C2: Use the data from the regression model training dataset to train the Bi-LSTM neural network to obtain the Bi-LSTM neural network regression model; Step D1: Construct a classification model training dataset from the pulsed thermal imaging data sequence of each pixel point in the defective and non-defective regions as a function of time and the corresponding defect categories. Step D2: Use the data from the training dataset of the classification model to train the Bi-LSTM neural network to obtain the Bi-LSTM neural network classification model.

6. The training method according to claim 5, characterized in that, The training dataset for the regression model is: in, Let s be the normalized defect depth at the known position corresponding to the s-th pixel, where s = 1, 2, ..., S; In the training dataset for the classification model, if the pixel corresponds to a defect, the defect category is defined as 1; otherwise, the defect category is 0. The training dataset for the classification model is: (SEQ D,1 ,1),...,(SEQ D,s ,1),...,(SEQ D,S ,1) (SEQ U,1 ,0),...,(SEQ U,t ,0),...,(SEQ U,T ,0).

7. The training method according to claim 6, characterized in that, The non-defect region is defined as an annular region outside the defect region, and the number of pixels T in the non-defect region satisfies: 0.5S≤T≤1.5S; and / or Steps B1 and B2 further include: dividing the 1×N pulsed thermal imaging data sequence into W segments of 1×N / W pulsed thermal imaging data subsequences in chronological order; in each training step C2 and D2, the W segments of 1×N / W pulsed thermal imaging data subsequences are independently input into the Bi-LSTM neural network for training, where 2≤W≤5.

8. The training method according to claim 6, characterized in that: Step B1 further includes: multiplying each frame of infrared thermal image with the digital mask image of the defect area to filter out the pixels of the defect area. In the digital mask image of the defect area, the pixel value of the defect area is 1, and the pixel value of other areas is 0. Step B2 further includes: multiplying each frame of infrared thermal image with the digital mask image of non-defect areas to filter out the pixels of non-defect areas. In the digital mask image of non-defect areas, the pixel value of non-defect areas is 1, and the pixel value of other areas is 0. Before steps B1 and B2, the method further includes: acquiring a visible light digital photograph of the test surface of the training workpiece; performing image segmentation on the visible light digital photograph to obtain a binary digital photograph of the defect region on the test surface of the training workpiece; registering the binary digital photograph of the defect region with an infrared thermal image to obtain a digital mask of the defect region; selecting an annular region around the defect region in the registered two-dimensional digital photograph of the defect region as the non-defect region to obtain a digital mask of the non-defect region.

9. The training method according to claim 6, characterized in that, The normalized defect temperature in step C1 is obtained through the following method: Among them, D D,s D represents the physical depth of the defect at the pixel location. mean D max D min These represent the average, maximum, and minimum values ​​of the defect depth in the training workpiece, respectively. Steps C2 and D2 include: feeding the training data from the classification model training dataset into the Bi-LSTM neural network, and inputting the feature vector O extracted by the forward LSTM. f and the feature vector O extracted by inverse LSTM b The feature vector O is obtained by concatenating the features. Step C2 further includes: multiplying the feature vector O with the learning matrix V to obtain the deep regression value y. gp , the deep regression value y gp Inversely mapped to its true scale depth y gm y gm =y gp ×(D max -D min )+D mean During training, the L1 distance loss function F is used. L1 :F L1 =|y gp -y g |,y g For true depth; Step D2 further includes: multiplying the feature vector O with the learning matrix U to obtain the probability vector. The normalized probability vector y with dimension 1×C is obtained by applying the softmax function. cp : Among them, y cp The values ​​in dimension C are summed to 1. The values ​​in dimension C represent the probability of predicting a defect or not. We choose y. cp The index value with the highest probability in dimension C is the predicted category, and the following loss function is used: Among them, y c This represents the ground truth value for the defect category in the training data.

10. An apparatus, characterized in that, include: Digital memory; as well as A processor, electrically coupled to the digital memory, is configured to execute the dual-model defect detection method as described in any one of claims 1 to 4, or the training method as described in any one of claims 5 to 9, based on instructions stored in the digital memory.

Images