Defect recognition method of deep learning combined with space-time domain features in ultrasonic infrared technology

Abstract

The present application relates to a kind of defect identification method of deep learning in ultrasonic infrared technology combined with space-time domain features, belong to nondestructive testing field, comprising: S1: the test piece is applied excitation, obtains the infrared thermal image sequence of measured object surface;S2: select the optimal foreground image number and background image number, carry out background reduction processing to thermal image sequence data, obtain temperature rise thermal image sequence;S3: select the image at a moment to carry out threshold segmentation;S4: all test pieces are divided into training set, verification set and test set;S5: to binary image is marked;S6: in training set and verification set according to temperature rise thermal image sequence, threshold segmented binary image and the annotation file of binary image, extract the one-dimensional space-time domain signal of white pixel point;In test set, extract the one-dimensional space-time domain signal of all white region;S7: space-time domain signal is input into defect identification model, obtain the binary image containing white region, white is defect.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of non-destructive testing and relates to a deep learning defect recognition method combining time and space domain features in ultrasonic infrared technology. BACKGROUND

[0002] In ultrasonic infrared non-destructive testing technology, ultrasonic waves with certain frequency are applied to the test object. During the detection process, the length of time, power and pressure of the ultrasonic pulse can be adjusted according to different test objects. However, for most applications, such as different test pieces made of the same material, fixed parameter settings can be used. The test object generates vibration under ultrasonic excitation. The surface of the defect generates heat due to friction and other reasons. An infrared thermal imager is used to record the temperature change process of the object surface, including the initial equilibrium temperature, the temperature rise and the temperature drop to reach the equilibrium temperature. Subsequently, image processing and other methods can be used to realize the detection of internal defects of the object. The present application relates to a method for automatically recognizing defect signals in ultrasonic infrared non-destructive testing technology by combining one-dimensional time domain signals and one-dimensional space domain signals as deep learning input signals.

[0003] For active infrared thermal imaging technology using excitation such as pulse, ultrasonic, electromagnetic, etc., the current main input for defect recognition and detection algorithm is two-dimensional image, which has the advantages of intuitive image and easy access. However, when the signal is weak or the difference between the defect signal and the normal signal is small, the defect signal is not obvious in the two-dimensional image and needs to be enhanced. At the same time, when part of the interference signal is similar to the defect in the two-dimensional image, the two-dimensional image as input cannot effectively distinguish the defect and the interference signal. Some researches use one-dimensional time domain signal as the input signal of deep learning. The main problem is that when the interference signal or noise signal is close to the defect signal, it cannot be effectively distinguished, so in the final defect detection result, there are some non-defect areas that are similar to the defect.

[0004] For ultrasonic excitation infrared thermal imaging technology, in addition to defect signals, there are heat reflection, paint defects and support positions and their interference signals. From the one-dimensional time domain signal, the heat reflection and the support position are different from the defect signal, and for the paint defect, not only the one-dimensional time domain signal is close to the defect signal, but also from the two-dimensional thermal image, it also behaves similarly to the defect. The patent 201210065546.6 extracts the characteristic value from the time domain signal and the two-dimensional thermal image, and realizes the defect recognition by using the support vector machine method. The method needs to manually select the characteristic value, and when the experimental parameters change, it cannot be applied, and its effectiveness is also limited by the data amount. Whether the one-dimensional time domain or the two-dimensional spatial domain image signal is used as the input signal, it is difficult to effectively realize the defect signal detection, therefore, the one-dimensional time domain signal and the one-dimensional spatial signal at multiple time points are used as the input signal of deep learning. Since the defect signal and the paint defect signal are relatively small in the experimental results, in order to effectively save these signals, it is very time-consuming to manually select the region for saving, and it is easy to repeat. For ultrasonic excitation infrared thermal image segmentation, the patent ZL201210039599.0 utilizes the histogram peak value and the left and right half-height width positions for image segmentation. The method processes the average subtraction result of the selected foreground image number and background image number, but since the threshold value only depends on the values of the selected three points, it is easy to be affected by noise and other interference factors. Since the test piece is a pure noise signal at the beginning of the detection, the infrared thermal image sequence needs to be converted into a temperature rise thermal image sequence by processing a number of foreground image numbers and a number of background image numbers, and the foreground image number and the background image number will also affect the accuracy of the final defect recognition. SUMMARY

[0005] Therefore, the present application aims to provide a deep learning defect recognition method combining time and spatial domain features in ultrasonic infrared technology.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0007] A deep learning defect recognition method combining time and spatial domain features in ultrasonic infrared technology, comprising the following steps:

[0008] S1: using ultrasonic pulse to excite the test piece, using ultrasonic infrared thermal imaging device to obtain the infrared thermal image sequence of the surface of the test object, and storing the thermal image sequence in the memory;

[0009] S2: selecting the optimal foreground image number and background image number, subtracting the background from the thermal image sequence data to obtain the temperature rise thermal image sequence f(u,v,t);

[0010] S3: selecting an image f(u,v,t0) at a time t0 in the temperature rise thermal image sequence f(u,v,t) for threshold segmentation to obtain a binary image fbw ;

[0011] S4: divide all test pieces into training set, validation set and test set;

[0012] S5: label the binary image f bw , divide the white area into four categories of defects, support columns, heat reflection and bad paint; in the training set and the validation set, select representative positions for the white areas of heat reflection and support columns, and select all white areas of defects and bad paint; in the test set, select all white areas of the four categories;

[0013] S6: in the training set and the validation set, extract one-dimensional space-time domain signals of white pixel points according to the temperature rise heat map sequence, the binary image after threshold segmentation and the label file of the binary image; in the test set, extract one-dimensional space-time domain signals of all white areas;

[0014] S7: input the time domain signal and the space domain signal into the first stage and the second stage models of the defect recognition model respectively, and finally obtain a binary image containing white areas, and the white color is the defect.

[0015] Further, step S2 specifically includes the following steps:

[0016] S21: obtain noise values of the foreground image number minus the background image number under different foreground image numbers b1 and background image numbers b2, obtain different noise values according to different combinations of b1 and b2, and the greater the values of b1 and b2, the smaller the generated noise values;

[0017] S22: select a time point t0, t0 takes a certain time in the ultrasonic excitation time;

[0018] S23: obtain different temperature rise heat map sequences through different combinations of b1 and b2, select a temperature rise heat map at t0 in the temperature rise heat map sequence, and the greater b1 and b2, the greater the generated heat diffusion range; in combination with step S21, select the optimal b1 and b2;

[0019] S24: use the selected optimal b1 and b2 to perform background reduction on the obtained heat map sequence T(u, v, t), thereby obtaining a temperature rise heat map sequence f(u, v, t):

[0020] wherein 1≤t≤120-b1

[0021] wherein u represents a spatial horizontal coordinate, v represents a vertical coordinate, and t represents time.

[0022] Further, step S3 specifically includes the following steps:

[0023] S31: Calculate the histogram of the image f(u, v, t0): find the peak value M point, the abscissa X1 of the M point represents the temperature rise value corresponding to the peak value, and the corresponding position of twice X1 is X2; then store the values in the image f(u, v, t0) whose temperature rise value is less than or equal to X2 in array A, take the absolute value of all values in array A to obtain array B, calculate the number s1 of values greater than 0 in array A and the length s2 of array A; finally, calculate the mean value e and the mean square deviation σ of array B;

[0024] S32: Calculate the value of the coefficient k:

[0025]

[0026] Wherein, s1 is the number of values greater than 0 in array A, and s2 is the number of temperature rise values contained in array A;

[0027] S33: Calculate the threshold value T based on the histogram corresponding to the ultrasonic infrared temperature rise thermogram:

[0028] T = e + k x σ

[0029] S34: Perform global thresholding on the image f(u, v, t0) to obtain the threshold segmented binary image f bw :

[0030]

[0031] Further, step S7 specifically comprises:

[0032] S71: First, observe and analyze the temperature rise images at each time, then select a plurality of temperature rise images at different times that can reflect the different heat diffusion ranges caused by the increase of defect signals and bad paint signals with time, store the t value of each selected time in array C, then select the width l of the spatial domain signal, and finally, according to the selected plurality of temperature rise thermogram sequences, the threshold segmented binary image, and the annotation file of the binary image, obtain all the spatial domain signals of the white region of all pixel points required in each data set; according to the temperature rise thermogram sequence, the threshold segmented binary image, and the annotation file of the binary image, obtain the time domain signal of all pixel points of the white region required in each data set;

[0033] S72: When training the defect recognition model in the first stage, input the time domain signal of all data into the model for training;

[0034] S73: When training the defect recognition model in the second stage, input the spatial domain signals of defects and bad paint into the model for training;

[0035] S74: During testing, a zero matrix MM of the same size as the temperature rise heat map is generated. The time-domain signal is extracted from the test sample and input into the trained first-stage model. If the result after passing the first-stage model is heat reflection or support column type, the result of the next sample is directly judged. If the result after passing the first-stage model is defect or paint defect type, the spatial domain signal is extracted from these samples and input into the trained second-stage model for testing. After passing the second-stage model, if it is judged as a defect, the horizontal and vertical coordinates of the sample are extracted and the value of the same position in the MM matrix is ​​set to 255. If it is judged as paint defect type, the result of the next sample is judged.

[0036] S75: The target area identified after processing by steps S71-S74 is a defect; the time domain signal and spatial domain signal of all white areas of a specimen are input into the first stage model and the second stage model respectively. After the two stages of the model, the defect is represented as a white area in the matrix MM. If there is a defect in the specimen, the matrix MM is white at the defect position. If there is no defect in the specimen, the matrix MM is a completely black image.

[0037] Further, step S71, which involves obtaining the spatiotemporal domain signal of all pixels in the desired region based on the temperature rise heat map sequence, the threshold-segmented binary image, and the annotation file of the binary image, specifically includes:

[0038] S711: Calculate one-dimensional time-domain signals:

[0039] TT[t] = f(x0, y0, t), where 0 ≤ t < 120 - b1

[0040] Where b1 is the number of foreground images, and x0 and y0 are the horizontal and vertical coordinates of the required pixels;

[0041] S712: Calculate c j One-dimensional spatial signal at time:

[0042] Where 0 ≤ i < 2 × l, c j ∈C

[0043] Where x0 and y0 are the horizontal and vertical coordinates of the required pixels, h is the width of the temperature rise image, and l is the range value representing the space obtained by analyzing the space curves according to different materials.

[0044] The time-domain signal TT is normalized, and then the one-dimensional spatial signal K at each time step is normalized. The normalization formula is as follows:

[0045]

[0046] wherein G is an array of one-dimensional signal formation, g1 is the minimum value in the array G, and g2 is the maximum value in the array;

[0047] Finally, the time domain signal and the spatial domain signals at multiple time points are spliced, and the splicing formula is as follows:

[0048] Tk=[TT,K1,K2,…,K p ]。

[0049] Further, the defect recognition model is divided into two stages, in the first stage, one-dimensional time domain signals are used as inputs of the defect recognition model, data belonging to the time domain signals, i.e. TT, are taken out from the space-time domain data set, wherein defects and paint defects are one class, and heat reflection and support columns are one class; in the second stage, one-dimensional spatial domain signals of defects and paint defects are used as inputs of the defect recognition model, and the spliced multiple time point spatial domain signal data, i.e. [K1,K2,…,K p ] are taken out from the space-time domain data set, wherein defects are one class, and paint defects are one class;

[0050] The defect recognition model comprises a global feature extraction module, a cross double-branch feature extraction module, an attention feature extraction module and a classification output module;

[0051] The global feature extraction module is composed of three layers of long short-term memory networks (LSTM) and is used for extracting the most original time information as global features;

[0052] The cross double-branch feature extraction module uses convolution layers to constitute multi-level feature extraction, each layer is composed of two convolution kernels of different sizes, the features obtained through the two convolution layers are merged according to the channel dimension, and the cross double-branch convolutional neural network is jointly composed, which is used for outputting the cross-fused features;

[0053] The attention feature extraction module is composed of a time and location multi-headed attention network (TLMANet) and a channel attention network (Channel Attention) to jointly constitute a time sequence attention mechanism network (TLMCA);

[0054] The classification output module needs to perform global average pooling on the features after the attention feature extraction module, then merge the features after global average pooling with the output features at the last moment after global feature extraction according to the channel dimension, and finally give the classification result through the softmax layer.

[0055] Further, the first layer of the cross double branch feature extraction module uses one-dimensional convolution with a convolution kernel size of 9 and 17, a frame expansion size of 4 and 8, and an output channel number of 128 respectively to extract features, and inputs the extracted features into a batch normalization layer (BN) and then into an activation layer (ReLU);

[0056] The second layer of the convolution layer is similar to the first layer and uses one-dimensional convolution with a convolution kernel size of 9 and 17, a frame expansion size of 4 and 8, and an output channel number of 256 respectively to extract features;

[0057] The third layer of the convolution layer is the same as the first layer of the convolution layer;

[0058] The second layer input is the output generated by the first layer through a small convolution kernel, and the third layer input is the output of the small convolution kernel in the second layer;

[0059] The features extracted by each layer of different branches need to be spliced according to the channel dimension respectively, and then the merged features in each layer are input into one-dimensional convolution with a convolution kernel size of 1, a frame expansion of 0, and an output channel number of 128 for feature extraction, to obtain the cross fusion features of each layer. Then the cross fusion features are sequentially input into a batch normalization layer (BN) and an activation layer (ReLU), and then input into a random inactivation (Dropout) with a value of 0.2. Finally, the cross fusion features obtained by each layer are merged according to the channel dimension, and the merged features are sequentially input into a batch normalization layer (BN) and an activation layer (ReLU) to obtain the features of the cross double branch feature extraction module.

[0060] Further, the attention feature extraction module first inputs the information after feature extraction into a network composed of a positional encoding composed of sin and cos and a time relationship network composed of a long short-term memory network (LSTM), and converts the single feature information into feature information with time sequence and position.

[0061] The positional encoding composed of sin and cos is used to give a unique position to the input sequence, and a position is encoded in the form of sin and cos, where sin processes even positions and cos processes odd positions.

[0062] The time relationship network composed of a long short-term memory network is composed of a 1-layer LSTM network for capturing historical information of a sequence; output information of each time point is composed of information of the time point and before the time point;

[0063] Then, information with time sequence features and position features is input into a multi-head attention network, and finally, information is input into a channel attention network to automatically obtain importance of each channel; the channel attention network uses an SENet network structure.

[0064] Further, the softmax layer provides a probability of each category, which is encoded into a one-hot code format and delivered to an output.

[0065] The beneficial effects of the present application are that (1) the selection of the threshold value, since the corresponding temperature rise value of the weak defect signal may overlap with the temperature rise value of the noise, the present application mainly studies all temperature rise values in the range of the temperature rise value less than or equal to twice the temperature rise value corresponding to the peak value, and the appropriate threshold value is analyzed from this range, which is more beneficial to the weak defect signal.

[0066] (2) Spatio-temporal signal, in the previous invention of automatic defect signal recognition, most of them take time domain signal or spatial domain signal as input, but in the case of weak defect signal or similar defect signal and interference signal or noise signal, only taking time domain signal or spatial domain signal as input will not be able to identify the defect signal well. Therefore, the present application combines the time domain signal and the spatial domain signal into a one-dimensional spatio-temporal signal, which can obtain more information from the temperature rise image sequence and is more beneficial to the automatic recognition of the defect signal.

[0067] (3) Defect recognition model, for the previous invention, the present application designs a time sequence attention mechanism network, which can obtain position and time information of the sequence, then input the features with time and position into a network combined by multi-head attention and SENet to obtain important information in space and channel, and improve the accuracy of defect recognition. The present application also designs a cross double-branch feature extraction network, which can obtain feature information of different scales from the original sequence data, then fuse the feature information of different scales, which can well prevent information leakage, effectively protect the original information, and increase the defect recognition efficiency. Other advantages, objects and features of the present application will be described in the subsequent specification to some extent, and to some extent, it will be obvious to those skilled in the art based on the study of the following text or can be taught from the practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the following specification. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to make the purposes, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be given below with reference to the drawings, in which:

[0069] Figure 1 A flow chart for defect recognition combining spatial and temporal features in ultrasonic infrared technology;

[0070] Figure 2 A sequence diagram of temperature rise heat maps corresponding to different values of b1 and b2;

[0071] Figure 3 A temperature rise heat map when the number of foreground images and background images at t0=25 ms is 6;

[0072] Figure 4 A histogram curve of the temperature rise heat map corresponding to 25 ms;

[0073] Figure 5 A reduced view of Figure 4 ;

[0074] Figure 6 A temperature rise heat map corresponding to Figure 3 ;

[0075] Figure 7 One-dimensional time domain signal graphs of defects, paint spraying defects, thermal reflections and support columns of the tested sample in the embodiment;

[0076] Figure 8 One-dimensional spatial domain signal graphs of defects and paint spraying defects of the tested sample in the embodiment;

[0077] Figure 9 A network structure diagram of the defect recognition model;

[0078] Figure 10 A structure diagram of the cross double-branch feature extraction module in the defect recognition model network;

[0079] Figure 11 A structure diagram of the attention feature extraction module in the defect recognition model network;

[0080] Figure 12 A result diagram after automatic defect recognition in the embodiment. DETAILED DESCRIPTION

[0081] The present application is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Wherein, the figures are only used for example description, the representation is only schematic diagram, not real object figure, cannot be understood as the limitation of the present application; in order to better illustrate the embodiment of the present application, some components of the figure will be omitted, enlarged or reduced, and does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their description in the figure can be omitted.

[0083] The same or similar reference numerals in the drawings of the embodiments of the present application correspond to the same or similar components; in the description of the present application, it is understood that if the terms 'upper', 'lower', 'left', 'right', 'front', 'back' and the like indicate the orientation or positional relationship shown in the drawings, only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the positional relationship described in the drawings is only used for example description, cannot be understood as the limitation of the present application, for those skilled in the art, the specific meaning of the above terms can be understood according to the specific situation.

[0084] Figure 1 The figure is the flow chart of the defect recognition process of the ultrasonic infrared technology combined with space-time domain features, and also the structure schematic diagram of the actual system applying the method of the present application.

[0085] The process of the defect recognition method of the ultrasonic infrared technology combined with space-time domain features will be described below in combination with the embodiment. In this embodiment, 153 samples made of titanium and chromium-nickel-iron alloy are used, and the sample size is about: 200 mm long, 30 mm wide, and 6 to 8 mm thick. Before the experiment, each sample is manually sprayed with paint to eliminate surface heat reflection, and there are several unknown defects and bad paint caused by spraying among the 153 samples.

[0086] First, the ultrasonic welding gun is used to emit ultrasonic pulses to the test piece under certain conditions of fixed frequency, pulse length, power and pressure, etc. The test piece vibrates under ultrasonic excitation, the ultrasonic infrared thermal imager records the temperature change process of the object surface, obtains the thermal image sequence of the measured object surface, and saves it by computer. The surface of the defect generates heat due to friction and other reasons, resulting in a brighter area.

[0087] The basic principle of ultrasonic infrared thermal imaging technology is that under the action of ultrasonic waves, the surface of the defect generates heat due to friction and other reasons, causing the corresponding surface temperature to rise, while the normal area does not have temperature rise due to no heat generation. Therefore, in the ultrasonic infrared thermal imaging technology, the main concern is the temperature change, i.e. the temperature rise value. In order to obtain this value, the signal before ultrasonic excitation needs to be subtracted. The signal before ultrasonic excitation is pure noise. In order to reduce the influence of noise on the final obtained temperature rise signal, appropriate foreground image number b1 and background image number b2 need to be selected, and then the temperature mean value of the foreground image number is subtracted from the temperature mean value of the background image number to obtain the threshold segmentation and depth learning data set temperature rise thermal image sequence. The formula for converting the original infrared thermal image sequence T(u,v,t) to the temperature rise thermal image sequence f(u,v,t) through the foreground image number b1 and the background image number b2 is as follows:

[0088] where 1≤t≤120-b1 (1)

[0090] where u, v, t are spatial horizontal coordinate, vertical coordinate, and time, respectively.

[0091] For the selection of parameters b1 and b2, the traditional method is to randomly select two values. According to the noise values obtained by different foreground image numbers and background image numbers, and the effects exhibited by the bright areas in the infrared images obtained by observing different foreground image numbers and background image numbers, the most suitable combination of foreground image number b1 and background image number b2 is selected.

[0092] The specific method of obtaining noise values according to different foreground image numbers b1 and background image numbers b2 is as follows: first, an approximate range of foreground image number and background image number variation is given according to experience, for example, the foreground image number variation range is Y1, and the background image number variation range is Y2; second, a first value in Y2 is obtained as the background image number b2, the noise value q2 is calculated, the foreground image number b1 is sequentially obtained from Y1, the noise value q1 of each foreground image number b1 is calculated, and the noise value of each foreground noise q1 minus the background noise q2 is calculated respectively; third, a second value in Y2 is obtained as the background image number b2, and the noise values of all foreground images under this background image number are calculated according to the manner of the second step; fourth, the b2 is sequentially obtained from Y2 according to the foregoing manner, and the noise values of all foreground image numbers in Y1 minus the background noise are calculated. In this way, the noise values of different foreground image numbers and background image numbers are obtained.

[0093] According to the experimental data, it can be found that the greater the foreground image number and the background image number, the smaller the noise value. However, according to the observation of the gray scale images obtained under different foreground image numbers and background image numbers, it can be found that the greater the foreground image number and the background image number, the larger the bright area of the defect or other interference signal, and the greater the diffusion, which will be not conducive to the identification of the defect in the later stage. In view of the above reasons, it is found that the foreground image number and the background image number cannot be too large or too small, and through comparison, it is found that when the background image number b1 and the foreground image number b2 are equal, it is more appropriate.

[0094] In the embodiment, the foreground image number b1 has Y1 = [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], and the background image number b2 has Y2 = [5, 6, 7, 8, 9, 10]. Different noise values are obtained according to the combination of different values of b1 and b2, as shown in Table 1. Through experimental data analysis, it can be known that the greater the values of b1 and b2, the smaller the generated noise value.

[0095] Table 1

[0096]

[0097] The time point t0 is selected, t0 is a certain moment in the ultrasonic excitation time, t0 = 25 ms.

[0098] Different temperature rise thermograph sequences are obtained through the combination of different values of b1 and b2, and the t0 time temperature rise thermograph is selected in the temperature rise thermograph sequence, as shown in Figure 2

[0099] ​Observing the temperature rise thermogram reveals that larger values ​​for b1 and b2 are not always better. Larger values ​​result in a wider thermal diffusion range, which is detrimental to defect identification. Therefore, based on the steps described above, a value of b1 = b2 = 6 is optimal.

[0100] Using formula (1) and b1 and b2, the obtained heat map sequence T(u,v,t) is subjected to background subtraction processing to obtain the temperature rise heat map sequence f(u,v,t). Defects and other interference signals appear as bright areas, such as Figure 3 As shown.

[0101] After obtaining the temperature rise heat map sequence f(u,v,t), in order to better obtain the defect signal, it is necessary to select the temperature rise heat map f(u,v,t0) with the earlier ultrasonic excitation from the temperature rise heat map sequence f(u,v,t) as the original image for threshold segmentation.

[0102] After determining the temperature rise heatmap f(u,v,t0), calculate its histogram, as shown in the attached figure. Figure 4 As shown, to more easily observe the changes around the peak, data that is too far to the right of the peak are temporarily discarded, forming a supplementary data set. Figure 5 Appendix Figure 5 It is attached Figure 4 The image shows a magnified view with a peak value and portions of data before and after it. Under ultrasonic excitation, only defects, support columns, heat reflections, and poor painting conditions generate heat in the temperature rise thermal map, appearing as bright areas. Other areas do not generate energy and represent noise signals. Moreover, most areas in the temperature rise thermal map are noise signals; therefore, the histogram of the ultrasonic infrared thermal map has only one peak value, T. m The x-coordinate corresponding to the peak value is X1, and the peak value is shown in the attached figure. Figure 5 Point M is shown in the diagram. Since the temperature rise thermogram is obtained after background subtraction, some noise signals may be negative. The left side of this peak is the noise signal, and the right side of the peak consists of a portion of defect signals and interference signals or noise signals. When the defect signal is weak, the microcrack defect signal and noise signal obtained by ultrasonic infrared thermography are similar. The key to the thresholding technique is to select the position corresponding to twice the temperature rise value of X1, i.e., the temperature rise value corresponding to X2. Although previous patents have also used histograms to obtain thresholds, histograms are greatly affected by noise and fluctuate significantly. In traditional methods, the peak value or a certain position is used to calculate the threshold. However, in this embodiment, based on the analysis of the noise signal, the mean and root mean square error of the signal within the X2 range are used as parameters to calculate the threshold, which is less affected by noise.

[0103] Before calculating the threshold, all data less than or equal to X2 in the thermal image of temperature rise needs to be composed into an array A, and then the global threshold required for threshold segmentation is obtained according to the array A. The specific way is as follows: in the array A, since there is a signal less than 0 in the thermal image of temperature rise, the data less than 0 needs to be taken as its absolute value when calculating the mean and variance of the array A. After taking all the data as absolute values, a new array B is formed, the number of data greater than 0 in the array A s1 and the length of the array A s2 are calculated, and finally the mean e and the mean square deviation σ of B are calculated.

[0104] The threshold formula is as follows:

[0105] T = e + k x σ (2)

[0106] Wherein, the coefficient k is determined according to the number of temperature rise values in the array A and the number of data greater than 0 in A. The calculation formula of k is as follows:

[0107]

[0108] Wherein, s1 is the number of data greater than 0 in the array A, and s2 is the number of temperature rise values contained in the array A.

[0109] After the threshold is determined, the global threshold processing of the thermal image sequence of temperature rise is as follows:

[0110]

[0111] As shown in Figure 6 , after threshold processing, the binary image f bw obtained is the result of threshold segmentation processing, wherein the target region includes the defect region and other high temperature regions, that is, the segmented target region contains defects and other interference signals in the thermal image of temperature rise which appear as bright regions.

[0112] Since the white regions in the binary image not only have defects, but also have poor paint, support columns, heat reflection and other interference signals, as shown in Figure 7 , therefore, the binary image needs to be labeled, and the defects and poor paint are labeled out. All test pieces are divided into training set, validation set and test set. According to the obtained binary image f bwThe binary image is labeled, and the white area is mainly labeled and classified into four categories, i.e., defects, support columns, heat reflection, and poor paint spraying. In the training set and the validation set, representative positions of the white area of the heat reflection and the support column are selected, and the white area of the defects and the poor paint spraying is selected. In the test set, the white area of the four categories is selected. Since the defect signal and the poor paint spraying interference signal are very similar in one-dimensional time domain characteristics under ultrasonic excitation, but the defect signal and the poor paint spraying signal have different heat diffusion ranges on the test piece, the defect heat diffusion range is relatively large, therefore, not only the time domain signal of the white area needs to be extracted, but also the spatial domain signal needs to be extracted, such as Figure 8 Specifically, in the training set and the validation set, the one-dimensional time-space domain signal of the white pixel is extracted according to the temperature rise thermal image sequence, the binary image after threshold segmentation, and the label file of the binary image. In the test set, the one-dimensional time-space domain signal of the entire white area is extracted.

[0113] In the present embodiment, in the ultrasonic excitation infrared thermal imaging technology, there are currently two main ways to identify defect signals, one is based on two-dimensional images, and the other is based on one-dimensional time signals. In the ultrasonic excitation infrared thermal image sequence, in addition to defect and noise signals, there are also interference such as heat reflection, poor paint spraying, etc. It is difficult to correctly identify the defect signal from these interference signals only from two-dimensional images or one-dimensional time signals. Therefore, the present patent proposes a method of combining the spatial signals corresponding to multiple time points in the neighborhood of the pixel point based on the one-dimensional time signal to splice a one-dimensional signal as an input signal.

[0114] The one-dimensional time domain signal can distinguish the defect signal and the poor paint spraying signal from the heat reflection signal and the support column signal. Since the one-dimensional time domain signals of the defect signal and the poor paint spraying signal are very similar, but the heat diffusion range caused by the defect after ultrasonic excitation is relatively larger than that of the poor paint spraying, the one-dimensional spatial signals at multiple time points can be used to distinguish the defect signal from the poor paint spraying signal.

[0115] Since most of the temperature rise thermal images are noise signals, and the present embodiment is not mainly aimed at noise signals, therefore, in order to reduce the data amount of the data set, the threshold segmentation method is used to exclude most of the noise signals, and in the present experiment, since the defect signal and the poor paint spraying signal are relatively less than the heat reflection signal and the support column signal, the binary image f bw after threshold segmentation is also needed to reduce the data amount of the heat reflection signal and the support column. Therefore, the time-space domain data set used in the present embodiment is made of the binary image f bw after threshold segmentation and the temperature rise thermal image sequence. One sample in the data set needs to be selected from the binary image f bwextract a required pixel point, then extract a one-dimensional time domain signal and a one-dimensional space domain signal at multiple time points corresponding to the pixel point from the thermal image sequence, and finally combine the time domain signal and the space domain signal at multiple time points to form a time-space domain data set. In the ultrasonic excitation infrared thermal imaging technology, in addition to the defect signal, there are thermal reflection, paint spraying defects and support columns and other interference signals. The embodiment mainly distinguishes the defect signal from the three kinds of interference signals of thermal reflection, paint spraying defects and support columns, so the defect signal and the three kinds of interference signals need to be extracted in the time-space domain data set. The defect and the three kinds of interference signals are white regions in the binary image f bw , and the white regions of thermal reflection and support columns are larger, so if all the regions are extracted, the number of defect and paint spraying defects will be too different from that of thermal reflection and support columns, so the regions of defect and paint spraying defects are all extracted in each test piece, and only part of the regions of thermal reflection and support columns are extracted.

[0116] The time-space domain data set is made as follows: first, in order to extract the required region in the data set, the binary image f bw after threshold processing needs to be labeled, and each white region is labeled as a corresponding region. In this embodiment, there are four types of white regions, which are defect, support column, thermal reflection and bad paint. Then, according to the binary image f bw and the label of the binary image, find all the pixel points of the labeled white regions, and the time domain signal TT of a pixel point (x0, y0) is obtained by formula (5). The space domain signal at multiple time points is extracted, first, p time points for space domain signal extraction need to be selected, and the time values of the p time points are stored in array C, and the space domain signal K of the pixel point (x0, y0) at a certain time point is obtained by formula (6). First, observe and analyze the temperature rise images at each time point, select multiple temperature rise images at different time points which can reflect the different heat diffusion ranges caused by the increase of defect signal and bad paint signal with time, in this embodiment, p = 5 temperature rise images are selected, and the value of each time point t is stored in array C, C = [16, 20, 24, 28, 32], then since titanium and nickel are used in this embodiment, according to the experimental data, the width of the space domain signal is relatively appropriate, that is, l = 9, but if it is other composite materials, it needs to be adjusted according to the actual situation.

[0117] The one-dimensional time domain signal formula is as follows:

[0118] TT[t] = f(x0, y0, t), where 0 ≤ t < 120-b1 (5)

[0119] Where b1 is the number of foreground images, x0 and y0 are the horizontal and vertical coordinates of the required pixel point.

[0120] c jThe one-dimensional spatial signal of each time is shown in the following formula:

[0121]

[0122] where x0 and y0 are the horizontal and vertical coordinates of the required pixel point, h is the width of the temperature rise image, and l needs to be obtained according to different materials. The range value representing the space is obtained by analyzing the spatial curve.

[0123] In order to better identify defects, the data needs to be normalized to the same range, so the data needs to be normalized. First, normalize the time domain signal TT according to formula (7), then normalize each one-dimensional spatial signal K of each time according to formula (7), and finally splice the time domain signal and the spatial signal of multiple times according to formula (8).

[0124] The normalization formula is as follows:

[0125]

[0126] where G is an array formed by one-dimensional signals, g1 is the minimum value in the array G, and g2 is the maximum value in the array.

[0127] The splicing formula of the time domain signal and the spatial signal is as follows:

[0128] Tk=[TT,K1,K2,…,K p ] (8)

[0129] Defects and bad paint are very similar in time domain characteristics, and have subtle differences in spatial characteristics. Thermal reflection and support columns are different from defects and bad paint in time domain characteristics. Therefore, in order to better identify defects in the test piece, the network model is trained in two stages. In the first stage, one-dimensional time domain signals are used as inputs of the defect recognition model. The data belonging to the time domain signal TT is taken from the time-space domain data set, in which defects and bad paint are one class, and thermal reflection and support columns are one class. In the second stage, one-dimensional spatial signals of defects and bad paint are used as inputs of the defect recognition model. The data of the spatial signal of multiple times is taken from the time-space domain data set, that is, [K1, K2, …, K p ], in which defects are one class and bad paint is one class.

[0130] The defect recognition model is shown in the attached Figure 9 The structure mainly includes a global feature extraction module, a cross double-branch feature extraction module, an attention feature extraction module, and a classification output module.

[0131] The global feature extraction module mainly consists of a long short-term memory (LSTM) network. Since the temperature of the test piece slightly rises with the excitation time, the temperature rise value also slightly changes with time, so the original data contains time information. In order to better obtain the overall information of the sequence, a long short-term memory network with a 3-layer structure is used in the embodiment to extract the most original time information as the global feature.

[0132] The network structure of the cross double-branch feature extraction module is as shown in FIG. 4. Figure 10As shown, mainly using convolutional layers to constitute multi-level feature extraction, each layer is composed of two different size convolution kernels, and the features of the two convolution layers in each layer are also merged according to the channel dimension, which jointly constitute the cross double branch convolutional neural network (CDBCNN). In the first layer of the model, the convolution kernel size is 9 and 17, the frame expansion size is 4 and 8, and the output channel number is 128. One-dimensional convolution (Conv1d) is used to extract features, and two different size receptive fields can capture different scale time information, in which the larger convolution kernel can better obtain the global information of the data and master the overall trend of the data, and the smaller convolution kernel can obtain more details to prevent information omission. After the convolution operation, the extracted features are input into the batch normalization (BN) layer to avoid the problem of gradient disappearance. Then input into the activation (ReLU) layer to increase the non-linear relationship between the network and reduce the occurrence of overfitting. Because the small convolution kernel retains more time information, in order to extract more abstract and complex information, only the features generated by the small convolution kernel of the previous layer need to be extracted again, so the input of the second layer of the model is the output of the first layer through the small convolution kernel. Similarly, the input of the third layer is the output of the small convolution kernel in the second layer. The convolution layer of the second layer is similar to the first layer, except that the output channel number in the second layer is 256. The convolution layer of the third layer is the same as the convolution layer of the first layer. The features of different branches extracted through each layer need to be spliced respectively, and the splicing method is to splice according to the channel dimension, and then the merged features in each layer are input into the one-dimensional convolution with a convolution kernel size of 1, a frame expansion of 0, and an output channel number of 128 for feature extraction, to obtain the cross fusion features of each layer. Then the cross fusion features are sequentially input into the batch normalization layer (BN) and the activation layer (ReLU). Cross fusion features can provide more protection for information integrity, especially for small defect signals. After passing through the batch normalization layer (BN) and the activation layer (ReLU), in order to prevent overfitting, it also needs to be input into the random inactivation (Dropout) with a value of 0.2. After three convolution layers, the cross fusion features obtained by each layer need to be merged again, and the merging method is still to splice according to the channel dimension, and then the merged features are sequentially input into the batch normalization layer (BN) and the activation layer (ReLU) to obtain the features of this module.

[0133] The network structure of the attention feature extraction module is as shown in the accompanying Figure 11As shown, the main part is composed of a time and location multi-headed attention network (Time Location Multi-headed Attention Net, TLMANet) and a channel attention network (Channel Attention) to form a time sequence attention mechanism network (Time Location Multi-headed Attention Net and Channel Attention, TLMCA). This module first needs to input the information extracted after the feature into a network composed of a positional encoding composed of sin and cos and a time relationship network composed of a long short-term memory network (LSTM), and convert the separate feature information into feature information with time sequence and location, which can better utilize the time sequence of the data. The positional encoding composed of sin and cos is the same as the positional encoding method (Sinusoidal functions) in the Transformer network, which can give a unique position to the input sequence. The sin and cos method gives a position encoding, where sin handles even positions and cos handles odd positions, which makes good use of the sequence information of the data. The time relationship network composed of a long short-term memory network (LSTM) is composed of a 1-layer LSTM network, which can capture the historical information of the sequence. The output information at each time will be composed of the information at this time and before this time, which makes good use of the time of the data. Then, the information with time sequence and position features is input into the multi-headed attention network, where 8 multi-attention mechanisms are used in this embodiment to obtain more rich information. Finally, the information is input into the channel attention network to automatically obtain the importance of each channel, which can enhance useful features and suppress features that are not useful for the current task according to the importance. The channel attention network uses the SENet network structure.

[0134] The classification output module needs to perform global average pooling on the features after the attention feature extraction module, and then concatenate the features with the output of the last time after global feature extraction in the channel dimension. Finally, the total features are given a classification result through a softmax layer. The softmax layer provides the probability of each class, which is encoded into a one-hot code format and passed to the output.

[0135] When training the defect recognition model in the first stage, the time domain signals of all data are input into the model for training. When training the defect recognition model in the second stage, the spatial domain signals of defects and poor painting are input into the model for training.

[0136] In the test, a full zero matrix MM of the same size as the temperature rise heat map is generated, the time domain signal is extracted from the test sample and input into the trained first stage model, if the result after passing the first stage model is heat reflection and support column type, the next sample result is directly judged; if the result after passing the first stage model is defect and paint spraying defect type, the spatial domain signal is continuously extracted from the sample and input into the trained second stage model for testing, after the second stage model, if it is judged as a defect, the horizontal and vertical coordinates of the binary image to which the sample belongs are taken, and the value of the same position in the MM matrix is set to 255, if it is judged as a paint spraying defect type, the next sample result is judged.

[0137] After the above steps, the identified target area is a defect. The time domain signal and the spatial domain signal of all white areas of a test piece are input into the first stage model and the second stage model respectively, and the result after passing the two stage models is white area in the matrix MM, if the test piece has a defect, the matrix MM is white at the defect position, if the test piece has no defect, the matrix MM is a full black image, and the final result is as shown in Figure 12

[0138] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should be covered in the scope of the claims of the present application.​

Claims

1. A deep learning-based defect identification method combining spatiotemporal features in ultrasonic infrared technology, characterized in that: Includes the following steps: S1: Apply an ultrasonic pulse to the test piece, use an ultrasonic infrared thermography device to obtain an infrared thermal image sequence of the surface of the test object, and store the thermal image sequence in the memory; S2: Select the optimal number of foreground images and background images, and perform background subtraction processing on the heat map sequence data to obtain the temperature rise heat map sequence. f(u,v,t) ; S3: In the temperature rise heatmap sequence f(u,v,t) Select a certain moment t Image at 0 f(u,v,t 0 ) Thresholding segmentation is performed to obtain a binary image. f bw Step S3 specifically includes the following steps: S31: Calculate the image f(u,v,t 0 ) Histogram: Find its peak value M point, M x-coordinate of a point X 1 represents the temperature rise corresponding to the peak value, twice. X The corresponding position of 1 is X 2; then the image f(u,v,t 0 ) The temperature rise value is less than or equal to X The value of 2 is stored in an array. A For arrays A The absolute value of all values ​​in the array is obtained by taking the absolute value of the array. B Calculate array A The number of values ​​greater than 0 s 1, and arrays A length s 2; Finally, calculate the array. B mean e and mean square deviation ; S32: Calculation coefficients k Value: in, s 1 is an array A The number of values ​​greater than 0 in the middle. s 2 is an array A The number of temperature rise values ​​included in it; S33: Calculate the threshold based on the histogram corresponding to the ultrasonic infrared temperature rise thermogram. T : S34: For the image f(u,v,t 0 ) Perform global thresholding to obtain a thresholded binary image. f bw : S4: Divide all specimens into training set, validation set and test set; S5: For binary images f bw The white areas were labeled and categorized into four types: defects, support columns, heat reflection, and poor paint application. Representative locations of the white areas for heat reflection and support columns were selected from the training and validation sets, while all white areas for defects and poor paint application were selected. All white areas of the four categories were selected from the test set. S6: Extract the one-dimensional spatiotemporal domain signal of white pixels from the temperature rise heat map sequence, the binary image after threshold segmentation, and the annotation file of the binary image in the training set and validation set; extract the one-dimensional spatiotemporal domain signal of all white areas in the test set. S7: Input the time domain signal and the spatial domain signal into the first stage and the second stage of the defect identification model, respectively, and finally obtain a binary image containing white areas, where white represents the defect.

2. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21: For different numbers of foreground images b 1 and the number of background images b 2. Obtain the noise value by subtracting the number of background images from the number of foreground images. b 1 and b 2. Different combinations of values ​​result in different noise values. b 1 and b The larger the value of 2, the smaller the noise value generated; S22: Select a time point t 0, t 0. Take a certain moment earlier in the ultrasonic excitation time; S23: Pass b 1 and b Different combinations of values ​​for 2 result in different temperature rise heat map sequences. From these sequences, select... t Temperature rise heat map at 0:00, when b 1 and b The larger the value of 2, the greater the range of heat diffusion; combining with step S21, select the optimal value. b 1 and b 2; S24: Use the selected optimal... b 1 and b 2. For the obtained heatmap sequence T(u,v,t) Background reduction processing is performed to obtain the temperature rise heat map sequence. f(u,v,t) : in u Represents the horizontal coordinate in space. v Represents the ordinate, t Indicates time.

3. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 1, characterized in that: Step S7 specifically includes: S71: First, observe and analyze the temperature rise images at various times. Then, select multiple temperature rise images at different times that can reflect the different thermal diffusion ranges caused by the increase of defect signals and bad paint signals over time. (The sentence is incomplete and requires further context.) t Values ​​are stored in an array C Then, select the width of the spatial signal. l Finally, based on the selected temperature rise heat map sequences at multiple time points, the threshold-segmented binary images, and the annotation files of the binary images, the spatial domain signals of all pixels in the white region required in each dataset are obtained; based on the temperature rise heat map sequences, the threshold-segmented binary images, and the annotation files of the binary images, the temporal domain signals of all pixels in the white region required in each dataset are obtained. S72: When training the defect identification model in the first stage, input the time-domain signals of all data into the model for training; S73: When training the defect identification model in the second stage, the spatial signals of defects and poor painting are input into the model for training. S74: During testing, a zero matrix of the same size as the temperature rise heatmap is generated. MM The time-domain signal is extracted from the test samples and input into the trained first-stage model. If the result after passing the first-stage model is heat reflection or support column type, it directly proceeds to the judgment of the result of the next sample; if the result after passing the first-stage model is defect or paint defect type, then the spatial domain signal is extracted from these samples and input into the trained second-stage model for testing. After passing the second-stage model, if it is judged as a defect, the horizontal and vertical coordinates of the sample in the binary image are extracted and... MM The values ​​at the same positions in the matrix are set to 255. If the result is determined to be a defective paint spraying, the process proceeds to the next sample result for evaluation. S75: The target area identified after steps S71-S74 is the defect; the time-domain and spatial-domain signals of all white areas of a specimen are input into the first-stage model and the second-stage model, respectively. After passing through the two-stage models, the areas judged as defects are in the matrix. MM The white area is represented in the matrix; if the specimen has defects, then the matrix... MM The defect location is white; if the specimen has no defects, then the matrix... MM The image is completely black.

4. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 3, characterized in that: Step S71, which involves obtaining the spatiotemporal domain signals of all pixels in the desired region based on the temperature rise heatmap sequence, the threshold-segmented binary image, and the annotation file of the binary image, specifically includes: S711: Calculate one-dimensional time-domain signals: in b 1 represents the number of foreground images. u 0 and v 0 represents the x and y coordinates of the desired pixel. S712: Calculation c j One-dimensional spatial signal at time: in u 0 and v 0 represents the x and y coordinates of the desired pixel. h It is the width of the temperature rise image. l Different experimental space curves are obtained based on different materials, and the range values ​​representing the space are obtained by analyzing the space curves. Time domain signal TT Normalize the signal, and then convert the one-dimensional spatial signal at each time step. K Normalization is performed using the following formula: in, GN It is a normalized array. G It is an array formed by one-dimensional signals. g 1 is an array G The minimum value in, g 2 is the maximum value in the array; Finally, the time-domain signal and the spatial signals from multiple time points are concatenated using the following concatenation formula: 。 5. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 3, characterized in that: The defect identification model consists of two stages. The first stage uses a one-dimensional time-domain signal as input to the defect identification model, extracting data belonging to the time-domain signal from the spatiotemporal domain dataset. TT Defects and poor painting are classified into one category, while heat reflection and support columns are classified into another. In the second stage, one-dimensional spatial signals of defects and poor painting are used as input to the defect identification model. Spatial signal data from multiple pre-assembled time points are extracted from the spatiotemporal dataset, i.e., […]. K 1, K 2,…, K p [The categories are as follows: defects are classified as one type, and poor painting is classified as another type;] The defect identification model includes a global feature extraction module, a cross-branch feature extraction module, an attention feature extraction module, and a classification output module; The global feature extraction module consists of a 3-layer long short-term memory network, used to extract the most primitive time information as global features; The cross-branch feature extraction module uses convolutional layers to form a multi-level feature extraction. Each layer consists of two convolutional kernels of different sizes. The features obtained after passing through the two convolutional layers are merged according to the channel dimension to form a cross-branch convolutional neural network, which is used to output the cross-fused features. The attention feature extraction module consists of a temporal attention mechanism network composed of a multi-head attention network with time and location information and a channel attention network. The classification output module performs global average pooling on the features after the attention feature extraction module, then merges the features after global average pooling with the output features at the last time step after global feature extraction according to the channel dimension, and finally gives the classification result by passing the total features through the softmax layer.

6. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 5, characterized in that: In the first convolutional layer of the cross-branch feature extraction module, one-dimensional convolutions with kernel sizes of 9 and 17, bounding box expansion sizes of 4 and 8, and output channel numbers of 128 are used to extract features. The extracted features are then input into the batch normalization layer and then into the activation layer. The second convolutional layer is similar to the first layer, using one-dimensional convolution with kernel sizes of 9 and 17, bounding padding sizes of 4 and 8, and 256 output channels to extract features. The third convolutional layer is the same as the first convolutional layer; The input to the second layer is the output generated by the small convolutional kernel in the first layer, and the input to the third layer is the output of the small convolutional kernel in the second layer. The features extracted from different branches in each layer need to be concatenated according to the channel dimension. Then, the merged features from each layer are input into a one-dimensional convolution with a kernel size of 1, a bounding box expansion of 0, and an output channel count of 128 for feature extraction. This yields the cross-fusion features of each layer. The cross-fusion features are then sequentially input into the batch normalization layer and the activation layer, and then into a random deactivation layer with a value of 0.

2. Finally, the cross-fusion features obtained from each layer are merged according to the channel dimension, and the merged features are sequentially input into the batch normalization layer and the activation layer to obtain the features of the cross-branch feature extraction module.

7. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 6, characterized in that: The attention feature extraction module first inputs the extracted feature information into the... sin and cos In a network composed of positional encoding and a temporal relational network composed of long short-term memory networks, individual feature information is transformed into feature information with temporal and positional characteristics. The by sin and cos The positional encoding is used to assign a unique position to an input sequence, using sin and cos The method is used to encode a position, where sin It deals with even-numbered positions. cos It deals with odd-numbered positions; The temporal relation network, composed of a long short-term memory network, is a single-layer LSTM network used to capture historical information of the sequence; the output information at each time step consists of information from that time step and the time steps before that time step. Then, information with temporal and positional features is input into a multi-head attention network, and finally, the information is input into a channel attention network to automatically obtain the importance of each channel; The channel attention network uses the SENet network structure.

8. The deep learning defect identification method based on combined spatiotemporal features in ultrasonic infrared technology according to claim 5, characterized in that: The softmax layer provides the probability for each class, which is encoded in a one-hot code format and passed to the output.

Images