A Deep Learning-Based Weld Defect Detection Method and System

Abstract

A method and system for detecting weld defects based on deep learning, relating to the field of metal welding, comprising the following steps: receiving ultrasonic signals of a weld and image information of the weld; the ultrasonic signals including one-dimensional UT waveform signals; extracting time features of the weld based on the one-dimensional UT waveform signals, and predicting a first probability of the defect category of the weld based on the time features; extracting spatial features of the weld based on the image information; predicting a second probability of the defect category of the weld based on the spatial features; calculating a third probability of the defect category by combining the first probability and the second probability based on soft voting, and determining the defect category of the weld based on the third probability.

Description

Technical Field

[0001] The present application relates to the field of metal welding, and particularly to a weld defect detection method and system based on deep learning. Background Art

[0002] Ultrasonic testing is applicable to non-destructive testing of weld defects. However, ultrasonic testing is a technology highly dependent on operation experience and skills. Different inspectors have different levels of ability and experience in operating equipment, interpreting signals, and judging defects. This results in different conclusions even under the same testing conditions for different inspectors. Moreover, the interpretation and judgment of ultrasonic signals often involve subjective understanding of signal characteristics. Characteristics such as the shape, intensity, and propagation speed of the signals are affected by internal defects, and the recognition and interpretation of these characteristics require inspectors to have certain professional knowledge and experience. Different inspectors have different understandings and judgments of these characteristics, leading to different inspection results.

[0003] In summary, due to differences in operation experience, skill level, subjective understanding, and visual perception ability, etc., different inspectors will result in different subjective inspection results when performing ultrasonic testing. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a weld defect detection method and system based on deep learning to solve the technical problem of how to avoid different subjective inspection results caused by differences in operation experience, skill level, subjective understanding, and visual perception ability, etc., among different inspectors when performing ultrasonic testing.

[0005] The present application discloses a weld defect detection method based on deep learning, including the following steps:

[0006] S1. Receive the ultrasonic signal of the weld and the image information of the weld; the ultrasonic signal includes a one-dimensional UT waveform signal;

[0007] S2. Extract the time features of the weld based on the one-dimensional UT waveform signal, and predict the first probability of the defect category of the weld based on the time features; extract the spatial features of the weld based on the image information; predict the second probability of the defect category of the weld based on the spatial features;

[0008] S3. Calculate the third probability of the defect category by combining the first probability and the second probability based on soft voting, and determine the defect category of the weld based on the third probability.

[0009] In some possible implementation manners, predicting the first probability of the defect category of the weld based on the time features includes:

[0010] S201. Identify and capture the peak position and shape from the one-dimensional UT waveform signal based on the time characteristics; S202. Predict the first probability of the defect category of the weld seam based on the peak position and shape.

[0011] In some possible implementation manners, the predicting the first probability of the defect category of the weld seam based on the peak position and shape includes: obtaining, through a plurality of sequentially repeated combinations each including a one-dimensional convolutional layer, an activation layer, a pooling layer, and a dropout layer, the first probability of predicting the defect type of the weld seam based on the softmax function.

[0012] In some possible implementation manners, the image information includes a stereoscopic scan map; the extracting the spatial characteristics of the weld seam based on the image information includes: processing the image information by a two-dimensional convolutional network model with an encoder-decoder symmetric structure to extract the spatial characteristics of the weld seam; the two-dimensional convolutional network model with an encoder-decoder symmetric structure includes 23 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 fusion layers.

[0013] In some possible implementation manners, the predicting the second probability of the defect category of the weld seam based on the spatial characteristics includes:

[0014] S211. Receive the stereoscopic scan map of the weld seam by the encoder part of the two-dimensional convolutional network model;

[0015] S212. Sample the scan image of the newly verified weld seam by a sliding window cropping of 800 pixels × 800 pixels and number them sequentially;

[0016] S213. Move the sliding window in steps of 600 pixels so that there is an overlapping area of 200 pixels in length between different samples;

[0017] S214. Crop the obtained semantic segmentation image from a 100-pixel-wide area within the overlapping area, and then automatically stitch the cropped images according to the image numbers.

[0018] As a second aspect of the present application, a weld defect detection system based on deep learning is also provided, including: a defect data receiving module, a defect probability prediction module, and a defect category determination module; the defect data receiving module is used to receive the ultrasonic signal of the weld and the image information of the weld; the ultrasonic signal includes a one-dimensional UT waveform signal; the defect probability prediction module includes a time feature extraction unit, a first probability prediction unit, a spatial feature extraction unit, and a second probability prediction unit. Among them, the time feature extraction unit is used to extract the time feature of the weld based on the one-dimensional UT waveform signal, and the first probability prediction unit is used to predict the first probability of the defect category of the weld based on the time feature; the spatial feature extraction unit is used to extract the spatial feature of the weld based on the image information; the second probability prediction unit is used to predict the second probability of the defect category of the weld based on the spatial feature; the defect category determination module is used to calculate the third probability of the defect category by combining the first probability and the second probability based on soft voting, and determine the defect category of the weld based on the third probability.

[0019] In some possible implementation manners, the first probability prediction unit includes a first sub-unit, a second sub-unit, and a third sub-unit; the first sub-unit is used to identify and capture the peak position and shape from the one-dimensional UT waveform signal based on the time feature; the second sub-unit is used to predict the first probability of the defect category of the weld based on the peak position and shape.

[0020] In some possible implementation manners, predicting the first probability of the defect category of the weld based on the peak position and shape includes: passing through a plurality of sequential and repeated layers including a one-dimensional convolutional layer, an activation layer, a pooling layer, and a dropout layer, and obtaining the first probability of predicting the weld defect type based on the softmax function.

[0021] In some possible implementation manners, the image information includes a stereoscopic scan map; extracting the spatial feature of the weld based on the image information includes: processing the image information based on a two-dimensional convolutional network model with an encoder-decoder symmetric structure to extract the spatial feature of the weld; the two-dimensional convolutional network model with an encoder-decoder symmetric structure includes 23 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 fusion layers.

[0022] In some possible implementation manners, predicting the second probability of the defect category of the weld based on the spatial feature includes:

[0023] S211. Receiving the stereoscopic scan map of the weld based on the encoder part of the two-dimensional convolutional network model;

[0024] S212. Sample the scanned images of the newly verified weld seams by means of sliding window cropping based on 800 pixels × 800 pixels, and number them sequentially;

[0025] S213. Move the sliding window in steps of 600 pixels, so that there is an overlapping area of 200 pixels in length between different samples;

[0026] S214. Crop the obtained semantic segmentation images from the 100-pixel-wide area within the overlapping area, and then automatically stitch the cropped images according to the image numbers.

[0027] Beneficial effects: This method collects ultrasonic information through sensors and develops a multi-branch deep fusion network model. By integrating one-dimensional and two-dimensional convolutional neural networks, it can capture the temporal and spatial features of weld seams simultaneously, and can judge whether there are defects in the weld seams and determine the types of defects without damaging the integrity of the finished products.

[0028] Other advantages, objectives and features of this application will, to some extent, be described in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of this application. The objectives and other advantages of this application can be realized and obtained through the following specification. Brief Description of the Drawings

[0029] The embodiments described below with reference to the accompanying drawings are exemplary, intended to explain and illustrate this application, and should not be construed as limiting the protection scope of this application.

[0030] Figure 1 is the system flow chart of this application;

[0031] Figure 2 is the one-dimensional UT waveform signal diagram of qualified samples;

[0032] Figure 3 is the one-dimensional UT waveform signal diagram of porosity samples;

[0033] Figure 4 is the one-dimensional UT waveform signal diagram of crack samples;

[0034] Figure 5 is the one-dimensional UT waveform signal diagram of lack of fusion samples;

[0035] Figure 6 is the feature extraction network diagram;

[0036] Figure 7 is the feature enhancement network diagram;

[0037] Figure 8 is the system structure diagram of this application;

[0038] Among them: 1. Defect data receiving module; 2. Defect probability prediction module; 3. Defect category determination module. Specific implementation mode

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Generally, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0041] It should be noted that: Similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0042] In the above description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0043] As Figure 1 shown, the present application discloses a weld defect detection method based on deep learning, including the following steps:

[0044] S1. Receive the ultrasonic signal of the weld and the image information of the weld; the ultrasonic signal includes a one-dimensional UT waveform signal, and the image information of the weld includes a three-dimensional scan map of the weld; collect the one-dimensional UT waveform signal of the weld and the image information of the same weld through a sensor, and develop a multi-branch deep fusion network model, and process the ultrasonic signal of the weld and the image information of the weld by integrating a one-dimensional convolutional neural network and a two-dimensional convolutional neural network respectively.

[0045] S2. Extract the time features of the weld seam based on the one-dimensional UT waveform signal, and predict the first probability of the defect category of the weld seam based on the time features. The first probability is obtained through a one-dimensional CNN model. The input parameter of the CNN model is ultrasonic waveform data, that is, a defect classification model based on ultrasonic waveforms is obtained by learning the ultrasonic waveform data of defects through the one-dimensional CNN model. Time-varying features such as oscillation and spark are extracted through the one-dimensional CNN model, and the vector is normalized by Softmax before output to obtain the probability of each type of crack, incomplete penetration, lack of fusion, and porosity; extract the spatial features of the weld seam based on the image information; predict the second probability of the defect category of the weld seam based on the spatial features. The second probability is obtained through a two-dimensional CNN model, and the input parameter is picture data. Spatial features such as position and shape are extracted through the model, that is, a defect classification model based on picture data is obtained by learning the picture data of defects through the two-dimensional CNN model. Spatial features are extracted through the two-dimensional CNN model, and the vector is normalized by Softmax before output to obtain the probability of each type of crack, incomplete penetration, lack of fusion, and porosity; that is, the first probability is the probability of predicting each type of defect in the crack, incomplete penetration, lack of fusion, and porosity through time features, and the second probability is the probability of predicting each type of defect in the crack, incomplete penetration, lack of fusion, and porosity through spatial features.

[0046] In some embodiments, the predicting the first probability of the defect category of the weld seam based on the time features includes:

[0047] S201. Identify and capture the peak position and shape from the one-dimensional UT waveform signal based on the time features. Due to external inclusions such as pores or slag along the ultrasonic path, as well as improper welding conditions that cause incomplete tissue, the phase and amplitude of the reflected ultrasonic signal will change. These changes appear as peaks in the one-dimensional UT waveform. Therefore, different weld defects have one-dimensional UT waveforms of different shapes, so it is possible to judge which defect type the current UT waveform belongs to by learning the waveform data. The one-dimensional convolutional network is used to extract time features and can identify and capture the peak position and shape from the one-dimensional UT waveform data; through the peak position and shape and resolve the specific defect location and defect type, where , is the defect depth, is the propagation distance of the ultrasonic wave, is the detection angle, is the propagation speed of the ultrasonic wave, is the propagation time of the ultrasonic wave. The ultrasonic information is collected through a probe and a pulse receiver, and the ultrasonic analog signal is converted into digital data by an analog-to-digital converter. After conversion, the horizontal axis is the transmission time of the current signal, and the vertical axis is the amplitude of the signal; as Figures 2 - 5Among them, (a) the 1D waveform of a qualified sample, where the peak signal appears at approximately 65 on the time axis. According to the formula in the text, it can be calculated that it is approximately 7.6 mm deep. By the peak at a certain depth + the amplitude within a certain range, those not within this range belong to abnormal situations; (b) is the waveform of a porous sample, showing that the peak caused by the defect is at 50 (about 5.8 mm deep); (c) is a crack, and the defect peaks starting from 50 (about 5.8 mm deep) are in a distributed state; (d) is lack of fusion, shown as a large peak at around 55 (about 6.4 mm deep).

[0048] S202. The first probability of predicting the defect category of the weld based on the peak position and shape. In the two-dimensional picture data, because the original image is too large, the deep learning model cannot directly process the image, and it is necessary to perform cropping first. In this embodiment, a sliding window image cropping method is proposed to obtain the training and test samples of the model. First, the entire scanned image is marked with welds and different welding defects to improve the speed of label image acquisition. Then, a sliding window of 800 pixels × 800 pixels is used to move along the center of the original image, and each time the window is moved 100 pixels to the right for sampling to obtain a number of sample images. Finally, on the marked image, the same sampling method is used to obtain the labels of the corresponding samples. To better train the segmentation model, according to the characteristics of the sampled images, image horizontal flipping, vertical flipping, and horizontal flipping followed by vertical flipping are used for data augmentation, and the same processing method is applied to the obtained marked sample images to obtain more and richer data. The first probability of predicting the defect category of the weld based on the peak position and shape includes: along the time feature of the effective peak on the time axis of the one-dimensional UT waveform signal, normalizing the one-dimensional ultrasonic waveform to the range of 0 to 1, and introducing random noise to further normalize the signal to improve the generalization ability of the model. After passing through multiple one-dimensional convolutional blocks including one-dimensional convolutional layers, activation layers, pooling layers, and dropout layers, the global features, local features, and sub-local features of the one-dimensional UT waveform are extracted to obtain the feature vectors of three branches, which are then concatenated. After passing through normalization + activation layer + dropout layer and then through softmax, the first probability of the type predicted for the current weld is obtained. Among them, the global feature is the time feature of the entire ultrasonic signal, the local feature is the time feature of the ultrasonic signal in the defect segment, and the sub-local feature is the time feature of the signal before and after the defect segment. In this embodiment, the filter size of the one-dimensional convolutional block of the deep learning model is , using ReLU as the activation function, using max pooling, and the parameter setting of the dropout layer is 0.5. The numbers of filters are respectively , , and There is a residual connection between the input and output of the one-dimensional convolutional block to solve the problem of gradient disappearance.

[0049] In some embodiments, extracting the spatial features of the weld seam based on the image information includes: processing the image information based on a two-dimensional convolutional network model with an encoder-decoder symmetric structure to extract the spatial features of the weld seam; the two-dimensional convolutional network model with an encoder-decoder symmetric structure includes 23 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 fusion layers, and the optimized numbers of convolutional kernels are 64, 128, 256, 512, 512, 256, 128, and 64 respectively. As Figures 6 - 7 described, the left half of the two-dimensional convolutional network model is the encoder part, which belongs to the feature extraction network. The encoder part performs four downsamplings on the input image. The stride of the convolutional layer and the value of zero padding are 1, so the feature size remains unchanged; the right half is the decoder part, which belongs to the feature enhancement network. Through four upsampling operations, the feature map gradually restores to the size of the input image. After receiving the image input, the feature extraction network uses a convolutional network with a convolutional kernel + ReLU activation function to expand to a dimension of 64, and then uses a filter with a stride of 1 for max pooling for downsampling to obtain a vector with the same number of channels but reduced height and width. Then, through the same processing steps, it expands to a dimension of 128. And so on until 4 downsamplings are completed; for the feature enhancement network, taking the output of the previous network as the original input, using a transposed convolution for upsampling to obtain a first result, splicing the first results of the same layer in the extraction network together, and processing with a two-layer convolutional network with a convolutional kernel + ReLU activation function to obtain a second result, and so on until 4 upsamplings are completed; finally, the network will obtain a 64-dimensional vector, and the classification result is obtained through normalization processing using the Softmax function.

[0050] During upsampling, stack with the feature layer copied from the same layer position to better reconstruct the details of the feature map and ensure that the corresponding spatial information dimension remains unchanged. And in the last layer of the two-dimensional convolutional network model, use a convolutional operation to map each 64-component feature vector to different classifications of defect categories. The two-dimensional convolutional network model uses the cross-entropy loss function to measure the error between the predicted second probability and the true sample label. The formula is as follows:

[0051] ;

[0052] where gt represents the true label, p represents the predicted second probability, is the cross-entropy loss function.

[0053] In some embodiments, predicting a second probability of a defect category of the weld seam based on the spatial features includes:

[0054] S211. The encoder part of the two-dimensional convolutional network model receives the stereoscopic scan image of the weld seam;

[0055] S212. Sample the scan image of the newly verified weld seam based on a sliding window cropping of 800 pixels × 800 pixels and number them in sequence;

[0056] S213. Move the sliding window in steps of 600 pixels so that there is an overlapping area of 200 pixels in length between different samples;

[0057] S214. Crop the obtained semantic segmentation image from a 100-pixel-wide area within the overlapping area, and then automatically stitch the cropped images according to the image numbers. The stitched image obtained based on steps S211 - S214 is the same size as the original scan image, which can effectively solve the segmentation problem at the stitching edge of the cropped images.

[0058] After being trained, the two-dimensional convolutional network model can process input images of various sizes, and the size of the obtained semantic segmentation image is also 800 pixels × 800 pixels. To obtain the classification result of the complete image, an image fusion method based on the features of the segmentation image is used to process and analyze the verified weld seam after the model training is completed. Sample the scan image of the newly verified weld seam based on a sliding window cropping of 800 pixels × 800 pixels and number them in sequence; Move the sliding window in steps of 600 pixels so that there is an overlapping area of 200 pixels in length between different samples; Crop the obtained semantic segmentation image from a 100-pixel-wide area within the overlapping area, and then automatically stitch the cropped images according to the image numbers. This makes the obtained stitched image the same size as the original scan image, which can effectively solve the segmentation problem at the stitching edge of the cropped images.

[0059] S3. Calculate the third probability of the defect category by combining the first probability and the second probability based on soft voting, and determine the defect category of the weld seam based on the third probability. To obtain the final decision, soft voting is used to combine the probabilities of the predicted defect categories output by the two models respectively. The soft voting method is a combination strategy for classification problems in ensemble learning. By integrating multiple models, the variance is reduced, thereby improving the robustness of the model. In this embodiment, the third probability is the weighted sum of the first probability and the second probability, that is, the third probability = a * the first probability + b * the second probability, where a + b = 1. The third probability is the result of combining the first probability and the second probability using soft voting. For example, the prediction result of a one-dimensional convolutional neural network for a certain position is that the probability of the normal type is 89% and the probability of the crack defect is 11%. The result of the two-dimensional convolutional neural network for the same position is that the probability of the normal type is 75% and the probability of the crack defect is 9%. Then the average probability of the normal type at this position is 82%, and the probability of the crack defect is 10%.

[0060] Ultrasonic information is collected through sensors and a multi-branch deep fusion network model is developed. By integrating one-dimensional and two-dimensional convolutional neural networks, the temporal and spatial features of the weld seam are captured simultaneously, which can determine whether there are defects in the weld seam and judge the defect type without damaging the integrity of the finished product. Moreover, it avoids the problem that one-dimensional UT transmits and receives pulse signals through a single sound wave at a fixed angle and frequency, which is prone to signal distortion due to various factors such as probe-workpiece contact, probe position, and direction.

[0061] As Figure 8 shown, as the second aspect of the present application, a weld defect detection system based on deep learning is further provided, including: a defect data receiving module 1, a defect probability prediction module 2, and a defect category determination module 3; the defect data receiving module is used to receive the ultrasonic signal and the image information of the weld seam; the ultrasonic signal includes a one-dimensional UT waveform signal; the defect probability prediction module includes a temporal feature extraction unit, a first probability prediction unit, a spatial feature extraction unit, and a second probability prediction unit. Among them, the temporal feature extraction unit is used to extract the temporal features of the weld seam based on the one-dimensional UT waveform signal, and the first probability prediction unit is used to predict the first probability of the defect category of the weld seam based on the temporal features; the spatial feature extraction unit is used to extract the spatial features of the weld seam based on the image information; the second probability prediction unit is used to predict the second probability of the defect category of the weld seam based on the spatial features; the defect category determination module is used to calculate the third probability of the defect category by combining the first probability and the second probability based on soft voting, and determine the defect category of the weld seam based on the third probability.

[0062] In some embodiments, the first probability prediction unit includes a first subunit and a second subunit; the first subunit is configured to identify and capture the peak position and shape from the one-dimensional UT waveform signal based on the time feature; the second subunit is configured to predict a first probability of the defect category of the weld seam based on the peak position and shape.

[0063] In some embodiments, predicting the first probability of the defect category of the weld seam based on the peak position and shape includes: passing through a plurality of sequential and repeated convolutional modules each including a one-dimensional convolutional layer, an activation layer, a pooling layer, and a dropout layer, and finally obtaining the final output using a softmax function.

[0064] In some embodiments, the image information includes a stereoscopic scan map; extracting the spatial feature of the weld seam based on the image information includes: processing the image information based on a two-dimensional convolutional network model with an encoder-decoder symmetric structure to extract the spatial feature of the weld seam; the two-dimensional convolutional network model with an encoder-decoder symmetric structure includes 23 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 fusion layers.

[0065] In some embodiments, predicting the second probability of the defect category of the weld seam based on the spatial feature includes:

[0066] S211. The encoder part of the two-dimensional convolutional network model receives the stereoscopic scan map of the weld seam;

[0067] S212. Sampling the scan image of the newly verified weld seam based on a sliding window cropping of 800 pixels × 800 pixels and numbering them in sequence;

[0068] S213. Moving the sliding window in steps of 600 pixels to have an overlapping area of 200 pixels in length between different samples;

[0069] S214. Cropping out the obtained semantic segmentation image from a 100-pixel-wide area within the overlapping area, and then automatically stitching the cropped images according to the image numbers.

[0070] Those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that it is within the scope of the present application and forms different embodiments.

[0071] Those skilled in the art can understand that the descriptions of the various embodiments have their respective focuses. For parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0072] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations all fall within the scope defined by the appended claims. The above is only the specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for detecting weld defects based on deep learning, characterized in that, Including the following steps: S1. Receive the ultrasonic signal of the weld seam and the image information of the weld seam; the ultrasonic signal includes a one-dimensional UT waveform signal; S2. Extract the time features of the weld seam based on the one-dimensional UT waveform signal. The time features include the time position and shape of the peak in the waveform diagram of the one-dimensional UT waveform signal. Analyze the position of the defect based on the time position and the defect depth, and predict the first probability of the defect category of the weld seam based on the position of the defect and the shape of the peak. The calculation of the defect depth is shown in the following formula: , is the propagation distance of the ultrasonic wave, , is the detection angle, is the propagation speed of the ultrasonic wave, is the propagation time of the ultrasonic wave; Extract the spatial features of the weld seam based on the image information; predict the second probability of the defect category of the weld seam based on the spatial features; the predicting the second probability of the defect category of the weld seam based on the spatial features includes: performing cropping and sampling operations on the three-dimensional scanned image of the weld seam based on a sliding window, and inputting the spatial features of the processed three-dimensional scanned image into Softmax to obtain the probability of each defect category among crack, incomplete penetration, lack of fusion, and porosity; the predicting the first probability of the defect category of the weld seam based on the position and shape of the peak of the defect includes: Along the time position and shape of the effective peak on the time axis of the waveform diagram of the one-dimensional UT waveform signal, normalize it to the range of 0 to 1. After passing through multiple one-dimensional convolutional blocks including one-dimensional convolutional layers, activation layers, pooling layers, and dropout layers, extract the feature vectors corresponding to the global features, local features, and sub-local features of the one-dimensional UT waveform signal, and then splice them. Finally, use softmax to obtain the first probability of the defect category of the weld seam, where the global feature is the time feature of the entire one-dimensional UT waveform signal, the local feature is the time feature of the one-dimensional UT waveform signal of the defect segment, and the sub-local feature is the time feature of the one-dimensional UT waveform signal before and after the defect segment; the filter size of the one-dimensional convolutional block is , the parameter setting of the dropout layer is 0.5, and the numbers of filters are respectively , , and ; S3. Combine the first probability and the second probability based on a soft voting model to calculate the third probability of the defect category, and determine the defect category of the weld seam based on the third probability.

2. The method for detecting weld defects based on deep learning according to claim 1, characterized in that The extracting the spatial features of the weld seam based on the image information includes: extracting the spatial features of the weld seam from the image information based on a two-dimensional convolutional network model with an encoder-decoder symmetric structure; the two-dimensional convolutional network model with an encoder-decoder symmetric structure includes 23 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 fusion layers.

3. A weld defect detection system based on deep learning, characterized in that, Including: A defect data receiving module, a defect probability prediction module, and a defect category determination module; The defect data receiving module is used to receive the ultrasonic signal of the weld seam and the image information of the weld seam; the ultrasonic signal includes a one-dimensional UT waveform signal; the defect probability prediction module includes a time feature extraction unit, a first probability prediction unit, a spatial feature extraction unit, and a second probability prediction unit. Among them, the time feature extraction unit is used to extract the time features of the weld seam based on the one-dimensional UT waveform signal. The time features include the time position and shape of the peak in the waveform diagram of the one-dimensional UT waveform signal. The first probability prediction unit is used to analyze the position of the defect based on the time position and the defect depth, and predict the first probability of the defect category of the weld seam based on the position of the defect and the shape of the peak. The calculation of the defect depth is shown in the following formula: , is the propagation distance of the ultrasonic wave, , is the detection angle, is the propagation speed of the ultrasonic wave, is the propagation time of the ultrasonic wave; the spatial feature extraction unit is used to extract the spatial features of the weld seam based on the image information; the second probability prediction unit is used to predict the second probability of the defect category of the weld seam based on the spatial features. The second probability of predicting the defect category of the weld seam based on the spatial features includes: performing cropping and sampling operations on the three-dimensional scanned image of the weld seam based on a sliding window, and inputting the spatial features of the processed three-dimensional scanned image into Softmax to obtain the probability of each defect category among crack, incomplete penetration, lack of fusion, and porosity; the first probability of predicting the defect category of the weld seam based on the position of the defect and the shape of the peak includes: normalizing the time position and shape of the effective peak along the time axis of the waveform diagram of the one-dimensional UT waveform signal to the range of 0 to 1, passing through multiple one-dimensional convolutional blocks including a one-dimensional convolutional layer, an activation layer, a pooling layer, and a dropout layer, extracting the feature vectors corresponding to the global feature, local feature, and sub-local feature of the one-dimensional UT waveform signal, splicing them, and finally obtaining the first probability of the defect category of the weld seam by using softmax. Among them, the global feature is the time feature of the entire one-dimensional UT waveform signal, the local feature is the time feature of the one-dimensional UT waveform signal of the defect segment, and the sub-local feature is the time feature of the one-dimensional UT waveform signal before and after the defect segment; the filter size of the one-dimensional convolutional block is , the parameter setting of the dropout layer is 0.5, and the numbers of filters are respectively , , and ; the defect category determination module is used to combine the first probability and the second probability based on a soft voting model to calculate the third probability of the defect category, and determine the defect category of the weld seam based on the third probability.

4. The weld defect detection system based on deep learning according to claim 3, wherein, The extracting the spatial features of the weld seam based on the image information includes: extracting the spatial features of the weld seam from the image information based on a two-dimensional convolutional network model with an encoder-decoder symmetric structure; the two-dimensional convolutional network model with an encoder-decoder symmetric structure includes 23 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 fusion layers.

Images