Ship weld defect detection method based on convolutional confidence generative adversarial network model

By using a convolutional belief generative adversarial network model and deep clustering technology, the problems of data imbalance and feature selection in ship weld defect detection were solved, achieving efficient and accurate weld defect identification.

CN114694028BActive Publication Date: 2024-11-01JIANGSU AUTOMATION RESEARCH INSTITUTE +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210340710.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-02
Publication Date
2024-11-01
Estimated Expiration
2042-04-02

AI Technical Summary

Technical Problem

Existing methods for detecting defects in ship welds suffer from problems such as data imbalance, complex selection of defect features, and difficulty in manual annotation, resulting in insufficient model generalization ability and low recognition accuracy.

Method used

A convolutional belief generative adversarial network (CGA) model is adopted. By constructing a deep convolutional belief generative adversarial neural network, adversarial learning between the generator and the discriminator is used to generate a balanced dataset for weld defect detection. Deep clustering technology is then combined to extract deep hidden features.

Benefits of technology

It effectively solves the problem of data imbalance, improves the efficiency of weld defect identification, enhances the stability and scalability of the model, and improves the identification accuracy and recall rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114694028B_ABST
    Figure CN114694028B_ABST
Patent Text Reader

Abstract

The present invention discloses a ship weld defect detection method based on a convolutional confidence generative adversarial network. A generative adversarial network is constructed, and the adversarial game learning characteristics of the network are utilized to expand a data set based on unbalanced samples to generate a balanced data set for ship weld defect detection. A generator and a discriminator based on a deep convolutional confidence network are constructed, so that both can learn rich ship weld features, solving the problem of poor training of the generative adversarial network; the confidence layer proposed therein constitutes a deep clusterer, and clusters the deep hidden features output in the convolutional layer and the fully connected layer, which can improve the detection rate by capturing the connection between similar samples. The network is pre-trained layer by layer using an unsupervised greedy idea, and when the training of one layer of the network is optimized, the training of the next layer of the network begins. This method effectively solves the technical difficulties such as unbalanced weld data, difficulty in labeling defect features, and the existence of multiple defects, and realizes efficient and effective detection of ship weld defects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and relates to a ship weld defect detection technology based on X-ray images, and in particular to a ship weld defect detection method based on a convolutional confidence generative adversarial network model. Background Art

[0002] Welding is the main process in shipbuilding. However, since the welding process is a complex phase change process coupled with multiple factors, it is easily affected by external environment and human factors. In the process of weld formation, unpredictable welding defects such as slag inclusion, cracks, pores, and incomplete penetration are usually generated. These defects will seriously reduce the mechanical properties of welded components, thus often causing great adverse effects on the safety performance of key ship components. In order to timely detect defects in welds, non-destructive testing of welding defects has become the main technical means, and X-ray non-destructive testing technology has been widely used in the actual welding defect detection process due to its advantages of non-destructive, accurate, and fast.

[0003] With the development of computer vision, weld defect detection technology based on X-ray images has been rapidly developed and applied. Existing weld defect detection can be roughly divided into two categories: defect detection based on digital image processing and defect detection based on machine learning. The former includes: edge extraction based on Canny operator, local feature extraction, edge detection based on OSTU, etc. These methods are generally susceptible to noise, similar background colors and mixed defects; the latter includes: artificial neural networks, support vector machines, etc., but their learning performance is highly dependent on the representative ability of the sample library for defects and the accuracy of the learning model.

[0004] Thanks to the development of big data and the improvement of powerful computing power, deep learning has become a hot machine learning method in the field of weld defect identification because of its advantages of automatic feature learning and the ability to reduce the possibility of human-designed features by integrating learning into the model building process. It has also begun to be used in ship weld defect detection. However, this method is highly dependent on data sets and still has the following technical difficulties in application:

[0005] (1) In the actual welding process of ships, the probability of each defect (such as slag inclusion, crack, porosity, and incomplete penetration) is not equal. Therefore, the images obtained by X-ray detection may usually have serious imbalance problems. Unbalanced defect data will affect model learning, thereby reducing the generalization ability of the model and making it prone to underfitting.

[0006] (2) Since ship weld defect detection involves multiple defects and serious data imbalance, if you expect to achieve good sampling results, you need to have sufficient prior knowledge of ship weld defects; and since multiple defect types are mixed in one image, it is more difficult to manually label the categories. If the above problems are not handled properly, it is easy to cause underfitting of the model.

[0007] (3) The existing feature extraction of ship welding defects is mainly based on the geometry, strength and background characteristics of the weld, but the features are numerous and complex, which has a great impact on the training of the deep learning network model and directly affects the recognition accuracy.

[0008] Therefore, ship weld defect detection based on deep learning needs to overcome technical difficulties such as data imbalance, defect feature selection, and difficulty in manual labeling in order to achieve efficient and effective detection of ship weld defects. Summary of the invention

[0009] The purpose of the present invention is to address the problems existing in the prior art and provide a ship weld defect detection method based on a convolutional belief generative adversarial network model, which integrates the convolutional belief generative adversarial neural network model of unbalanced data and deep clustering, thereby improving the efficiency of ship weld defect recognition.

[0010] The technical solution to achieve the purpose of the present invention is: a ship weld defect detection method based on a convolutional confidence generative adversarial network model, the method comprising the following steps:

[0011] Step 1, obtain several ship weld images and build a weld dataset;

[0012] Step 2: Construct a convolutional belief generation adversarial network model;

[0013] Step 3, using the weld data set to train a convolutional confidence generative adversarial network model to generate a convolutional confidence generative adversarial network model for weld defect detection;

[0014] Step 4: Collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

[0015] Furthermore, step 2 constructs a convolutional belief generative adversarial network model, specifically: using a deep convolutional belief neural network as the generator G and discriminator D of the generative adversarial network.

[0016] Furthermore, the deep convolutional belief neural network includes 5 convolutional layers, 2 fully connected layers and 1 belief layer. The input image passes through 5 convolutional layers and 2 fully connected layers in sequence and is output through the belief layer.

[0017] Furthermore, the basic component unit of the confidence layer is a restricted Boltzmann machine (RBM), which includes a visible layer and a hidden layer. The hidden layer of the previous RBM is the visible layer of the next RBM, and the output of the previous RBM is the input of the next RBM.

[0018] The visible layer includes several dominant neurons, referred to as visible neurons. A single visible neuron is represented by v i The hidden layer includes several hidden neurons, referred to as hidden units, and a single hidden unit is represented by h i Indicates that i=1, 2, 3…;

[0019] The energy function of the restricted Boltzmann machine is expressed as:

[0020] E(v,y,h)=-h T Wv-a T vb T hc T yh T Uy

[0021] Where, v=(v1,v2,…,v m ), h=(h1,h2,…,h k ) and y=(y1,y2,…,y p ) represent the states of the visible layer, hidden layer and output layer respectively, m, k, p represent the number of neurons in the visible layer, hidden layer and output layer respectively; θ = {W, a, b, c, U} is the structural parameter to be learned in training, W is the connection weight between the visible layer nodes and the hidden layer nodes; a is the bias of the visible layer nodes; b is the bias of the hidden layer nodes; c is the bias of the output layer nodes; U is the connection weight between the hidden layer and the output layer;

[0022] The joint probability distribution is obtained from the above energy function:

[0023]

[0024] Where Z(θ) is the partition function, which is expressed as follows:

[0025]

[0026] Furthermore, in step 3, the convolutional confidence generative adversarial network model is trained using the weld data set to generate a convolutional confidence generative adversarial network model for weld defect detection. The specific process includes:

[0027] Step 3-1, initialize the parameters of the generator G and discriminator D networks;

[0028] Step 3-2, extract n samples from the weld dataset as real samples, add random noise z to the generator G to generate n samples as fake samples, and mark the defect category;

[0029] Step 3-3, fix the generator G and train the discriminator D based on the deep convolutional belief network. Specifically: train layer by layer, and when the training of one layer of network is optimized, start the training of the next layer of network; for the training of the belief layer, continuously adjust the number and structural parameters of the restricted Boltzmann machine in the hidden layer until the training of the belief layer reaches the required optimal state;

[0030] Step 3-4, update the discriminator D k times through the above process, and then update the generator G once in the same way;

[0031] Step 3-5, after the update iteration reaches the preset number of times, determine whether the probability that the discriminator distinguishes the ship weld image as a real sample or a false sample is 0.5. If so, the training is completed, otherwise return to step 3-2.

[0032] A ship weld defect detection system based on a convolutional confidence generative adversarial network model, the system comprising:

[0033] A data set construction module is used to obtain a number of ship weld images and construct a weld data set;

[0034] Model building module, used to build convolutional confident generative adversarial network models;

[0035] A model training module is used to train a convolutional confidence generative adversarial network model using a weld data set to generate a convolutional confidence generative adversarial network model for weld defect detection;

[0036] The detection module is used to collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

[0037] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0038] Step 1, obtain several ship weld images and build a weld dataset;

[0039] Step 2: Construct a convolutional belief generation adversarial network model;

[0040] Step 3, using the weld data set to train a convolutional confidence generative adversarial network model to generate a convolutional confidence generative adversarial network model for weld defect detection;

[0041] Step 4: Collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0043] Step 1, obtain several ship weld images and build a weld dataset;

[0044] Step 2: Construct a convolutional belief generation adversarial network model;

[0045] Step 3, using the weld data set to train a convolutional confidence generative adversarial network model to generate a convolutional confidence generative adversarial network model for weld defect detection;

[0046] Step 4: Collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

[0047] Compared with the prior art, the present invention has the following significant advantages:

[0048] (1) The unbalanced processing method for ship weld defects based on the generative adversarial network model can expand the data set based on known unbalanced samples and use the generated balanced data set for ship weld defect detection. The convolutional confidence generative adversarial network adopted is an unsupervised learning model that can find good features without category information; and can select some representative pictures from the ship weld sample set and annotate them for discriminator training. The model effectively avoids the problem of over-reliance on the lack of prior knowledge, and also effectively solves the problem of high difficulty in manual category labeling or high cost of manual category labeling.

[0049] (2) A generative adversarial network based on a deep convolutional belief neural network is proposed. The deep convolutional belief neural network is used as the generator and discriminator of the generative adversarial network to expand the performance of the generator and discriminator, making the generative adversarial network more stable and enhancing the effectiveness of ship weld defect detection.

[0050] (3) A confidence layer with deep clustering function. The confidence layer extracts the deep hidden features of the ship weld image from the fully connected layer and performs clustering analysis on the deep hidden features. The deep clustering method performs clustering tasks very efficiently in complex high-dimensional data sets and can improve the clustering effect of noise and abnormal data points by capturing the connection between similar samples. The output of the confidence layer is used as the classification detection of ship weld defects, which makes the network more scalable and stable.

[0051] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is the overall flow chart of the ship weld defect detection method based on the convolutional belief neural network model.

[0053] Figure 2 This is the basic framework diagram of the convolutional confidence generation adversarial network model.

[0054] Figure 3 This is the structure diagram of the deep convolutional belief network.

[0055] Figure 4 It is the structure diagram of restricted Boltzmann machine.

[0056] Figure 5 This is the confidence layer structure diagram.

[0057] Figure 6 Flowchart of unsupervised greedy thinking.

[0058] Figure 7 The CBGAN model detection results of fine pores and incomplete penetration defects.

[0059] Figure 8 This is the CBGAN model detection result diagram of multi-defect ship welds. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] Ship welding often causes weld defects due to factors such as welding methods, process sequence and welding equipment, and the main defects include slag inclusions, cracks, pores and incomplete penetration. However, in the actual welding process, the probability of each defect is not the same, so the images obtained by X-ray detection may have serious imbalance problems. Table 1 shows the 500 general data sets of welding X-ray internal detection obtained. It can be seen from the table that the number of pores is the largest, followed by cracks, followed by slag inclusions, and the number of incomplete penetration is very small, less than 10% of the total number of samples, which belongs to the minority class samples, resulting in data imbalance, which will reduce the generalization ability of the neural network model used for ship weld defect detection and make it prone to underfitting. For this reason, before training the convolutional confidence generation adversarial network model, it is necessary to first carry out data imbalance processing.

[0062] Table 1 Main defects

[0063] type quantity Slag inclusion 207 crack 375 Pores 422 Incomplete penetration 36

[0064] In order to solve the problem of imbalanced training sets, oversampling or undersampling techniques are often used in engineering. Since there are many inclusion defects in ship welds, when sampling technology is used to deal with imbalanced problems, the multi-defect problem will interfere with the sampling of minority classes, often resulting in poor processing results. If you expect to achieve good sampling results, you need to have sufficient prior knowledge of ship weld defects; and since multiple defect types are mixed in one picture, it is more difficult to manually label categories. If oversampling technology is used to process unbalanced data sets, some of the same samples will appear repeatedly, causing the model to overfit to a certain extent; using undersampling technology to process unbalanced data sets will lose some effective information, so that the ship weld detection model can only learn part of the overall characteristics, resulting in model underfitting.

[0065] The problem of ship weld detection with multiple defects is a complex classification problem in the field of machine learning and deep learning. In the face of complex classification problems, traditional methods often use model pre-training and feature engineering to enhance the recognition accuracy of the model. Due to the serious data imbalance problem in ship weld defect detection and the lack of some effective features, the use of traditional methods for learning is prone to model underfitting.

[0066] In view of the above problems, based on game theory and using the basic viewpoints of probability and mathematical statistics, a ship weld defect detection method based on convolutional belief generative adversarial network (CBGAN, Convolutional Belief Generative Adversarial Networks) is proposed. The method includes the following steps:

[0067] Step 1, obtain several ship weld images and build a weld dataset;

[0068] Step 2: Construct a convolutional belief generation adversarial network model;

[0069] Step 3, using the weld data set to train a convolutional confidence generative adversarial network model to generate a convolutional confidence generative adversarial network model for weld defect detection;

[0070] Step 4: Collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

[0071] Generative Adversarial Networks (GAN) is an unsupervised deep learning method on complex distributions. The GAN model requires at least two modules: a generator G (Generative Model) and a discriminator D (Discriminative Model). The generator generates realistic images by mapping images from a small dimensional space to a pixel-level image space and minimizing random noise. The discriminator discriminates the images generated by the generator. When the input is a real image, the output of the discriminator D is 1; when the input is a generated image, the output of the discriminator D is 0. When training the generative adversarial network, n samples are extracted from the training set as real samples, and n noise samples generated by the generator based on random noise are used as fake samples to train the discriminator D, so that it can distinguish between true and false samples as much as possible.

[0072] like Figure 2 As shown in the figure, CBGAN inputs random noise z that obeys Gaussian distribution or uniform distribution into the generator G, and outputs the generated image G(z); the generated image G(z) and the real image x are input into the discriminator D, and the output is the probability that G(z) is true.

[0073] Within a certain batch of samples, the model calculates and feeds back the loss based on the output probability, so that the parameters of the generator D and the discriminator G are continuously optimized. The objective functions of the generator and the discriminator can be expressed as the minimization objective function V(D,G):

[0074]

[0075] In the formula, p data (x) is the true sample probability distribution function, p z (z) is the probability distribution function of generating noise samples, and E[·] represents the expected value of each sample.

[0076] It can be seen from the above formula that the optimization goals of the generator and the discriminator are different, forming an "adversarial" learning relationship. For the discriminator D, it is expected to be able to identify real samples, that is, D(x) is as close to 1 as possible, and the larger log[D(x)] is, the better. During the training process, the goal of the generator G is to generate real pictures as much as possible to deceive the discriminator D, and the goal of D is to distinguish the pictures generated by G from the real pictures as much as possible. In this way, G and D constitute a dynamic game process. In the most ideal state, G can generate pictures G(z) that are "real" enough. For D, it is difficult to determine whether the pictures generated by G are real, so the following discriminator optimization objective function D is obtained: G (x):

[0077]

[0078] In the formula, pG (x) represents the probability distribution function of the real sample on the generator G.

[0079] As training improves, the distribution of real samples in the generator G tends to be the same as the distribution of real samples, that is, the following relationship exists:

[0080] p G (x)→p data (x)

[0081]

[0082] Therefore, the generator G and the discriminator D in the trained and perfected generative adversarial network are in a Nash equilibrium state, which is the result of adversarial game. Due to the lack of actual ship welding technology, there are serious imbalance problems in the types of defects, and there are also mixed problems of multiple weld defects. The generator G of the invention generates a data set by using CBGAN. The generated ship weld defect data set is richer than the original data set, and the number of ship weld defect images is increased; the discriminator D improves the defect detection performance in continuous adversarial learning. The model can expand the data set based on known unbalanced samples, and use the generated balanced data set for ship weld defect detection.

[0083] The generative adversarial network adopted by the invention is an unsupervised learning model that can find good features without category information; and can select some representative pictures from the ship weld sample set and annotate them for discriminator training. This model effectively avoids the problem of over-reliance on lack of prior knowledge, and also effectively solves the problem of high difficulty or high cost of manual category labeling.

[0084] In the original generative adversarial network theory, it is not required that both the generator G and the discriminator D are neural networks, but only that they can fit the corresponding generation and discrimination functions. An excellent generative adversarial network model requires a good training method, otherwise it will lead to unsatisfactory results in ship weld quality inspection. In order to solve the problem of unstable GAN training, Goodfellow pioneered the combination of convolutional neural networks (CNN) and generative adversarial networks in 2014 and proposed deep convolution generative adversarial networks (DCGAN). Through a large amount of training and testing on a large number of different data sets, it is fully demonstrated that the generator G and discriminator D of DCGAN can learn rich features, and the model has very good generalization ability.

[0085] Ship weld defect detection has the problem of multiple defect types mixed and serious defect type imbalance. In order to further solve the problem of poor training of generative adversarial networks, the concept of belief layer (BL) is proposed based on the Bayesian probability point of view using the restricted deep Boltzmann machine. Combined with DCGAN, the most promising research method in the current deep learning field, the deep convolutional belief network (DCBN) is proposed as the generator and discriminator of the generative adversarial network. The structure of the deep convolutional belief neural network has 5 convolutional layers, 2 fully connected layers, and 1 belief layer. Each layer takes the output of the previous layer as input, such as Figure 3 shown.

[0086] The basic unit of the confidence layer is the restricted Boltzmann machine (RBM), which has a visible layer (VL) and a hidden layer (HL). Correspondingly, the visible layer is composed of several visible units (VU), which can be referred to as visible units in the following. A single visible unit can be represented by v i (i=1,2,3…) indicates that the hidden layer is composed of several hidden neurons (HU, Hidden Unit), which can be referred to as hidden units in the following. A single hidden unit can be represented by h i (i=1,2,3...) indicates that the specific structure is as follows Figure 4 shown.

[0087] RBM is an energy-based model, and its energy function can be expressed as:

[0088] E(v,y,h)=-h T Wv-a T vb T hc T yh T Uy

[0089] Where: v=(v1,v2,…,v m ), h=(h1,h2,…,h k ) and y=(y1,y2,…,y p ) represent the states of the visible layer, hidden layer and output layer respectively; θ = {W, a, b, c, U} is the structural parameter that needs to be learned in training, W is the connection weight between the visible layer nodes and the hidden layer nodes; a is the bias of the visible layer nodes; b is the bias of the hidden layer nodes; c is the bias of the category layer nodes; U is the connection weight between the hidden layer and the category layer. Given the RBM energy function, the joint probability distribution can be obtained:

[0090]

[0091] In the formula, the partition function Z(θ) can ensure the validity of the joint probability distribution. The expression of the partition function is as follows:

[0092]

[0093] Using restricted Boltzmann machines, according to Bayesian probability view and clustering idea, a confidence layer is proposed. The confidence layer is formed by connecting several restricted Boltzmann machines in series. The model consists of three layers of restricted Boltzmann machines. The hidden layer of the previous RBM is the visible layer of the next RBM, and the output of the previous RBM is the input of the next RBM. The specific structure is as follows: Figure 5 shown.

[0094] The confidence layer can be understood as the application of the Bayesian probability model. The main theoretical basis of the Bayesian probability model is the Bayesian theorem, which is a conditional probability theorem about random events A and B. Suppose the sample space of experiment E is S, A is the event of E, B1, B2,…, B n is a partition of S, and P(A)>0,P(B i )>0(i=0,1,2,…,n), the Bayesian formula is expressed as:

[0095]

[0096] In the formula, j=0,1,2,…,n.

[0097] The confidence layer consists of a visible input layer, a visible output layer, and multiple layers of random hidden layers. The top layer is the visible output layer. The essence of the middle layer is to stack restricted Boltzmann machines of different numbers. The bottom layer is the visible input layer. Combining the Bayesian formula with the restricted Boltzmann machine theory, we can get the joint probability distribution:

[0098] P(v,h 1 ,h 2 ,...,h l )=P(v|h 1 )P(h 1 |h 2 )...P(h 1-1 |h 1 )

[0099] In the formula, h=(h 1 ,h 2 ,…,h l ) represents the hidden layer state in the confidence layer. If there are three hidden layers, then l = 3.

[0100] The hidden layer is composed of several hidden elements, which can function as feature detectors. Therefore, each restricted Boltzmann machine can be used as a clusterer alone. The confidence layer is a layer composed of several restricted Boltzmann machines connected in series, which is essentially a deep clusterer. Compared with traditional clustering algorithms such as distance-based K-means, deep clustering methods can perform clustering tasks very efficiently in complex high-dimensional data sets, and can improve the clustering effect of noise and abnormal data points by capturing the connection between similar samples.

[0101] In the deep convolutional belief neural network structure, the confidence layer extracts the deep hidden features of the ship weld image from the fully connected layer, performs cluster analysis on the deep hidden features, and uses the output of the confidence layer as the classification detection of ship weld defects, so that the network has better scalability and stability.

[0102] The training of the generative adversarial network model is crucial, and the quality of its training directly determines the performance of the generator G and the discriminator D. An excellent generative adversarial network model requires a good training method, otherwise the final classification and detection effect will be unsatisfactory. We choose to use the unsupervised greedy idea to pre-train the convolutional confidence generative adversarial network layer by layer, so that both the generator G and the discriminator D can learn rich features, and the model has very good generalization ability. The unsupervised greedy idea means that when solving a problem, when faced with several sub-problems, we always make the best choice that seems to be the best at the moment, and there is no need for any intervention or guidance on the optimization method in the process.

[0103] The specific algorithm flow chart is shown in Figure 6:

[0104] (1) Initialize the parameters of the generator G and the discriminator D networks.

[0105] (2) For the generator G, n samples are extracted from the training set and random noise z is added to generate n samples.

[0106] (3) The generator G is fixed, and the discriminator D based on the deep convolutional belief network is trained using an unsupervised greedy approach. The deep convolutional belief neural network consists of 5 convolutional layers, 2 fully connected layers, and 1 belief layer. The training is performed layer by layer, and when the training of one layer of the network is optimized, the training of the next layer of the network begins. For the training of the belief layer, the number and structural parameters of the restricted Boltzmann machine in the hidden layer are continuously adjusted to make the training of the belief layer reach the optimal state.

[0107] (4) After cyclically updating the discriminator D k times, update the generator G once. The adjustment method of the generator G network parameters is the same as the training method of the discriminator D.

[0108] (5) After multiple updates and iterations, the discriminator D is finally unable to distinguish whether the weld image comes from the real training sample set or from the sample generated by the generator G. At this time, the discriminator's discrimination probability is 0.5, and the training is completed.

[0109] In order to evaluate the effectiveness of the weld defect detection model, accuracy, precision, recall and balance score F1 (F1 Score) are selected as evaluation indicators. Obviously, accuracy is the most important indicator of the model, so the following accuracy Acc is defined:

[0110]

[0111] Where Num(.) represents the count, j represents the test sample number, V P Represents the model's predicted value for the sample, V R Represents the true value of the sample.

[0112] In order to accurately express other evaluation indicators, the following definitions are made:

[0113] TP: Predict the correct number of positive images

[0114] FN: The number of errors in predicting positive images

[0115] FP: The number of negative class images predicted incorrectly

[0116] TN: The number of correct predictions for negative classes

[0117] The precision rate P indicates the proportion of positive class samples that are correctly classified by the model, and is calculated as follows:

[0118]

[0119] The recall rate R indicates the completeness of the model's correct classification of positive class samples, and is calculated as follows:

[0120]

[0121] The balanced score F1 (F1 Score), which is defined as the harmonic mean of precision and recall, is defined as follows:

[0122]

[0123] In order to facilitate the study of the detection effect of the model in the context of unbalanced weld image data, it is stipulated that the minority defect incomplete penetration images are positive classes, and the other three types of defect slag inclusion, crack, and porosity images are negative classes.

[0124] In order to verify the effectiveness of the CBGAN model in processing unbalanced data with multiple defect inclusions, the neural network models US-CNN, SMOTE-CNN, 11Features-DNN, and K-CNN based on undersampling technology, oversampling technology, multi-features, and K-means clustering were tested and compared with the CBGAN model. The results are shown in Table 2.

[0125] Table 2 Experimental comparison results

[0126] Model Acc(%) P(%) R(%) F1 US-CNN 88.70 82.94 97.35 0.896 SMOTE-DNN 90.46 96.24 82.16 0.886 11Features-DNN 92.10 97.43 90.77 0.940 K-CNN 91.09 89.94 96.35 0.930 CBGAN 96.87 96.10 97.03 0.971

[0127] As can be seen from Table 2, the CBGAN model has a high recognition accuracy of 96.87% in weld defect detection. Compared with the undersampling technology, the recognition accuracy of the CBGAN model is improved by 8.17%; the recall rate R, the CBGAN model is slightly inferior; but the precision rate R, the CBGAN model is significantly higher than the undersampling model by 13.16%. Compared with other weld defects, the geometric configuration of the positive class of incomplete penetration is relatively simple. With the undersampling technology, the number of correct predictions of positive class images TP is relatively large, and the number of incorrect predictions of positive class images FN is relatively small, so it performs well in the recall rate R; but due to the use of undersampling technology, some effective features are lost, and the model performs poorly when learning images with mixed and coexisting weld defects, resulting in a significant increase in the number of incorrect predictions of negative class images FP, causing the precision rate R to drop significantly. The precision rate R of the CBGAN model is higher than that of the oversampling model, and the recall rate R is slightly lower than that of the oversampling model. The F1 score is used for comprehensive evaluation: the CBGAN model is 0.075 points higher than the undersampling model. Similarly, the oversampling technology cannot accurately handle minority class problems. The model overfits when learning images with mixed weld defects, and its performance is not as good as that of the CBGAN model.

[0128] In summary, the sampling technology performs poorly in the complex weld defect detection and classification problem, while the detection method based on the convolutional confidence generative adversarial network performs well. If you want to generate minority class samples in a refined manner, the sampling rule requires more prior knowledge. The CBGAN method uses the game idea and does not require too much prior knowledge to continuously improve the performance of the generator G. The weld images generated by the generator G through game confrontation have clear geometric features.

[0129] It can be analyzed from Table 2 that the weld defect detection method based on feature engineering 11Features-DNN and K-CNN are not as good as CBGAN overall. The method of improving the performance of deep learning models based on feature engineering requires an accurate grasp of the features, otherwise it will bring poor results. Proposing too many features is prone to overfitting of neural networks, resulting in poor generalization ability of the model; using the clustering processing feature method, it is necessary to remove abnormal points and have a certain grasp of effective features. The CBGAN model adopted by the invention is an unsupervised learning method that can find good features without category information; and can select some representative pictures from the ship weld sample set and annotate them for discriminator training. Each individual hidden element in the confidence layer proposed by the invention essentially acts as a feature detector, and each restricted Boltzmann machine can be essentially regarded as a clusterer. The confidence layer of multiple restricted Boltzmann machines in series plays a role of deep clustering. The introduction of the confidence layer actually optimizes the clustering analysis ability of features. Therefore, the generative adversarial network based on the deep convolutional belief neural network has a stronger grasp of features in the unsupervised learning mode.

[0130] In order to further verify the effectiveness of the CBGAN model in defect detection when multiple defects coexist, the data set processed by the generator G is substituted into the CNN and 11Features-DNN models mentioned above for testing; the original data set is substituted into the DCGAN and CBGAN models for testing; the data set processed by DCGAN is substituted into the CNN model for testing. The results are shown in Table 3.

[0131] Table 3 Experimental comparison results

[0132] Model Acc(%) <![CDATA[Acc1(%)]]> <![CDATA[Acc2(%)]]> CNN 92.67 93.91 90.21 11Features-DNN 93.67 94.65 91.71 DCGAN 90.33 92.28 87.95 DCGAN-CNN 91.33 92.54 89.14 CBGAN 96.87 98.10 94.47

[0133] In the table, Acc represents the overall recognition accuracy, Acc1 represents the recognition accuracy of a single weld defect, and Acc2 represents the recognition accuracy of two or more weld defects. It can be seen from Table 3 that compared with the other four models, the generative adversarial network model based on the deep convolutional belief network performs well, reaching the highest overall recognition accuracy of 96.87%. Compared with the data in Table 2, the convolutional neural network model is also used as a weld detection classifier. The test results of the data set processed by the generator G are good, which is better than the data set processed by the undersampling and oversampling techniques in Table 2, and the recognition accuracy is increased by 3.97% and 2.21% respectively. Similarly, on the weld defect detection method 11Features-DNN based on feature engineering, the data set processed by the generator G has a good effect, which is better than directly using the original data set, and the recognition accuracy is increased by 3.21%. In summary, the generator G based on the deep convolutional belief network has excellent performance.

[0134] Further analysis of Table 3 shows that among the three model test results based on generative adversarial networks, the generative adversarial network model based on deep convolutional belief network still performs best. Compared with the DCGAN model, the generator G is also used to expand the data set based on known imbalanced samples, and then the discriminator D is used to detect ship weld defects. The GAN based on deep convolutional belief network is superior to the GAN based on convolutional neural network, and the recognition accuracy is improved by 6.54%. Focusing on the DCGAN-CNN model, using DCGAN to process the imbalanced data set and then using CNN for weld defect classification detection, its effect is not as good as using the generative adversarial network model based on deep convolutional belief network to process the imbalanced data set and then using CNN for weld defect classification detection, which once again proves that the performance of the generator G based on deep convolutional belief network is better than that of the generator G based on convolutional neural network. Compared with the original DCGAN, the recognition accuracy of DCGAN-CNN and CBGAN increased by 1.00% and 6.54% respectively. The CBGAN model uses the restricted Boltzmann mechanism to construct a deep clusterer to cluster the deep hidden information output by the convolutional layer and the fully connected layer, thereby improving the performance of the generative adversarial network. In addition, it can be seen from the test data that the selection of the generator G and the discriminator D is crucial. The effect of the generative adversarial network depends on the game-based learning effect of G and D. G and D are in a Nash equilibrium state, and the model has a strong generalization ability. This partially confirms the feasibility of Goodfellow's various optimization schemes based on this framework.

[0135] Compared with other models, the CBGAN model performs better when there are multiple defects in ship weld defect detection, and the recognition accuracy of two or more weld defects reaches the highest 94.47%. Figure 7 The weld defect pores shown in the figure are subtle in shape, and are accompanied by a few types of defects such as incomplete penetration, which are difficult to distinguish. The other model methods in Table 3 cannot accurately detect all the subtle pores and incomplete penetration defects, but the CBGAN model can accurately detect these two defects.

[0136] In order to further verify the effectiveness of the convolutional confidence-generating adversarial network in ship weld defect detection, 150 X-ray internal inspection images of ship welds were first collected, and the distribution of weld defects is shown in Table 4. Then they were substituted into the model for testing, and the detection results were still compared with the detection results of the CNN, 11Features-DNN, DCGAN and DCGAN-CNN models mentioned above. The results are shown in Table 5.

[0137] Table 4 Distribution of ship weld dataset

[0138] Total number of samples 150 Number of single defects 47 Number of two or more defects 103

[0139] Table 5 Experimental comparison results

[0140] Model Acc(%) <![CDATA[Acc1(%)]]> <![CDATA[Acc2(%)]]> CNN 89.33 93.62 87.38 11Features-DNN 90.00 93.62 88.35 DCGAN 87.33 91.49 84.44 DCGAN-CNN 88.67 89.36 88.35 CBGAN 94.67 95.74 94.17

[0141] It can also be seen from Table 5 that the overall recognition accuracy of the CBGAN model for ship weld defect detection is still the highest. In the detection of a single weld defect, the recognition accuracy of the three generator G detection models based on deep convolutional belief networks is similar, among which the CBGAN model has the highest recognition accuracy of 95.74%, which is significantly better than the detection efficiency of the generator G based on the convolutional neural network. In the detection of two or more weld defects, the CBGAN model has an absolute advantage, reaching 94.17%. Comparing the data in Table 5 with Table 3, the recognition accuracy of the other four models in the detection of two or more ship weld defects has dropped significantly, while the CBGAN model has stable detection function, high efficiency, and good generalization ability. Figure 8 There are three kinds of defects in the weld metal, namely slag inclusion, crack and incomplete penetration. The CBGAN model can accurately detect all three defects, while other model methods in the table cannot accurately detect all of them, which further verifies the effectiveness of the invention.

[0142] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A ship weld defect detection method based on a convolutional confidence generative adversarial network model, characterized in that: The method comprises the following steps: Step 1, obtain several ship weld images and build a weld dataset; Step 2: Construct a convolutional belief generative adversarial network model; specifically: use the deep convolutional belief neural network as the generator G and discriminator D of the generative adversarial network; The deep convolutional belief neural network includes 5 convolutional layers, 2 fully connected layers and 1 belief layer. The input image passes through 5 convolutional layers and 2 fully connected layers in sequence and is output through the belief layer. The basic component unit of the confidence layer is a restricted Boltzmann machine (RBM), which includes a visible layer and a hidden layer. The hidden layer of the previous RBM is the visible layer of the next RBM, and the output of the previous RBM is the input of the next RBM. The visible layer includes several dominant neurons, referred to as visible neurons. A single visible neuron is represented by v i The hidden layer includes several hidden neurons, referred to as hidden units, and a single hidden unit is represented by h i Indicates that i=1, 2, 3…; The energy function of the restricted Boltzmann machine is expressed as: E(v,y,h)=-h T Wv-a T v-b T h-c T y-h T Uy Where, v=(v1,v2,…,v m ), h=(h1,h2,…,h k ) and y=(y1,y2,…,y p ) represent the states of the visible layer, hidden layer and output layer respectively, m, k, p represent the number of neurons in the visible layer, hidden layer and output layer respectively; θ = {W, a, b, c, U} is the structural parameter to be learned in training, W is the connection weight between the visible layer nodes and the hidden layer nodes; a is the bias of the visible layer nodes; b is the bias of the hidden layer nodes; c is the bias of the output layer nodes; U is the connection weight between the hidden layer and the output layer; The joint probability distribution is obtained from the above energy function: Where Z(θ) is the partition function, which is expressed as follows: Step 3, using the weld data set to train the convolutional confidence generative adversarial network model to generate a convolutional confidence generative adversarial network model for weld defect detection; wherein, during the training process of the convolutional confidence generative adversarial network model: the generator G is fixed, and the discriminator D based on the deep convolutional confidence network is trained using the unsupervised greedy idea; Step 4: Collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

2. The ship weld defect detection method based on the convolutional confidence generative adversarial network model according to claim 1 is characterized in that: In step 3, the convolutional confidence generative adversarial network model is trained using the weld data set to generate a convolutional confidence generative adversarial network model for weld defect detection. The specific process includes: Step 3-1, initialize the parameters of the generator G and discriminator D networks; Step 3-2, extract n samples from the weld data set as real samples, add random noise z to the generator G to generate n samples as fake samples, and mark the defect category; Step 3-3, fix the generator G and train the discriminator D based on the deep convolutional belief network. Specifically: train layer by layer, and when the training of one layer of network is optimized, start the training of the next layer of network; for the training of the belief layer, continuously adjust the number and structural parameters of the restricted Boltzmann machine in the hidden layer until the training of the belief layer reaches the required optimal state; Step 3-4, update the discriminator D k times through the above process, and then update the generator G once in the same way; Step 3-5, after the update iteration reaches the preset number of times, determine whether the probability that the discriminator distinguishes the ship weld image as a real sample or a false sample is 0.

5. If so, the training is completed, otherwise return to step 3-2.

3. A ship weld defect detection system based on a convolutional belief generative adversarial network model based on the method according to any one of claims 1 to 2, characterized in that: The system comprises: A data set construction module is used to obtain a number of ship weld images and construct a weld data set; Model building module, used to build convolutional confident generative adversarial network models; A model training module is used to train a convolutional confidence generative adversarial network model using a weld data set to generate a convolutional confidence generative adversarial network model for weld defect detection; The detection module is used to collect images of the ship welds to be inspected and use the trained convolutional confidence generative adversarial network model to perform weld defect detection.

4. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.