Semi-supervised wood defect detection method based on generative adversarial network

Abstract

According to the semi-supervised wood defect detection method based on the generative adversarial network provided by the invention, effective information in an unlabeled wood defect image which is easy to obtain is effectively explored by adopting an edge discriminator network algorithm and a segmentation network algorithm of a semi-supervised learning framework of the generative adversarial network GAN; the semi-supervised framework uses a DeepLabV3 + semantic segmentation network as a generator; using a full convolutional network as a discriminator, inputting an image segmentation mask pair and outputting a corresponding confidence map, and judging whether the segmentation mask is true or false; the edge discriminator network and the segmentation network perform adversarial training in a mutual game mode, when the discriminator reaches the required discrimination capability through stage training, the generator generates a segmentation result of a label-free image and an image label pair formed by corresponding original images, and then inputs the segmentation result and the image label pair into the discriminator; a label part which is verified to be true through a discriminator is used as a supervision signal, and a data set is added for training of a segmentation network; the reliability and segmentation performance of the detection model can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of industrial wood defect detection, and in particular to a semi-supervised wood defect detection method based on a generative adversarial network. Background Art

[0002] Wood is very susceptible to various types of damage during its growth and processing, making it inevitable that various types of defects will appear on the surface of the wood boards used for processing. The presence of surface defects seriously affects the appearance and structural properties of wood and its finished products, greatly reducing its value and wood utilization. However, traditional manual wood defect detection methods are subjective and inefficient, making it difficult to achieve stable product quality management and improve production efficiency. Therefore, achieving accurate and efficient automated wood defect detection is an important part of the automated processing of industrial wood.

[0003] Wood defect detection methods based on traditional computer vision extract features such as texture, color, and shape of material surface defects, and manually design defect features to achieve automatic detection of wood defects. The detection performance of this type of method is highly dependent on the subjective experience of researchers, and has poor robustness and versatility. The wood defect detection method based on deep learning is adopted, and the generative adversarial network is introduced to achieve semi-supervised learning, which can realize the automatic extraction of sample defect features, reduce the manual burden, and improve the robustness and recognition accuracy of detection.

[0004] The task of semantic segmentation of wood defects requires a large number of high-quality semantic labels, while the definition of wood defects requires certain experience and the definition criteria are affected by subjective factors, which makes it difficult and costly to obtain labeled data, while unlabeled defect images are easy to obtain. By designing a semi-supervised learning framework based on generative adversarial networks (GANs), effective information in unlabeled defect images can be effectively discovered, the network's dependence on labeled data can be reduced, the labor cost can be effectively reduced, and the reliability of the detection algorithm can be improved.

[0005] The Chinese patent application number is: CN117876663A, and the name is: A wood defect detection method based on deep neural network. A target detection model based on YOLOv8 was constructed to detect wood defects. By replacing the Conv module of the Neck network with the RFCA module, adding a small target decoupling detection head to the Head part, and introducing the SiLU activation function and WiseIoU loss function, accurate detection of wood defects is achieved. This method uses a detection box to determine the rectangular boundary of the defect, and cannot output the exact boundary position of the defect, which cannot meet the needs of high-precision industrial wood processing.

[0006] The Chinese patent application number is: CN202310578316.8, and the name is: Wood floor color classification and defect detection method based on semi-supervised multi-task detection. Use a small number of labeled images as a data set to train the Color RCNN model for color classification and defect detection. Input the remaining unlabeled images into the trained prediction model, and use the prediction results as pseudo labels. At the same time, train a semi-supervised prediction monitoring network to determine the accuracy of the pseudo labels, retain the accurate detection frames and add them to the data set, and reuse them for model training to achieve semi-supervised defect detection. This method trains the prediction model and the discrimination model independently, which makes it difficult to effectively balance the prediction and discrimination performance of the two models, and cannot guarantee the stability and reliability of model training to achieve optimal utilization of unlabeled data. Summary of the invention

[0007] The present invention proposes a semi-supervised wood defect detection method based on a generative adversarial network, which is applied to the task of automatic wood defect detection. It can significantly reduce the amount of labeled data required for training a reliable defect segmentation network, solve the problem of insufficient data volume of high-quality labeled data sets faced in the actual production process, greatly improve the reliability and segmentation performance of the detection model, and thus effectively reduce the labor cost of industrial production and improve production efficiency.

[0008] The present invention adopts the following technical solutions.

[0009] A semi-supervised wood defect detection method based on a generative adversarial network (GAN) is proposed. The method effectively explores effective information in easily accessible unlabeled wood defect images by using an edge detector network algorithm and a segmentation network algorithm in a semi-supervised learning framework of a generative adversarial network (GAN).

[0010] The semi-supervised framework includes a generator G(.) and a discriminator D(.); the DeepLabV3+ semantic segmentation network is used as the generator to generate pixel-level segmentation results of wood defect images; a fully convolutional network is used as the discriminator, which inputs image segmentation mask pairs and outputs corresponding confidence maps to judge the authenticity of the segmentation masks; the edge detector network and the segmentation network are trained adversarially in a game-like manner to automatically learn the data distribution of the sample set; when the discriminator reaches the required discrimination ability after stage training, the generator generates an image label pair consisting of the segmentation result of the unlabeled image and the corresponding original image, which is then input into the discriminator, and the label part verified to be true by the discriminator is used as a supervisory signal and added to the dataset for training the segmentation network.

[0011] The method comprises the following steps:

[0012] Step S1, obtaining a wood defect dataset, and dividing the defect dataset into a labeled dataset and an unlabeled dataset;

[0013] Step S2, model creation, i.e. creating a segmentation network and a discriminator network to form a semi-supervised adversarial learning model and initializing model parameters;

[0014] Step S3, image preprocessing;

[0015] Step S4, training the generator;

[0016] Step S5, training the discriminator;

[0017] Step S6: adversarial training;

[0018] Step S7: semi-supervised learning;

[0019] Step S8: deploy the trained segmentation network to perform defect detection on the wood image.

[0020] The step S1 specifically includes: acquiring wood image data through a linear array camera, semantically annotating 1 / M defect images using annotation software, and not annotating the remaining (M-1) / M images, and dividing the defect data set into an annotated data set and an unannotated data set. The annotated data set includes wood defect images and their corresponding manual semantic annotations, and the unannotated data set only includes defect images.

[0021] Step S3 specifically includes: eliminating noise and interference in the wood image through a filtering algorithm, and performing data enhancement by random cropping, random inversion, etc.

[0022] Step S4 is as follows: Input the labeled data set into the DeepLabV3+ network for T iterations of training, and minimize the multi-classification cross entropy loss function. Guide the segmentation network of the generator to generate prediction results similar to the distribution of manually annotated data, so that the segmentation network can obtain the required prediction ability;

[0023] The generator cross entropy loss function formula is as follows:

[0024]

[0025] in represents the multi-classification cross entropy loss, X and Y represent wood images and their corresponding manual annotations, respectively;

[0026] The generator uses the obtained prediction ability to predict the unlabeled dataset to obtain the prediction results.

[0027] Step S5 is as follows: the manual labels and the prediction results of the generator are spliced ​​with the corresponding defect images in the channel direction to obtain image mask pairs, which are respectively sent to the discriminator for T iterations of training to minimize the discriminant loss function composed of the binary cross entropy loss. The discriminator is encouraged to distinguish between the true and false input segmentation masks. When the discriminant network input comes from the true label, the discriminator is encouraged to output a prediction confidence of 1; when the input comes from the generator prediction result, the discriminator is encouraged to output a confidence of 0;

[0028] The discriminator loss function formula is as follows:

[0029]

[0030] Formula 2:

[0031] in represents the binary cross entropy loss, Represents the concatenation of the wood image and the corresponding segmentation mask in the channel direction.

[0032] Step S6 improves the prediction ability of the generator and the discriminator through the adversarial learning method of mutual game, specifically: by minimizing the adversarial loss function Encourage the generator to output more accurate prediction results to deceive the discriminator; at the same time, force the discriminator to further learn to distinguish the true and false input segmentation masks, thereby improving its own discrimination ability and outputting more accurate confidence maps;

[0033] The formula for the adversarial loss function is:

[0034]

[0035] In step S7, after N iterations of training, both the generator and the discriminator can output reliable prediction results. The introduction of the semi-supervised learning process is as follows: the image mask consisting of the unlabeled image and the segmentation mask obtained by the generator is sent to the discriminator for prediction to obtain a confidence map representing the truth or falsehood of the input mask. The generated confidence map is binarized by a preset confidence threshold γ. If the confidence is greater than the threshold, it is set to 1, otherwise it is set to 0, so as to retain the segmentation results corresponding to the mask area with a confidence greater than the threshold, that is, the area is judged to be true; finally, the unlabeled image segmentation result judged to be true is used as a pseudo-label, added to the training data set, and the semi-supervised loss is used. Train the generator;

[0036] The formula for the semi-supervised loss function is:

[0037]

[0038] in is the pseudo-label obtained by applying the softmax function to the generator output S(X) in the category dimension, and F(.) is the decision function, defined as:

[0039] In the detection method, the discriminator network is only effective in the model training stage, and the detection task only needs to use the segmentation network.

[0040] In step S8, the trained segmentation network is deployed on a computer to perform defect detection on the wood image.

[0041] The beneficial effects of the present invention are

[0042] 1) The present invention adopts the advanced semantic segmentation network DeepLabV3+ as the defect detection model, which can obtain accurate pixel-level classification results and meet the needs of high-precision wood industry production.

[0043] The adoption of a semi-supervised learning framework based on a generative adversarial network can effectively utilize the information in unlabeled images as a supervisory signal, reduce the detection model's demand for labeled data, greatly improve the defect detection effect of the model under insufficient sample conditions, improve production efficiency, and reduce production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0045] Attached Figure 1 It is a schematic diagram of the process of the present invention;

[0046] Attached Figure 2 Schematic diagram of the semi-supervised wood defect detection framework based on generative adversarial network;

[0047] Attached Figure 3 Schematic diagram of the discriminator network structure;

[0048] Attached Figure 4 A schematic diagram of the test results. DETAILED DESCRIPTION

[0049] The above and other advantages of the present invention will become more apparent through the following detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings.

[0050] As shown in the figure, a semi-supervised wood defect detection method based on a generative adversarial network is described. The method effectively discovers effective information in easily accessible unlabeled wood defect images by using an edge detector network algorithm and a segmentation network algorithm in a semi-supervised learning framework of a generative adversarial network GAN.

[0051] The semi-supervised framework includes a generator G(.) and a discriminator D(.); the DeepLabV3+ semantic segmentation network is used as the generator to generate pixel-level segmentation results of wood defect images; a fully convolutional network is used as the discriminator, which inputs image segmentation mask pairs and outputs corresponding confidence maps to judge the authenticity of the segmentation masks; the edge detector network and the segmentation network are trained adversarially in a game-like manner to automatically learn the data distribution of the sample set; when the discriminator reaches the required discrimination ability after stage training, the generator generates an image label pair consisting of the segmentation result of the unlabeled image and the corresponding original image, which is then input into the discriminator, and the label part verified to be true by the discriminator is used as a supervisory signal and added to the dataset for training the segmentation network.

[0052] The method comprises the following steps:

[0053] Step S1, obtaining a wood defect dataset, and dividing the defect dataset into a labeled dataset and an unlabeled dataset;

[0054] Step S2, model creation, i.e. creating a segmentation network and a discriminator network to form a semi-supervised adversarial learning model and initializing model parameters;

[0055] Step S3, image preprocessing;

[0056] Step S4, training the generator;

[0057] Step S5, training the discriminator;

[0058] Step S6: adversarial training;

[0059] Step S7: semi-supervised learning;

[0060] Step S8: deploy the trained segmentation network to perform defect detection on the wood image.

[0061] The step S1 specifically includes: acquiring wood image data through a linear array camera, semantically annotating 1 / M defect images using annotation software, and not annotating the remaining (M-1) / M images, and dividing the defect data set into an annotated data set and an unannotated data set. The annotated data set includes wood defect images and their corresponding manual semantic annotations, and the unannotated data set only includes defect images.

[0062] Step S3 specifically includes: eliminating noise and interference in the wood image through a filtering algorithm, and performing data enhancement by random cropping, random inversion, etc.

[0063] Step S4 is as follows: Input the labeled data set into the DeepLabV3+ network for T iterations of training, and minimize the multi-classification cross entropy loss function. Guide the segmentation network of the generator to generate prediction results similar to the distribution of manually annotated data, so that the segmentation network can obtain the required prediction ability;

[0064] The generator cross entropy loss function formula is as follows:

[0065]

[0066] in represents the multi-classification cross entropy loss, X and Y represent wood images and their corresponding manual annotations, respectively;

[0067] The generator uses the obtained prediction ability to predict the unlabeled dataset to obtain the prediction results.

[0068] Step S5 is as follows: the manual labels and the prediction results of the generator are spliced ​​with the corresponding defect images in the channel direction to obtain image mask pairs, which are respectively sent to the discriminator for T iterations of training to minimize the discriminant loss function composed of the binary cross entropy loss. The discriminator is encouraged to distinguish between the true and false input segmentation masks. When the discriminant network input comes from the true label, the discriminator is encouraged to output a prediction confidence of 1; when the input comes from the generator prediction result, the discriminator is encouraged to output a confidence of 0;

[0069] The discriminator loss function formula is as follows:

[0070]

[0071] in represents the binary cross entropy loss, Represents the concatenation of the wood image and the corresponding segmentation mask in the channel direction.

[0072] Step S6 improves the prediction ability of the generator and the discriminator through the adversarial learning method of mutual game, specifically: by minimizing the adversarial loss function Encourage the generator to output more accurate prediction results to deceive the discriminator; at the same time, force the discriminator to further learn to distinguish the true and false input segmentation masks, thereby improving its own discrimination ability and outputting more accurate confidence maps;

[0073] The formula for the adversarial loss function is:

[0074]

[0075] In step S7, after N iterations of training, both the generator and the discriminator can output reliable prediction results. The introduction of the semi-supervised learning process is as follows: the image mask consisting of the unlabeled image and the segmentation mask obtained by the generator is sent to the discriminator for prediction to obtain a confidence map representing the truth or falsehood of the input mask. The generated confidence map is binarized by a preset confidence threshold γ. If the confidence is greater than the threshold, it is set to 1, otherwise it is set to 0, so as to retain the segmentation results corresponding to the mask area with a confidence greater than the threshold, that is, the area is judged to be true; finally, the unlabeled image segmentation result judged to be true is used as a pseudo-label, added to the training data set, and the semi-supervised loss is used. Train the generator;

[0076] The formula for the semi-supervised loss function is:

[0077]

[0078] in is the pseudo-label obtained by applying the softmax function to the generator output S(X) in the category dimension, and F(.) is the decision function, defined as:

[0079] In the detection method, the discriminator network is only effective in the model training stage, and the detection task only needs to use the segmentation network.

[0080] In step S8, the trained segmentation network is deployed on a computer to perform defect detection on the wood image.

[0081] Example:

[0082] This example includes the following procedures:

[0083] Obtain a wood defect dataset; obtain 2,000 wood image data using a linear array camera, use annotation software to semantically annotate 100 defect images, and leave the remaining 1,900 images unannotated, and divide the defect dataset into an annotated dataset and an unannotated dataset. The annotated dataset contains wood defect images and their corresponding manual semantic annotations, and the unannotated dataset only contains defect images;

[0084] Model creation: Create a segmentation network and a discriminator network to form a semi-supervised adversarial learning model and initialize model parameters. The designed semi-supervised wood defect detection framework based on generative adversarial network is shown in the attached figure. Figure 2 As shown in the attached figure, the discriminator network Figure 3 As shown;

[0085] Image preprocessing: eliminate noise and interference in wood images through filtering algorithms, and use random cropping, random inversion and other methods to enhance data;

[0086] Training generator: input the labeled dataset into the DeepLabV3+ network for 20,000 iterations of training, by minimizing the multi-classification cross entropy loss function Guide the segmentation network to generate prediction results similar to the distribution of manually annotated data, so that the segmentation network can obtain a certain prediction ability. The generator cross entropy loss function formula is as follows:

[0087]

[0088] in represents the multi-classification cross entropy loss, X and Y represent wood images and their corresponding manual annotations, respectively;

[0089] Train the discriminator; splice the manual labels and the prediction results of the generator with their corresponding defect images in the channel direction to obtain image mask pairs, which are sent to the discriminator for 20,000 trainings respectively to minimize the discriminant loss function composed of binary cross entropy loss The discriminator is encouraged to distinguish between the true and false input segmentation masks. When the discriminant network input comes from the true label, the discriminator is encouraged to output a prediction confidence of 1; when the input comes from the generator prediction result, the discriminator is encouraged to output a confidence of 0. The discriminator loss function formula is as follows:

[0090]

[0091] in represents the binary cross entropy loss, It means that the wood image and the corresponding segmentation mask are concatenated in the channel direction;

[0092] Adversarial training: Based on the original loss function, an adversarial learning mechanism is added to minimize the adversarial loss function. The generator is encouraged to output more accurate prediction results to deceive the discriminator. At the same time, the discriminator is forced to further learn to distinguish the true and false input segmentation masks, thereby improving its own discrimination ability and outputting more accurate confidence maps. Through this mutual game of adversarial learning, the two networks automatically extract effective information from the data set, and the prediction capabilities of both the generator and the discriminator are improved. The adversarial loss function formula is as follows:

[0093]

[0094] Semi-supervised learning; after 5000 iterations of training, both the generator and the discriminator can output relatively reliable prediction results, and the semi-supervised learning process is introduced. The image mask pair consisting of the unlabeled image and the segmentation mask obtained by the generator is sent to the discriminator for prediction to obtain a confidence map representing the true or false of the input mask. The generated confidence map is binarized by a preset confidence threshold. If the confidence is greater than 0.3, it is set to 1, otherwise it is set to 0, so as to retain the segmentation results corresponding to the mask area with a confidence greater than the threshold, that is, the area is judged to be true. Finally, the mask part judged as false is discarded and not used for training. The unlabeled image segmentation result judged as true is used as a pseudo-label and added to the training data set, and a semi-supervised loss composed of a multi-category cross entropy loss is passed. Train the generator. The semi-supervised loss function formula is as follows:

[0095]

[0096] in is the pseudo-label obtained by applying the softmax function to the generator output S(X) in the category dimension, and F(.) is the decision function, defined as:

[0097] Defect detection: The discriminator network is only effective in the model training stage, and the detection task only needs to use the segmentation network. Deploy the trained segmentation network to the computer and perform defect detection on the wood image. The detection result diagram is as follows: Figure 4 shown.

Claims

1. A semi-supervised wood defect detection method based on generative adversarial network, characterized in that: The method adopts a discriminator network algorithm and a segmentation network algorithm of a semi-supervised learning framework of a generative adversarial network (GAN) to effectively explore effective information in easily accessible unlabeled wood defect images, wherein the semi-supervised framework includes a generator G(.) and a discriminator D(.); a DeepLabV3+ semantic segmentation network is used as a generator to generate pixel-level segmentation results of wood defect images; a fully convolutional network is used as a discriminator, an image segmentation mask pair is input and a corresponding confidence map is output to judge the authenticity of the segmentation mask; the discriminator network and the segmentation network are adversarially trained in a game-like manner to automatically learn the data distribution of the sample set; when the discriminator reaches the required discrimination ability after stage training, the generator generates an image label pair consisting of a segmentation result of an unlabeled image and a corresponding original image, which is then input into the discriminator, and the label part verified as true by the discriminator is used as a supervisory signal and added to a data set for training the segmentation network.

2. The semi-supervised wood defect detection method based on generative adversarial network according to claim 1, characterized in that: The method comprises the following steps: Step S1, obtaining a wood defect dataset, and dividing the defect dataset into a labeled dataset and an unlabeled dataset; Step S2, model creation, i.e. creating a segmentation network and a discriminator network to form a semi-supervised adversarial learning model and initializing model parameters; Step S3, image preprocessing; Step S4, training the generator; Step S5, training the discriminator; Step S6: adversarial training; Step S7, semi-supervised learning; Step S8: deploy the trained segmentation network to perform defect detection on the wood image.

3. The semi-supervised wood defect detection method based on generative adversarial network according to claim 2, characterized in that: The step S1 specifically includes: acquiring wood image data through a linear array camera, semantically annotating 1 / M defect images using annotation software, and not annotating the remaining (M-1) / M images, and dividing the defect data set into an annotated data set and an unannotated data set. The annotated data set includes wood defect images and their corresponding manual semantic annotations, and the unannotated data set only includes defect images.

4. The semi-supervised wood defect detection method based on generative adversarial network according to claim 2, characterized in that: Step S3 specifically includes: eliminating noise and interference in the wood image through a filtering algorithm, and performing data enhancement by random cropping, random inversion, etc.

5. The semi-supervised wood defect detection method based on generative adversarial network according to claim 2, characterized in that: Step S4 is as follows: Input the labeled data set into the DeepLabV3+ network for T iterations of training, and minimize the multi-classification cross entropy loss function. Guide the segmentation network of the generator to generate prediction results similar to the distribution of manually annotated data, so that the segmentation network can obtain the required prediction ability; The generator cross entropy loss function formula is as follows: Where lmce(.) represents the multi-classification cross entropy loss, X and Y represent wood images and their corresponding manual annotations, respectively; The generator uses the obtained prediction ability to predict the unlabeled dataset to obtain the prediction results.

6. The semi-supervised wood defect detection method based on generative adversarial network according to claim 5, characterized in that: Step S5 is as follows: the manual labels and the prediction results of the generator are spliced ​​with the corresponding defect images in the channel direction to obtain image mask pairs, which are respectively sent to the discriminator for T iterations of training to minimize the discriminant loss function composed of the binary cross entropy loss. The discriminator is encouraged to distinguish between the true and false input segmentation masks. When the discriminant network input comes from the true label, the discriminator is encouraged to output a prediction confidence of 1; when the input comes from the generator prediction result, the discriminator is encouraged to output a confidence of 0; The discriminator loss function formula is as follows: Among them l bce (.) represents the binary cross entropy loss, Represents the concatenation of the wood image and the corresponding segmentation mask in the channel direction.

7. The semi-supervised wood defect detection method based on generative adversarial network according to claim 6 is characterized in that: Step S6 improves the prediction ability of the generator and the discriminator through the adversarial learning method of mutual game, specifically: by minimizing the adversarial loss function Encourage the generator to output more accurate prediction results to deceive the discriminator; at the same time, force the discriminator to further learn to distinguish the true and false input segmentation masks, thereby improving its own discrimination ability and outputting more accurate confidence maps; The formula for the adversarial loss function is:

8. The semi-supervised wood defect detection method based on generative adversarial network according to claim 7, characterized in that: In step S7, after N iterations of training, both the generator and the discriminator can output reliable prediction results. The introduction of the semi-supervised learning process is as follows: the image mask consisting of the unlabeled image and the segmentation mask obtained by the generator is sent to the discriminator for prediction to obtain a confidence map representing the truth or falsehood of the input mask. The generated confidence map is binarized by a preset confidence threshold γ. If the confidence is greater than the threshold, it is set to 1, otherwise it is set to 0, so as to retain the segmentation results corresponding to the mask area with a confidence greater than the threshold, that is, the area is judged to be true; Finally, the unlabeled image segmentation results that are judged as true are added to the training dataset as pseudo labels, and the semi-supervised loss is used to calculate the true segmentation results. Train the generator; The formula for the semi-supervised loss function is: in is the pseudo-label obtained by applying the softmax function to the generator output S(X) in the category dimension, and F(.) is the decision function, defined as:

9. The semi-supervised wood defect detection method based on generative adversarial network according to claim 7, characterized in that: In the detection method, the discriminator network is only effective in the model training stage, and the detection task only needs to use the segmentation network.

10. The semi-supervised wood defect detection method based on generative adversarial network according to claim 2, characterized in that: In step S8, the trained segmentation network is deployed on a computer to perform defect detection on the wood image.

Images