A semi-supervised small sample polaroid flaw detection method and device and storage medium

Abstract

The present application relates to a kind of semi-supervised small sample polaroid flaw detection method, device and storage medium, method steps include: obtaining polaroid dataset, the dataset includes labeled data and unlabeled data;Initialize polaroid flaw detection network, use labeled data as initial training set, train flaw detection network until convergence;Unlabeled data is input into the converged flaw detection network, according to the confidence threshold of classification branch output and expansion training set;Reinitialize polaroid flaw detection network, use the training set after expansion to train flaw detection network until convergence;According to the iteration number set, repeat the above two steps, obtain the final polaroid flaw detection network;The image to be detected is input into flaw detection network, and whether there is flaw according to the result of classification branch is judged.Compared with prior art, the present application can more effectively extract image features, and the detection network has better generalization ability, improves the accuracy of polaroid classification.

Description

Technical Field

[0001] This invention relates to the field of polarizer defect detection technology, and in particular to a semi-supervised method and apparatus for detecting defects in small samples of polarizers. Background Technology

[0002] Liquid crystal displays (LCDs) rely on polarized light for imaging, thus requiring polarizers and liquid crystal glass to form the liquid crystal panel. Although the manufacturing process for polarizers is simple and technologically mature, defects can still occur during production and transportation. Surface defects such as creases, unevenness, and streaks directly affect the quality of the polarizer, thereby impacting image quality. Therefore, polarizer defect detection is the final and crucial step before product shipment.

[0003] Initially, defect detection of polarizers was done manually, with specialized quality inspectors responsible for the quality checks. While defects were rare, the inspectors had to meticulously examine each polarizer, making it a tedious and repetitive task with low efficiency and a high rate of missed defects.

[0004] With the continuous improvement of computer performance, defect detection in polarizers has gradually been treated as a computer vision task. Especially in recent years, deep learning has demonstrated its powerful capabilities in computer vision problems, and some methods have applied deep learning to defect detection in polarizers. Deep learning methods require a large amount of labeled data to train network parameters. Some semi-supervised methods use a portion of labeled and unlabeled data to train the model, setting a certain confidence threshold to add pseudo-labels to the unlabeled data for training. However, existing methods require setting a large confidence threshold to maximize the accuracy of the pseudo-labels on the unlabeled data added to training. Therefore, the amount of unlabeled data that can be added to training is limited, failing to fully utilize unlabeled data and hindering the network from learning better feature representations. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for detecting defects in small sample polarizers.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] As a first aspect of the present invention, a method for detecting defects in small-sample polarizers is provided, the method comprising the following steps:

[0008] Step 1: Obtain the polarizer dataset, which includes labeled and unlabeled data;

[0009] Step 2: Initialize the polarizer defect detection network. Use labeled data as the initial training set and train the defect detection network until convergence. The network includes a feature extraction part, a classification branch for outputting the probability of polarizer images belonging to positive and negative examples, and an auxiliary branch for restoring the data to the original input size image.

[0010] Step 3: Input the unlabeled data into the converged defect detection network. If the output of the classification branch exceeds the confidence threshold, use the category corresponding to the result as the pseudo-label of the unlabeled sample and add the sample to the training set. If it does not exceed the confidence threshold, add the sample directly to the training set.

[0011] Step 4: Reinitialize the polarizer defect detection network and train the defect detection network using the expanded training set until convergence;

[0012] Step 5: Repeat steps 3 to 4 according to the set number of iterations. The converged network is the final polarizer defect detection network. Input the polarizer image to be detected into the polarizer defect detection network and determine whether there are defects based on the results of the classification branches.

[0013] Furthermore, the defect detection network specifically comprises:

[0014] The backbone network selected for the feature extraction part is ResNet, WideResNet, DenseNet, or EfficientNet.

[0015] The network structure of the classification branch is a fully connected layer. The number of nodes in the last layer is equal to the number of categories in the classification task. The classification branch takes the feature vector obtained from the feature extraction part, passes it through the fully connected layer, and then uses the Softmax function to obtain the probability that the input polarizer image belongs to positive or negative examples.

[0016] The auxiliary branch uses transposed convolution to restore the feature vectors obtained from the feature extraction part to image data of the original input size.

[0017] Furthermore, the input to the polarizer defect detection network consists of two images: one image is the polarizer image to be detected, and the other image is the average image of all defect-free samples obtained by summing and averaging the pixel values ​​of all defect-free samples in the labeled data; both images are RGB images, and the two images are concatenated before being input into the polarizer defect detection network.

[0018] Furthermore, the specific process of step 2 includes:

[0019] Input labeled image data into the polarizer defect detection network;

[0020] The classification branch obtains the probability that the image belongs to a positive or negative example, and calculates the classification branch loss function Loss1;

[0021] The auxiliary branch outputs an image of the original dimension after transpose convolution, and the auxiliary branch loss function Loss2 is calculated.

[0022] The loss function of the network is obtained by fitting the classification branch loss function Loss1 and the auxiliary branch loss function Loss2. The network parameters are updated by backpropagation until the loss function converges to a certain value or the training reaches a set number of times.

[0023] Furthermore, the specific process of step 3 includes:

[0024] Input the unlabeled data into the polarizer defect detection network and perform forward calculation of the classification branch results;

[0025] For each unlabeled sample, the network's classification branch outputs the probability that the sample belongs to a positive or negative example. The larger probability is compared with the set confidence threshold. If it is greater than or equal to the confidence threshold, the class corresponding to the probability is used as the pseudo-label of the sample, and the sample is added to the training set. If it is less than the confidence threshold, the unlabeled sample is directly added to the training set.

[0026] Furthermore, the specific process of step 4 includes:

[0027] Reinitialize the polarizer defect detection network and train it using the expanded training set;

[0028] During training, if the sample has a label or a pseudo-label, the classification branch and auxiliary branch are computed forward, and the classification branch loss function Loss1 and the auxiliary branch loss function Loss2 are fitted as the network loss function. The network parameters are updated through backpropagation.

[0029] If the sample is newly added unlabeled data and does not have a pseudo-label, then only the auxiliary branch is computed forward. The loss function Loss2 of the auxiliary branch is processed and used as the loss function. The parameters of the network are then updated through backpropagation.

[0030] Furthermore, the classification branch loss function Loss1 is obtained by calculating the cross-entropy between the image label and the classification branch result:

[0031] Loss1=-((1-T)log(Y1)+(T)log(Y2))

[0032] In the formula, T is the label of the image, Y1 represents the probability that the polarizer image is a negative example, and Y2 represents the probability that the polarizer image is a positive example.

[0033] The auxiliary branch loss function Loss2 uses the auxiliary branch to output the result X. output , to input image X input The absolute value of the difference between the two images is used as the label, and the L2 norm of the difference between the two images is calculated.

[0034]

[0035] Furthermore, in step 3, the defect detection network trained in step 2 is used in the first iteration to input all unlabeled samples into the network and determine whether they meet the confidence threshold requirement; in each subsequent iteration, the defect detection network that converged in the previous iteration is used to re-input all unlabeled samples into the network and determine whether they meet the confidence threshold requirement.

[0036] As a second aspect of the present invention, an electronic device is provided, comprising:

[0037] One or more processors;

[0038] Memory, used to store one or more programs;

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement the defect detection method as described above.

[0040] As a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the defect detection method described above.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1) This invention uses a dual-branch network structure. When the confidence of an unlabeled sample does not exceed the threshold, its pseudo-label reliability is not high, but it can still be added to the training set. The network parameters are optimized through the auxiliary branch, making full use of all unlabeled samples, so that the network has better generalization ability and improves the accuracy of the network as much as possible.

[0043] 2) This invention uses the constraint of auxiliary branches to make the feature extraction part pay more attention to the abnormal parts in the image to be detected, learn better feature representations, improve the learning efficiency of the network and the accuracy of the classification branch. Attached Figure Description

[0044] Figure 1 This is a flowchart of the polarizer defect detection method according to an embodiment of the present invention;

[0045] Figure 2 A schematic diagram illustrating the optimization of network parameters for labeled data and unlabeled data with pseudo-labels;

[0046] Figure 3 A schematic diagram illustrating the optimization of network parameters for input data without labels or pseudo-labels. Detailed Implementation

[0047] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0048] Example 1

[0049] This embodiment discloses a semi-supervised method for detecting defects in polarizers with small sample sizes. The task of this example is to train a polarizer defect detection classification network using a small amount of labeled data and a large amount of unlabeled data, and then apply the trained network to a polarizer image to be detected to determine whether the image contains defects.

[0050] Figure 1 This is a flowchart of the polarizer defect detection method according to an embodiment of the present invention.

[0051] The polarizer dataset was obtained by capturing images of polarizers at fixed stations on the polarizer production line. The dataset includes images of flawless polarizers (negative examples) and images of polarizers with defects (positive examples). Labels T were manually assigned to some data points. A label T = 1 indicates a positive example, and a label T = 0 indicates a negative example. The labeled data should be balanced between positive and negative examples. Finally, the polarizer dataset used for training was obtained, including a small amount of labeled data X. label and a large amount of unlabeled data X unlabel .

[0052] After obtaining the polarizer dataset, use the labeled data X label As the initial training set for the polarizer defect detection network, the network comprises a feature extraction section, a classification branch, and an auxiliary branch. The feature extraction section uses commonly used backbone networks such as ResNet, WideResNet, DenseNet, and EfficientNet. The classification branch uses a fully connected layer where the number of nodes in the last layer corresponds to the number of classes in the classification task (2, representing no defects and defects). The classification branch passes the feature vectors obtained from the feature extraction section through a fully connected layer and then uses the Softmax function to obtain the probability that the input polarizer image belongs to a positive or negative example. The auxiliary branch uses transposed convolutions to restore the feature vectors obtained from the feature extraction section to the original input image size. The input to the polarizer defect detection network consists of two images, one of which is the polarizer image X to be detected. inputThe other image is the average flawless image obtained by summing and averaging the pixel values ​​of all flawless samples in the labeled data. Both images are RGB images (H×W×3). The data (H×W×6) obtained by concatenating the two images in the third channel is then input into the polarizer defect detection network.

[0053] After inputting the (H×W×6) data into the polarizer defect detection network, the feature extraction part obtains the image feature vector D. 1×n eigenvector D 1×n After passing through the fully connected layer and using the Softmax function in the classification branch, the probability y = [Y1, Y2] of the image belonging to a positive or negative example is obtained, where Y1 represents the probability that the polarizer image belongs to a negative example, and Y2 represents the probability that the polarizer image belongs to a positive example. The cross-entropy is calculated using the image label and the result of the classification branch as the classification branch loss function, specifically expressed as:

[0054] Loss1=-((1-T)log(Y1)+(T)log(Y2))

[0055] The auxiliary branch, after being transposed and convolved, yields an output X of dimension (H×W×3). output Using the absolute value of the difference between the used image and the flawless average image (H×W×3) as the label, the L2 norm of the difference between the two is calculated as the auxiliary branch loss function, specifically expressed as:

[0056]

[0057] Using Loss = λ1Loss1 + λ2Loss2 as the loss function, the network parameters are updated through backpropagation. A schematic diagram of this process is shown below. Figure 2 As shown, the training process in step 2 ends when the loss function converges to a certain value or the training reaches a set number of iterations.

[0058] After training in step 2, the unlabeled data is input into the polarizer defect detection network. Here, only the result of the classification branch is computed forward; the auxiliary branch is not required. For each unlabeled sample, the network's classification branch outputs the probability y = [Y1, Y2] that the sample belongs to a positive or negative example. A larger confidence level is compared with a set confidence threshold. If it is greater than or equal to the confidence threshold, the category corresponding to this probability is used as the pseudo-label for the sample, and the sample is added to the training set. If it is less than the confidence threshold, the unlabeled sample is directly added to the training set, and such samples do not have pseudo-labels. Specifically, in this embodiment, the confidence threshold is set to 0.7. When an unlabeled sample is input into the network classification branch, y1 = [0.95, 0.05] is obtained. The larger probability value of 0.95 is taken. Since 0.95 ≥ 0.7, the category corresponding to 0.95, i.e., Y = 0, is taken as the pseudo-label of the sample, and this sample is added to the training set. When an unlabeled sample is input into the network classification branch, y2 = [0.65, 0.35] is obtained. Since 0.65 < 0.7, it does not meet the threshold requirement, so it is impossible to assign a pseudo-label to this sample. This sample should be directly added to the training set.

[0059] Step 4: Reinitialize the polarizer defect detection network and train it using the expanded training set. This training set now includes labeled data and all added unlabeled data. Since some unlabeled data have pseudo-labels and some do not, during training, if a sample has a label or pseudo-label, the classification branch and auxiliary branch are computed forward, using Loss = λ1Loss1 + λ2Loss2 as the loss function. The network parameters are updated through backpropagation. This process is as follows: Figure 2 As shown; if the sample is newly added unlabeled data and does not have a pseudo-label, then only the auxiliary branch is computed forward, using Loss = λ²Loss² as the loss function, and the network parameters are updated through backpropagation. This process is as follows: Figure 3 As shown.

[0060] Repeat steps 3-4. Specifically, in step 3, during the first iteration, the defect detection network trained in step 2 is used to input all unlabeled samples into the network to determine whether they meet the confidence threshold requirement. In each subsequent iteration, step 3 needs to use the network that converged in the previous iteration to re-input all unlabeled samples into the network and determine whether they meet the confidence threshold requirement. If the confidence threshold is greater than or equal to the confidence threshold, the category corresponding to this higher confidence level is used as the pseudo-label for the sample, and the sample is added to the training set. If the confidence threshold is less than the confidence threshold, the unlabeled sample is directly added to the training set. After iterating to the set number of times, the converged network is the final polarizer defect detection network. The polarizer image to be detected is input into the network to determine whether defects exist based on the results of the classification branch.

[0061] Example 2

[0062] As a second aspect of the present invention, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors 11 implement the defect detection method described above. In addition to the processors, memory, and interfaces described above, any data processing device in the embodiments of the present invention may also include other hardware depending on the actual function of the data processing device, which will not be elaborated further.

[0063] Example 3

[0064] As a third aspect of the present invention, this application also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the defect detection method described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0065] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for detecting defects in small-sample polarizers, characterized in that, The method includes: Step 1: Obtain the polarizer dataset, which includes labeled samples and unlabeled samples; Step 2: Initialize the polarizer defect detection network. Use labeled samples as the initial training set and train the network until convergence. The polarizer defect detection network includes a feature extraction part, a classification branch for outputting the probability of the polarizer image belonging to positive or negative examples, and an auxiliary branch for restoring the data to the original input size image. The input to the polarizer defect detection network consists of two images: one image is the polarizer image to be detected, and the other image is the average image of all defect-free samples obtained by summing and averaging the pixel values ​​of all defect-free samples in the labeled samples. Both images are RGB images. The two images are concatenated and then input into the polarizer defect detection network. Step 3: Input the unlabeled sample into the converged defect detection network. If the output of the classification branch exceeds the confidence threshold, use the category corresponding to the result as the pseudo-label of the unlabeled sample and add the unlabeled sample with the pseudo-label to the training set. If it does not exceed the confidence threshold, directly add the unlabeled sample to the training set. Step 4: Reinitialize the polarizer defect detection network, and train the defect detection network using the expanded training set until convergence. The specific process includes: Reinitialize the polarizer defect detection network and train it using the expanded training set; During training, if the samples in the training set have labels or pseudo-labels, the classification branch and auxiliary branch are computed forward, and the classification branch loss function is fitted. With auxiliary branch loss function As a network loss function, the network parameters are updated through backpropagation; If the samples in the training set are newly added unlabeled samples and do not have pseudo-labels, then only the auxiliary branch is computed forward, and the loss function of the auxiliary branch is calculated. After processing, it is used as the loss function to update the network parameters through backpropagation; The classification branch loss function It is obtained by calculating the cross-entropy of the image's label and the results of the classification branch: In the formula, For image labels, This represents the probability that the polarizer image is a negative example. This represents the probability that the polarizer image is a positive example; The auxiliary branch loss function Output results using auxiliary branches , to input image And flawless average image The absolute value of the difference is used as the label, and the L2 norm of the difference is calculated. Step 5: Repeat steps 3 to 4 according to the set number of iterations. The converged network is the final polarizer defect detection network. Input the polarizer image to be detected into the polarizer defect detection network and determine whether there are defects based on the results of the classification branches.

2. The method for detecting defects in small-sample polarizers according to claim 1, characterized in that, The defect detection network is specifically as follows: The backbone network selected for the feature extraction part is ResNet, WideResNet, DenseNet, or EfficientNet. The classification branch uses a fully connected layer in its network structure. The number of nodes in the last layer is equal to the number of categories in the classification task. The classification branch passes the feature vectors obtained from the feature extraction part through the fully connected layer and then uses the Softmax function to obtain the probability that the input polarizer image belongs to positive or negative examples. The auxiliary branch uses transposed convolution to restore the feature vectors obtained from the feature extraction part to image data of the original input size.

3. The method for detecting defects in small-sample polarizers according to claim 1, characterized in that, The specific process of step 2 includes: Input the image data of the labeled samples into the polarizer defect detection network; The classification branch obtains the probability that the image data belongs to a positive or negative example, and calculates the classification branch loss function. ; The auxiliary branch outputs an image of the original dimension after transpose convolution, and the loss function of the auxiliary branch is calculated. ; Fitting the classification branch loss function With auxiliary branch loss function The loss function of the network is obtained, and the network parameters are updated through backpropagation until the loss function converges to a certain value or the training reaches a set number of times.

4. The method for detecting defects in small-sample polarizers according to claim 1, characterized in that, The specific process of step 3 includes: Input the unlabeled samples into the polarizer defect detection network and perform forward calculation of the classification branch results; For each unlabeled sample, the network's classification branch outputs the probability that the unlabeled sample belongs to a positive or negative example. The larger probability is compared with the set confidence threshold. If it is greater than or equal to the confidence threshold, the class corresponding to the probability is used as the pseudo-label of the unlabeled sample, and the unlabeled sample with the pseudo-label is added to the training set. If it is less than the confidence threshold, the unlabeled sample is directly added to the training set.

5. The method for detecting defects in small-sample polarizers according to claim 1, characterized in that, In step 3, during the first iteration, the defect detection network trained in step 2 is used to input all unlabeled samples into the network to determine whether they meet the confidence threshold requirement. In each subsequent iteration, the defect detection network that converged in the previous iteration is used to re-input all unlabeled samples into the network to determine whether they meet the confidence threshold requirement.

6. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the defect detection method as described in any one of claims 1-5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the defect detection method as described in any one of claims 1-5.

Images