A real-time detection method for polarizer appearance defects

Abstract

The present invention relates to a polarizer appearance defect detection technology, specifically a polarizer appearance defect real-time detection method. The present invention solves the problem of poor real-time detection of existing polarizer appearance defect detection methods. A polarizer appearance defect real-time detection method, the method is implemented by the following steps: Step 1: first use a camera to take a polarizer picture, and then perform data expansion on the taken polarizer picture, thereby obtaining a polarizer data set; Step 2: Improve the backbone network and CSP BLOCK module of the YOLOv4‑Tiny network model; Step 3: First train the YOLOv4‑Tiny‑C network model, and then test the detection accuracy of the YOLOv4‑Tiny‑C network model; Step 4: Prune the YOLOv4‑Tiny‑C network model; Step 5: Divide the original large image of the polarizer in the polarizer data set into small polarizer images, and then input the small polarizer images into the Pruning‑YOLOv4‑Tiny‑C network model for defect detection. The present invention is suitable for polarizer appearance defect detection.

Description

Technical Field

[0001] The invention relates to a polarizer appearance defect detection technology, and in particular to a polarizer appearance defect real-time detection method. Background Art

[0002] Polarizer is a common polarization optical element, which is widely used in liquid crystal displays and various imaging devices and instruments. During the production and transportation process, polarizers will inevitably form appearance defects such as dirt, scratches, bubbles, etc., which will affect the performance and quality of the polarizer. Therefore, in order to ensure the performance and quality of the polarizer, it is necessary to perform polarizer appearance defect detection.

[0003] With the development of artificial intelligence technology, the polarizer appearance defect detection method based on deep learning has become the mainstream method for polarizer appearance defect detection due to its advantage of high detection accuracy. However, due to the limitation of its own principle, this method still has the problem of slow detection speed, which leads to poor real-time detection. Based on this, it is necessary to invent a real-time detection method for polarizer appearance defects to solve the problem of poor real-time detection of existing polarizer appearance defect detection methods. Summary of the invention

[0004] In order to solve the problem that the existing polarizer appearance defect detection method has poor real-time detection performance, the present invention provides a polarizer appearance defect real-time detection method.

[0005] The present invention is achieved by adopting the following technical solutions:

[0006] A method for real-time detection of appearance defects of polarizers is implemented by the following steps:

[0007] Step 1: first use a camera to take a polarizer picture, and then perform data expansion on the taken polarizer picture, thereby obtaining a polarizer data set; the polarizer data set includes a training set, a validation set, a test set, and an original large image of the polarizer;

[0008] Step 2: Improve the backbone network and CSP BLOCK module of the YOLOv4-Tiny network model to obtain the YOLOv4-Tiny-C network model;

[0009] Step 3: First use the training set in the polarizer dataset to train the YOLOv4-Tiny-C network model, then use the test set in the polarizer dataset to test the detection accuracy of the YOLOv4-Tiny-C network model, and output the prediction graph;

[0010] Step 4: Use model pruning technology to prune the YOLOv4-Tiny-C network model to obtain the Pruning-YOLOv4-Tiny-C network model;

[0011] Step 5: First write an image segmentation script, and then use the image segmentation script to segment the original large polarizer image in the polarizer dataset into small polarizer images, and then input the small polarizer images into the Pruning-YOLOv4-Tiny-C network model for defect detection, and then recombine the small polarizer images with detection information into the original large polarizer image for output and storage.

[0012] In the step 1, the polarizer images taken include 500 defect-free images and 1,100 images with defect features, and the polarizer data set includes a training set consisting of 3,000 images with a resolution of 200×200 and defect features, a verification set consisting of 600 images with a resolution of 200×200 and defect features, a test set consisting of 200 images with a resolution of 200×200 and defect features, and 32 original large images of polarizers with a resolution of 4096×3796.

[0013] In the step 1, the defect characteristics include dirt, scratches, and bubbles.

[0014] In the step 2, the improvement steps are as follows:

[0015] First, add a layer of Conv3×3 before the first convolutional layer of the YOLOv4-Tiny network model to perform convolution operations on the input image to extract richer features and enhance the nonlinear expression ability of the network;

[0016] Secondly, after the first CBL downsampling, the Conv1×1+Conv3×3 convolution layer combination is added to further improve the network depth and enhance the feature extraction capability; the Conv1×1+Conv3×3 convolution layer combination mode refers to the DenseBlock module for feature fusion, so that the number of channels of the output feature map is increased to 64, which is consistent with the size of the feature map of the 6th layer;

[0017] Then, a layer of Conv3×3 is added after the Cnov1×1 operation on the right branch of the original CSP BLOCK structure to increase the effective receptive field and enrich the context information;

[0018] Finally, the k-means++ clustering algorithm was used to regenerate 6 groups of anchor box parameter values ​​[(28,22), (33,57), (75,40), (40,105), (100,87), (141,153)] according to the cluster centers and data frame distribution. These groups were used for defect detection algorithm training to prevent a large number of missed detections and false detections.

[0019] In the step 3, before training the YOLOv4-Tiny-C network model, the hyperparameters in the .cfg file are set as follows: ① the maximum number of iterations is set to 50,000; ② the learning rate is set using a distribution strategy: the initial learning rate is set to 0.0261, the first 30,000 learning rate is maintained at 0.0261, the learning rate is decayed by 0.1 times from 30,000 to 42,000 times, and the learning rate is decayed by 0.1 times from 42,000 to 50,000 times; ③ the memory factor is set to 0.9, the weight decay term is set to 0.005, and the batch training scale is set to 128.

[0020] In step 4, the pruning operation steps are as follows:

[0021] First, sparse training is performed based on the YOLOv4-Tiny-C network model, and the contribution of different channels in the YOLOv4-Tiny-C network model to the prediction results is calculated as a scale factor;

[0022] After the sparse training is completed, all channels in the YOLOv4-Tiny-C network model are sorted by the scale factor, and the channels with smaller scale factors are proportionally deleted to generate the pruned YOLOv4-Tiny-C network model;

[0023] Finally, the pruned YOLOv4-Tiny-C network model is fine-tuned to obtain the Pruning-YOLOv4-Tiny-C network model.

[0024] In step 4, when performing sparse training, in the YOLOv4-Tiny-C network model structure, only the input convolution layer of the YOLO layer does not have a bn layer, and all other convolution layers have a bn layer. The calculation formula is:

[0025]

[0026] Where: andσ 2 Set to the mean and variance of the same batch, γandβ are the trainable scale factor and bias respectively; γ is used directly to measure the importance of the channel, and the importance of γ is measured using L1 regression. The objective formula of sparsity training is:

[0027] L=Loss yolo +α∑ y∈τ f(γ);

[0028] Represents L1 regression, which is used to balance the two losses. The non-smooth L1 penalty term is optimized using the negative gradient method, and the value is set to 0.0001;

[0029] After the sparse training is completed, the channels to be pruned are selected by substituting the global threshold, and the global threshold is used to control all the highest pruning values. At the same time, in order to avoid excessive pruning of channels within a convolutional layer, a local threshold is introduced to completely preserve the connection structure of the network. The global threshold is set to 0.7 and the local threshold is set to 0.3.

[0030] In step 5, when performing defect detection, the image is input into the Pruning-YOLOv4-Tiny-C network model and first divided into S×S small grids, each of which can represent the local coordinates of the prior anchor box. The coordinate offset, object confidence, and category confidence score predicted by the network can be trained and calculated in these small grids; the NMS algorithm can calculate the optimal bounding box category and coordinates for all anchor boxes;

[0031] The loss function is as follows:

[0032] LOSS=L xywh +L confidence +L classes ;

[0033]

[0034]

[0035]

[0036] Where: L xywh is the sum of the center point and width and height errors between the predicted box and the real box, λcoord is the coordinate coefficient, L confidence is the target confidence error, λ obj and λ noobj are the confidence coefficients of the presence and absence of objects respectively; L classes is the target classification loss, It is represented as the matching status of the jth anchor box of the i-th grid.

[0037] In the step five, the detection information includes defect label information and defect location information.

[0038] Compared with the existing polarizer appearance defect detection method, the real-time detection method of polarizer appearance defects described in the present invention realizes polarizer appearance defect detection by adopting a new principle, thereby having the following advantages: First, the present invention improves the backbone network and CSP BLOCK module of the YOLOv4-Tiny network model on the one hand, and trains the YOLOv4-Tiny-C network model on the other hand, thereby having the advantage of high detection accuracy (experiments show that the average accuracy of detecting three appearance defects is 98.28%). Second, the present invention prunes the YOLOv4-Tiny-C network model on the one hand, and divides the original large polarizer image into small polarizer images on the other hand, thereby having the advantage of fast detection speed (experiments show that the detection speed on an ordinary CPU can reach 1 image / s, and the detection speed on a GPU can reach 3 images / s), thereby having the advantage of strong real-time detection.

[0039] The present invention effectively solves the problem of poor real-time detection in existing polarizer appearance defect detection methods, and is suitable for polarizer appearance defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the polarizer data set in the present invention.

[0041] Figure 2 It is a schematic diagram of the YOLOv4-Tiny-C network model in the present invention.

[0042] Figure 3 Schematic diagram of the CSP BLOCK module of the YOLOv4-Tiny-C network model in the present invention. DETAILED DESCRIPTION

[0043] A method for real-time detection of appearance defects of polarizers is implemented by the following steps:

[0044] Step 1: first use a camera to take a polarizer picture, and then perform data expansion on the taken polarizer picture, thereby obtaining a polarizer data set; the polarizer data set includes a training set, a validation set, a test set, and an original large image of the polarizer;

[0045] Step 2: Improve the backbone network and CSP BLOCK module of the YOLOv4-Tiny network model to obtain the YOLOv4-Tiny-C network model;

[0046] Step 3: First use the training set in the polarizer dataset to train the YOLOv4-Tiny-C network model, then use the test set in the polarizer dataset to test the detection accuracy of the YOLOv4-Tiny-C network model, and output the prediction graph;

[0047] Step 4: Use model pruning technology to prune the YOLOv4-Tiny-C network model to obtain the Pruning-YOLOv4-Tiny-C network model;

[0048] Step 5: First write an image segmentation script, and then use the image segmentation script to segment the original large polarizer image in the polarizer dataset into small polarizer images, and then input the small polarizer images into the Pruning-YOLOv4-Tiny-C network model for defect detection, and then recombine the small polarizer images with detection information into the original large polarizer image for output and storage.

[0049] In the step 1, the polarizer images taken include 500 defect-free images and 1,100 images with defect features, and the polarizer data set includes a training set consisting of 3,000 images with a resolution of 200×200 and defect features, a verification set consisting of 600 images with a resolution of 200×200 and defect features, a test set consisting of 200 images with a resolution of 200×200 and defect features, and 32 original large images of polarizers with a resolution of 4096×3796.

[0050] In the step 1, the defect characteristics include dirt, scratches, and bubbles.

[0051] In the step 2, the improvement steps are as follows:

[0052] First, add a layer of Conv3×3 before the first convolutional layer of the YOLOv4-Tiny network model to perform convolution operations on the input image to extract richer features and enhance the nonlinear expression ability of the network;

[0053] Secondly, after the first CBL downsampling, the Conv1×1+Conv3×3 convolution layer combination is added to further improve the network depth and enhance the feature extraction capability; the Conv1×1+Conv3×3 convolution layer combination mode refers to the DenseBlock module for feature fusion, so that the number of channels of the output feature map is increased to 64, which is consistent with the size of the feature map of the 6th layer;

[0054] Then, a layer of Conv3×3 is added after the Cnov1×1 operation on the right branch of the original CSP BLOCK structure to increase the effective receptive field and enrich the context information;

[0055] Finally, the k-means++ clustering algorithm was used to regenerate 6 groups of anchor box parameter values ​​[(28,22), (33,57), (75,40), (40,105), (100,87), (141,153)] according to the cluster centers and data frame distribution. These groups were used for defect detection algorithm training to prevent a large number of missed detections and false detections.

[0056] In the step 3, before training the YOLOv4-Tiny-C network model, the hyperparameters in the .cfg file are set as follows: ① the maximum number of iterations is set to 50,000; ② the learning rate is set using a distribution strategy: the initial learning rate is set to 0.0261, the first 30,000 learning rate is maintained at 0.0261, the learning rate is decayed by 0.1 times from 30,000 to 42,000 times, and the learning rate is decayed by 0.1 times from 42,000 to 50,000 times; ③ the memory factor is set to 0.9, the weight decay term is set to 0.005, and the batch training scale is set to 128.

[0057] In step 4, the pruning operation steps are as follows:

[0058] First, sparse training is performed based on the YOLOv4-Tiny-C network model, and the contribution of different channels in the YOLOv4-Tiny-C network model to the prediction results is calculated as a scale factor;

[0059] After the sparse training is completed, all channels in the YOLOv4-Tiny-C network model are sorted by the scale factor, and the channels with smaller scale factors are proportionally deleted to generate the pruned YOLOv4-Tiny-C network model;

[0060] Finally, the pruned YOLOv4-Tiny-C network model is fine-tuned to obtain the Pruning-YOLOv4-Tiny-C network model.

[0061] In step 4, when performing sparse training, in the YOLOv4-Tiny-C network model structure, only the input convolution layer of the YOLO layer does not have a bn layer, and all other convolution layers have a bn layer. The calculation formula is:

[0062]

[0063] Where: andσ 2 Set to the mean and variance of the same batch, γandβ are the trainable scale factor and bias respectively; γ is used directly to measure the importance of the channel, and the importance of γ is measured using L1 regression. The objective formula of sparsity training is:

[0064] L=Loss yolo +α∑ y∈τ f(γ);

[0065] Represents L1 regression, which is used to balance the two losses. The non-smooth L1 penalty term is optimized using the negative gradient method, and the value is set to 0.0001;

[0066] After the sparse training is completed, the channels to be pruned are selected by substituting the global threshold, and the global threshold is used to control all the highest pruning values. At the same time, in order to avoid excessive pruning of channels within a convolutional layer, a local threshold is introduced to completely preserve the connection structure of the network. The global threshold is set to 0.7 and the local threshold is set to 0.3.

[0067] In step 5, when performing defect detection, the image is input into the Pruning-YOLOv4-Tiny-C network model and first divided into S×S small grids, each of which can represent the local coordinates of the prior anchor box. The coordinate offset, object confidence, and category confidence score predicted by the network can be trained and calculated in these small grids; the NMS algorithm can calculate the optimal bounding box category and coordinates for all anchor boxes;

[0068] The loss function is as follows:

[0069] LOSS=L xywh +L confidence +L classes ;

[0070]

[0071]

[0072]

[0073] Where: L xywh is the sum of the center point and width and height errors between the predicted box and the real box, λcoord is the coordinate coefficient, L confidence is the target confidence error, λ obj and λ noobj are the confidence coefficients of the presence and absence of objects respectively; L classes is the target classification loss, It is represented as the matching status of the jth anchor box of the i-th grid.

[0074] In the step five, the detection information includes defect label information and defect location information.

[0075] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A real-time detection method for polarizer appearance defects, Features: This method is implemented by the following steps: Step 1: first use a camera to take a polarizer picture, and then perform data expansion on the taken polarizer picture, thereby obtaining a polarizer data set; the polarizer data set includes a training set, a validation set, a test set, and an original large image of the polarizer; Step 2: Improve the backbone network and CSP BLOCK module of the YOLOv4-Tiny network model to obtain the YOLOv4-Tiny-C network model; Step 3: First use the training set in the polarizer dataset to train the YOLOv4-Tiny-C network model, then use the test set in the polarizer dataset to test the detection accuracy of the YOLOv4-Tiny-C network model, and output the prediction graph; Step 4: Use model pruning technology to prune the YOLOv4-Tiny-C network model to obtain the Pruning-YOLOv4-Tiny-C network model; Step 5: First write an image segmentation script, and then use the image segmentation script to segment the original large polarizer image in the polarizer dataset into small polarizer images, and then input the small polarizer images into the Pruning-YOLOv4-Tiny-C network model for defect detection, and then reassemble the small polarizer images with detection information into the original large polarizer image for output and storage; In step 2, the improvement steps are as follows: First, add a layer of Conv3×3 before the first convolutional layer of the YOLOv4-Tiny network model to perform convolution operations on the input image to extract richer features and enhance the nonlinear expression ability of the network; Secondly, after the first CBL downsampling, the Conv1×1+Conv3×3 convolution layer combination is added to further improve the network depth and enhance the feature extraction capability; the Conv1×1+Conv3×3 convolution layer combination mode refers to the Dense Block module for feature fusion, so that the number of channels of the output feature map is increased to 64, which is consistent with the size of the feature map of the 6th layer; Then, a layer of Conv3×3 is added after the Cnov1×1 operation on the right branch of the original CSP BLOCK structure to increase the effective receptive field and enrich the context information; Finally, the k-means++ clustering algorithm was used to regenerate 6 groups of anchor box parameter values ​​[(28,22), (33,57), (75,40), (40,105), (100,87), (141,153)] according to the cluster centers and data frame distribution. These groups were used for defect detection algorithm training to prevent a large number of missed detections and false detections.

2. A method for real-time detection of appearance defects of a polarizer according to claim 1, Features: In the step 1, the polarizer images taken include 500 defect-free images and 1,100 images with defect features, and the polarizer data set includes a training set consisting of 3,000 images with a resolution of 200×200 and defect features, a verification set consisting of 600 images with a resolution of 200×200 and defect features, a test set consisting of 200 images with a resolution of 200×200 and defect features, and 32 original large images of polarizers with a resolution of 4096×3796.

3. A method for real-time detection of appearance defects of a polarizer according to claim 2, Features: In the step 1, the defect characteristics include dirt, scratches, and bubbles.

4. The method for real-time detection of appearance defects of a polarizer according to claim 1, Features: In the step 3, before training the YOLOv4-Tiny-C network model, the hyperparameters in the .cfg file are set as follows: ① the maximum number of iterations is set to 50,000; ② the learning rate is set using a distribution strategy: the initial learning rate is set to 0.0261, the first 30,000 learning rate is maintained at 0.0261, the learning rate is decayed by 0.1 times from 30,000 to 42,000 times, and the learning rate is decayed by 0.1 times from 42,000 to 50,000 times; ③ the memory factor is set to 0.9, the weight decay term is set to 0.005, and the batch training scale is set to 128.

5. The method for real-time detection of appearance defects of a polarizer according to claim 1, Features: In step 4, the pruning operation steps are as follows: First, sparse training is performed based on the YOLOv4-Tiny-C network model, and the contribution of different channels in the YOLOv4-Tiny-C network model to the prediction results is calculated as a scale factor; After the sparse training is completed, all channels in the YOLOv4-Tiny-C network model are sorted by the scale factor, and the channels with smaller scale factors are proportionally deleted to generate the pruned YOLOv4-Tiny-C network model; Finally, the pruned YOLOv4-Tiny-C network model is fine-tuned to obtain the Pruning-YOLOv4-Tiny-C network model.

6. A method for real-time detection of appearance defects of a polarizer according to claim 5, Features: In step 4, when performing sparse training, in the YOLOv4-Tiny-C network model structure, only the input convolution layer of the YOLO layer does not have a bn layer, and all other convolution layers have a bn layer. The calculation formula is: Where: andσ 2 Set to the mean and variance of the same batch, γandβ are the trainable scale factor and bias respectively; γ is used directly to measure the importance of the channel, and the importance of γ is measured using L1 regression. The objective formula of sparsity training is: L=Loss yolo +α∑ y∈τ f(γ); Represents L1 regression, which is used to balance the two losses. The non-smooth L1 penalty term is optimized using the negative gradient method, and the value is set to 0.0001; After the sparse training is completed, the channels to be pruned are selected by substituting the global threshold, and the global threshold is used to control all the highest pruning values. At the same time, in order to avoid excessive pruning of channels within a convolutional layer, a local threshold is introduced to completely preserve the connection structure of the network. The global threshold is set to 0.7 and the local threshold is set to 0.

3.

7. The method for real-time detection of appearance defects of a polarizer according to claim 1, Features: In step 5, when performing defect detection, the image is input into the Pruning-YOLOv4-Tiny-C network model and first divided into S×S small grids, each of which can represent the local coordinates of the prior anchor box. The coordinate offset, object confidence, and category confidence score predicted by the network can be trained and calculated in these small grids; the NMS algorithm can calculate the optimal bounding box category and coordinates for all anchor boxes; The loss function is as follows: LOSS=L xywh +L confidence +L classes ; Where: L xywh is the sum of the center point and width and height errors between the predicted box and the real box, λcoord is the coordinate coefficient, L confidence is the target confidence error, λ obj and λ noobj are the confidence coefficients of the presence and absence of objects respectively; L classes is the target classification loss, It is represented as the matching status of the jth anchor box of the i-th grid.

8. The method for real-time detection of appearance defects of a polarizer according to claim 1, Features: In the step five, the detection information includes defect label information and defect location information.

Images