A visual inspection method for defects in large-size light guide plates

Abstract

The present invention discloses a method for visual inspection of defects of large-size light guide plates, including using a high-resolution line scan camera to collect images at the end of a light guide plate production line, and then inputting the obtained light guide plate images into a host computer to intercept them into a group of small images with a resolution of 416×416, and inputting them into a large-size light guide plate defect detection model to obtain light guide plate images with defect marks and confidence scores; the large-size light guide plate defect detection model includes a backbone network, a feature pyramid, an improved RFB module, and three prediction heads Y1, Y2, and Y3 connected in sequence. The method of the present invention can simultaneously predict the type and location of defects, solving the problem of only being able to classify whether there are defects in the past. Experimental results show that compared with the existing YOLOv3 algorithm, the detection accuracy of defects in large-size light guide plates is higher.

Description

Technical Field

[0001] The present invention belongs to the field of light guide plate image recognition, and in particular is a method for visually detecting defects of large-size light guide plates. Background Art

[0002] Light guide plate (LGP) is made by making light guide points on acrylic plate by screen printing, laser engraving and other technologies. Due to the advantages of ultra-thin, high transparency, high reflection, uniform and bright light guide, light guide plate (LGP) is widely used in LCD screens such as mobile phones, tablets, and computers. As a key component of the LCD backlight module, the quality of the light guide plate directly affects the quality of the picture. However, during the production process of the light guide plate, due to factors such as raw materials, temperature and processing technology, it is inevitable that processing defects such as point damage, line scratches, and scratches will occur, which will cause display abnormalities and affect user use. Therefore, it is very necessary to detect defects. According to the shape of the defect, it is divided into point defects and line defects. Point defects refer to point defects formed inside the light guide plate. In the process of forming the light guide plate, the raw materials are not completely melted and there is a lot of dust on the mold, which will cause point defects. Linear defects refer to scratches and scratches on the surface of the light guide plate, which are mainly linear in the image. They are mainly caused by impurities on the surface of the parts and relative friction during the transmission process.

[0003] At present, defect detection of light guide plates is mainly done manually, but manual detection has certain limitations, mainly in the following aspects: (1) Manual detection is too subjective, and there are large differences in detection between individuals, making it difficult to form a unified evaluation standard; (2) Manual detection requires long-term observation under strong light, which seriously damages the eyesight of employees, and the detection speed is slow, making it impossible to achieve 7×24 hours of continuous and stable operation; (3) The background of the light guide plate is complex and there are many types of defects, making it difficult for employees to master the relevant detection technology.

[0004] At present, manufacturers mainly conduct defect detection of light guide plates manually. They view the light guide plates from multiple angles under special lighting to observe whether there are defects such as point damage, line scratches, and scrapes on the light guide plates, and remove defective light guide plates. Due to the various limitations of manual defect detection, the accuracy, efficiency, and stability of manual detection are difficult to meet the requirements of enterprises. In order to obtain high-resolution light guide plate detection images, high-precision line scan cameras are required to capture images, and the size of the obtained images is about 468MB. At industrial sites, companies require that defect detection of a light guide plate be completed within 6 seconds, which also puts higher requirements on the efficiency of defect detection. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a large-size light guide plate defect visual detection method for simultaneously completing the classification and positioning detection of large-size light guide plate defects.

[0006] In order to solve the above technical problems, the present invention provides a large-size light guide plate defect visual inspection method, comprising: using a high-resolution line scan camera at the end of the light guide plate production line to collect images, and then inputting the obtained light guide plate images into a host computer to intercept them into a group of small images with a resolution of 416×416, and inputting them into a large-size light guide plate defect detection model to obtain a light guide plate image with defect marks and confidence scores;

[0007] The large-size light guide plate defect detection model includes a backbone network, a feature pyramid, an improved RFB module, a first prediction head Y1, a second prediction head Y2 and a third prediction head Y3 connected in sequence. The backbone network extracts and outputs (13, 13), (26, 26) and (52, 52) feature layers through downsampling, and then the feature pyramid is used to strengthen the feature extraction of the feature layers extracted by the backbone network. The 52×52 scale feature map obtained by the feature pyramid is further strengthened by the improved RFB module and then input into the third prediction head Y3 for prediction. The 26×26 scale feature map obtained by the feature pyramid is further strengthened by the improved RFB module and then input into the second prediction head Y2 for prediction. The 13×13 scale feature map obtained by the feature pyramid is input into the first prediction head Y1 for prediction, thereby obtaining a prediction result. The anchor frame linearly scaled by the improved K-means algorithm is used to decode the prediction result, and finally the anchor frame with the highest score is selected through non-maximum screening.

[0008] As an improvement of the visual inspection method for defects of a large-size light guide plate of the present invention:

[0009] The improved RFB module includes first reducing the dimension through 1×1 convolution, and then performing 1×1, 3×3 and 5×5 convolutions respectively. Convolution kernels of different sizes correspond to the hollow convolutions with dilation rates of 1, 3, and 5 in series. Finally, the outputs of each branch are concatenated. The shortcut connection method in the ResNet network is also adopted in the RFB module, and two cascaded 3×3 convolutions are used instead of 5×5 convolutions.

[0010] As a further improvement of the large-size light guide plate defect visual inspection method of the present invention:

[0011] The improved K-means algorithm clusters the anchor frame sizes that match the large-size light guide plate defects. The anchor frames used on the first prediction head Y1 are (56, 46), (78, 95), and (89, 49). The anchor frames used on the second prediction head Y2 are (40, 55), (52, 31), and (53, 66). The anchor frames used on the third prediction head Y3 are (27, 29), (31, 52), and (36, 37).

[0012] The anchor boxes used on the second prediction head Y2 and the third prediction head Y3 are linearly scaled as follows:

[0013] x′1=αx1 x′6=βx6

[0014]

[0015] Among them, x i ,y i is the width and height of the original anchor box, x′ i , y′ i are the width and height values ​​of the anchor box after linear scaling, the value of α is 0.9, the value of β is 1.2, the value of x1 is 27, x′1 is the minimum anchor box width value after linear scaling, the value of x6 is 89, and x′6 is the maximum anchor box width value after linear scaling.

[0016] As a further improvement of the large-size light guide plate defect visual inspection method of the present invention:

[0017] The backbone network is a Darknet-53 network, including 1, 2, 8, 8, and 4 residual modules, which extract and output the (13, 13), (26, 26), and (52, 52) feature layers;

[0018] The feature pyramid includes convolving the (13, 13) feature layer, and obtaining a 13×13 scale feature map, which is convolved and up-sampled with the (26, 26) feature layer to obtain a 26×26 scale feature map, and the 26×26 scale feature map is further convolved and up-sampled with the (52, 52) feature layer to obtain a 52×52 scale feature map.

[0019] As a further improvement of the large-size light guide plate defect visual inspection method of the present invention:

[0020] The training and testing process of the large-size light guide plate defect detection model is as follows: a high-resolution line scan camera is used to capture images of the large-size light guide plate, and then the captured images are randomly intercepted at angles to capture two types of defect images, namely, point and line defect images, and the intercepted defect images are preprocessed by defect category and area labeling. Then, for each defect type in the preprocessed images, the preprocessed images are divided into training set, verification set and test set in a ratio of 7:1:2. In the training stage, a stochastic gradient descent optimization algorithm is used, and training parameters including initial learning rate, final learning rate, momentum size, weight attenuation coefficient and batch size are set. After 100 training iterations, a trained large-size light guide plate defect detection model is obtained, and then the large-size light guide plate defect detection model is obtained after verification with the test set.

[0021] The beneficial effects of the present invention are mainly reflected in:

[0022] Compared with the prior art, the present invention proposes a visual detection method for defects in large-size light guide plates based on an improved new YOLOv3 algorithm, which makes it possible to simultaneously predict the type and location of defects, solving the problem of only being able to classify whether there are defects. Experimental results show that compared with the existing YOLOv3 algorithm, the detection accuracy of defects in large-size light guide plates is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0024] Figure 1 It is a schematic diagram of the overall structure of the large-size light guide plate defect detection model of the present invention;

[0025] Figure 2 for Figure 1 A schematic diagram of the structure of the improved RFB module;

[0026] Figure 3 This is a sample display of the large-size light guide plate defect dataset in Example 1;

[0027] Figure 4 This is the curve of the loss of the training set and the validation set changing with the number of iterations during the training process;

[0028] Figure 5 This is a diagram showing the comparison of the prediction effects of different algorithms in Experiment 1. DETAILED DESCRIPTION

[0029] The present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto:

[0030] Embodiment 1, a large-size light guide plate defect visual detection method, based on the improved YOLOv3 algorithm to classify and locate large-size light guide plate defects, solves the problem that the existing defect detection method based on deep learning cannot simultaneously complete the classification and positioning of large-size light guide plate defects and can only determine the presence or absence of defects, and solves the problem that the detection accuracy of large-size light guide plate defects using the YOLOv3 algorithm is not high, including the process as follows:

[0031] S1. Improve and build a large-size light guide plate defect detection model based on the YOLOv3 network, such as Figure 1As shown, it includes a backbone network, a feature pyramid, an improved RFB module and a prediction head connected in sequence. The backbone network extracts feature layers through multiple downsampling, and then the feature pyramid is used to strengthen the feature extraction of the (13, 13), (26, 26) and (52, 52) feature layers extracted by the backbone network. The feature maps of different scales generated by the feature extraction are further enhanced through the improved RFB module and sent to the prediction head to realize the classification and positioning of defects.

[0032] S101, build a backbone network to extract multi-scale features;

[0033] Darknet-53 is used as the backbone network of the large-size light guide plate defect detection model, including 1, 2, 8, 8, and 4 residual modules. The residual structure can alleviate the gradient vanishing problem during deep network training and enhance the convergence effect. Through multiple downsampling, the (13, 13), (26, 26), and (52, 52) feature layers are extracted and output respectively.

[0034] S102, construct feature pyramid to enhance feature extraction;

[0035] The (13, 13) feature layer is convolved (Convolutional set), and the obtained 13×13 scale feature map is convolved (Convolutional set), upsampled (UpSampling) and concatenated (Concat) with the (26, 26) feature layer output from the backbone network to obtain a 26×26 scale feature map. The 26×26 scale feature map is then convolved (Convolutional set), upsampled (UpSampling) and concatenated (Concat) with the (52, 52) feature layer output from the backbone network to obtain a 52×52 scale feature map.

[0036] S103, introducing an improved RFB module to further enhance feature extraction;

[0037] The 26×26 scale feature map and the 52×52 scale feature map obtained in step S102 are respectively introduced into the improved RFB module convolution to further enhance feature extraction.

[0038] The improved RFB module structure, such as Figure 2As shown in the figure, first, 1×1 convolution is used to reduce the dimension; secondly, 1×1, 3×3 and 5×5 convolutions are performed respectively to simulate receptive fields of different scales, and convolution kernels of different sizes are connected (in series) to corresponding dilation rates of 1, 3, and 5. Finally, the outputs of each branch are concatenated (Concat) to fuse different features and improve the network model's ability to express target features of defects of different sizes. In addition, the shortcut connection method in the ResNet network is also used in the RFB module, which can effectively reduce the gradient disappearance, enable the gradient to be well transferred to the shallow layer, reduce the complexity of the network, and improve the training effect of the network. Under the premise of ensuring the network receptive field, in order to further reduce the amount of calculation, two cascaded 3×3 convolutions are used instead of 5×5 convolutions.

[0039] The improved RFB module is composed of convolutional layers with multiple convolutional kernels of different sizes. Each branch uses a combination of convolutional kernels of different scales and dilation rates, so that the receptive field of each branch is expanded to different degrees to enhance feature extraction.

[0040] S104. Improve the K-means algorithm and perform linear scaling on the clustered small and medium-scale anchor frames to make them closer to the real frame and improve the detection accuracy.

[0041] The idea of ​​anchor boxes is introduced in the YOLOv3 algorithm. The sizes of these anchor boxes are based on the COCO dataset, which is mainly for multi-scale targets such as pedestrians, cars and daily objects. The sizes of the species are diverse, which is too different from the size of the light guide plate defect target. Therefore, the K-means algorithm is used to cluster the anchor box sizes that match the light guide plate defect target. They are divided into three groups according to size. The largest group is used on the 13×13 scale prediction head (i.e., the first prediction head Y1), and the corresponding anchor boxes are (56, 46), (78, 95), (89, 49). The smallest group is used on the 52×52 scale prediction head (i.e., the third prediction head Y3), and the corresponding anchor boxes are (27, 29), (31, 52), (36, 37). The corresponding anchor boxes on the 26×26 scale prediction head (i.e., the second prediction head Y2) are (40, 55), (52, 31), (53, 66).

[0042] In the large-scale light guide plate defect detection model, feature fusion is used to output three scale feature maps for prediction. Large targets are detected in small-scale feature maps, medium-sized targets are detected in medium-scale feature maps, and small targets are detected in large-scale feature maps. In the light guide plate defect dataset, small target defects account for a high proportion, and the scale size is evenly concentrated, which does not reflect the advantages of multi-scale prediction of the YOLOv3 network (Shao Huixiang / Zeng Dan Classification and Recognition of Underwater Small Targets Based on Improved YOLOv3 Algorithm [J] Journal of Shanghai University (Natural Science Edition), 2021, 27(03): 481-491). According to the defect imaging characteristics of large-size light guide plates, the anchor frames corresponding to the 26×26 scale and 52×52 scale obtained by clustering are linearly scaled. The central idea is to make the anchor frame close to the real frame, and to enlarge and reduce the width and height at a certain ratio based on the center point of the anchor frame, so that the small anchor frame size is smaller and the large anchor frame size is larger. The linear scaling formula is as follows:

[0043] x′1=αx1 x′6=βx6

[0044]

[0045] Among them, x i ,y i is the width and height of the original anchor box, x′ i , y′ i is the width and height of the anchor box after linear scaling, α is 0.9, and β is 1.2. The value of x1 is 27, x′1 is the smallest anchor box width after linear scaling, and the value of x6 is 89, x′6 is the largest anchor box width after linear scaling.

[0046] S105. Use the feature maps of different scales generated in step S102 and step S103 to enter the prediction head to classify and locate the defects; the 13×13 scale feature map obtained in step S102 enters the first prediction head Y1, the 26×26 scale feature map obtained in step S103 enters the second prediction head Y2, and the 52×52 scale feature map enters the third prediction head Y3. The three prediction heads predict the type and location of the defects at three scales respectively, thereby obtaining the prediction results. Use the anchor frame after linear scaling in step S104 to decode the prediction results, and finally select the anchor frame with the highest score through non-maximum screening.

[0047] S106. Training and testing models

[0048] To verify the present invention, the following Figure 1 The network structure test shown in the figure. First, a large-size light guide plate defect dataset is constructed. The images contained in the dataset are divided into two types of defects: points and lines. Pytorch is used to build a large-size light guide plate defect detection model for training and testing.

[0049] At the end of the large-size light guide plate production line, a light guide plate image acquisition device is set up, and a high-resolution line scan camera is used for image acquisition. Then the acquired large-size light guide plate image data set is transmitted to the host computer for processing. Based on the acquired original image, images containing point and line defects are randomly intercepted at a random angle. The resolution of each image is 416′416. The intercepted defective images are preprocessed for defect category and area annotation. Then, for each defect type in the preprocessed image, the preprocessed image is divided into a training set, a validation set, and a test set in a ratio of 7:1:2. The specific composition is shown in Table 1 below.

[0050] Table 1 Composition of large-size light guide plate defect dataset

[0051] training validation test Total Point 879 126 251 1256 Line 686 98 196 980

[0052] The defect samples in the dataset are as follows Figure 3 As shown, Figure 3 (a) is a point defect, Figure 3 (b) is a line defect.

[0053] In the training phase, the stochastic gradient descent (SGD) optimization algorithm is used, the input image size is set to 416×416, the initial learning rate and the final learning rate are set to 0.01 and 0.0001, the momentum size is 0.9, the weight decay coefficient is 0.0005, and the batch size is 16. In the network training process, the loss curves of the training set and the validation set change with the number of iterations are shown in the figure below: Figure 4 shown.

[0054] The loss function of the large-size light guide plate defect detection model consists of three parts: coordinate position loss L1, confidence loss L2, and category loss L3, as shown below:

[0055] Loss = L1 + L2 + L3

[0056] The coordinate position loss L1 is as follows:

[0057]

[0058] Among them, λ c It is expressed as the weight of coordinate error, and its value is 5; s 2 is the number of feature map grids, s is 7; B is the number of predicted bounding boxes for each grid, and its value is 9; Determine whether the jth prediction box of the i-th cell contains the detected target. If the image contains the target, then otherwise (x, y, w, h) are the center coordinates, width, and height values ​​of the actual bounding box; are the center coordinates, width, and height values ​​of the prediction box.

[0059] The confidence loss L2 is as follows:

[0060]

[0061] Among them, c i is the actual confidence score of the object, is the predicted confidence score of the object, λ noobj is the confidence error weight value that the bounding box does not contain the target, λ noobj =0.5;

[0062] The class loss L3 is as follows:

[0063]

[0064] Among them, p i (c) is the true probability that the target detected by the i-th network belongs to category C, The predicted probability that the target detected by the i-th network belongs to category C.

[0065] After 100 training iterations, the loss value gradually stabilized and the model tended to converge, and a trained large-size light guide plate defect detection model was obtained. The trained large-size light guide plate defect detection model was then tested and verified using the test set, thereby finally obtaining a large-size light guide plate defect detection model that can be used in online production.

[0066] S2. Online defect detection and output results

[0067] At the end of the light guide plate production line, a high-resolution line scan camera is used to obtain the light guide plate image, which is then cut into a group of small images with a resolution of 416′416 and input into the large-size light guide plate defect detection model finally obtained in step S106. The anchor frame is linearly scaled in step S103, and then the anchor frame with the highest score is selected through non-maximum screening.

[0068] Finally, a light guide plate image with defect marks and confidence scores is obtained, and the above results are output and stored in the host computer for real-time display results and subsequent query analysis.

[0069] Experiment 1:

[0070] The large-size light guide plate image dataset constructed in step S106 of Example 1 was used as input, and the large-size light guide plate defect detection model of the present invention was compared with detection algorithms such as Faster R-CNN, SSD, and YOLOv3. The accuracy, average accuracy, and F1 value were used as algorithm evaluation indicators, and the definition of mAP is as follows:

[0071]

[0072]

[0073] In the formula, AP is the prediction accuracy of each type of defect, and mAP is the average AP of multiple types of defects;

[0074] F1 is a comprehensive indicator for evaluating the model and is defined as the harmonic mean of precision and recall. The formula is as follows:

[0075]

[0076] In the formula, Refers to the accuracy, Refers to the recall rate. TP represents the positive samples predicted by the model as positive, FP represents the negative samples predicted by the model as positive, and FN represents the positive samples predicted by the model as negative.

[0077] The comparison of prediction results of different algorithms is shown in Table 1:

[0078] Table 1 Comparison of experimental results of different algorithms

[0079]

[0080] As shown in Table 1, compared with YOLOv3, the large-size light guide plate defect detection model has improved the point defect accuracy (point / %), line defect accuracy (line / %) and F1 by 7.71%, 11.24% and 10.76% respectively, and the mAP has increased by 8.55% to 98.92%. Compared with the SSD and Faster R-CNN algorithms, the improved YOLOv3 algorithm has higher detection accuracy. Figure 5 As shown, the excellent performance of the large-size light guide plate defect detection method based on the improved YOLOv3 algorithm proposed in the present invention is demonstrated.

[0081] Finally, it should be noted that the above examples are only some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. A method for visually inspecting defects of a large-size light guide plate, characterized in that include: At the end of the light guide plate production line, a high-resolution line scan camera is used to collect images. The obtained light guide plate images are then input into the host computer to be intercepted into a set of small images with a resolution of 416×416, and then input into the large-size light guide plate defect detection model to obtain light guide plate images with defect marks and confidence scores. The large-size light guide plate defect detection model comprises a backbone network, a feature pyramid, an improved RFB module, a first prediction head Y1, a second prediction head Y2 and a third prediction head Y3 connected in sequence, the backbone network extracts and outputs (13, 13), (26, 26) and (52, 52) feature layers by downsampling, and then the feature layer extracted by the backbone network is enhanced by the feature pyramid. The 52×52 scale feature map obtained by the feature pyramid is further enhanced by the improved RFB module and then input into the third prediction head Y3 for prediction. The 26×26 scale feature map obtained by the feature pyramid is further enhanced by the improved RFB module and then input into the second prediction head Y2 for prediction. The 13×13 scale feature map obtained by the feature pyramid is input into the first prediction head Y1 for prediction, thereby obtaining a prediction result. The prediction result is decoded by using an anchor frame linearly scaled by the improved K-means algorithm, and finally the anchor frame with the highest score is selected by non-maximum screening. The improved RFB module includes firstly reducing the dimension through 1×1 convolution, then performing 1×1, 3×3 and 5×5 convolutions respectively, and connecting convolution kernels of different sizes in series with dilation rates of 1, 3 and 5, and finally concatenating the outputs of each branch; the shortcut connection method in the ResNet network is also adopted in the RFB module, and two cascaded 3×3 convolutions are used instead of 5×5 convolutions; The improved K-means algorithm clusters the anchor frame sizes that match the large-size light guide plate defects. The anchor frames used on the first prediction head Y1 are (56, 46), (78, 95), and (89, 49). The anchor frames used on the second prediction head Y2 are (40, 55), (52, 31), and (53, 66). The anchor frames used on the third prediction head Y3 are (27, 29), (31, 52), and (36, 37). The anchor boxes used on the second prediction head Y2 and the third prediction head Y3 are linearly scaled as follows: x1'=αx1 x'6=βx6 Among them, x i ,y i is the width and height of the original anchor box, x i ',y i ' is the width and height of the anchor box after linear scaling, the value of α is 0.9, the value of β is 1.2, the value of x1 is 27, the value of x1' is the minimum width of the anchor box after linear scaling, the value of x6 is 89, and x'6 is the maximum width of the anchor box after linear scaling.

2. A large-size light guide plate defect visual inspection method according to claim 1, characterized in that: The backbone network is a Darknet-53 network, including 1, 2, 8, 8, and 4 residual modules, which extract and output the (13, 13), (26, 26), and (52, 52) feature layers; The feature pyramid includes convolving the (13, 13) feature layer, and obtaining a 13×13 scale feature map, which is convolved and up-sampled with the (26, 26) feature layer to obtain a 26×26 scale feature map, and the 26×26 scale feature map is further convolved and up-sampled with the (52, 52) feature layer to obtain a 52×52 scale feature map.

3. A large-size light guide plate defect visual inspection method according to claim 2, characterized in that: The training and testing process of the large-size light guide plate defect detection model is as follows: a high-resolution line scan camera is used to capture images of the large-size light guide plate, and then the captured images are randomly intercepted at angles to capture two types of defect images, namely, point and line defect images, and the intercepted defect images are preprocessed by defect category and area labeling. Then, for each defect type in the preprocessed images, the preprocessed images are divided into training set, verification set and test set in a ratio of 7:1:

2. In the training stage, a stochastic gradient descent optimization algorithm is used, and training parameters including initial learning rate, final learning rate, momentum size, weight attenuation coefficient and batch size are set. After 100 training iterations, a trained large-size light guide plate defect detection model is obtained, and then the large-size light guide plate defect detection model is obtained after verification with the test set.

Images