Evaluation method of conductive particle detection deep learning network model based on stability

Abstract

The invention relates to an evaluation method of a conductive particle detection deep learning network model based on stability, which can overcome various defects in evaluation indexes of an existing conductive particle detection method, and is combined with the characteristics of a particle detection task: counting conductive particles, and adding the fluctuation of the number of the conductive particles into the evaluation indexes; the stability evaluation of the detection method is considered: besides common model method evaluation indexes, the difference of detection results of different images collected under the same environment condition is also considered, and the difference of the same group of detection result images is compared by using an image difference index SSIM; comprising the following steps: step 1, obtaining a conductive particle area image intercepted from a TFT-LCD panel of an industrial production line, a detection result image predicted through a trained deep learning network, and a standard result image marked manually; step 2, calculating evaluation indexes based on stability, namely repeatability and reproducibility; and step 3, summarizing and comparing evaluation results of different network models.

Description

Technical Field

[0001] The present invention relates to the technical field of semiconductor panel conductive particle detection, and in particular to an evaluation method for a conductive particle detection deep learning network model based on stability. Background Art

[0002] The manufacturing process of TFT-LCD panels usually uses the COG process. During the hot pressing process of the ACF and the panel in the COG process, the conductive particles on the ACF are attached to the panel under pressure and captured by the ITO electrode, forming conductive particles on the panel. The number, position and distribution of conductive particles will affect the circuit connection performance. The conductivity of the TFT-LCD panel also directly affects its performance. The test of the conductivity of the panel in the industry is mainly through indirect measurement of the number of conductive particles on its conductive channel. Therefore, the detection accuracy of conductive particles directly affects the accuracy of the quality assessment of the TFT-LCD panel, so it is necessary to evaluate the method of conductive particle detection.

[0003] At present, the evaluation indicators of existing conductive particle detection methods are usually the same as those for evaluating deep learning networks: accuracy, precision, and recall. Accuracy refers to the ratio of the number of samples correctly predicted by the method to the total number of samples; precision refers to the proportion of samples predicted as positive examples by the method as true positive examples, indicating the prediction accuracy of the detection method; recall refers to the proportion of samples predicted as positive examples (correct prediction) by the method as true positive examples, indicating the ability of the detection method to find positive examples. In addition, since conductive particle detection is an image detection field, image-related features are also used in the evaluation indicators. DICE regards the predicted results and the true results as sets, and calculates the ratio of the intersection and sum between two to evaluate the similarity between the predicted results and the true results. This indicator is often used in image segmentation tasks; IoU (intersection over union) calculates the ratio of the intersection and union of the predicted target box and the true target box, indicating the overlap between the predicted results and the true results, and is generally used for comparison between detection boxes of target detection. However, the above methods only evaluate the ability of the model to identify targets, lacking specificity for the conductive particle detection task. In addition, in actual industrial sites, it may be necessary to measure the same target multiple times. In the process of acquiring images of the target multiple times, it is very easy to introduce various accidental factors, such as noise, blur, light intensity, etc. These tiny accidental factors may also introduce large errors to the method, causing the detection method to give results with large deviations when it should give the same detection results when detecting images of the same target acquired in different batches. However, the existing evaluation indicators only consider the detection ability of the model method, and do not consider the stability of the detection method, that is, the method's ability to resist accidental factors in the target group to be detected. The evaluation of the detection method is relatively one-sided and cannot fully meet the actual needs of the industrial site. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an evaluation method for a deep learning network model for conductive particle detection based on stability, which can overcome various drawbacks existing in the evaluation indicators of existing conductive particle detection methods, and is combined with the characteristics of particle detection tasks: it is necessary to count the conductive particles, and add the fluctuation of the number of conductive particles to the evaluation indicators; consider the stability evaluation of the detection method: in addition to the commonly used model method evaluation indicators, the differences in the detection results of different images collected under the same environmental conditions are also considered, and the differences in the images of the same group of detection results are compared using the image difference index SSIM; the definition and specific calculation method of the repeatability and reproducibility indicators are proposed.

[0005] In view of the deficiencies of the prior art, the technical solution adopted by the present invention to solve the technical problems is: an evaluation method for a deep learning network model for conductive particle detection based on stability, comprising the following steps:

[0006] Step 1, obtaining a conductive particle region image captured from a TFT-LCD panel on an industrial production line, a detection result image predicted by a trained deep learning network, and a manually annotated standard result image;

[0007] Step 2, calculate the stability-based evaluation indicators: repeatability and reproducibility;

[0008] Step 3: Summarize and compare the evaluation results of different network models.

[0009] Preferably, the conductive particle area image captured from the panel in step 1 is: aligning the panel according to the existing markers on the panel, capturing the corresponding image area at the selected panel channel position, and capturing several conductive particle areas of the same channel from different panels as a test image for calculating the evaluation index.

[0010] Preferably, the repeatability in step 2 refers to the comparison of images obtained from different batches at the same position of the panel, and the reproducibility is the comparison of images obtained from different positions of the same batch, and the image comparison is the difference in output images after comparing image detection.

[0011] Preferably, the difference of the output images is quantitatively measured by pixel difference, number of detected particles and difference of detection accuracy index; the number of detected particles count is the number of particles actually detected in a single particle image; the detection accuracy index is the precision and recall rate commonly used accuracy indicators of deep learning network models, which are calculated by the detection results and manual labeling results; the pixel difference is the degree of difference between the two images at the pixel level, which is calculated by the image similarity index SSIM between the detection results and the manual labeling results.

[0012] As a preferred calculation method, the original image is calculated as follows: The image to be compared In terms of brightness, contrast and structure Differences in:

[0013]

[0014] Where μ x ,μ y Represents the zero-order feature of the image, which is the average intensity of the image and is calculated using the pixel mean; σ x ,σ y Represents the second-order characteristics of the image, which is the standard deviation of the image pixels and is calculated using an unbiased estimate in discrete form; σ xy Represents the relationship between images, calculated using the mutual correlation coefficient. SSIM is the product of:

[0015] SSIM(x,y)=[l(x,y)]·[c(x,y)]·[s(x,y)].

[0016] Preferably, the repeatability S repeat The calculation is based on the pixel difference and the fluctuation degree of the number of detected particles. The specific calculation method is:

[0017]

[0018] Where S SSIM It represents the mean SSIM value obtained by comparing all result images of the same position in different batches. count The coefficient of variation is the standard deviation of the number of detected particles σ count and mean count Ratio:

[0019]

[0020] The reproducibility S reproduce The detection accuracy of the image samples in the group is calculated by the mean of the detection accuracy and recall rate. The specific calculation method is:

[0021] S reproduce =0.5*(mean precisi on+mean recall ).

[0022] The beneficial effects of the present invention are as follows: first, a new evaluation method for conductive particle detection methods taking into account detection stability is proposed, and the concepts used in industry to evaluate stability: repeatability and reproducibility are introduced and redefined, which not only meets the evaluation of the accuracy of the detection model method, but also meets the demand for stability in the industrial field, improves the objectivity and coverage of the evaluation indicators, and makes the evaluation indicators more in line with industrial needs; combined with the characteristics of the particle detection task, the evaluation indicators of the detection method are more targeted and specific, and more suitable for the current task. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is an overall flow chart for evaluating the conductive particle detection model method of the present invention.

[0024] Figure 2 This is an example of a conductive particle channel captured from a TFT-LCD panel. DETAILED DESCRIPTION

[0025] like Figure 1 , Figure 2 As shown, an evaluation index of a deep learning network model for conductive particle detection based on stability includes the following steps: Step 1: Obtain a conductive particle area image intercepted from a TFT-LCD panel of an industrial production line, a detection result image predicted by a trained deep learning network, and a manually annotated standard result image. The method of intercepting an image on a panel is as follows: align the panel according to the existing flags on the panel, intercept the corresponding image area at the selected panel channel position, intercept several conductive particle areas of the same channel from different panels, and use them as test images for calculating evaluation indicators. The preferred number of panels and channels in this embodiment is 5, so a total of 25 particle channel images are involved in the test.

[0026] like Figure 1As shown, step two: calculate the evaluation indicators based on stability: repeatability and reproducibility. Repeatability in the usual sense refers to multiple measurements of the same object, and reproducibility refers to multiple measurements of the same object by different operators. The present invention uses the difference in the detection results of different images collected at the same position and under the same environmental conditions to measure stability, that is, repeatability is the comparison of images obtained from different batches at the same position of the panel (horizontal comparison), and reproducibility is the comparison of images obtained from different positions of the same batch (vertical comparison). Image comparison is to compare the differences in the output images after image detection. The differences in the output images can be further quantitatively measured by the differences in pixel differences, the number of detected particles, and the detection accuracy index. The number of detected particles is an intuitive indicator based on the characteristics of the particle detection task, which is obtained directly from the image to be tested by the detection model or method; the detection accuracy index is an intuitive indicator for the model, which can be expressed by a series of accuracy indicators obtained by comparing the detection results with manual markings. The accuracy indicators used in the present invention are precision and recall:

[0027]

[0028] Where TP represents the correctly predicted positive sample pixels, FP represents the incorrectly predicted positive sample pixels, and FN represents the incorrectly predicted negative sample pixels; pixel difference refers to the difference between two images at the pixel level, and usually only image pairs with similar structures are meaningful. Its representation can refer to the image similarity index SSIM calculation:

[0029] SSIM(x,y)=[l(x,y)]·[c(x,y)]·[s(x,y)]

[0030] Where x, y represent the compared images, l, c, s represent the three aspects of image comparison, namely brightness comparison, contrast comparison, and structure comparison:

[0031]

[0032] Where μ x ,μ y Represents the zero-order feature of the image, which is the average intensity of the image and is calculated using the pixel mean:

[0033]

[0034] σ x ,σ y Represents the second-order features of the image, which is the standard deviation of the image pixels and is calculated using an unbiased estimate in discrete form:

[0035]

[0036] σ xyRepresents the relationship between images, calculated using the mutual correlation coefficient:

[0037]

[0038] Repeatability repeat The calculation is based on the pixel difference and the fluctuation degree of the number of detected particles. The specific calculation method is:

[0039]

[0040] Where S SSIM It represents the mean SSIM value obtained by comparing all result images of the same position in different batches. count The coefficient of variation is the standard deviation of the number of detected particles σ count and mean count Ratio:

[0041]

[0042] Reproducibility reproduce The detection accuracy of the image samples in the group is calculated by the mean of the detection accuracy and recall rate. The specific calculation method is:

[0043] S reproduce =0.5*(mean precision +mean recall )

[0044] like Figure 1 As shown in the figure, step three is to summarize and compare the evaluation results of different network models.

[0045] The present invention introduces and redefines the concepts used in industry to evaluate stability: repeatability and reproducibility, which not only meets the evaluation of the accuracy of the detection model method, but also meets the demand for stability in the industrial field, improves the objectivity and coverage of the evaluation indicators, and makes the evaluation indicators more in line with industrial needs; combined with the characteristics of the particle detection task, the evaluation indicators of the detection method are more targeted and specific, and more suitable for the current task.

Claims

1. A stability-based evaluation method for a deep learning network model for conductive particle detection, characterized in that: The following steps are involved: Step 1, obtaining a conductive particle region image captured from a TFT-LCD panel on an industrial production line, a detection result image predicted by a trained deep learning network, and a manually annotated standard result image; Step 2, calculate the stability-based evaluation indicators: repeatability and reproducibility; Step 3: Summarize and compare the evaluation results of different network models.

2. The evaluation method of the deep learning network model for conductive particle detection based on stability as claimed in claim 1 is characterized in that: The conductive particle area image captured from the panel in step 1 is: align the panel according to the existing mark position on the panel, capture the corresponding image area at the selected panel channel position, and capture several conductive particle areas of the same channel from different panels as a test image for calculating the evaluation index.

3. The evaluation method of the deep learning network model for conductive particle detection based on stability as claimed in claim 1, characterized in that: The repeatability in step 2 refers to the comparison of images obtained from different batches at the same position of the panel, and the reproducibility is the comparison of images obtained from different positions of the same batch. The image comparison is the difference in the output images after comparing the image detection.

4. The evaluation method of the deep learning network model for conductive particle detection based on stability as claimed in claim 1, characterized in that: The difference of the output images is quantitatively measured by the difference of pixel difference, number of detected particles and detection accuracy index; the number of detected particles count is the number of particles actually detected in a single particle image; the detection accuracy index is the precision and recall rate commonly used in deep learning network models, which are calculated from the detection results and manual labeling results; The pixel difference is the difference between two images at the pixel level, which is calculated by the image similarity index SSIM between the detection result and the manual labeling result.

5. The evaluation method of the deep learning network model for conductive particle detection based on stability as claimed in claim 4, characterized in that: The calculation method is: calculate the difference between the original image x and the image to be compared y in terms of brightness l, contrast c and structure s: Where μ x ,μ y Represents the zero-order feature of the image, which is the average intensity of the image and is calculated using the pixel mean; σ x ,σ y Represents the second-order characteristics of the image, which is the standard deviation of the image pixels and is calculated using an unbiased estimate in discrete form; σ xy Represents the relationship between images, calculated using the mutual correlation coefficient. SSIM is the product of: SSIM(x,y)=[l(x,y)]·[c(x,y)]·[s(x,y)].

6. The evaluation method of the deep learning network model for conductive particle detection based on stability as claimed in claim 5, characterized in that: The repeatability S repeat The calculation is based on the pixel difference and the fluctuation degree of the number of detected particles. The specific calculation method is: Where S SSIM It represents the mean SSIM value obtained by comparing all result images of the same position in different batches. count The coefficient of variation is the standard deviation of the number of detected particles σ count and mean count Ratio: The reproducibility S reproduce The detection accuracy of the image samples in the group is calculated by the mean of the detection accuracy and recall rate. The specific calculation method is: S reproduce =0.5*(mean precision +mean recall )。

Images