A method, device and storage medium for evaluating a defect detection algorithm

Abstract

The application discloses an evaluation method and device of a defect detection algorithm, equipment and a storage medium, and belongs to the technical field of evaluation of the defect detection algorithm, and comprises the following steps: determining a sample quantity of an optoelectronic device according to specificity and sensitivity of the optoelectronic device; determining sample distribution according to a type of the optoelectronic device and a preset distribution rule; determining a test data set according to the sample quantity and the sample distribution; obtaining a binary classification label and a defect prediction probability of the optoelectronic device based on a target defect detection algorithm and the test data set; determining a confusion matrix according to the binary classification label, the defect prediction probability and a real defect label; and calculating a performance index of the target defect detection algorithm according to the confusion matrix. The application determines test data by combining specificity and sensitivity of the optoelectronic device with sample distribution, so that the test data can comprehensively cover test content, and the performance index of the detection algorithm is evaluated by using the confusion matrix, thereby realizing accurate and comprehensive evaluation of the defect detection algorithm.

Description

Technical Field

[0001] This invention relates to the field of optimization technology for evaluating defect detection algorithms, and more particularly to an evaluation method, apparatus, device, and storage medium for defect detection algorithms. Background Technology

[0002] Automated surface defect inspection equipment, as one of the core technologies of modern manufacturing, plays an irreplaceable role in improving production efficiency, ensuring product quality, and reducing labor costs. Under the trend of "high performance, low cost, and large-scale" manufacturing of optoelectronic devices, automated visual inspection has become a key link in ensuring the yield, efficiency, and reliability of optoelectronic devices. Its value lies not only in replacing manual labor, but also in driving the leap from "experience-driven" to "model-driven" optoelectronic device manufacturing through a closed-loop iteration of data, algorithms, and hardware. In automated visual inspection equipment, software algorithms are the core elements determining inspection accuracy, efficiency, and adaptability. However, domestic equipment has significant shortcomings at the algorithm level, such as insufficient generalization ability in complex scenarios, an imbalance between model efficiency and accuracy, and issues with real-time performance and system coordination. This not only directly restricts equipment performance but also puts us in a passive position in high-end market competition.

[0003] In the existing technology, the evaluation system for defect detection algorithms uses traditional statistical methods to determine the sample size, ignoring the design of the sample distribution. At the same time, the testing method is singular, resulting in a lack of accuracy in the test evaluation results.

[0004] Therefore, there is an urgent need for an evaluation method, device, equipment, and storage medium for defect detection algorithms that can solve the technical problem that existing technologies cannot accurately and comprehensively evaluate defect detection algorithms. Summary of the Invention

[0005] In view of this, it is necessary to provide an evaluation method, apparatus, device and storage medium for defect detection algorithms, which can solve the technical problems of being unable to accurately identify multiple forms of annotation content and being unable to accurately convert drawing annotations into electronic annotations.

[0006] To address the aforementioned technical problems, this invention provides, on the one hand, an evaluation method for defect detection algorithms, comprising:

[0007] The sample size of the optoelectronic device is determined based on its specificity and sensitivity.

[0008] The sample distribution is determined based on the type of optoelectronic device and the preset distribution rules;

[0009] The test dataset is determined based on the sample size and sample distribution.

[0010] The target defect detection algorithm is tested based on the test dataset to obtain the defect prediction probability of the optoelectronic device, and a binary classification label is determined based on the defect prediction probability.

[0011] The confusion matrix is ​​determined based on the binary classification labels, the predicted defect probability, and the actual defect labels.

[0012] The performance index of the target defect detection algorithm is calculated based on the confusion matrix, and the target defect detection algorithm is evaluated based on the performance index.

[0013] In one possible implementation, the sample size of the optoelectronic device is determined based on its specificity and sensitivity, including:

[0014] The first sample size of the optoelectronic device is calculated based on its sensitivity.

[0015] The second sample size of the optoelectronic device is calculated based on the specificity of the optoelectronic device;

[0016] The first sample size and the second sample size are combined to obtain the sample size of optoelectronic devices.

[0017] In one possible implementation, the formula for calculating the first sample size based on the sensitivity of the optoelectronic device is as follows:

[0018] ,

[0019] in, This is the first sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the sensitivity of optoelectronic devices, d It is half the width of the confidence interval.

[0020] In one possible implementation, the formula for calculating the second sample size based on the specificity of the optoelectronic device is as follows:

[0021] ,

[0022] in, This is the second sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the specificity of optoelectronic devices, d It is half the width of the confidence interval.

[0023] In one possible implementation, the test dataset is determined based on the sample size and sample distribution, including:

[0024] Based on the sample size and sample distribution, obtain the actual production data of optoelectronic devices;

[0025] Based on the actual production data, Gaussian filtering is used to simulate focusing inaccuracy, generating the first interference data;

[0026] Based on the actual production data, a second interference data is generated by adding salt and pepper noise or Gaussian noise;

[0027] A third interference data is generated by randomly occluding image regions on the actual production data;

[0028] The actual production data is randomly transformed to generate a fourth interference data;

[0029] The test dataset consists of the real production data, the first interference data, the second interference data, the third interference data, and the fourth interference data.

[0030] In one possible implementation, determining the binary classification label based on the defect prediction probability includes:

[0031] The prediction type of each sample is determined based on the defect prediction probability of each sample in the test dataset and a preset threshold. The prediction type includes positive and negative classes.

[0032] The binary classification label is determined based on the predicted type and the true defect label of the sample.

[0033] In one possible implementation, the performance metrics of the target defect detection algorithm include accuracy, precision, recall, F1 score, specificity, and false positive rate.

[0034] Secondly, the present invention also provides an evaluation device for a defect detection algorithm, comprising:

[0035] The test dataset determination module is used to determine the sample size of optoelectronic devices based on their specificity and sensitivity; determine the sample distribution based on the type of optoelectronic devices and preset distribution rules; and determine the test dataset based on the sample size and sample distribution.

[0036] The confusion matrix calculation module is used to test the target defect detection algorithm based on the test dataset, obtain the defect prediction probability of the optoelectronic device, determine the binary classification label based on the defect prediction probability, and determine the confusion matrix based on the binary classification label, the defect prediction probability, and the true defect label.

[0037] The algorithm evaluation module is used to calculate the performance index of the target defect detection algorithm based on the confusion matrix, and evaluate the target defect detection algorithm based on the performance index.

[0038] Thirdly, an evaluation device for a defect detection algorithm includes a cloud database, a communication component, a display panel, and a tooling fixture, wherein the communication component is used to implement the evaluation method for the aforementioned defect detection algorithm.

[0039] Fourthly, a computer-readable storage medium stores computer program instructions that, when executed by a computer, implement an evaluation method based on the aforementioned defect detection algorithm.

[0040] The beneficial effects of this invention are as follows: First, the sample size of the optoelectronic devices is determined based on their specificity and sensitivity, ensuring that the test data fully reflects the characteristics of the devices without being too large and resulting in excessive testing costs. Then, the sample distribution is determined by combining the type of optoelectronic device and preset distribution rules, ensuring that the test data is representative and covers all possibilities of the optoelectronic devices, thereby improving the comprehensiveness, robustness, and accuracy of the test results. Next, the test dataset is determined based on the sample size and distribution. Then, the target defect detection algorithm is tested based on the test dataset to obtain the defect prediction probability of the optoelectronic devices, and a binary classification label is determined based on the defect prediction probability. Then, a confusion matrix is ​​determined based on the binary classification label, defect prediction probability, and true defect label, allowing for an intuitive evaluation of the algorithm's performance. Finally, the performance index of the target defect detection algorithm is calculated based on the confusion matrix, and the algorithm is evaluated based on the performance index. This invention determines the test sample size by considering the specificity and sensitivity of the optoelectronic devices, while also taking into account the sample distribution, ensuring that the test data comprehensively covers the test content. The confusion matrix is ​​determined by comparing the algorithm's prediction results with the true results, and the performance index of the detection algorithm is evaluated using the confusion matrix, achieving an accurate and comprehensive evaluation of the defect detection algorithm. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A schematic flowchart of an embodiment of the evaluation method for the defect detection algorithm provided by the present invention;

[0043] Figure 2 For the present invention Figure 1 A schematic flowchart of an embodiment of step S101;

[0044] Figure 3 For the present invention Figure 1 A schematic flowchart of an embodiment of step S103;

[0045] Figure 4 For the present invention Figure 1 A schematic flowchart of an embodiment of step S105;

[0046] Figure 5 A schematic diagram of the structure of an embodiment of the evaluation device for the defect detection algorithm provided by the present invention;

[0047] Figure 6 This is a schematic diagram of an embodiment of the evaluation device for the defect detection algorithm provided by the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0049] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0050] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0051] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] This invention provides an evaluation method, apparatus, device, and storage medium for defect detection algorithms, which are described below.

[0053] Figure 1 A schematic flowchart of an embodiment of the evaluation method for the defect detection algorithm provided by the present invention is shown below. Figure 1 As shown, the evaluation methods for defect detection algorithms include:

[0054] S101. Determine the sample size of the optoelectronic devices based on their specificity and sensitivity.

[0055] It should be noted that the expected specificity (the ability to distinguish irrelevant signals) and sensitivity (the ability to detect the smallest light signal) of an optoelectronic device are its core performance indicators. Sensitivity reflects the optoelectronic device's ability to respond to light signals, including monochromatic sensitivity, integral sensitivity, and spectral sensitivity. Monochromatic sensitivity is the response to light signals of a specific wavelength and is used to evaluate spectral selectivity. Integral sensitivity is the overall response capability to continuous light signals. Spectral sensitivity is the response difference to light of different wavelengths and is usually expressed as relative spectral sensitivity. Specificity is the optoelectronic device's ability to distinguish target signals from non-target interference signals, and is mainly reflected by flux threshold, spectral selectivity, noise equivalent power, on / off ratio, and environmental adaptability.

[0056] S102. Determine the sample distribution based on the type of optoelectronic device and the preset distribution rules;

[0057] It should be noted that the sample distribution should be determined based on the type of optoelectronic device. The test dataset must include a certain proportion of interfering and low-quality images. Furthermore, the test dataset should have diverse attributes, such as different manufacturers or production lines, and the distribution of each attribute in the test data should correspond to actual market data.

[0058] Specifically, firstly, the product types, market share, and key variables of optoelectronic devices are clarified. Product categories include LED chips, photovoltaic modules, and optoelectronic sensors, while key variables include production lines (old / new), defect types (scratches, bubbles, corrosion), and interference types (blurring, noise, obstruction). Then, sample distribution rules are determined based on real market data. These rules include distribution rules for manufacturer origin, production line type, defect type, interference data ratio, and quality level. In this embodiment, the sample distribution rules are shown in Table 1, which is the sample distribution rule provided by this invention.

[0059] Table 1 Sample Distribution Rules

[0060]

[0061] S103. Determine the test dataset based on the sample size and sample distribution;

[0062] Specifically, real production data is obtained from various manufacturers proportionally based on sample size and distribution. This data includes images of different production lines (old / new) to ensure that equipment differences are covered. At the same time, a corresponding number of interference data are generated based on the obtained real production data, and test data with corresponding sample numbers are generated for each quality level.

[0063] S104. Test the target defect detection algorithm based on the test dataset to obtain the defect prediction probability of the optoelectronic device, and determine the binary classification label based on the defect prediction probability.

[0064] Specifically, the target defect detection algorithm is tested in a closed and secure testing environment using a test dataset to obtain the defect prediction probability. Based on the binary classification method, the binary classification label is determined according to the defect prediction probability and a preset threshold.

[0065] S105. Determine the confusion matrix based on the binary classification labels, defect prediction probabilities, and actual defect labels;

[0066] S106. Calculate the performance index of the target defect detection algorithm based on the confusion matrix, and evaluate the target defect detection algorithm based on the performance index.

[0067] Specifically, this embodiment uses a system architecture design to evaluate target defect detection algorithms, including a data loading layer, an algorithm adaptation layer, an indicator calculation layer, and a visualization layer. The data loading layer supports parsing COCO, VOC, and custom data formats; the algorithm adaptation layer uses inference interfaces that encapsulate frameworks such as PyTorch, TensorFlow, and ONNX; the indicator calculation layer integrates multi-task indicator calculation libraries (such as TorchMetrics and COCO API); and the visualization layer generates PDF / HTML test reports and dynamically displays indicator radar charts and confusion matrices.

[0068] This embodiment determines the sample size of optoelectronic devices based on their specificity and sensitivity, ensuring that the test data fully reflects the characteristics of the devices without being too large and incurring excessive testing costs. It also determines the sample distribution by combining the type of optoelectronic device with pre-defined distribution rules, ensuring the test data is representative and covers all possibilities of the optoelectronic devices, thereby improving the comprehensiveness, robustness, and accuracy of the test results. The test dataset is determined based on the sample size and distribution. The target defect detection algorithm is tested based on the test dataset to obtain the defect prediction probability of the optoelectronic devices, and binary classification labels are determined based on the defect prediction probability. A confusion matrix is ​​determined based on the binary classification labels, defect prediction probability, and true defect labels, allowing for a direct evaluation of the algorithm's performance. The performance index of the target defect detection algorithm is calculated based on the confusion matrix, and the algorithm is evaluated based on this performance index. This embodiment determines the test sample size by considering the specificity and sensitivity of the optoelectronic devices, while also taking into account the sample distribution, ensuring that the test data comprehensively covers the test content. The confusion matrix is ​​determined by comparing the algorithm's prediction results with the true results, and the performance index of the detection algorithm is evaluated using the confusion matrix, achieving an accurate and comprehensive evaluation of the defect detection algorithm.

[0069] In some embodiments of the present invention, such as Figure 2As shown, Figure 2 Provided by the present invention Figure 1 A flowchart of one embodiment of step S101 includes:

[0070] S201. Calculate the first sample size of the optoelectronic device based on its sensitivity.

[0071] The formula for calculating the first sample size based on the sensitivity of the optoelectronic device is as follows:

[0072] ,

[0073] in, This is the first sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the sensitivity of optoelectronic devices, d It is half the width of the confidence interval.

[0074] S202. Calculate the second sample size of the optoelectronic device based on its specificity.

[0075] The formula for calculating the second sample size based on the specificity of optoelectronic devices is as follows:

[0076] ,

[0077] in, This is the second sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the specificity of optoelectronic devices, d It is half the width of the confidence interval.

[0078] S203. Obtain the sample size of the optoelectronic device based on the first sample size and the second sample size.

[0079] In some embodiments of the present invention, such as Figure 3 As shown, Figure 3 Provided by the present invention Figure 1 A flowchart of one embodiment of step S103 includes:

[0080] S301. Obtain real production data of optoelectronic devices based on sample size and sample distribution;

[0081] Specifically, based on the sample size and sample distribution, the sample size of each test data in the sample distribution is calculated, real production data is obtained from each manufacturer, and samples are drawn proportionally (e.g., manufacturer A provides 50% of the test data), covering images of different production lines (old lines / new lines) to ensure that equipment differences are covered.

[0082] S302. Based on real production data, Gaussian filtering is used to simulate focusing inaccuracy and generate the first interference data.

[0083] S303. Generate second interference data by adding salt and pepper noise or Gaussian noise based on real production data;

[0084] S304. Generate third interference data by randomly occluding image areas from the actual production data;

[0085] S305. Randomly transform the real generated data to generate fourth interference data;

[0086] It should be noted that a portion of the test data is randomly subjected to basic transformations to test the product's robustness. These basic transformations include cropping the original image by 5% of its side length, flipping horizontally, flipping vertically, increasing contrast (5%), decreasing contrast (5%), increasing brightness (5%), decreasing brightness (5%), and adding Gaussian white noise of a certain signal amplitude (5%). At least three of these transformations should be included in the actual test; the specific transformations used are selected by the product claims. Data undergoing basic transformations should comprise 10% of the total test sample. The transformed test data and the remaining untransformed data are fed into a closed sandbox, and the performance indicators are calculated to analyze whether there are statistically significant differences.

[0087] The test dataset consists of S306, real production data, first interference data, second interference data, third interference data, and fourth interference data.

[0088] This embodiment simulates focusing inaccuracies by applying Gaussian filtering to multi-source real-generated data, covering the central area of ​​the image or random locations, thus enhancing the model's robustness to blurred scenes. Salt-and-pepper noise is used to randomly replace some pixels with extreme black and white values ​​to simulate sensor anomalies. Global Gaussian noise is added to simulate sensor noise in low-light environments. Occlusion is simulated by randomly generating rectangular or circular occlusion blocks, the distribution of which must conform to actual occlusion scenarios (such as the operating area of ​​a robotic arm). Low-contrast images are generated by adjusting the Gamma value (0.5~1.5), or exposure parameters are set outside the normal range (overexposure +2EV, underexposure -2EV). Low-quality data is uniformly mixed into the training set according to categories to avoid model overfitting to specific defect patterns. This ensures that the test dataset comprehensively covers different test conditions, achieving comprehensiveness, robustness, and generalization of the test results.

[0089] In some embodiments of the present invention, such as Figure 4 As shown, Figure 4 Provided by the present invention Figure 1 A flowchart of one embodiment of step S105 includes:

[0090] S401. Determine the prediction type of each sample based on the defect prediction probability and preset threshold of each sample in the test dataset. The prediction type includes positive class and negative class.

[0091] S402. Determine the binary classification result based on the predicted type and the true defect label of the sample.

[0092] Specifically, the defect prediction probability is compared with a preset threshold. If the defect prediction probability is greater than or equal to the preset threshold, a positive class label is added to the prediction result of the corresponding sample; otherwise, a negative class label is added to the prediction result of the corresponding sample. For each sample, the predicted type is matched with the sample's true defect label to obtain a binary classification result, i.e.:

[0093] True Positive Cases (TP): The number of cases that are both true positive and predicted as positive.

[0094] False positives (FP): The number of instances that are actually negative but are predicted as positive;

[0095] True negative examples (TN): The number of instances that are actually negative and predicted to be negative;

[0096] False negatives (FN): The number of cases that are actually positive but are predicted as negative;

[0097] The binary classification result is converted into a confusion matrix, which is shown in Table 2 below.

[0098] Table 2 Confusion Matrix

[0099]

[0100] In some embodiments of the present invention, the performance metrics of the target defect detection algorithm include accuracy, precision, recall, F1 score, specificity, and false detection rate.

[0101] Specifically, accuracy reflects the overall correct prediction ratio of the defect detection algorithm, thus evaluating its global predictive ability. Precision assesses the reliability of positive class predictions, recall assesses the coverage of positive class predictions, F1 score balances precision and recall, further evaluating algorithm performance, specificity assesses the algorithm's ability to correctly identify negative classes, and false positive rate assesses the risk of negative samples being misclassified as positive. In summary, accuracy corresponds to overall photoelectric efficiency, precision is related to noise control, recall depends on quantum efficiency, specificity is related to the on / off ratio, and false positive rate is affected by noise. The formula for calculating accuracy is: ,

[0102] The formula for calculating accuracy is:

[0103] The formula for calculating recall rate is:

[0104] The formula for calculating the F1 score is:

[0105] The formula for calculating specificity is:

[0106] The formula for calculating the false positive rate is:

[0107] in, TP This represents the number of instances where the actual value is positive and the prediction is also positive. FP This represents the number of cases where the actual value is negative but the prediction is positive. TN This represents the number of instances where the actual value is negative and the predicted value is negative. FN This represents the number of instances that are actually positive but are predicted as negative.

[0108] To better implement the evaluation method of the defect detection algorithm in the embodiments of the present invention, based on the evaluation method of the defect detection algorithm, correspondingly, as follows: Figure 5 As shown, this embodiment of the invention also provides an evaluation device 500 for a defect detection algorithm, comprising:

[0109] The test dataset determination module 501 is used to determine the sample size of optoelectronic devices based on their specificity and sensitivity; determine the sample distribution based on the type of optoelectronic devices and preset distribution rules; and determine the test dataset based on the sample size and sample distribution.

[0110] The confusion matrix calculation module 502 is used to test the target defect detection algorithm based on the test dataset, obtain the defect prediction probability of the optoelectronic device, determine the binary classification label based on the defect prediction probability, and determine the confusion matrix based on the binary classification label, the defect prediction probability, and the true defect label.

[0111] The algorithm evaluation module 503 is used to calculate the performance index of the target defect detection algorithm based on the confusion matrix, and evaluate the target defect detection algorithm based on the performance index.

[0112] like Figure 6 As shown, the present invention also provides an evaluation device 600 for a defect detection algorithm. The evaluation device 600 for the defect detection algorithm includes a cloud database 601, a communication component 602, a display panel 603, and a tooling fixture 604. Figure 6 Only some components of the evaluation device 600 for the defect detection algorithm are shown. However, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0113] Specifically, the cloud database 601 is used to store the test dataset, the communication component 602 is used to obtain the prediction results of the defect detection algorithm, evaluate the performance of the defect detection algorithm based on the prediction results, and finally display the evaluation results through the display panel 603. The tooling fixture 604 is used to fix the evaluation device on the defect detection device, which integrates the defect detection algorithm.

[0114] Accordingly, embodiments of the present invention also provide a storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions in the evaluation method of the defect detection algorithm provided in the above-described method embodiments.

[0115] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0116] The evaluation method, apparatus, device, and storage device of the defect detection algorithm provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An evaluation method for a defect detection algorithm, wherein the defect detection algorithm is an algorithm for automatically detecting surface defects of optoelectronic devices, characterized in that, The method includes: The sample size of the optoelectronic device is determined based on its specificity and sensitivity. The sample distribution is determined based on the type of optoelectronic device and the preset distribution rules; The test dataset is determined based on the sample size and sample distribution. The defect detection algorithm is tested based on the test dataset to obtain the defect prediction probability of the optoelectronic device, and a binary classification label is determined based on the defect prediction probability. The confusion matrix is ​​determined based on the binary classification labels, the predicted defect probability, and the actual defect labels. The performance index of the defect detection algorithm is calculated based on the confusion matrix, and the defect detection algorithm is evaluated based on the performance index. The sample size of optoelectronic devices is determined based on their specificity and sensitivity, including: The first sample size of the optoelectronic device is calculated based on its sensitivity. The formula for calculating the first sample size is as follows: ， in, This is the first sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the sensitivity of optoelectronic devices, d It is half the width of the confidence interval; The second sample size of the optoelectronic device is calculated based on its specificity. The formula for calculating the second sample size is as follows: ， in, This is the second sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the specificity of optoelectronic devices, d It is half the width of the confidence interval; The first sample size and the second sample size are combined to obtain the sample size of optoelectronic devices.

2. The evaluation method for the defect detection algorithm according to claim 1, characterized in that, The test dataset is determined based on the sample size and sample distribution, including: Based on the sample size and sample distribution, obtain the actual production data of optoelectronic devices; Based on the actual production data, Gaussian filtering is used to simulate focusing inaccuracy, generating the first interference data; Based on the actual production data, a second interference data is generated by adding salt and pepper noise or Gaussian noise; A third interference data is generated by randomly occluding image regions on the actual production data; The actual production data is randomly transformed to generate a fourth interference data; The test dataset consists of the real production data, the first interference data, the second interference data, the third interference data, and the fourth interference data.

3. The evaluation method for the defect detection algorithm according to claim 1, characterized in that, Determining binary labels based on the defect prediction probability includes: The prediction type of each sample is determined based on the defect prediction probability of each sample in the test dataset and a preset threshold. The prediction type includes positive and negative classes. The binary classification label is determined based on the predicted type and the true defect label of the sample.

4. The evaluation method for the defect detection algorithm according to claim 1, characterized in that, The performance metrics of the defect detection algorithm include accuracy, precision, recall, F1 score, specificity, and false positive rate.

5. An evaluation device for a defect detection algorithm, characterized in that, include: The test dataset determination module is used to determine the sample size of optoelectronic devices based on their specificity and sensitivity. The sample distribution is determined based on the type of optoelectronic device and the preset distribution rules; the test dataset is determined based on the sample size and sample distribution. The confusion matrix calculation module is used to test the defect detection algorithm based on the test dataset, obtain the defect prediction probability of the optoelectronic device, determine the binary classification label based on the defect prediction probability, and determine the confusion matrix based on the binary classification label, the defect prediction probability, and the true defect label. The algorithm evaluation module is used to calculate the performance index of the defect detection algorithm based on the confusion matrix, and evaluate the defect detection algorithm based on the performance index. The sample size of optoelectronic devices is determined based on their specificity and sensitivity, including: The first sample size of the optoelectronic device is calculated based on its sensitivity. The formula for calculating the first sample size is as follows: ， in, This is the first sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the sensitivity of optoelectronic devices, d It is half the width of the confidence interval; The second sample size of the optoelectronic device is calculated based on its specificity. The formula for calculating the second sample size is as follows: ， in, This is the second sample size. This represents the upper quantile of the standard normal distribution corresponding to the optoelectronic device. For the specificity of optoelectronic devices, d It is half the width of the confidence interval; The first sample size and the second sample size are combined to obtain the sample size of optoelectronic devices.

6. An evaluation device for a defect detection algorithm, comprising a cloud database, communication components, a display panel, and tooling fixtures, characterized in that, The communication component is used to implement the evaluation method of the defect detection algorithm according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a computer, implement the evaluation method of the defect detection algorithm according to any one of claims 1 to 4.

Images