Threshold estimation method and electronic device

Abstract

This case discloses a threshold estimation method and electronic device. The electronic device includes a processing unit and a storage unit. The storage unit stores complex negative sample data. The processing unit is electrically connected to the storage unit and performs threshold estimation, including: reading the negative sample data and extracting complex feature values; compressing the feature values ​​into complex training data; calculating and recording the complex outlier scores of the training data to obtain an outlier score distribution map, which includes a regional extreme value and a global extreme value; estimating a probability distribution model corresponding to the outlier score distribution map based on the regional extreme value and the global extreme value; calculating a confidence interval based on the probability distribution model; and obtaining a threshold with the significance of the negative sample probability distribution based on the upper limit value of the confidence interval.

Description

[Technical Field]

[0001] This case relates to a threshold estimation method for unsupervised defect detection and an electronic device for performing the threshold estimation method. [Previous Technology]

[0002] During the training process of the unsupervised defect detection artificial intelligence model, since no positive samples containing defects are added, there is no effective way to estimate the threshold used to distinguish between positive and negative samples when making inferences. Users need to adjust it themselves based on the online inference samples.

[0003] The general approach is to require users to provide one or more datasets containing positive samples, and to determine the threshold based on the anomaly score of the positive samples. However, this method is prone to false negatives due to sampling bias in the selected positive samples, and the threshold has no statistical significance, varying with the number and type of positive samples provided by the user. Another approach is to artificially generate defects and then let the model predict them. Repeated sampling can yield the probability distribution of positive sample anomaly scores. However, this probability distribution is based on artificially generated defects and is usually not representative of the probability distribution of real defects, and it consumes additional computing power and time. [Summary of the Invention]

[0004] This case provides a threshold estimation method, comprising: reading complex negative sample data and extracting complex feature values; compressing the feature values ​​into complex training data; calculating and recording complex outlier scores of the training data to obtain an outlier score distribution map, which includes a regional extreme value and a global extreme value; estimating a probability distribution model corresponding to the outlier score distribution map based on the regional extreme value and the global extreme value; calculating a confidence interval based on the probability distribution model; and obtaining a threshold with the significance of the probability distribution of negative samples based on the upper limit value of the confidence interval.

[0005] This application further provides an electronic device comprising a storage device and a processing device. The storage device stores a plurality of negative sample data. The processing device is electrically connected to the storage device. The processing device reads the negative sample data and extracts complex feature values, compresses the feature values ​​into complex training data, and calculates and records the complex outlier scores of the training data to obtain an outlier score distribution map. This outlier score distribution map includes a regional extreme value and a global extreme value. Based on the regional extreme value and the global extreme value, the processing device estimates a probability distribution model corresponding to the outlier score distribution map; calculates a confidence interval based on the probability distribution model; and obtains a threshold with the significance of the negative sample probability distribution based on the upper limit value of the confidence interval.

[0006] In summary, this invention proposes a threshold estimation method and electronic device. During the training process of an unsupervised defect detection AI model, the threshold is effectively estimated using negative sample data to propose a consistency standard based on probability distribution. Furthermore, the threshold obtained through the probability distribution model derived from the probabilityization of negative sample anomaly scores directly represents the model's prediction result. Therefore, this invention does not require additional computational power and time for AI model estimation; the calculation can be completed simultaneously with the completion of the unsupervised model training process.

Implementation Method

[0008] The following provides a detailed description of preferred embodiments; however, these embodiments are merely illustrative and do not limit the scope of protection sought in this application. Furthermore, some elements are omitted from the drawings in the embodiments to clearly show the technical features of this application. The same reference numerals will be used to denote the same or similar elements in all drawings.

[0009] Please refer to Figure 1. An electronic device 10 includes a processing device 12 and a storage device 14. The processing device 12 is electrically connected to the storage device 14. The storage device 14 stores a plurality of negative sample data. The processing device 12 reads the negative sample data from the storage device 14 and directly estimates the threshold based on the negative sample data. Furthermore, the electronic device 10 also includes a graphics processor 16, which is electrically connected to the processing device 12. When the processing device 12 performs calculations or training, the graphics processor 16 assists the processing device 12 in performing relevant calculations, thereby accelerating the overall calculation speed. In one embodiment, the electronic device 10 is a personal computer, a laptop computer, or a tablet computer, etc., which can independently perform artificial intelligence training calculations, but this invention is not limited to this.

[0010] In one embodiment, the processing device 12 is a central processing unit (CPU), or other general-purpose or special-purpose microprocessor, microcontroller, micro control unit (MCU), digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), or other similar elements or combinations of the above elements, and this invention is not limited thereto.

[0011] In one embodiment, the storage device 14 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or other similar elements or combinations thereof, for storing any model or parameter data required by the processing device 12, but this application is not limited thereto.

[0012] In the electronic device 10, the processing device 12 executes the threshold estimation method using software. Referring to Figures 1 and 2, as shown in step S10, the processing device 12 reads complex negative sample data from the storage device 14 and extracts complex feature values ​​from the negative sample data using a deep learning model. As shown in step S12, the processing device 12 compresses these feature values ​​to compress them into complex training data through a dimension reduction method. As shown in step S14, the processing device 12 calculates and records the complex anomaly scores in the compressed training data using a defect detection algorithm to obtain an anomaly score distribution map. This anomaly score distribution map includes a regional extreme value and a global extreme value. These anomaly scores are stored in the storage device 14, but this is not a limitation of the present invention. In one embodiment, the defect detection algorithm may have different methods for calculating anomaly scores depending on the algorithm itself. For example, the PaDiM (Patch Distribution Modeling) model calculates the anomaly score based on the Mahalanobis distance per pixel. However, this application is not limited to this method, and any method for calculating anomaly scores may be applied.

[0013] As shown in step S16, based on the regional extreme value and the global extreme value, the processing device 12 estimates a probability distribution model corresponding to the anomaly score distribution map, wherein this probability distribution model is a thick-tailed probability model. In one embodiment, the regional extreme value is an unbiased estimate of the global extreme value. Unsupervised defect detection algorithms typically predict an anomaly score distribution map (also known as a defect score distribution map or defect score heat map) with the same resolution as the original force at the end. In this case, an area of ​​1 / 4 of the original size is randomly selected from the anomaly score distribution map, and the extreme value of its anomaly score is taken as the regional extreme value. An anomaly score distribution map is repeatedly selected 10 times, and the regional extreme values ​​are recorded. Next, kernel density estimation (KDE) is performed on the extreme value distributions of regional extreme values ​​and global extreme values ​​using Gaussian distribution and Gamma distribution respectively, in order to perform curve fitting, as shown in the Cumulative Density Function (CDF) in Figure 3. In the figure, the X-axis represents the extreme values ​​of the negative sample anomaly scores, and the Y-axis represents the cumulative probability distribution. Real represents the true extreme value distribution, Est_Gamma is the extreme value-cumulative probability distribution curve simulated by the Gamma function, and Est_Normal is the extreme value-cumulative probability distribution curve simulated by the Gaussian function. The probability distribution of negative samples is obtained from the curve with the best fit, and the function with the best fit is taken as the probability distribution model.

[0014] As shown in step S18, the processing device 12 calculates a confidence interval based on the probability distribution model. Specifically, this invention can provide an error probability value, for example, a Type 1 error probability value, denoted as α, with a preset value of 0.01, representing a 1% probability of mistaking a negative sample (normal sample) for a flawed sample. The error probability value is input into the probability distribution model to obtain the confidence interval. If the error probability value is 0.01, the confidence interval is the limit value corresponding to the probability [0, 0.99], including an upper limit value (right limit value) and a lower limit value (left limit value).

[0015] Finally, as shown in step S20, the processing device 12 obtains a threshold with the significance of the probability distribution of negative samples based on the limit value above the confidence interval. Since the fitted function represents the "probability distribution of negative sample outlier scores", the extreme value obtained by the confidence interval represents the "possible outlier score of negative samples within the confidence interval". In one embodiment, if the limit value corresponding to the probability [0, 0.99] of the confidence interval is [3, 57], it means that 99% of the outlier scores of negative sample data will fall within this interval of 3 to 57. Therefore, setting the threshold to 57 means that only 1% of flawless negative sample data will be misclassified as flawed positive samples.

[0016] Referring to Figures 1 and 4, after the processing device 12 obtains the threshold with the meaning of negative sample probability distribution through steps S10 to S20, as shown in step S22, the processing device 12 can further perform binarization analysis on the outlier distribution map based on this threshold to generate a binarized output map, thereby determining whether the binarized output map is negative sample data or positive sample data. In one embodiment, when the error probability value (α value) is set to 0.012, the corresponding threshold value is 57. Blocks on the outlier distribution map with outlier scores greater than 57 are set as the number 1 (labeled 1) representing defects, and blocks on the outlier distribution map with outlier scores less than 57 are set as the number 0 (labeled 0) representing no defects. Finally, after generating the binarized output image, if the binarized output image does not contain the number 1, it means that the image is flawless negative sample data. Conversely, if the binarized output image contains the number 1, it means that there is a defect in the prediction of this block location, and the image is a positive sample data.

[0017] In summary, this invention proposes a threshold estimation method and electronic device. During the training process of an unsupervised defect detection AI model, the threshold is effectively estimated using negative sample data to establish a consistency standard based on probability distribution. Therefore, users do not need to provide positive samples, and there is no need to collect additional data or worry about the provided positive samples being biased data that might affect defect settings. Furthermore, the threshold and probability distribution obtained through the probability distribution model derived from the probability of negative sample anomaly scores directly represent the model's prediction results. Therefore, users can set the error probability value (α value) according to their acceptable type-one error, or use the statistically common p=0.05 as the preset value. Thus, this invention does not require additional computational power and time for AI model estimation; the calculation can be completed simultaneously with the completion of the unsupervised model training process.

[0018] The above-described embodiments are only for illustrating the technical ideas and features of this case. Their purpose is to enable those skilled in this technology to understand the content of this case and implement it accordingly. They should not be used to limit the scope of the patent in this case. That is, all equivalent changes or modifications made in accordance with the spirit disclosed in this case should still be covered within the scope of the patent application in this case. [Simplified Explanation of the Diagram]

[0007] Figure 1 is a block diagram of an electronic device according to an embodiment of the present invention. Figure 2 is a flowchart illustrating the threshold estimation method performed by the electronic device according to an embodiment of the present invention. Figure 3 is a graph of the cumulative probability distribution function generated by the electronic device according to an embodiment of the present invention. Figure 4 is a flowchart illustrating the threshold estimation method performed by the electronic device according to another embodiment of the present invention.

Claims

1. A threshold estimation method, comprising: reading complex negative sample data and extracting complex feature values; compressing the feature values ​​into complex training data; calculating and recording complex outlier scores of the training data to obtain an outlier score distribution map, the outlier score distribution map including a regional extreme value and a global extreme value; estimating a probability distribution model corresponding to the outlier score distribution map based on the regional extreme value and the global extreme value; calculating a confidence interval based on the probability distribution model; and obtaining a threshold with the significance of the negative sample probability distribution based on an upper limit value of the confidence interval.

2. The threshold estimation method as described in claim 1, wherein in the step of extracting the feature values, a deep learning model is used to extract the feature values ​​from the negative sample data.

3. The threshold estimation method as described in claim 1, wherein the feature values ​​are compressed into the training data through a dimensionality reduction method.

4. The threshold estimation method as described in claim 1, wherein the training data are used to calculate the outlier scores through a defect detection algorithm.

5. The threshold estimation method as described in claim 1, wherein the probability distribution model is a thick-tailed probability model.

6. The threshold estimation method as described in claim 5, wherein the regional extreme value is an unbiased estimate of the global extreme value, and the distributions of the regional extreme value and the global extreme value are estimated by kernel density using Gaussian distribution and Gamma distribution respectively, so as to perform curve fitting, and the function with the best fit is taken as the probability distribution model.

7. The threshold estimation method as described in claim 1, wherein the step of calculating the confidence interval further includes inputting an error probability value into the probability distribution model to obtain the confidence interval.

8. The threshold estimation method as described in claim 7, wherein the error probability value is a Type 1 Error probability value.

9. The threshold estimation method as described in claim 1 further includes: Based on this threshold, the distribution map of the anomaly scores is binarized to produce a binarized output map.

10. An electronic device comprising: a storage device storing a plurality of negative sample data; and a processing device electrically connected to the storage device, the processing device reading the negative sample data and extracting a plurality of feature values, compressing the feature values ​​into a plurality of training data, the processing device calculating and recording a plurality of outlier scores of the training data to obtain an outlier score distribution map, the outlier score distribution map including a regional extreme value and a global extreme value; the processing device estimating a probability distribution model corresponding to the outlier score distribution map based on the regional extreme value and the global extreme value, calculating a confidence interval based on the probability distribution model, and obtaining a threshold having the significance of a negative sample probability distribution based on an upper limit value of the confidence interval.

11. The electronic device as claimed in claim 10, wherein the processing device extracts the feature values ​​from the negative sample data using a deep learning model.

12. The electronic device as claimed in claim 10, wherein the processing device compresses the feature values ​​into the training data through a dimensionality reduction method.

13. The electronic device as claimed in claim 10, wherein the training data is used to calculate the abnormal scores through a defect detection algorithm.

14. The electronic device as claimed in claim 10, wherein the probability distribution model is a thick-tailed probability model.

15. The electronic device as claimed in claim 14, wherein the regional extremum is an unbiased estimator of the global extremum, and the distributions of the regional extremum and the global extremum are respectively estimated by kernel density using Gaussian distribution and Gamma distribution to perform curve fitting, and the function with the best fit is taken as the probability distribution model.

16. The electronic device as claimed in claim 10, wherein the processing device inputs an error probability value into the probability distribution model to obtain the confidence interval.

17. The electronic device as claimed in claim 16, wherein the error probability value is a Type 1 Error probability value.

18. The electronic device as claimed in claim 10, wherein the processing device further performs binarization analysis on the anomaly score distribution map according to the threshold to produce a binarized output map.

19. The electronic device as claimed in claim 10 further includes a graphics processor electrically connected to the processing device, the graphics processor assisting the processing device in performing calculations.

Images