Chip defect detection method based on unsupervised learning and evaluation method thereof

Abstract

The application discloses a chip defect detection method based on unsupervised learning and an evaluation method thereof, and comprises the following steps: 1, training data preprocessing; 2, constructing an unsupervised defect detection network model, including a repair subnetwork, a discriminative subnetwork, an analog anomaly generator and reliability evaluation of uncertainty; 3, training the unsupervised defect detection network model by using the automatically generated chip defect data set; 4, processing the chip defect data set by using the trained unsupervised defect detection network model, and obtaining anomaly detection results and uncertainty results therefrom. The application can significantly improve the PCB detection efficiency and accuracy, and introduces uncertainty as a reliability evaluation index, so that the model performance can be better evaluated.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to an unsupervised defect detection method and its evaluation method. Background Technology

[0002] With the rapid development of integrated circuit manufacturing technology, traditional packaging can no longer meet the ever-increasing demands of integrated circuits. Advanced packaging forms such as 2.5D / 3D integration and wafer-level packaging have become the main direction of packaging technology development. These are characterized by increasingly smaller package sizes and continuously narrowing metal wiring widths. To achieve higher chip interconnect density, typical interconnect linewidths have decreased from 30-50 μm to 2-5 μm, and the bump height of interlayer interconnects has decreased from 150-600 μm to 20-40 μm. As interconnect linewidths develop towards the micrometer scale, higher requirements are placed on the accuracy of detecting two-dimensional defects such as deformation, cracks, and misalignment in metal wiring. Any short circuit, open circuit, or loose connection caused by defects will directly affect the quality of the entire chip. As linewidths shrink to the micrometer level, the size of two-dimensional defects is also developing towards the sub-micrometer scale. Simultaneously, with the reduction in the size and spacing of bumps connecting chips, interconnect short circuits caused by solder deformation are becoming increasingly apparent. To improve reliability, the need for chip defect detection is becoming more urgent. However, current PCB micro-surface defect detection still suffers from many problems, such as small sample size, unbalanced sample types, and defects with multiple scales. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention proposes a chip defect detection method and its evaluation method based on unsupervised learning, aiming to enable the identification network to recognize and label different chip defects, thereby significantly improving the efficiency and accuracy of chip defect detection.

[0004] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0005] The present invention provides a chip defect detection method based on unsupervised learning, characterized by the following steps:

[0006] Step 1: Preprocessing of training data:

[0007] Obtain a dataset of defective PCB images in, This represents the PCB image of the t-th defect type, and the t-th defect type is denoted as e. t Thus, the real defect set F = {e1, e2, ..., e} is composed of all defect types. t ,...e T}; T represents the number of images;

[0008] Obtain a defect-free PCB image dataset I = {I1, I2, ..., I...} t ,...,I T}; where I t This represents the t-th defect-free PCB image;

[0009] Step 2: Construct an unsupervised defect detection network, including: a simulated anomaly Φ and a restorative subnet θ. R Discrimination subnet θ D Surface anomaly detector ξ;

[0010] The restorative subnet θ R and the discriminative subnet θ D Both are encoder-decoder structures;

[0011] The restorative subnet θ R The encoder in the code consists of C1 convolutional layers with kernel size a1×a1 and a max pooling layer with dimension b1×b1.

[0012] The restorative subnet θ R The decoder in the code consists of C2 convolutional layers with kernel size a2×a2 and a max pooling layer with dimension b2×b2.

[0013] The discriminative subnet θ D The encoder in the code consists of C3 convolutional layers with kernel size a3×a3 and a max pooling layer with dimension b3×b3.

[0014] The discriminative subnet θ D The decoder in the code consists of C4 convolutional layers with kernel size a4×a4 and a max pooling layer with dimension b4×b4.

[0015] The encoder and decoder are connected via a jump link;

[0016] Step 3: Training the unsupervised defect detection network:

[0017] Step 3.1, Processing of the simulated anomaly Φ:

[0018] Step 3.1.1: Randomly select several types of defects from the real defect set F and overlay them onto the t-th defect-free PCB board image I. t Above, an image M containing several types of defects is formed. t ;

[0019] Step 3.1.2, for M t After binarization, the defect mask M is obtained. a,t ;

[0020] Step 3.1.3: Use equation (1) to generate the t-th PCB image I containing several types of defects. a,t :

[0021]

[0022] In equation (1), It is M a,t Inverting pixel values;

[0023] Step 3.2, the restorative subnet θ R Reconstruct defect-free images;

[0024] Step 3.2.1, the restorative subnet θ R to I a,t Perform reconstruction processing and output the t-th reconstructed defect-free PCB image I. r,t ;

[0025] Step 3.2.2, based on I r,t with I t Constructing a restorative subnet θ R Mean square error loss L MSE (I t ,I r,t );

[0026] Step 3.2.3, based on I r,t with I t Using equation (2), a restorative subnet θ is constructed. R Structural similarity loss function L SSIM (I t ,I r,t ):

[0027]

[0028] In equation (2), μ and μ r They are I t and I r,t The weighted mean of I, σ is t The variance, σ r isI r,t variance isI t and I r,t The covariance; C1 and C2 represent two constants;

[0029] Step 3.2.4: Construct the restorative subnet θ using equation (3). R Total loss function L rec (I a,t ,I r,t ):

[0030] Lrec (I t ,I r,t )=λL SSIM (I t ,I r,t )+L MSE (I t ,I r,t (3)

[0031] In equation (3), λ is a hyperparameter that adjusts the loss balance;

[0032] Step 3.3, the discriminative subnet θ D Processing:

[0033] The discriminative subnet θ D It contains two output channels m1 and m2, which will output I r,t with I a,t The result of the subtraction is fed into the discriminative subnet θ. D Defect segmentation is performed during the process, and the t-th segmented image is output through the first output channel m1. The second output channel m2 outputs the uncertainty result σ of the t-th data. t ;

[0034] Step 3.4: The surface anomaly detector ξ uses a mean filter to... After smoothing, the image-level anomaly score η for the t-th image is obtained using equation (4). t :

[0035]

[0036] In equation (4), f sf×sf This represents a mean filter with dimensions sf×sf, where * represents the convolution operator;

[0037] Step 3.5, when the abnormal score η t When the pixel-level threshold is exceeded, I a,t The abnormal location information present in the PCB is binarized to obtain the abnormal mask M0, which is then used to determine the location of the PCB defect.

[0038] Step 3.6: Construct the segmentation loss function L using equation (5). seg (M a,t M t ):

[0039] L seg (M a,t M t )=-α(1-p t ) γ log(p t (5)

[0040] Where, p t α is the predicted probability of the model, α is the weight of each class, and γ is an adjustable parameter used to control the impact of low-probability predictions.

[0041] Step 3.7: Construct the initial training loss function L1(I) using equation (6). t ,I r,t M a,t M t ):

[0042] L1(I t ,I r,t M a,t M t ) = L rec (I t ,I r,t )+L seg (M a,t M t (6)

[0043] Step 4: Train the unsupervised defect detection network using gradient descent and calculate the total loss function L1(I t ,I r,t M a,t M t To update the network parameters until the initial training loss function L1(I) is reached. t ,I r,t M a,t M t The training continues until the model converges or reaches the maximum number of training iterations, thus obtaining a trained unsupervised defect detection model for detecting defects in PCB images.

[0044] The present invention provides an evaluation method for chip defect detection based on unsupervised learning, characterized by evaluating the trained unsupervised defect detection model according to the following steps:

[0045] Step 5: Calculate the uncertainty result σ;

[0046] Step 5.1: Define the current model index as j, initialize j = 1, and set the total number of models to J;

[0047] The trained unsupervised defect detection model is used as the j-th model; and the j-th model is used to segment the t-th image. Data uncertainty result σ t,j and the j-th model for the t-th segmented image Predicted label images

[0048] Step 5.2: Train the j-th model according to the process in Step 4 to obtain the (j+1)-th trained model; and calculate the segmentation of the (j+1)-th model on the t-th image. Data uncertainty result σ t,j+1 and predicted label images

[0049] Step 5.3: After assigning j+1 to j, check if j>J holds true. If true, save the J trained models and their segmentation of the t-th image. Data uncertainty results {σ t,1 ,σ t,2 ,...,σ t,j ,...,σ t,J} and calculate the average value of the label images output by the J models. Otherwise, return to step 5.2;

[0050] Step 5.4: Calculate the discriminative network θ using equation (7). D For the segmented image of the t-th image The total uncertainty result σ t ;

[0051]

[0052] In equation (7), σ t,a Represents the segmented image of the t-th image. The model uncertainty, σ D Let represent the uncertainty of the data, and we have:

[0053]

[0054]

[0055] Step 5.5: Construct the uncertainty loss function L using equation (10). uc :

[0056]

[0057] Step 6: Construct the total loss function L using equation (11) all (I t ,I r,t M a,t M t ):

[0058] L all (I t ,I r,t M a,t M t ) = L rec (It ,I r,t )+L seg (M a,t M t )+L uc (M a,t M t (11)

[0059] Step 7: Train J models using gradient descent and calculate the total loss function L. all (I t ,I r,t M a,t M t To update the network parameters until the total loss function L is reached. all (I t ,I r,t M a,t M t The training continues until the model converges or reaches the maximum number of training iterations, thus obtaining an evaluation model for PCB defect detection and enabling the evaluation of PCB defect detection results.

[0060] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the method, and the processor is configured to execute the program stored in the memory.

[0061] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program is executed by a processor to perform the steps of the method.

[0062] Compared with existing technologies, the beneficial effects of this invention are reflected in:

[0063] 1. This invention proposes an unsupervised learning method for detecting minute PCB defects. It eliminates the need for manual image annotation and processing, and simulates the generation of real defects by adding multi-scale noise, thus solving the problems of insufficient sample size and sample imbalance in the field of PCB detection.

[0064] 2. This invention introduces uncertainty as an automatic evaluation index for the detection results in the PCB defect detection model, and provides a quantitative description of the reliability of the defect detection results; thereby improving the robustness of the model.

[0065] 3. This invention improves upon the traditional repair subnetwork and discrimination subnetwork by autonomously adding noise to simulate the generation of real defects and cascading the two networks, thereby improving detection accuracy and training efficiency.

[0066] 4. Compared to supervised learning methods, this invention does not require labeling a large number of chip defect samples; only a small number of defect-free samples are needed as the network training set to achieve the detection of multiple categories of chip defects. Its detection accuracy is also significantly improved compared to existing unsupervised chip defect detection methods. Attached Figure Description

[0067] Figure 1 This is a flowchart of the method of the present invention;

[0068] Figure 2 This is a structural diagram of the model of the present invention;

[0069] Figure 3 This is a flowchart of the simulated anomaly device of the present invention;

[0070] Figure 4 This is an image used for uncertainty evaluation in this invention. Detailed Implementation

[0071] In this embodiment, as Figure 1 As shown, a chip defect detection method based on unsupervised learning includes the following steps:

[0072] Step 1: Preprocessing of training data:

[0073] Obtain a dataset of defective PCB images in, This represents the PCB image of the t-th defect type, and the t-th defect type is denoted as e. t Thus, the real defect set F = {e1, e2, ..., e} is composed of all defect types. t ,...e T}; T represents the number of images;

[0074] This invention uses the chip defect dataset from the Human-Computer Interaction Open Laboratory of Peking University as a prototype to generate a chip defect dataset, including 693 images, with defect types divided into 6 categories. The chip defect dataset is divided into an average of 256×256 pixels. In this experiment, the defect dataset I is obtained through data augmentation by segmentation. D It contains 2278 images, including T types of defects.

[0075] Step 2: Construct an unsupervised defect detection network, including: a simulated anomaly Φ and a restorative subnet θ. R Discrimination subnet θ D Surface anomaly detector ξ, such as Figure 2 As shown;

[0076] Repair subnet θ R and the discriminative subnet θ D Both are encoder-decoder structures;

[0077] Repair subnet θ R The encoder in the code consists of C1 convolutional layers with kernels of a1×a1 and a max pooling layer of b1×b1.

[0078] Repair subnet θ R The decoder in the code consists of C2 convolutional layers with kernels of a2×a2 and one max pooling layer of b2×b2.

[0079] Discrimination subnet θ D The encoder in the code consists of C3 convolutional layers with kernels of a3×a3 and a max pooling layer of b3×b3.

[0080] Discrimination subnet θ D The decoder in the code consists of C4 convolutional layers with kernels of a4×a4 and one max pooling layer of b4×b4.

[0081] The encoder and decoder are connected via jump links;

[0082] Step 3: Training the unsupervised defect detection network:

[0083] Step 3.1: The analog anomaly generator Φ generates analog noise, such as... Figure 3 As shown:

[0084] Step 3.1.1: Randomly select several types of defects from the real defect set F and overlay them onto the t-th defect-free PCB board image I. t Above, an image M containing several types of defects is formed. t ;

[0085] Step 3.1.2, for M t After binarization, the defect mask M is obtained. a,t ;

[0086] Step 3.1.3: Apply the defect mask M a,t Invert the pixel value and then use it with I t Perform dot product, and further generate I using equation (1). a,t :

[0087]

[0088] In equation (1), · is equivalent to ⊙. It is M a,t The pixel is inverted, and ⊙ is the element-wise multiplication operation.

[0089] The above describes the process of simulating anomaly generation. In this method, the generated training set includes original images without anomalies and images with real anomalies and defects. Comparing the PCB defect image P generated by Burmester noise, it can be clearly observed that the artificially generated PCB defect proposed in this invention is closer to the real defect.

[0090] Step 3.2, Repair subnet θ R Reconstruct defect-free images;

[0091] Step 3.2.1, Repair subnet θ R to I a,t Perform reconstruction processing and output the t-th reconstructed defect-free PCB image I. r,t The instigation subnet uses the U-Net network, which is an encoder-decoder structure. By introducing skip connections, it can simultaneously utilize image feature information from different levels, allowing the network to better restore edges and subtle structures, thereby improving the accuracy of reconstruction.

[0092] Step 3.2.2, based on I r,t with I t Using equation (2), a restorative subnet θ is constructed. R Mean square error loss L MSE (I t ,I r,t ):

[0093]

[0094] Step 3.2.3, based on I r,t with I t Using equation (3), a restorative subnet θ is constructed. R Structural similarity loss function L SSIM (I t ,I r,t ):

[0095]

[0096] In equation (3), μ and μ r They are I t and I r,t The weighted mean of I, σ is t The variance, σ r isI r,t The variance, σ I,Ir isI t and I r,t The covariance; C1 and C2 represent two constants;

[0097] Step 3.2.4: Construct the restorative subnet θ using equation (4). R Total loss function Lrec (I t ,I r,t ):

[0098] L rec (I t ,I r,t )=λL SSIM (I t ,I r,t )+L MSE (I t ,I r,t (4)

[0099] In equation (4), λ is a hyperparameter that adjusts the loss balance;

[0100] Step 3.3, Discrimination subnet θ D Processing:

[0101] Step 3.3.1: The discriminative subnet has one output channel m1. Add another output channel m2 to the network structure of the discriminative subnet. (The last part, "I," appears to be an unrelated instruction and is left as is.) r,t with I a,t After subtraction, the result is fed into the discriminative subnet θ. D Defect segmentation is performed, and the m1 output channel outputs the t-th segmented image. The uncertainty result σ of the output data from the m2 output channel.

[0102] Step 3.4: The surface anomaly detector ξ utilizes a mean-filtered convolutional layer to... Smoothing is performed to aggregate local anomaly information, thereby obtaining the t-th image-level anomaly score η using equation (5). t :

[0103]

[0104] In equation (5), f sf×sf This represents a mean filter with dimensions sf×sf, and * represents the convolution operator.

[0105] Step 3.5, when the abnormal score η t If the pixel-level threshold is exceeded, then the t-th original defect-free PCB image I will be removed. t The location information of the anomaly generated by the simulation is binarized.

[0106] Step 3.6: Construct the segmentation loss function L using equation (6). seg (M a,t M t ):

[0107] L seg (M a,t M t)=-α(1-p t ) γ log(p t (6)

[0108] Where, p t α is the predicted probability of the model, α is the weight of each class, and γ is an adjustable parameter used to control the impact of low-probability predictions.

[0109] Step 3.7: Construct the initial training loss function L1(I) using equation (7). t ,I r,t M a,t M t ):

[0110] L1(I t ,I r,t M a,t M t ) = L rec (I t ,I r,t )+L seg (M a,t M t (7)

[0111] Step 4: Train the unsupervised defect detection network using gradient descent and calculate the initial training loss function L1(I t ,I r,t M a,t M t To update the network parameters until the initial training loss function L1(I) is reached. t ,I r,t M a,t M t The training continues until the model converges or reaches the maximum number of training iterations, thus obtaining the trained unsupervised defect detection model.

[0112] Step 5: Calculate the uncertainty result σ;

[0113] Step 5.1: Define the current model index as j and initialize j = 1; set the total number of models to J; use the trained unsupervised defect detection model as the j-th model; and calculate the segmentation effect of the j-th model on the t-th image. Data uncertainty result σ t,j and the j-th model for the t-th segmented image Predicted label images

[0114] Step 5.2: Train the j-th model according to the process in step 4 to obtain the (j+1)-th trained model and its t-th segmented image. Data uncertainty result σt,j+1 ;

[0115] Step 5.3: After assigning j+1 to j, check if j>J holds true. If true, save the J trained models and their t-th segmented image. Data uncertainty results {σ t,1 ,σ t,2 ,...,σ t,j ,...,σ t,J} and calculate the average value of the segmented images output by the J models. Otherwise, return to step 5.2.

[0116] Step 5.4: Calculate the total uncertainty result σ;

[0117] Using equation (8), calculate J models for the segmentation of the t-th image. Model uncertainty σ t,a :

[0118]

[0119] Construct the discriminative subnetwork θ using equation (9) D For the segmented image of the t-th image Total uncertainty σ t :

[0120]

[0121] Among them, the data uncertainty σ D As shown in equation (10):

[0122]

[0123] Step 5.5: Construct the uncertainty loss function L using equation (11). uc (M a,t M t ):

[0124]

[0125] Step 6: Combining equations (10) and (11), use equation (12) to construct the total loss function for the j-th training iteration obtained after model fitting in step 4:

[0126] L all (I t ,I r,t M a,t M t ) = L rec (I t ,I r,t )+L seg (Ma,t M t )+L uc (M a,t M t (12)

[0127] Step 7: Train J models using gradient descent and calculate the total loss function L. all (I t ,I r,t M a,t M t To update the network parameters until the total loss function L is reached. all (I t ,I r,t M a,t M t The training continues until the model converges or reaches the maximum number of training iterations, thus obtaining an evaluation model for PCB defect detection and enabling the evaluation of PCB defect detection results.

[0128] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0129] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0130] Experimental evaluation of the method proposed in this invention:

[0131] This experiment extensively evaluates the proposed method on publicly available chip defect datasets and compares it with typical fully supervised learning models and the unsupervised DRAEM model. The experiments demonstrate that the proposed method outperforms current mainstream fully supervised defect detection methods and unsupervised DRAEM detection methods.

[0132] In the experiments conducted, the chip defect dataset was trained for 1000 epochs. The bitchsize was set to 5, and the learning rate to 0.0001. After 1000 epochs, the total loss approached 0, indicating model fit. To mitigate overfitting due to the small training data size, the training images were randomly rotated within a range of (-45, 45 degrees) to achieve data augmentation. This invention uses the PyTorch framework to implement our network, and related experiments were conducted on a workstation with a main configuration including an NVIDIA GeForce RTX 3080ti GPU, a 10900X CPU, and 128GB of RAM.

[0133] 1. Evaluation Indicators

[0134] To evaluate the experimental results, this invention uses AP and mAP as evaluation metrics. AP represents the shaded area enclosed by precision-recall curves (PRCs) and is used to evaluate the local performance of the learning model for each category. In this invention, AP is used to evaluate the average detection accuracy for each type of PCB defect. AP is suitable for highly imbalanced categories and plays an important role in evaluating PCB defect detection accuracy. mAP is the arithmetic mean of AP values ​​for all PCB defect categories and is used to evaluate the overall performance of the proposed method for all categories. The formulas for calculating AP and mAP are as follows:

[0135] (1) Precision and Recall:

[0136]

[0137]

[0138] True positives (TP) represent the number of correct detections for each defect type, false positives (FP) represent the number of incorrect detections for each defect type, and false negatives (FP) represent the number of unidentifiable detections for each defect type.

[0139] (2) Average precision of a single category and average precision of the whole:

[0140]

[0141]

[0142] Here, mean precision (AP) represents the shaded area enclosed by the precision-recall curves (PRCs), and mean average precision (mAP) is the arithmetic mean of the AP values ​​for each class. C is the number of defective classes in the test set. AP measures the local performance of the learning model for each class, while mAP measures the overall performance of the learning model across all classes.

[0143] 2. Evaluate the test results of this invention in conjunction with the following charts:

[0144] To demonstrate the performance improvement of the proposed unsupervised learning model compared to existing supervised methods and other unsupervised methods, this invention first presents the comparison results on a chip defect dataset as shown in Table 1.

[0145] Table 1 Comparison of mAP results obtained from the defect dataset of the test chip under different methods.

[0146]

[0147]

[0148] As shown in Table 1, the proposed method, as an unsupervised learning model, outperforms the current mainstream fully supervised learning models and the unsupervised model DRAEM on the chip defect dataset in terms of average accuracy (AP). Specifically, the proposed method achieves an average AP of 99.29%, with average accuracy exceeding 99% for all four defect types ("missing vias", "short circuits", "burrs", and "excess copper"). This demonstrates that the unsupervised learning model and proposed method of this invention exhibit more advanced performance compared to other current mainstream fully supervised learning models. Compared to the unsupervised learning model DRAEM, the proposed method shows a 6.02% improvement in the mAP for PCB defect detection, with improvements across all categories.

[0149] Tables 2 and 3 present the experimental results of replacing the instigation subnet with a standard encoder-decoder structure and replacing the artificially generated defect method with Burmester noise based on the UDDNET model. The AP results are shown in Table 2, and the pixel-level AP and AUC results are shown in Table 3.

[0150] Table 2. Experimental results under image-level metrics.

[0151]

[0152] Table 3. Experimental results under pixel-level performance indicators.

[0153]

[0154]

[0155] Experimental results show that replacing the generator and the noise generation method significantly reduces the detection accuracy for all types of PCB defects. Specifically, the map pixel count after generator replacement and noise replacement is 24.5% and 18.48% lower than the method proposed in this invention, respectively. Uneven recognition accuracy across categories also occurs. The highest AP pixel after generator replacement is 95.2%, while the lowest is as low as 0.46%. The highest AP pixel after noise replacement is 95.27%, while the lowest is as low as 0.4%. This demonstrates that the restorative subnet and artificial defect generation method proposed in this invention are extremely necessary. Simulating the generation and reconstruction process of PCB defects plays a crucial role in improving the performance of UDDNET in detecting PCB defects.

[0156] Finally, this invention uses a chip defect dataset as an example to demonstrate the significance of uncertainty in the reference evaluation of defect detection results.

[0157] Based on RoughSet, suspicious areas are defined as uncertain areas, that is, "third areas" in addition to normal and abnormal areas.

[0158] Figure 4 The first line intersects multiple test results of the model to obtain the defect region, and then takes the union to obtain all regions that may be defective, including the defective region. Subtracting the intersection region from the union region gives the regions that may be defective, and the boundaries between the defective region and the possible defective region, as well as between the possible defective region and the non-defective region, are drawn. Figure 4 The second row shows an image segmented into defective, potentially defective, and defect-free regions by calculating their uncertainties. (Comparison) Figure 4 The results in the first and second rows show an excellent similarity between the uncertainty image and the image in the second row, which uses intersection and union to determine the defined and uncertain regions. This invention directly displays the uncertainty, eliminating the need to evaluate the reliability of test results through intersection and union methods, thus greatly improving the efficiency of reliability evaluation for defect test results.

[0159] The experimental results show that, except for the second type of defect, all of the six types of defects are basically located in the defect area. That is, except for the second type of defect, all types of defects are identified as defects. It can be seen that the uncertainty of the detection results of the method proposed in this invention is very small.

[0160] Figure 4 The third row of the image simulates the uncertainty of each pixel, displaying the uncertainty at the pixel level. Each circle represents a pixel, indicating a pixel definitely without defects; the area represented by a circle is a region definitely without defects. Stars represent pixels definitely with defects, and the area represented by star-shaped pixels is a region definitely with defects. Diamonds represent anomalous pixels, i.e., pixels whose defect status is uncertain, and the area represented by diamond-shaped pixels is a region of uncertainty. Each pixel and region corresponds to the image pixels and regions in the second row, respectively. Figure 4 The images shown in the paper demonstrate that uncertainty can be used to more accurately evaluate the reliability of a model from a pixel-level perspective.

[0161] Figure 4 In the experimental results, only (b) rat bites showed a large potential defect area, while other types of defects were basically confirmed defect areas. This result shows that the detection results of the method proposed in this invention have extremely high reliability.

Claims

1. A chip defect detection method based on unsupervised learning, characterized in that, Includes the following steps: Step 1: Preprocessing of training data: Obtain a dataset of defective PCB images in, This represents the PCB image of the t-th defect type, and the t-th defect type is denoted as e. t Thus, the real defect set F = {e1, e2, ..., e} is composed of all defect types. t ,...e T }; T represents the number of images; Obtain a defect-free PCB image dataset I = {I1, I2, ..., I...} t ,...,I T }; where I t This represents the t-th defect-free PCB image; Step 2: Construct an unsupervised defect detection network, including: a simulated anomaly Φ and a repairable subnet θ. R Discrimination subnet θ D Surface anomaly detector ξ; The restorative subnet θ R and the discriminative subnet θ D Both are encoder-decoder structures; The restorative subnet θ R The encoder in the code consists of C1 convolutional layers with kernel size a1×a1 and a max pooling layer with dimension b1×b1. The restorative subnet θ R The decoder in the code consists of C2 convolutional layers with kernel size a2×a2 and a max pooling layer with dimension b2×b2. The discriminative subnet θ D The encoder in the code consists of C3 convolutional layers with kernel size a3×a3 and a max pooling layer with dimension b3×b3. The discriminative subnet θ D The decoder in the code consists of C4 convolutional layers with kernel size a4×a4 and a max pooling layer with dimension b4×b4. The encoder and decoder are connected via a jump link; Step 3: Training the unsupervised defect detection network: Step 3.1, Processing of the simulated anomaly Φ: Step 3.1.1: Randomly select several types of defects from the real defect set F and overlay them onto the t-th defect-free PCB board image I. t Above, an image M containing several types of defects is formed. t ; Step 3.1.2, for M t After binarization, the defect mask M is obtained. a,t ; Step 3.1.3: Use equation (1) to generate the t-th PCB image I containing several types of defects. a,t : In equation (1), It is M a,t Inverting pixel values; Step 3.2, the restorative subnet θ R Reconstruct defect-free images; Step 3.2.1, the restorative subnet θ R to I a,t Perform reconstruction processing and output the t-th reconstructed defect-free PCB image I. r,t ; Step 3.2.2, based on I r,t with I t Constructing a restorative subnet θ R Mean square error loss L MSE (I t ,I r,t ); Step 3.2.3, based on I r,t with I t Using equation (2), a restorative subnet θ is constructed. R Structural similarity loss function L SSIM (I t ,I r,t ): In equation (2), μ and μ r They are I t and I r,t The weighted mean of I, σ is t The variance, σ r isI r,t variance isI t and I r,t The covariance; C1 and C2 represent two constants; Step 3.2.4: Construct the restorative subnet θ using equation (3). R Total loss function L rec (I a,t ,I r,t ): THE rec (THE t ,THE r,t )=λL SSIM (THE t ,THE r,t )+L MSE (THE t ,THE r,t ) (3) In equation (3), λ is a hyperparameter that adjusts the loss balance; Step 3.3, the discriminative subnet θ D The handling: The discriminative subnet θ D It contains two output channels m1 and m2, which will output I r,t with I a,t The result of the subtraction is fed into the discriminative subnet θ. D Defect segmentation is performed during the process, and the t-th segmented image is output through the first output channel m1. The second output channel m2 outputs the uncertainty result σ of the t-th data. t ; Step 3.4: The surface anomaly detector ξ uses a mean filter to... After smoothing, the image-level anomaly score η for the t-th image is obtained using equation (4). t : In equation (4), f sf×sf This represents a mean filter with dimensions sf×sf, where * represents the convolution operator; Step 3.5, when the abnormal score η t When the pixel-level threshold is exceeded, I a,t The abnormal location information present in the PCB is binarized to obtain the abnormal mask M0, which is then used to determine the location of the PCB defect. Step 3.6: Construct the segmentation loss function L using equation (5). seg (M a,t M t ): L seg (M a,t ,M t )=-α(1-p t ) γ log(p t ) (5) Where, p t α is the predicted probability of the model, α is the weight of each class, and γ is an adjustable parameter used to control the impact of low-probability predictions. Step 3.7: Construct the initial training loss function L1(I) using equation (6). t ,I r,t M a,t M t ): L1(I t ,I r,t ,M a,t ,M t )=L rec (I t ,I r,t )+L seg (M a,t ,M t ) (6) Step 4: Train the unsupervised defect detection network using gradient descent and calculate the total loss function L1(I t ,I r,t M a,t M t To update the network parameters until the initial training loss function L1(I) is reached. t ,I r,t M a,t M t The training continues until the model converges or reaches the maximum number of training iterations, thus obtaining a trained unsupervised defect detection model for detecting defects in PCB images.

2. An evaluation method for chip defect detection based on unsupervised learning, characterized in that, The unsupervised defect detection model trained according to claim 1 is evaluated according to the following steps: Step 5: Calculate the uncertainty result σ; Step 5.1: Define the current model index as j, initialize j = 1, and set the total number of models to J; The trained unsupervised defect detection model is used as the j-th model; and the j-th model is used to segment the t-th image. Data uncertainty result σ t,j and the j-th model for the t-th segmented image Predicted label images Step 5.2: Train the j-th model according to the process in Step 4 to obtain the (j+1)-th trained model; and calculate the segmentation of the (j+1)-th model on the t-th image. Data uncertainty result σ t,j+1 and predicted label images Step 5.3: After assigning j+1 to j, check if j>J holds true. If true, save the J trained models and their segmentation of the t-th image. Data uncertainty results {σ t,1 ,σ t,2 ,...,σ t,j ,...,σ t,J } and calculate the average value of the label images output by the J models. Otherwise, return to step 5.2; Step 5.4: Calculate the discriminative network θ using equation (7). D For the segmented image of the t-th image The total uncertainty result σ t ; In equation (7), σ t,a Represents the segmented image of the t-th image. The model uncertainty, σ D Let represent the uncertainty of the data, and we have: Step 5.5: Construct the uncertainty loss function L using equation (10). uc : Step 6: Construct the total loss function L using equation (11) all (I t ,I r,t M a,t M t ): L all (I t ,I r,t ,M a,t ,M t )=L rec (I t ,I r,t )+L seg (M a,t ,M t )+L uc (M a,t ,M t ) (11) Step 7: Train J models using gradient descent and calculate the total loss function L. all (I t ,I r,t M a,t M t To update the network parameters until the total loss function L is reached. all (I t ,I r,t M a,t M t The training continues until the model converges or reaches the maximum number of training iterations, thus obtaining an evaluation model for PCB defect detection and enabling the evaluation of PCB defect detection results.

3. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports a processor in executing the method of claim 1 or 2, the processor being configured to execute the program stored in the memory.

4. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of the method of claim 1 or 2.

Images