An intelligent detection method for appearance defects of a chip test carrier

Abstract

The present invention discloses an intelligent detection method for appearance defects of a chip test vehicle. The method includes: obtaining a plurality of images of the chip test vehicle, performing image segmentation based on deep learning to obtain a plurality of first images; extracting features from the first images and the spatial structure information of the chip test vehicle through a multi-modal feature extraction network to obtain first image features; the first image features are high-dimensional feature vectors that fuse the spatial structure information and image information of the chip test vehicle; performing defect detection on the first image features through a defect detection network and outputting a defect detection result; the defect detection result includes the defect position, defect type, and the probability of the defect type of the chip test vehicle. The whole process of the present invention from image acquisition to defect recognition is automated, avoiding the subjective errors of manual detection, improving consistency and efficiency, and reducing costs.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly relates to an intelligent detection method for appearance defects of a chip test carrier. Background Art

[0002] During the use of a chip test carrier (Test Tray), defects such as wear and aging may occur, which will directly affect the test results of the product. Therefore, the Test Tray needs to be regularly inspected for defects.

[0003] 1. At present, most of the appearance defect detections of Test Trays are carried out manually. Due to individual differences and other reasons, there are situations of missed detections and misdetections in manual detection, and the detection efficiency is relatively low.

[0004] 2. The existing traditional vision detection methods have limited detection accuracy for Test Trays. Although the detection speed is better than that of manual detection, the detection rate of traditional algorithms for irregular defects is not high, and there are still problems of missed detections and misdetections.

[0005] 3. Due to the large number of areas that need to be detected on the Test Tray and the complex images that need to be processed by the existing detection equipment, the time taken to finally detect one Test Tray is relatively long. Summary of the Invention

[0006] Technical Objective: Aiming at the defects in the prior art, the present invention discloses an intelligent detection method for appearance defects of a chip test carrier, which is fully automated from image acquisition to defect recognition, avoids the subjective errors of manual detection, improves consistency and efficiency, and reduces costs.

[0007] Technical Solution: To achieve the above technical objective, the present invention adopts the following technical solutions.

[0008] An intelligent detection method for appearance defects of a chip test carrier includes the following steps:

[0009] S1. Obtain a plurality of images of the chip test carrier, and perform image segmentation based on deep learning to obtain a plurality of first images;

[0010] S2. Extract features from the first images and the spatial structure information of the chip test carrier through a multi-modal feature extraction network to obtain first image features; the first image features are high-dimensional feature vectors that fuse the spatial structure information and image information of the chip test carrier;

[0011] S3. Perform defect detection on the first image features through a defect detection network, and output defect detection results; the defect detection results include the defect positions, defect types, and probabilities of the defect types of the chip test carrier.

[0012] Advantageous Effects:

[0013] 1. The present invention combines deep learning with a multi-modal feature extraction network, integrating the spatial structure and image information of the chip test vehicle, enabling the appearance defect detection method of the present invention to accurately identify complex and irregular defects, significantly reducing the cases of missed detection and false detection.

[0014] 2. The defect detection network of the present invention adopts a cascaded detection network and a lightweight neural network, greatly improving the detection speed, and optimizing the calculation through a feature reuse mechanism, reducing unnecessary computational redundancy.

[0015] 3. The present invention performs image segmentation based on deep learning, can automatically adjust the detection strategy according to the characteristics of different chip test vehicles, has wide applicability, and adapts to different test scenarios and models.

[0016] 4. The whole process of the present invention from image acquisition to defect recognition is automated, avoiding the subjective errors of manual detection, improving consistency and efficiency, and reducing costs.

[0017] 5. The present invention can not only identify the defect type, but also accurately locate the defect position, and perform refined classification according to the hierarchical structure to ensure higher diagnostic accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of the method of the present invention;

[0019] Figure 2 is a schematic diagram of simulation verification of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] In order to enable those skilled in the art of the present technology to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0021] As shown in the Figure 1 accompanying drawings, an intelligent appearance defect detection method for a chip test vehicle of the present invention includes the following steps:

[0022] S1. Obtain several images of the chip test carrier, perform image segmentation based on deep learning, and obtain several first images. In view of the feature differences of different models of Test Tray, the present invention designs an adaptive image segmentation algorithm based on deep learning. This algorithm first uses an improved U-Net network to perform semantic segmentation on the overall structure of the Test Tray to identify each functional area. Then, it dynamically generates regions of interest (ROIs) according to the recognition results, and applies specific image enhancement and denoising techniques to each ROI. For example, for highly reflective areas, adaptive histogram equalization is used; for shadow areas, a shadow removal algorithm based on guided filtering is applied. This method significantly improves the pertinence and accuracy of subsequent defect detection.

[0023] S2. Extract features from the first image and the spatial structure information of the chip test carrier through a multi-modal feature extraction network to obtain first image features; the first image features are high-dimensional feature vectors that fuse the spatial structure information and image information of the chip test carrier;

[0024] In the multi-modal feature extraction network, the 3D CAD model of the chip test carrier is fused with the first image to extract features and output first image features; it includes the following steps:

[0025] S21. Construct a multi-modal feature extraction network; the multi-modal feature extraction network includes a CNN network for extracting features from the first image, a GCN network for extracting features from the spatial structure information of the chip test carrier, and a feature fusion module for fusing the first image features and spatial structure features;

[0026] The CNN network for extracting features from the first image uses an existing CNN network, with the first image as the input and image features as the output. The image features mainly represent the surface texture, illumination, and reflection information of the chip test carrier.

[0027] In the GCN network for extracting features from the spatial structure information of the chip test carrier, the 3D CAD model of the chip test carrier is the input and spatial structure features are the output. The spatial structure features capture the structural and spatial layout of the chip test carrier.

[0028] The construction of the GCN network includes the following steps:

[0029] S211. Convert the 3D CAD model of the chip test carrier into several 2D projection maps through multi-view projection for alignment with the first image. The angles of the projection maps are consistent with the viewing angle of the first image to ensure that the geometric information is distortion-free. The multi-view projection conversion process includes:

[0030] S2111. Convert the 3D CAD model of the chip test vehicle into a 2D projection diagram, i.e., M_2D = Proj(M_3D); where M_2D is the 2D projection diagram converted from M_3D, retaining the geometric information of the original 3D model but reduced to a 2D space, M_3D is the 3D CAD model of the chip test vehicle, which contains complete 3D structure information, and Proj is the projection operation symbol, indicating that the 3D model M_3D is converted into a 2D planar image through orthogonal projection;

[0031] S2112. Perform image registration on the projection diagram and the first image through affine transformation. By detecting and matching key points, correct the geometric differences between the first image and the projection diagram to align them spatially and ensure the consistency of feature extraction. The calculation formula of the affine transformation matrix includes:

[0032] T = arg min||I - T(M_2D)|| 2

[0033] where T is the affine transformation matrix, T(M_2D) is the image of M_2D processed by the affine transformation matrix, and I is the first image;

[0034] S212. Construct a graph convolutional network (GCN) to process the 3D CAD model of the chip test vehicle; including the following steps:

[0035] S2121. Construct a graph G, G = (V, E), where V is the key component, i.e., the physical components in the 3D CAD model of the chip test vehicle, and E is the component relationship, i.e., the positional connection relationship or functional connection relationship between physical components;

[0036] S2122. Construct a graph convolutional network GCN according to graph G. The update rule of GCN is:

[0037]

[0038] where H I+1 is the node feature matrix of the (I + 1)-th layer, σ is the activation function, D is the degree matrix, is the adjacency matrix with self-loops added, A is the adjacency matrix, H I is the node feature matrix of the I-th layer, W I is the weight matrix of the I-th layer, which is the trainable parameter of the graph convolutional network GCN;

[0039] A feature fusion module for fusing image features and spatial structure features, with the input being image features and spatial structure features and the output being the first image feature F_fused. Its calculation formula is:

[0040] F_fused = Concat[F_CNN, F_GCN]

[0041] Wherein, F_CNN = CNN(I), representing the image features output by the CNN network; F_GCN = GCN(G), representing the spatial structure features output by the GCN network.

[0042] S22. Train and optimize the multi-modal feature extraction network; First, pre-train the CNN network and the GCN network:

[0043] Pre-training of the CNN network: Pre-train the CNN network on a large-scale general image dataset to learn general image features such as edges, textures, and shapes. Through transfer learning, apply the pre-trained CNN to the images of the chip test vehicle to achieve fast convergence.

[0044] Pre-training of the GCN network: The GCN network is pre-trained on a graph dataset similar to the structure of the chip test vehicle. This process helps the network learn how to propagate and aggregate node information in the graph structure, especially for capturing spatial relationships.

[0045] Secondly, on the basis of pre-training, combine the CNN network and the GCN network and fine-tune them for the specific chip test vehicle data. By minimizing the loss function of the multi-modal features, learn the optimal feature fusion method. Design the loss function, which includes not only the classification loss of the 2D image, that is, the first image, but also the graph convolution loss for the 3D CAD model.

[0046] Thirdly, adopt a staged training method. First, train the respective networks of the CNN and the GCN so that they can stably extract 2D and 3D features. Then, after fusing the two, conduct joint training on the multi-modal feature extraction network. In the initial stage of training, freeze some weights of the CNN or the GCN to prevent the multi-modal feature extraction network from overfitting during the fine-tuning stage. At the same time, during the fusion training process of the multi-modal feature extraction network, gradually unfreeze the weights of each layer for overall optimization.

[0047] Use a learning rate scheduler during the training process to ensure that the multi-modal feature extraction network can converge more smoothly in the later stage of training. To prevent the multi-modal feature extraction network from overfitting, use the L2 regularization technique, especially in the CNN part, to prevent the CNN network from relying too much on specific image features. When training a deep network, use gradient clipping to avoid the problem of gradient explosion and ensure that the gradients are updated stably during training.

[0048] Finally, the multi-modal feature extraction network is evaluated using the validation set. By monitoring performance metrics such as accuracy and recall in the actual defect detection task of the chip test vehicle, the parameters of the multi-modal feature extraction network are adjusted. After the initial training is completed, the CNN and GCN are fine-tuned to adapt to the specific defect detection task, ensuring the effectiveness of the multi-modal feature extraction network for the actual application scenario. In the final deployment stage, model compression technology is used to optimize the trained multi-modal feature extraction network, reducing the model size and improving the inference speed to quickly perform defect detection tasks in the actual test environment.

[0049] S3. Perform defect detection on the first image feature through the defect detection network and output the defect detection result; the defect detection result includes: 1. The defect type and the probability of the defect type, identifying the specific appearance defect type of the chip test vehicle and the probability value for each appearance defect type; 2. The defect location, outputting the location information of the defect. Specifically, it includes the following steps:

[0050] S31. Construct a defect detection network. For the appearance defects of the chip test vehicle, construct a defect hierarchy. Based on the defect hierarchy, construct a cascade detection defect detection network and introduce a feature reuse and soft decision mechanism into the defect detection network;

[0051] First, based on the physical characteristics and visual similarities of defects, the defects of the Test Tray are defined as the following 14 types of defect categories: Insert buckle fracture, abnormal opening and closing angle of Insert buckle, missing Insert buckle, damaged Insert buckle, Insert bottom net detachment, damaged Insert bottom net, damaged Insert main body, deformed Insert main body, damaged appearance of HiFix socket, damaged conductive adhesive of HiFix socket, dirt adhesion on HiFix socket, fixing screw of HiFix socket, missing Pin of HiFix socket, damaged back connector of HiFix; the 14 types of defects are organized into a three-layer defect hierarchy. The top layer area is the main defect type, including structural defects and surface defects. The middle layer is further divided into more specific defect categories. Structural defects include Insert-related defects and HiFix-related defects. Insert-related defects correspond to buckle problems, bottom net problems, and main body problems. HiFix-related defects correspond to socket problems, connector problems, and component problems. Surface defects include appearance defects. The bottom layer is the final 14 types of specific defects. Among the Insert-related defects, the buckle problems include Insert buckle fracture, abnormal opening and closing angle of Insert buckle, missing Insert buckle, damaged Insert buckle. The bottom net problems include Insert bottom net detachment, damaged Insert bottom net. The main body problems include damaged Insert main body, deformed Insert main body. Among the HiFix-related defects, the socket problems include damaged appearance of HiFix socket, damaged conductive adhesive of HiFix socket. The connector problems include damaged back connector of HiFix. The component problems include fixing screw of HiFix socket, missing Pin of HiFix socket;

[0052] This structure allows the model to learn defect features at different levels, improving the accuracy and robustness of classification.

[0053] Based on the defect hierarchy, the defect detection network of the present invention is designed as a cascaded detection defect detection network. The cascaded detection defect detection network includes detectors in three stages, and each stage corresponds to a level of the classification system. The first stage quickly identifies the main defect types. The second stage performs more specific defect categories within the selected categories. The last stage performs the defect classification at the bottom layer. Each stage uses a specially trained lightweight network, greatly improving the detection efficiency. At the same time, cross-stage feature reuse and soft decision-making mechanisms are introduced to ensure high accuracy while maintaining computational efficiency.

[0054] The defect detection network is based on the defect hierarchy. Let x be the input image feature and y i be the output of the i-th stage, where i = 1, 2, 3, and its calculation formula is:

[0055] The first stage: quickly identify the main defect types:

[0056] P(y1|x), where y1 ∈ {structural defects, surface defects}

[0057] Among them, P(y1|x) represents the probability that the main defect type is y1 given the input feature x;

[0058] The first stage is designed as a lightweight convolutional neural network (CNN) for quickly identifying the main defect types of the chip test vehicle. The input is the first image feature, that is, the input feature x, and the output is P(y1|x), which is the preliminary prediction result of the main defect category, represented as the category label of the main defect type. It is used to distinguish large defect categories through quick classification, facilitating more detailed detection in the subsequent stages.

[0059] The second stage: identify more specific defect categories among the selected main defect types, that is, the middle-layer defect types:

[0060] P(y2|x, y1), where y2 ∈ {Insert-related defects, HiFix-related defects, appearance defects}

[0061] Among them, P(y2|x, y1) represents the probability that the main defect type is y1 and the more specific defect category is y2 given the input feature x;

[0062] The second stage is designed as a more complex classification network, still a lightweight convolutional network, but with more levels of convolution and pooling operations added to handle more detailed information. The input is the feature subset of the main defect category output by the first stage. The output is the prediction result of the more specific defect category, used to accurately classify to more specific defect categories through step-by-step refinement, providing a basis for the accurate classification in the third stage.

[0063] The third stage: identify the final 14 specific defect types at the bottom layer:

[0064] P(y3|x, y1, y2), where y3 ∈ {14 specific defect types}

[0065] Among them, P(y3|x, y1, y2) represents the probability that the main defect type is y1, the more specific defect category is y2, and the bottom-layer defect type is y3 given the input feature x;

[0066] The result output by the defect detection network, that is, the final classification probability, is calculated by the Bayesian rule, so there is:

[0067] P(y3|x) = P(y3|x, y1, y2) * P(y2|x, y1) * P(y1|x)

[0068] Among them, P(y3|x) represents the result output by the defect detection network, that is, the probability of the underlying defect type being y3 when the input feature x is given; this probability comprehensively considers the classifications at all levels and directly predicts the specific defect category.

[0069] The third stage is designed as a refined detection network after cascading, a fully convolutional network with more feature maps; the input is the detailed category prediction result output by the second stage and the regional features in the image. This stage combines specific regions and focuses on the initially identified defect regions; the output is the final 14 specific defect categories at the bottom layer, and the specific defect positions are marked. In addition, this stage may also output the defect confidence, that is, the probability of the defect type.

[0070] Objective: To provide the final detailed classification and accurate position detection, ensuring that the system outputs the specific defect type and its position in the image.

[0071] To improve efficiency and accuracy, the present invention also introduces a feature reuse and soft decision mechanism in the defect detection network:

[0072] For feature reuse, feature extraction functions are defined for the three stages respectively: f1(x), f2(x), f3(x); the feature extraction functions are implemented through a feature extraction network. Each feature extraction function can be represented as a neural network module, which consists of multiple convolutional layers or other deep learning modules and is specifically used to extract specific features and can be used as the input processing module for each stage in the defect detection network; the calculation formula for feature reuse is:

[0073]

[0074] Among them, g2 and g3 are lightweight feature transformation networks respectively, which are composed of shallow convolutional layers, depthwise separable convolutions, bottleneck layers, etc., and can maintain a low computational cost in different feature transformation stages. Before the classification task in each stage, lightweight feature transformation networks g2 and g3 are added, so that the features output by the previous stage can be effectively compressed or transformed to provide an efficient input for the classification task in the next stage. Through the feature sharing mechanism, g2 and g3 can reuse some of the features extracted in the previous stage, making the feature transfer between different stages of the network more efficient and avoiding unnecessary repeated calculations.

[0075] For the soft decision mechanism, a first threshold τ1 and a second threshold τ2 are introduced, and the judgment process of the soft decision mechanism is as follows:

[0076]

[0077] S32. Train and optimize the defect detection network;

[0078] First, pre-train the defect detection network. Pre-train the underlying feature extraction network on a large-scale general dataset, that is, the CNN network and GCN network in the multi-modal feature extraction network, and then fine-tune it using domain-specific data.

[0079] Secondly, perform progressive training on the defect detection network:

[0080] 1. First, train the first-stage network; corresponding to the feature extraction function f1(x), mainly dealing with the preliminary classification task;

[0081] 2. Fix the parameters of the first stage and train the second-stage network, corresponding to the feature extraction function f2(x), dealing with more fine-grained defect classification;

[0082] 3. Finally, fine-tune the entire defect detection network.

[0083] Thirdly, perform knowledge distillation on the defect detection network; use the ensemble model as the teacher network and introduce the distillation loss, where P teacher is the teacher model, P student is the student model, and the calculation formula for knowledge distillation is:

[0084] L_dist = KL(P teacher (y|x)||P student (y|x))

[0085] where L_dist is the knowledge distillation loss, KL is the Kullback-Leibler divergence, used to measure the difference between two probability distributions, P teacher (y|x) is the probability distribution of the teacher model predicting the class y given the input x, and P student (y|x) is the probability distribution of the student model predicting the class y given the input x;

[0086] Finally, optimize the defect detection network through multi-task learning;

[0087] Construct a multi-task loss function to optimize all stages of the defect detection network. The calculation formula for the multi-task loss function is:

[0088]

[0089] where L is the total loss function, α1, α2, and α3 are the balance factors for the first stage, second stage, and third stage respectively, y1, are the true labels of the first stage and the predicted outputs of the first-stage network respectively, y2, are the true labels of the second stage and the predicted outputs of the second-stage network respectively, y3, They are respectively the true label in the third stage and the predicted output of the network in the third stage. L1, L2, and L3 are respectively the cross-entropy loss functions in the first stage, the second stage, and the third stage. λ is the regularization coefficient, θ is the parameter of the model, and R(θ) is the regularization term;

[0090] S33. Perform defect detection on the first image feature according to the trained and optimized defect detection network, and output the defect detection result;

[0091] As Figure 2 shown, compared with the existing deep learning methods and traditional visual detections, the method of the present invention has higher accuracy values in multi-modal feature extraction, hierarchical cascade detection, feature fusion optimization, and final high-precision detection, that is, the method of the present invention has better effects in practical applications.

[0092] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for intelligently detecting appearance defects of a chip test carrier, characterized in that: The following steps are involved: S1. Acquire several images of a chip test carrier, perform image segmentation based on deep learning, and obtain several first images; S2, extracting features from the first image and the spatial structure information of the chip test carrier through a multimodal feature extraction network to obtain a first image feature; the first image feature is a high-dimensional feature vector that integrates the spatial structure information and image information of the chip test carrier; the multimodal feature extraction network in S2 includes a CNN network for extracting features from the first image, a GCN network for extracting features from the spatial structure information of the chip test carrier, and a feature fusion module for fusing the first image feature and the spatial structure feature; S3, performing defect detection on the first image feature through a defect detection network, and outputting a defect detection result; the defect detection result includes a defect position, a defect type, and a probability of the defect type of the chip test carrier; The defect detection network is a cascade detection defect detection network, which includes three stages of detectors. The first stage quickly identifies the main defect types, the second stage identifies more specific defect categories in the selected categories, and the last stage performs the underlying defect classification; each stage uses a lightweight network.

2. The method for intelligently detecting appearance defects of a chip test carrier according to claim 1, characterized in that: The CNN network for extracting features from the first image adopts an existing CNN network, with the first image as input and image features as output; In the GCN network that extracts the spatial structure information of the chip test carrier, the input is the 3D CAD model of the chip test carrier, and the output is the spatial structure feature; The feature fusion module performs feature fusion on the image feature and the spatial structure feature, and its input is the image feature and the spatial structure feature, and its output is the first image feature.

3. The method for intelligently detecting appearance defects of a chip test carrier according to claim 2, characterized in that: The construction of the GCN network includes the following steps: Converting the 3D CAD model of the chip test carrier into a plurality of 2D projection images through multi-view projection so as to align with the first image; Construct a GCN network to process the 3D CAD model of the chip test carrier; in the GCN network, first construct a graph G, G = (V, E), where V is the key component, that is, the physical component in the 3D CAD model of the chip test carrier, and E is the component relationship, that is, the positional connection relationship or functional connection relationship between the physical components; then construct a graph convolutional network GCN based on the graph G.

4. The method for intelligently detecting appearance defects of a chip test carrier according to claim 3, characterized in that: The update rule of GCN is: Among them, H I+1 is the node feature matrix of the I+1th layer, σ is the activation function, D is the degree matrix, For the adjacency matrix with self-loops added, A is the adjacency matrix, H I is the node feature matrix of layer I, W I is the weight matrix of the Ith layer.

5. The method for intelligently detecting appearance defects of a chip test carrier according to claim 1, characterized in that: S3 includes the following steps: S31. Build a defect detection network, build a defect hierarchy structure for the appearance defects of the chip test carrier, and build a defect detection network for cascade detection based on the defect hierarchy structure; S32, training and optimizing defect detection network; S33. Perform defect detection on the first image feature according to the trained and optimized defect detection network, and output the defect detection result.

6. The method for intelligently detecting appearance defects of a chip test carrier according to claim 5, characterized in that: The defect hierarchy includes: based on the physical characteristics and visual similarity of the defects, the defects of the Test Tray are defined as 14 types of defects: Insert buckle breakage, Insert buckle opening and closing angle abnormality, Insert buckle missing, Insert buckle damage, Insert bottom net detachment, Insert bottom net damage, Insert body damage, Insert body deformation, HiFix socket appearance damage, HiFix socket conductive glue damage, HiFix socket dirt attachment, HiFix socket fixing screws, HiFix socket Pin missing, HiFix back connector damage; the 14 types of defects are organized into a three-layer defect hierarchy; the top layer area is the main defect type, including structural defects and surface defects, and the middle layer is subdivided into more specific Defect categories, structural defects include Insert-related defects and HiFix-related defects. Insert-related defects correspond to buckle problems, bottom net problems and main body problems. HiFix-related defects correspond to socket problems, connector problems and component problems. Surface defects include appearance defects; the bottom layer is the final 14 types of specific defects; among Insert-related defects, buckle problems include Insert buckle breakage, Insert buckle opening and closing angle abnormality, Insert buckle missing, Insert buckle damage, bottom net problems include Insert bottom net detachment, Insert bottom net damage, main body problems include Insert main body damage, Insert main body deformation; HiFi x Among the related defects, socket problems include damaged appearance of HiFix socket and damaged conductive glue of HiFix socket; connector problems include damaged connector on the back of HiFix; component problems include missing fixing screws of HiFix socket and missing pins of HiFix socket.

7. The method for intelligently detecting appearance defects of a chip test carrier according to claim 6, characterized in that: The defect detection network is based on the defect hierarchy. Let x be the input image feature, y i is the output of the i-th stage, i=1,2,3, and its calculation formula includes: Phase 1: Quickly identify the main defect types: P(y1|x), y1∈{structural defect, surface defect} Among them, P(y1|x) represents the probability that the main defect type is y1 when the input feature x is given; Phase 2: Identify more specific defect categories within the selected major defect types: P(y2|x, y1), y2∈{Insert related defects, HiFix related defects, appearance defects} Among them, P(y2|x, y1) represents the probability that the main defect type is y1 and the more specific defect category is y2 when the input feature x is given; Phase 3: Identify the final 14 specific defects at the bottom level: P(y3|x,y1,y2), y3∈{14 types of specific defects} Where P(y3|x, y1, y2) represents the probability that the main defect type is y1, the more specific defect category is y2, and the underlying defect type is y3 given the input feature x; The output of the defect detection network, that is, the final classification probability, is calculated by the Bayesian rule, and then: P(y3|x)=P(y3|x, y1, y2)*P(y2|x, y1)*P(y1|x) Among them, P(y3|x) represents the result output by the defect detection network, that is, the probability that the underlying defect type is y3 when the input feature x is given.

8. The method for intelligently detecting appearance defects of a chip test carrier according to claim 7, characterized in that: Feature reuse is introduced into the defect detection network, and feature extraction functions are defined for the three stages: f1(x), f2(x), and f3(x). Each feature extraction function is represented by a neural network module as the input processing module of each stage.

9. The method for intelligently detecting appearance defects of a chip test carrier according to claim 7, characterized in that: A soft decision mechanism is introduced into the defect detection network. The judgment process of the soft decision mechanism includes: Among them, τ1 is the first threshold and τ2 is the second threshold.

10. The method for intelligently detecting appearance defects of a chip test carrier according to claim 7, characterized in that: In the training and optimization of the defect detection network of S32, a multi-task loss function is constructed to optimize all stages of the defect detection network. The calculation formula of the multi-task loss function is: Among them, L is the total loss function, α1, α2, and α3 are the balance factors of the first stage, the second stage, and the third stage respectively, y1, are the true labels of the first stage, the predicted output of the first stage network, y2, are the true label of the second stage, the predicted output of the second stage network, y3, are the true labels of the third stage and the predicted output of the third stage network, L1, L2, and L3 are the cross entropy loss functions of the first, second, and third stages, λ is the regularization coefficient, θ is the parameter of the model, and R(θ) is the regularization term.

Images