Chip defect detection method and system based on nuclear norm minimization and machine learning

Abstract

The chip defect detection method and system based on kernel norm minimization and machine learning, the method comprises the steps of: using the background separation method based on kernel norm minimization to separate the image foreground and background for the collected chip defect data; the pad position of the foreground image is identified through Huffman transformation, and the corresponding pixels are set to 0 to eliminate the pad image; the image processing techniques such as corrosion and expansion are used to process and eliminate noise points, realize defect area connection; all defect areas in the image are searched to obtain the minimum boundary box of the defect area; the K-Means clustering algorithm is used for defect labeling of different types of defect images to form a defect data set; finally, an end-to-end deep neural network chip defect classification cluster algorithm model is formed by using the training set and the test set. The method can realize real-time and accurate integrated circuit chip surface defect recognition, classification and positioning.

Description

TECHNICAL FIELD

[0001] The application relates to a chip defect detection method and system based on kernel norm minimization and machine learning, and belongs to the technical field of semiconductor quality detection. BACKGROUND

[0002] The aerospace integrated circuit refers to an integrated circuit applied to satellites, spaceships, missiles, launch vehicles and other aerospace tasks, and has special requirements such as a wide operating temperature range, high mechanical conditions and long service life. At present, the packaging of aerospace integrated circuits has shifted from multiple varieties and small batches to simultaneous development and production, and high-efficiency batch production. The process nodes of aerospace integrated circuits such as billion-gate FPGAs and multi-core SoCs have reached 28 nanometers, and the wafer surface is provided with complex structures such as micro-nano rewiring and millions of ultra-micro bumps. In order to ensure the packaging quality of the chip, high-precision detection of the special structures on the wafer surface is required. The feature line width / spacing size is less than or equal to 1 um, and the number of features exceeds ten million. In the existing packaging process, wafer detection is still mainly carried out through manual microscopy, and the detection of defects such as dirt, scratches and missing balls on a single wafer takes more than 60 minutes per piece. The detection efficiency is low, and the detection relies on subjective experience, which has the risk of missed detection and missed detection, affects the chip packaging qualification rate and quality consistency, and urgently needs to combine AI artificial intelligence vision technology with packaging detection to carry out research on the surface defect detection method and system of aerospace integrated circuits, break through key technologies such as image preprocessing and image data feature recognition, and provide packaging detection technical support for the development of aerospace integrated circuits.

[0003] The AI-based machine vision detection is a non-contact, non-destructive intelligent automatic detection technology. Its detection algorithm integrates a series of characteristics such as deep convolutional neural network nonlinearity, self-adaptation, self-organization, self-learning ability, non-locality, non-convexity and fault tolerance, which is an effective means to realize device automation, intelligence and precise control, and has outstanding advantages such as safety and reliability, stable detection standard, long-time work in harsh environment and high production efficiency. The aerospace integrated circuit chip surface defect intelligent detection method based on kernel norm minimization and clustering labeling proposed by the application is a core component of the AI machine vision detection system. On the basis of obtaining the surface image of the product by using a proper light source and an image sensor (CCD camera), the method is divided into three steps of defect feature engineering, training of an identification model and use of the model to identify defects. The defect feature engineering integrates the minimum kernel norm image foreground and background separation technology and the clustering labeling technology to realize image target locking and target labeling, and then extracts the feature information of the target in all collected image data. According to the feature information, AI machine learning technology is used to perform positioning, identification, grading and other operations of the aerospace integrated circuit chip surface defects, and an intelligent defect identification model is trained. When the model is used, the image foreground and background are separated, the foreground features are extracted, and the intelligent positioning, identification and grading of the defects are realized by inputting the defect identification model. SUMMARY

[0004] The technical problem solved by the present application is to overcome the shortcomings of the prior art and provide a chip defect detection method and system based on kernel norm minimization and machine learning, which greatly improves the detection accuracy and standard consistency while ensuring the detection efficiency.

[0005] The technical solution of the present application is a chip defect detection method based on kernel norm minimization and machine learning, comprising:

[0006] The collected chip defect pictures are separated into foreground and background using the kernel norm minimization background separation technology; the foreground image includes chip solder balls and chip defect images, and the background image includes chip wiring;

[0007] For the separated foreground image, the solder ball image is eliminated using Huffman transformation, the noise points are eliminated using erosion and expansion image processing method, and the minimum bounding box of the connected region is obtained using image connectivity algorithm, and the image in the minimum bounding box is the defect image; for the defect image, the K-Means clustering algorithm is used to label different types of defect images, forming n defect data, and forming a training data set represented as {di}, i = 1, … n; each defect data is represented by a binary tuple di = [I, c], wherein I is the original image framed by the bounding box of the defect, and c is the defect category; the training data set is input into a convolutional neural network classification algorithm cluster to complete the image classification model training of the convolutional neural network CNN, ResNet, and VGG, and the obtained image classification model is saved;

[0008] The kernel norm minimization background separation technology is used to separate the foreground and background of m new chip images to be judged; for the separated foreground image, the solder ball image is eliminated using Huffman transformation, the noise points are eliminated using erosion and expansion image processing method, and the minimum bounding box of the connected region is obtained using image connectivity algorithm, and the image in the minimum bounding box is the defect image in the m new chip images; the obtained defect image is input into the saved convolutional neural network CNN, ResNet, and VGG image classification model, so as to obtain the defect type and position of the m new chip images.

[0009] The kernel norm minimization background separation technology is used to separate the foreground and background of m new chip images to be judged; for the separated foreground image, the solder ball image is eliminated using Huffman transformation, the noise points are eliminated using erosion and expansion image processing method, and the minimum bounding box of the connected region is obtained using image connectivity algorithm, and the image in the minimum bounding box is the defect image in the m new chip images; the obtained defect image is input into the saved convolutional neural network CNN, ResNet, and VGG image classification model, so as to obtain the defect type and position of the m new chip images.

[0010] An image separation conditional objective function is established to solve the L1 norm minimum value of the image foreground and background, and the conditional objective function is as follows:

[0011]

[0012] s.t.A+B=C

[0013] where A is the foreground image, B is the background image, C is the original image converted to a grayscale image, ‖·‖ l1 is the L1 norm, i.e., the sum of the absolute values of each element of the matrix, ‖·‖ * is the nuclear norm, i.e., the sum of the singular values of the matrix; and γ is a pre-set hyperparameter;

[0014] The conditional target function is converted into a semi-definite programming form, and A can be obtained by using an SDP solver; the image A is input into a convolutional neural network (CNN), ResNet, and VGG to complete defect positioning.

[0015] The method for eliminating the solder ball image by using the Huffman transform comprises the following steps:

[0016] The foreground image is scanned, and a Canny edge detection algorithm is used to determine suspected edge position points All position points A circle with r as the radius is drawn, and the intersection points of multiple circles are candidate points for the center of the circle; according to a parameter voting strategy, when the number of intersection circles with (a, b) as the center and r as the radius is greater than a threshold A, the position of the solder pad with (a, b) as the center and r as the radius is determined, then the image of the solder pad is eliminated by setting the pixels inside the circle to 0, and finally the solder ball image is eliminated.

[0017] The method for eliminating noise points by using the erosion and dilation image processing method comprises the following steps:

[0018] A convolution kernel B is defined; the convolution kernel B is convolved with the separated foreground image A, and the maximum value and the minimum value of the pixels in the image A region covered by the convolution kernel B after convolution are calculated;

[0019] The calculated maximum value and minimum value are assigned to the pixels in the corresponding region of the image A, and the images after B erosion A and B dilation A are obtained.

[0020] The method for obtaining the minimum bounding box of the connected region by using the image connected algorithm comprises the following steps:

[0021] An edge detection operator is used to extract the edges of the separated foreground image, and each defect region is represented by a closed curve, i.e., an edge image; the obtained edge image is binarized to obtain a binary edge image; each binary edge image is an edge curve of a connected region, and the connected space of the binary edge image is used to approximately represent the region, i.e., a connected domain; for the connected domains that are horizontally adjacent, have a high approximation, and have a small connected area, further merging is required, and finally the minimum bounding box of the connected region is obtained.

[0022] The method for defect labeling of different types of defect images by using the K-Means clustering algorithm comprises the following steps:

[0023] (1) Normalization processing is performed on the size of all separated foreground images and K kinds of defect images provided in the "Wafer Bump Preparation Process Quality Inspection Specification" (hereinafter referred to as the specification);

[0024] (2) The K kinds of defect images contained in the specification are taken as K initial clustering centers;

[0025] (3) Each point of the RGB image has three attributes [r, g, b], according to the principle of the nearest distance to the clustering center, in addition to the K kinds of defect images contained in the "Wafer Bump Preparation Process Quality Inspection Specification", the Euclidean distance between the target pixel points of other foreground pictures and the K initial clustering center pixel points is calculated;

[0026] (4) The Euclidean distance sample closest to the same center is a class, thereby completing a clustering;

[0027] (5) For the foreground images in each class, the clustering center of these foreground images is calculated, which is taken as the new clustering center of the class, and the step (3) is returned for loop execution, and the algorithm is executed when the clustering centers of various classes no longer change, and the algorithm ends;

[0028] (6) For the same kind of image, the belonging defect label is used for marking to form an effective classification marked data set.

[0029] The chip defect detection system based on kernel norm minimization and machine learning includes an algorithm application module, a data storage module and a display module;

[0030] The data storage module stores the original data set of the optimized neural network algorithm model;

[0031] The algorithm application module reads the original data set stored in the data storage module, separates the chip defect picture in the original data set by using a background separation technology of kernel norm minimization, separates the foreground and background of the picture, and uses the Huffman transform to eliminate the solder ball image, uses the erosion and expansion image processing method to eliminate the noise points, and uses the image connection algorithm to obtain the minimum bounding box of the connected region, and the image in the minimum bounding box is the defect image; for the defect image, the K-Means clustering algorithm is used to mark the different types of defect images, and n defect data sets are formed; each defect data is represented by a binary tuple di=[I,c], wherein I is the original image of the defect framed by the bounding box, and c is the defect category; the training data set is represented as {di}, i=1,…n; the training data set is input into the neural network classification algorithm cluster, and the image classification model training of the convolutional neural network (CNN), ResNet and VGG is completed; m new chip images to be judged are subjected to the same defect positioning process as the training data process, and the image in the obtained bounding box is input into the image classification model obtained in the training process, so that the defect type and position of the m new chip images are obtained.

[0032] The display module is used for displaying the data sent by the algorithm application module, including the chip defect positioning accuracy and the chip defect classification accuracy.

[0033] The beneficial effects of the present application are:

[0034] 1. Based on the kernel norm minimization algorithm, the separation effect of the defect part in the chip image is improved, so that the overall performance of the system is improved.

[0035] 2. The classification of the defect image is based on the machine learning technology, and the efficiency of the defect detection is improved.

[0036] 3. A large amount of manual labeling data is not required, so that the problem of low accuracy of the label generated due to the influence of the subjective factors of manual labeling and the large workload and low processing efficiency of manual labeling in the application of machine learning technology in the field of intelligent manufacturing are solved. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The algorithm flowchart of the present application is shown in the figure;

[0038] Figure 2 The background separation schematic diagram is shown in the figure;

[0039] Figure 3 The Huffman circular identification diagram is shown in the figure;

[0040] Figure 4 The foreground image A and the convolution kernel B diagram are shown in the figure;

[0041] Figure 5 The image erosion diagram is shown in the figure;

[0042] Figure 6 is an image inflation diagram;

[0043] Figure 7 is a clustering schematic diagram.

[0044] Figure 8 is an image classification positioning model technical roadmap. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying Figures 1-8 The embodiment will be specifically explained.

[0046] I. Image preprocessing for defect positioning

[0047] The purpose of defect positioning is to locate the position of the defect from the original image, and for each defect, a minimum bounding box is obtained to determine the defect position.

[0048] The defect positioning process of the present application is divided into two parts: background separation based on kernel norm minimization and separated image preprocessing.

[0049] 1. Background separation based on kernel norm minimization

[0050] Background separation based on kernel norm minimization is an effective background separation technique in image processing. Its mathematical expression is

[0051]

[0052] s.t.A+B=C

[0053] Where A is the foreground image, B is the background image, C is the original image converted to a gray image, ‖·‖ l1 is the L1 norm (the sum of the absolute values of each element of the matrix), ‖·‖ * is the kernel norm (the sum of the singular values of the matrix). γ is a pre-set hyperparameter.

[0054] There are many solutions to problem (1). According to Sparse and Low-Rank Matrix Decompositions, one of the solutions is to convert this problem into a semi-definite programming (SDP) and use the SDP solver to obtain A and B.

[0055] Reference can be made to Figure 2 The original picture C has wiring, pads, defects, background, and after background separation, B and A are obtained, where B is the background element containing the background and wiring, and A is the foreground element containing the pads and defects. Further processing of the image A can obtain the positioning of the defects.

[0056] 2. Image pre-processing

[0057] The foreground image A obtained in the above step is processed to obtain the boundary box of the defect by the following steps:

[0058] (1) Identify all the circular shapes on A, i.e. the position of the pads, by the Huffman transformation, and then eliminate the images of the pads by setting the pixels inside the circular shapes to 0

[0059] (2) Perform erosion and dilation processing to eliminate noise and make the defect area as integrated as possible

[0060] (3) Search for the connected parts in the image, and obtain the minimum boundary box of the connected area. The image in the boundary box is the defect image.

[0061] The "identify all the circular shapes on A, i.e. the position of the pads, by the Huffman transformation, and then eliminate the images of the pads by setting the pixels inside the circular shapes to 0" in (1) is implemented as follows:

[0062] Scan each pixel of the foreground image, and determine the suspected edge position point by using the Canny edge detection algorithm

[0063] Since in the two-dimensional Cartesian coordinate system, the analytical equation of a circle is as follows:

[0064] (x-a) 2 +(y-b) 2 =r 2

[0065] where (x, y) is the coordinate of any point on the circle, (a, b) is the center, and r is the radius.

[0066] If the radius r of the circle is known, any point Pi(x, y) on the circle degenerates into a circle on the plane in the Huffman parameter space, i.e. the parameters (a, b) are distributed on a circle with Pi(x, y) as the center and r as the radius, as shown in Figure 3 .

[0067] Therefore, for all points, a circle with r as the radius is drawn, and the intersection points of multiple circles are the possible centers (a, b);

[0068] According to the parameter voting strategy, when the number of intersecting circles is greater than a certain threshold A, the position of the pad with (a, b) as the center and r as the radius is determined;

[0069] Then the image of the pads is eliminated by setting the circular inner pixels to zero.

[0070] (2) The "corrosion and expansion process to eliminate noise and make the defect area as integrated as possible" is described in detail as follows:

[0071] Suppose two images A and B, where A is the foreground image to be processed and B is the convolution kernel used to process A, as shown in Figure 4 .

[0072] Corrosion of A with the convolution kernel B is as follows:

[0073] Scan all the pixel points of image A with the convolution kernel B (the origin of B is used as the coordinate to scan each pixel point of A), and perform an "and" operation between the convolution kernel B and the covered A image (that is, perform an "and" operation between each corresponding point of the region covered by B and A);

[0074] If the convolution kernel B is a black point and the corresponding point of image A is also black, the pixel of the point is black, otherwise it is white;

[0075] The result of corrosion makes the original image shrink by one circle;

[0076] Image corrosion is shown in Figure 5 .

[0077] Dilation of A with the convolution kernel B is as follows:

[0078] Scan each pixel point of image A with the convolution kernel B;

[0079] If the region covered by B has a black point A corresponding to a black point B, the scanned point is black, otherwise it is white;

[0080] Image dilation is shown in Figure 6 .

[0081] (3) The "search for connected parts in the image and get the minimum bounding box of the connected region, and the image in this bounding box is the defect image" is described in detail as follows:

[0082] First, use an edge detection operator to perform edge extraction on the separated foreground image, so that each defect region can be represented by a closed curve, that is, an edge image;

[0083] Perform binaryzation processing on the obtained edge image to obtain a binary edge image;

[0084] Each binary edge image is the edge curve of a connected region, which can be approximated by the connected space of the connected set.

[0085] II. Image Clustering Annotation

[0086] For the defect images separated in the previous step, this invention employs the K-Means clustering algorithm. K-Means is a clustering algorithm that groups data according to their spatial distribution. Clustering is an unsupervised learning method that does not require any form of label; the purpose of this algorithm is to infer cluster labels based on the structure of the data itself.

[0087] Given that the "Quality Inspection Specification for the Manufacturing Process of Circular Wafer Bumps" specifies K typical defects that may occur during production, and provides one image corresponding to each defect, the basic process of image clustering is as follows:

[0088] (1) Normalize the size of all separated foreground images and the K types of defect images specified in the standard;

[0089] (2) Use the K types of defect images included in the specification as K initial cluster centers;

[0090] (3) For each point in an RGB image, there are three attributes [r, g, b]. Based on the principle of proximity to the cluster center, except for the K types of defective images included in the specification, the distance from the target pixel point M = [r, g, b] of other foreground images to the pixel point of the K cluster center image is calculated. The Euclidean distance is given by the following formula:

[0091]

[0092] (4) The samples closest to the same center belong to the same class, thus completing one clustering;

[0093] (5) For each category of foreground images, calculate the cluster centers of these foreground images and use them as the new cluster centers of that category. Return to step (3) and repeat the process. When the cluster centers of each category no longer change, the algorithm ends.

[0094] (6) For images of the same type, use the corresponding defect labels to label them, forming an effective classification label dataset.

[0095] For example, the target is clustered into K=3 types of defects, with squares as cluster centers and dots as other images to be labeled. The steps are as follows: Figure 7 As shown.

[0096] III. Machine Learning Defect Identification and Classification

[0097] like Figure 1As shown, the machine learning algorithm of the present application is composed of two parts: training and prediction.

[0098] The process of training is to use historical data to get the bounding box of each defect after the defect positioning process (composed of background separation based on kernel norm minimization and image preprocessing), and then train an image classification model using the image inside the bounding box and the corresponding defect type. For training data, we assume that n chip defect pictures are collected, each containing defects labeled by clustering algorithm, and each defect is represented by a binary tuple di=[I,c]. Where I is the original image of the defect framed by the bounding box, and c is the defect category. For simplicity, we can represent the training data set as {di}, i=1,…

[0099] The process of prediction is to input the image of the new chip Image new into the image model obtained in the training process after the same defect positioning process as the training process, so as to obtain the type and location of the chip defect.

[0100] The function of the image classification and positioning model is to output the category and positioning of the input image.

[0101] Common image classification models include convolutional neural network (CNN), ResNet, VGG, etc. The technical route of the image classification and positioning model of the present application is as shown in Figure 8 .

[0102] IV. Computer system architecture

[0103] The present application also provides a computer system, which is divided into an algorithm application module, a data storage module, and a display module. The functions of each module are described in detail below.

[0104] The data storage module stores the original data set of the optimized neural network algorithm model.

[0105] The algorithm application module reads the original data set stored in the data storage module, separates the foreground and background of the chip defect picture in the original data set using the background separation technology based on kernel norm minimization, eliminates the solder ball image using Huffman transformation, eliminates noise points using erosion and dilation image processing methods, and obtains the minimum bounding box of the connected region using the image connected algorithm. The image inside this bounding box is the defect image.

[0106] For the defect image, the algorithm of K-Means clustering is used to mark the defect of different kinds of defect images, and n defect data sets are formed. Each defect data is expressed by a binary group di=[I, c], wherein I is the original image of the defect framed by the bounding box, and c is the category of the defect. In order to be simple, the training data set can be expressed as {di}, i=1,…n.

[0107] The training data set is input into the neural network classification algorithm cluster, and the image classification model training of convolutional neural network (CNN), ResNet and VGG is completed. The m new chip images to be judged are subjected to the same defect positioning process as the training data process, the images in the obtained bounding box are input into the image classification model obtained in the training process, and thus the defect type and position of the m new chip images are obtained.

[0108] The display module is used for displaying the data sent by the algorithm application module, including the chip defect positioning accuracy and the chip defect classification accuracy.

[0109] The parts not described in detail in the present application belong to the known technology of those skilled in the art.

Claims

1. A chip defect detection method based on nuclear norm minimization and machine learning, characterized in that, include: The foreground and background of the acquired chip defect images are separated using a background separation technique that minimizes the nuclear norm, resulting in a foreground image and a background image. The foreground image includes chip solder balls and chip defect images, while the background image includes chip wiring. For the separated foreground image, Huffman transform is used to eliminate solder ball images, erosion and dilation image processing methods are used to eliminate noise, and the minimum bounding box of the connected region is obtained using the image connectivity algorithm. The image within the minimum bounding box is the defect image. For the defect images, K-Means clustering algorithm is used to label different types of defect images, forming n defect data, which are used to form a training dataset denoted as {di}, i=1,…n; Each defect data is represented by a tuple di = [I, c], where I is the original image bounded by the boundary box for this defect, and c is the category to which the defect belongs. The training dataset is input into the convolutional neural network classification algorithm cluster to complete the training of the image classification model of the convolutional neural network CNN, and the obtained image classification model is saved. Using a background separation technique that minimizes the nuclear norm, the foreground and background of m new chip images to be judged are separated. For the separated foreground image, Huffman transform is used to eliminate solder ball images, erosion and dilation image processing methods are used to eliminate noise, and the minimum bounding box of the connected region is obtained using the image connectivity algorithm. The image within the minimum bounding box is the defect image in m new chip images. The obtained defect images are input into the saved convolutional neural network (CNN) image classification model to obtain the defect type and location of m new chip images. The process of separating the foreground and background of the acquired chip defect images using a background separation technique that minimizes the nuclear norm includes: A conditional objective function for image separation is established, and the minimum L1 norm of the foreground and background of the image is obtained. The conditional objective function is as follows: stA+B=C Where A is the foreground image, B is the background image, and C is the original image converted to grayscale. l The L1 norm is the sum of the absolute values ​​of all elements in a matrix, ||·||. * γ is the nuclear norm, which is the sum of the singular values ​​of the matrix; γ is a pre-defined hyperparameter. The conditional objective function is transformed into a semidefinite programming form, and A can be obtained using an SDP solver; the image A is then input into a convolutional neural network (CNN) to complete defect localization. The method of eliminating solder ball images using Huffman transform includes: Scan each pixel of the foreground image and use the Canny edge detection algorithm to determine the suspected edge locations. For all location points Draw a circle with radius r, and the intersection of multiple circles is the candidate point of the circle center. According to the parameter voting strategy, when the number of intersecting circles with (a,b) as the center and r as the radius is greater than a certain threshold A, the position of the pad with (a,b) as the center and r as the radius can be determined. Then, the images of these pads are eliminated by setting the pixels inside the circle to 0, and finally the solder ball image is eliminated. The noise removal method using erosion and dilation image processing includes: Custom convolution kernel B; convolve kernel B with the separated foreground image A, and calculate the maximum and minimum values ​​of pixels in the region of image A covered by kernel B after the convolution. The calculated maximum and minimum values ​​are assigned to the pixels in the corresponding region of image A to obtain the images after B erodes A and B dilates A. The method of obtaining the minimum bounding box of the connected region using the image connectivity algorithm includes: Edge detection operators are used to extract edges from the separated foreground image. Each defect region is represented by a closed curve, which is the edge image. The obtained edge images are binarized to obtain binary edge images. Each binary edge image is the edge curve of a connected region. The connected space of the binary edge image is used to approximate the region, which is the connected region. For connected regions with similar horizontal heights and small connected areas, further merging is required to obtain the minimum bounding box of the connected region. The K-Means clustering algorithm is used to annotate defects in images of different types, including: (1) Normalize the size of all separated foreground images and the K types of defect images specified in the "Quality Inspection Specification for the Preparation Process of Circular Bumps"; (2) The K types of defect images contained in the "Quality Inspection Specification for the Preparation Process of Circular Plate Bumps" are used as K initial cluster centers; (3) For each point in an RGB image, there are three attributes [r, g, b]. Based on the principle of proximity to the cluster center, except for the K types of defect images included in the "Quality Inspection Specification for the Preparation Process of Circular Bumps", calculate the Euclidean distance from the target pixel of other foreground images to the K initial cluster center pixels. (4) The samples closest to the same center by Euclidean distance belong to one class, thus completing one clustering; (5) For each category of foreground images, calculate the cluster centers of these foreground images and use them as the new cluster centers of that category. Return to step (3) and repeat the process. When the cluster centers of each category no longer change, the algorithm ends. (6) For images of the same type, use the corresponding defect labels to label them, forming an effective classification label dataset.

2. A chip defect detection system based on nuclear norm minimization and machine learning, characterized in that, It includes an algorithm application module, a data storage module, and a display module; The data storage module stores the original dataset for optimizing the neural network algorithm model; The algorithm application module reads the original dataset stored in the data storage module and uses a background separation technique that minimizes the nuclear norm to separate the foreground and background of the chip defect images in the original dataset. For the foreground image, Huffman transform is used to eliminate solder ball images, and erosion and dilation image processing methods are used to eliminate noise. An image connectivity algorithm is then used to obtain the minimum bounding box of the connected regions; the image within this minimum bounding box is the defect image. For the defect images, K-Means clustering is used to label different types of defect images, forming n defect datasets. Each defect data point is represented by a tuple di = [I, c], where I is the original image bounded by the bounding box, and c is the defect category. The training dataset is represented as {di}, i = 1, ..., n. The training dataset is input into a neural network classification algorithm cluster to train a convolutional neural network (CNN) image classification model. The m new chip images to be judged undergo the same defect localization process as the training data. The images within the obtained bounding boxes are then input into the image classification model obtained during the training process, thus obtaining the defect type and location of the m new chip images. The display module is used to display the data sent by the algorithm application module, including the chip defect location accuracy and chip defect classification accuracy. The chip defect images in the original dataset are separated into foreground and background using a background separation technique that minimizes the nuclear norm. This includes: A conditional objective function for image separation is established, and the minimum L1 norm of the foreground and background of the image is obtained. The conditional objective function is as follows: stA+B=C Where A is the foreground image, B is the background image, and C is the original image converted to grayscale. l The L1 norm is the sum of the absolute values ​​of all elements in a matrix, ||·||. * γ is the nuclear norm, which is the sum of the singular values ​​of the matrix; γ is a pre-defined hyperparameter. The conditional objective function is transformed into a semidefinite programming form, and A can be obtained using an SDP solver; the image A is then input into a convolutional neural network (CNN) to complete defect localization. The method of obtaining the minimum bounding box of the connected region using the image connectivity algorithm includes: Edge detection operators are used to extract edges from the separated foreground image. Each defect region is represented by a closed curve, which is the edge image. The obtained edge images are binarized to obtain binary edge images. Each binary edge image is the edge curve of a connected region. The connected space of the binary edge image is used to approximate the region, which is the connected region. For connected regions with similar horizontal heights and small connected areas, further merging is required to obtain the minimum bounding box of the connected region. The K-Means clustering algorithm is used to annotate defects in images of different types, including: (1) Normalize the size of all separated foreground images and the K types of defect images specified in the "Quality Inspection Specification for the Preparation Process of Circular Bumps"; (2) The K types of defect images contained in the "Quality Inspection Specification for the Preparation Process of Circular Plate Bumps" are used as K initial cluster centers; (3) For each point in an RGB image, there are three attributes [r, g, b]. Based on the principle of proximity to the cluster center, except for the K types of defect images included in the "Quality Inspection Specification for the Preparation Process of Circular Bumps", calculate the Euclidean distance from the target pixel of other foreground images to the K initial cluster center pixels. (4) The samples closest to the same center by Euclidean distance belong to one class, thus completing one clustering; (5) For each category of foreground images, calculate the cluster centers of these foreground images and use them as the new cluster centers of that category. Return to step (3) and repeat the process. When the cluster centers of each category no longer change, the algorithm ends. (6) For images of the same type, use the corresponding defect labels to label them, forming an effective classification label dataset.

Images