A surface defect detection method and system for integrated circuit packaging processes

Abstract

This invention discloses a method and system for surface defect detection in the integrated circuit packaging process. The method includes the following steps: acquiring digital images of the surface of the integrated circuit packaging process; performing image preprocessing on the acquired digital images; extracting features from the preprocessed surface images of the integrated circuit packaging process; classifying the extracted images according to defect features; applying a corresponding defect detection method to each type of defect to effectively separate the normal background from the defect area; determining the defect area for each defect type; determining whether the defects meet industrial production requirements; and outputting the surface defect detection results. This invention preprocesses the surface defect images of the integrated circuit packaging process before performing defect detection, avoiding interference from noise, low contrast, and other factors on subsequent defect detection, thus improving the accuracy of defect detection.

Description

Technical Field

[0001] This invention relates to the field of circuit defect detection technology, and specifically to a method and system for detecting surface defects in the integrated circuit packaging process. Background Technology

[0002] Packaging is a crucial step in the integrated circuit industry. To ensure product yield, comprehensive inspection of the packaging process is essential. For integrated circuits with nanometer-level circuitry, high-magnification microscopes are required to clearly visualize surface defects and avoid image distortion during magnification. Typically, a complete integrated circuit needs to be divided into hundreds of regions for image acquisition, with slight overlap between these regions, making it difficult to obtain a standard template. Furthermore, the packaging process generates a wide variety of defects, such as short circuits, open circuits, blocked vias, broken vias, scratches, creases, indentations, speckles, oxidation, ink defects, and foreign matter, significantly increasing the difficulty of defect detection. With increasing integration levels and the growing workload, the inspection difficulty has greatly increased. Therefore, there is an urgent need to develop machine vision-related inspection technologies to achieve automated inspection. Summary of the Invention

[0003] To overcome the defects and shortcomings of existing technologies, this invention provides a method and system for surface defect detection in the integrated circuit packaging process. This invention first performs preprocessing operations on the surface defect images of the integrated circuit packaging process before defect detection, avoiding interference from factors such as noise and low contrast on subsequent defect detection and improving the accuracy of defect detection.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] This invention provides a method for detecting surface defects in the integrated circuit packaging process, comprising the following steps:

[0006] Acquire digital images of the surface of integrated circuits during the packaging process;

[0007] Perform image preprocessing on the acquired digital images of the surface;

[0008] Feature extraction is performed on the surface image of the preprocessed integrated circuit packaging process;

[0009] The defect type classification operation is performed on the feature-extracted image based on the defect characteristics;

[0010] For each type of defect, a corresponding defect detection method is used to achieve effective separation between the normal background and the defect area;

[0011] Identify the defect area for each defect type;

[0012] Determine whether the defects meet the requirements of industrial production and output the surface defect detection results.

[0013] As a preferred technical solution, the feature extraction of the surface image of the preprocessed integrated circuit packaging process is specifically performed by using a regularization method to optimize and improve the low-rank embedding algorithm to extract features from the surface image of the preprocessed integrated circuit packaging process.

[0014] As a preferred technical solution, the defect type classification operation of the image after feature extraction based on defect features is specifically based on the defect type classification operation of the image after feature extraction using a symptotic group classifier.

[0015] As a preferred technical solution, defect type classification is performed on the feature-extracted image based on a symplectic group classifier, specifically including:

[0016] Map the image dataset to Sp(2n), and take the n-dimensional row vectors of Sp(2n) to form a set. Specifically, it is expressed as follows:

[0017]

[0018] Where the vector (I1,I2,…,I) n ,) represents any image sample;

[0019] Select Q i ∈Sp(2n), transform Q into the corresponding symplectic matrix and find its singular values;

[0020] Q i When applied to an image dataset, it is represented as:

[0021]

[0022] Perform singular value decomposition on the training image samples, take the top k largest singular values, and construct a discriminant function.

[0023] Solve the discriminant function and output the category of the image dataset.

[0024] As a preferred technical solution, the defect types include: manufacturing defects related to line width or line spacing, manufacturing defects related to hole parameters, manufacturing defects related to irregularity, manufacturing defects related to unevenness, contamination-related color defects, and trauma-related scratch defects.

[0025] As a preferred technical solution, the step of employing a corresponding defect detection method for each type of defect to effectively separate the normal background from the defect area specifically includes:

[0026] Analyze the characteristics of various defects and the relationships and properties between them;

[0027] Detect manufacturing defects related to line width or line spacing dimensional parameters, and manufacturing defects related to hole parameters;

[0028] Region segmentation of trauma-type scratches and manufacturing-type non-uniform defects based on Jardon's curve theorem;

[0029] Region segmentation of pollution-type color defects based on the Riemannian manifold method;

[0030] The preprocessed surface image is optimized to improve the line edges, and the constant cross-sectional curvature is calculated. Based on the constant cross-sectional curvature, it is determined whether there are manufacturing irregularities.

[0031] As a preferred technical solution, region segmentation for trauma-type scratches and manufacturing-type non-uniform defects is performed based on Jardon's curve theorem, specifically including:

[0032] The surface image of the pre-processed integrated circuit packaging process is converted into an HSV color space image, and the brightness V component of the color space image is obtained.

[0033] Segmentation curves for surface images are constructed based on Jordan's curve theorem;

[0034] Region segmentation of trauma-type scratches and manufacturing-type uneven defects based on segmentation curves;

[0035] The presence of traumatic scratches and manufacturing unevenness defects is determined based on the segmented regions.

[0036] As a preferred technical solution, region segmentation for pollution-type color defects is based on the Riemannian manifold method, specifically including:

[0037] Obtain image background samples and map them to the Riemannian manifold space;

[0038] Construct a similarity model between the image background sample and the image to be tested, and calculate the similarity between the image background sample and the image to be tested;

[0039] The optimal threshold is obtained by utilizing the principle of low-probability events;

[0040] Regions larger than the optimal threshold are classified as background, and regions smaller than the optimal threshold are classified as defects, thus completing the region segmentation of pollution-type color defects.

[0041] As a preferred technical solution, optimizing the line edges of the preprocessed surface image specifically includes:

[0042] The preprocessed surface image is converted from RGB color space to Lab color space, and image processing is performed based on morphological methods to extract line edges in the L component of Lab color space.

[0043] Fitting operations are performed based on the principle of numerical approximation to optimize the line edges extracted from the L component.

[0044] The present invention also provides a surface defect detection system for the integrated circuit packaging process, for implementing the above-mentioned surface defect detection method for the integrated circuit packaging process. The system includes: a microscopic imaging acquisition device, an image preprocessing module, a defect detection module, and a surface defect detection result output module.

[0045] The microscopic imaging acquisition device is used to acquire surface digital images of the integrated circuit packaging process;

[0046] The image preprocessing module is used to perform image preprocessing operations on the acquired surface digital images;

[0047] The defect detection module is used to detect different types of defect images after preprocessing and determine the defect region, specifically including:

[0048] Feature extraction is performed on the surface image of the preprocessed integrated circuit packaging process;

[0049] The defect type classification operation is performed on the feature-extracted image based on the defect characteristics;

[0050] For each type of defect, a corresponding defect detection method is used to achieve effective separation between the normal background and the defect area;

[0051] Identify the defect area for each defect type;

[0052] The surface defect detection result output module is used to determine whether the defect meets the requirements of industrial production and output the surface defect detection result.

[0053] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0054] (1) The present invention first performs preprocessing operation on the surface defect image of the integrated circuit packaging process and then performs defect detection, which avoids the interference of noise, low contrast and other factors on subsequent defect detection and improves the accuracy of defect detection.

[0055] (2) This invention adopts Jardon curve correlation theory and designs a segmentation curve suitable for integrated circuit images, which effectively solves the detection of two types of defects: trauma-type scratches and manufacturing-type unevenness.

[0056] (3) This invention adopts the Riemannian manifold correlation theory and uses the Riemannian metric to effectively construct a similarity model between the background sample and the image to be tested, thereby achieving effective segmentation of pollution-type color defects.

[0057] (4) This invention combines the cross-sectional curvature correlation theorem to design an algorithm suitable for detecting manufacturing irregular defects, which not only preserves the edge features of the line to the greatest extent, but also improves the processing speed of the algorithm.

[0058] (5) The present invention improves the quality control of the diverse defects that exist in the integrated circuit packaging process and improves the reliability of the integrated circuit packaging process. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the surface defect detection system for integrated circuit packaging process according to the present invention;

[0060] Figure 2 A schematic diagram illustrating the implementation process for determining trauma-type scratches and manufacturing-type uneven defect regions in this invention;

[0061] Figure 3 This is a schematic diagram illustrating the implementation process for determining the color defect region of pollution in this invention;

[0062] Figure 4 This is a schematic diagram illustrating the implementation process for determining manufacturing-related irregularity defect areas in this invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0064] Example 1

[0065] like Figure 1 As shown, this embodiment provides a surface defect detection method for integrated circuit packaging processes, including the following steps:

[0066] S1: Acquiring surface digital images of the integrated circuit packaging process using high-magnification microscopy, specifically including:

[0067] S11: Set the current position and microscope parameters;

[0068] S12: Plan the acquisition path of the platform to ensure the integrity of image acquisition;

[0069] In step S12, a complete integrated circuit needs to be divided into hundreds of regions for image acquisition. There are always slight overlaps between the images of each region. Therefore, while ensuring the completeness of image acquisition during the acquisition process, it is also necessary to control the overlap of each region to avoid repeated operations in subsequent defect detection.

[0070] S13: The microscope acquires surface images of the integrated circuit packaging process. The microscope acquires RGB color space images.

[0071] S14: Move the stage to the next position, and acquire the surface image of the current position after the stage stabilizes;

[0072] S15: The stage continues to move and it is determined whether the stage has moved to the set destination position;

[0073] S16: If the target is reached, image acquisition is completed; otherwise, return to step S14 to continue the image acquisition operation.

[0074] S2: Perform image preprocessing on the acquired images;

[0075] In step S2, since the surface image of the integrated circuit packaging process is acquired using high-magnification microscopic imaging, the complexity of the environment can easily lead to poor image quality. For example, various unknown noises are introduced in the industrial environment, the overall image is dark, and the contrast is low. Regardless of the factors that affect the quality of the image, the effect of subsequent defect detection will be reduced. Only by performing preprocessing on the acquired image can the accuracy of defect detection be improved.

[0076] In this embodiment, the image preprocessing operation includes, but is not limited to, noise reduction, improving the overall brightness of the image, and enhancing the contrast, to obtain a preprocessed RGB color space image.

[0077] S3: Detect different types of defect images after preprocessing to determine defect regions, specifically including:

[0078] S31: Extract features from the surface image of the preprocessed integrated circuit packaging process;

[0079] Online acquisition of integrated circuit package surface images often introduces noise, and even after noise reduction operations are performed, the noise cannot be completely removed. Since low-rank embedding methods can effectively remove noise and improve data robustness, this embodiment uses regularization methods to optimize and improve the low-rank embedding algorithm to achieve surface defect feature extraction in the integrated circuit packaging process.

[0080] S32: Perform classification operations on the image after feature extraction;

[0081] In this embodiment, based on the defect characteristics, the defect types can be divided into: manufacturing defects related to line width / line spacing, manufacturing defects related to hole parameters, manufacturing defects related to irregularity, manufacturing defects related to unevenness, contamination-related color defects, and trauma-related scratch defects.

[0082] Because Lie group machine learning fully utilizes the advantages of manifold learning and the Lie group concept, it can describe data geometrically and provide solutions algebraically. Lie group machine learning classifiers include symplectic group classifiers and quantum group classifiers. Therefore, this embodiment combines the advantages of Lie group machine learning to design a symplectic group classifier suitable for surface images in the integrated circuit packaging process. This solves the problem of classifying diverse defects in the integrated circuit packaging process, achieving accurate classification of surface defect images and forming a defect type library.

[0083] The specific steps of the symplectic group classifier algorithm include:

[0084] S321: Map the image dataset to Sp(2n), and take the n-dimensional row vectors on Sp(2n) to form a set.

[0085]

[0086] Where the vector (I1,I2,…,I) n ,) represents any sample image.

[0087] S322: Select Q i ∈Sp(2n), transform Q into the corresponding symplectic matrix and find its singular values;

[0088] S323: Q i The sample dataset applied to the images, i.e.:

[0089]

[0090] Formula (2) can be used as a training example for the learning process.

[0091] S324: Perform singular value decomposition on the training samples, take the top k largest singular values, and construct a discriminant function;

[0092] S325: Solving the discriminant function will output the category of the sample.

[0093] S33: For each type of defect, a corresponding defect detection method is used to effectively separate the normal background from the defect area. Specific steps include:

[0094] S331: Analyze the characteristics of various defects and the relationships and properties between defects;

[0095] S332: For manufacturing defects related to linewidth / spacing and manufacturing defects related to vias, a surface defect detection algorithm for low-density integrated circuit packaging process is adopted.

[0096] S333: such as Figure 2As shown, for trauma-type scratches and manufacturing-type uneven defects, the defect region is segmented using Jardon curve correlation theory, specifically including:

[0097] (1) Convert the preprocessed RGB color space image into an HSV color space image and obtain the brightness V component of the color space image;

[0098] (2) Combining Jordan's curve theorem (i.e. Jordan's curve theorem), design a segmentation curve suitable for integrated circuit images;

[0099] In this embodiment, the Jordan curve metric is mapped from a two-dimensional Euclidean plane to a high-dimensional, high-genus surface;

[0100] In this embodiment, Jordan's curve theorem states: if C is a simple closed curve in a plane, then R... 2 The region \C has exactly two domains, each bounded by C. Region segmentation can be achieved by mapping a two-dimensional Euclidean plane to a high-dimensional, high-genus surface.

[0101] (3) Determine whether there are traumatic scratches and manufacturing-related unevenness defects;

[0102] Since this embodiment first performs defect classification on the image after feature extraction based on defect features, and then uses the corresponding defect detection method for each type of defect, it can be determined whether the corresponding defect exists after the region segmentation is completed.

[0103] S334: As Figure 3 As shown, for color defects related to pollution, the defect region is segmented using the Riemannian manifold method, specifically including:

[0104] (1) Collect a set number of image background samples online, and design a Riemann curvature algorithm suitable for integrated circuit images to establish an image background sample fitting model;

[0105] In this embodiment, since the image background is affected by factors such as lighting, multiple pixel values ​​will appear at the same position in Euclidean space. Therefore, the image background sample is first mapped to the Riemannian manifold space. Combined with the Riemann curvature, a function is designed to calculate the optimal pixel value at the same position, thereby realizing the construction of the sample fitting model.

[0106] (2) Construct a similarity model between the image background sample and the image to be tested by combining the relevant theories of Riemann metric, and calculate the similarity between the two;

[0107] In this embodiment, both the background and the image to be tested are mapped to a Riemannian manifold space. The Riemannian metric can be used not only to calculate curve lengths but also to define curvature, tangent vectors, and the volume of the Riemannian manifold. In this embodiment, the Riemannian metric is used to define the distance between the background and the image to be tested, and a similarity function is constructed between them to calculate their similarity.

[0108] (3) Obtain the optimal threshold by utilizing the principle of low probability events;

[0109] (4) Areas larger than the optimal threshold are classified as background, and areas smaller than the optimal threshold are classified as defects;

[0110] (5) Achieve region segmentation for color defects of pollution type.

[0111] S335: such as Figure 4 As shown, for manufacturing irregularities, defect detection is achieved by combining the theory of cross-sectional curvature, specifically including:

[0112] (1) The influence of integrated circuit texture structure on line edge extraction is removed by adaptive L*a*b color space morphological processing method;

[0113] In this embodiment, the preprocessed RGB color space image is first converted to the Lab color space, where the L component represents luminance, and a and b represent color opposite dimensions. This embodiment designs the correspondence between the RGB and Lab color spaces so that the L component is unaffected by the texture structure of the integrated circuit. Then, morphological methods are used to further eliminate the influence of the texture structure. Finally, edges are extracted from the L component. This process can remove the influence of the texture structure of the integrated circuit on the circuit edges.

[0114] (2) Optimize the line edges extracted from the L component of the integrated circuit image by combining the principle of numerical approximation;

[0115] In this embodiment, since the circuit edges of the integrated circuit have been obtained, but there may be discontinuities in the edges during the acquisition process, it is necessary to perform a fitting operation using the principle of numerical approximation to optimize the edges. The edge approximation problem can be described as follows:

[0116] For any f(x) ∈ C[a,b], find an element in the subspace Φ such that and It is minimized in some sense. The best approximation can be described as:

[0117] Given f(x)∈C[a,b], if P * (x)∈H n Error:

[0118]

[0119] At this point, we need to solve for P. * (x) is to find the maximum error on the interval [a,b]. a≤x≤b The polynomial that minimizes |f(x) - P(x)|. Solving for the polynomial allows for the optimization of integrated circuit circuit edges.

[0120] (3) Calculate the curvature of a constant section by combining the relevant theorems on the curvature of the cross section;

[0121] In this embodiment, a common manufacturing irregularity defect is a hole defect. The Riemannian manifold with constant cross-sectional curvature is the simplest type, which can be divided into negative curvature, zero curvature, and positive curvature. Among them, positive curvature is used to describe elliptical geometry. In this case, the approximate condition of the hole can be determined based on the value of constant cross-sectional curvature.

[0122] (4) Determine whether there are manufacturing irregularities based on the curvature of the constant cross section;

[0123] S34: Determine the defect area for each defect type.

[0124] S4: Determine whether the defect meets the requirements of industrial production.

[0125] In this embodiment, the surface defects in the integrated circuit packaging process are diverse, and the production requirements for each type of defect are different. After the defects are segmented in step S3, it is determined whether they meet the industrial production requirements based on the shape, size, and other attributes of the defect area. If they meet the industrial production requirements, they are put into subsequent production; otherwise, they are discarded.

[0126] Example 2

[0127] This embodiment provides a surface defect detection system for the integrated circuit packaging process, which implements the surface defect detection method for the integrated circuit packaging process in Embodiment 1. The system includes: a microscopic imaging acquisition device, an image preprocessing module, a defect detection module, and a surface defect detection result output module.

[0128] In this embodiment, the microscopic imaging acquisition device is used to acquire surface digital images of the integrated circuit packaging process;

[0129] In this embodiment, the microscopic imaging acquisition device is a high-magnification metallurgical microscope;

[0130] In this embodiment, the image preprocessing module is used to perform image preprocessing operations on the acquired surface digital image;

[0131] In this embodiment, the defect detection module is used to detect different types of defect images after preprocessing and determine the defect region, specifically including:

[0132] Feature extraction is performed on the surface image of the preprocessed integrated circuit packaging process;

[0133] The defect type classification operation is performed on the feature-extracted image based on the defect characteristics;

[0134] For each type of defect, a corresponding defect detection method is used to achieve effective separation between the normal background and the defect area;

[0135] Identify the defect area for each defect type;

[0136] In this embodiment, the surface defect detection result output module is used to determine whether the defect meets the requirements of industrial production and output the surface defect detection result.

[0137] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for detecting surface defects in integrated circuit packaging processes, characterized in that, Includes the following steps: Acquire digital images of the surface of integrated circuits during the packaging process; Perform image preprocessing on the acquired digital images of the surface; Feature extraction is performed on the surface image of the preprocessed integrated circuit packaging process; Based on the defect features, the image after feature extraction is classified into defect types, including: manufacturing defects related to line width or line spacing scale parameters, manufacturing defects related to hole parameters, manufacturing defects related to irregularity, manufacturing defects related to non-uniformity, contamination-related color defects, and trauma-related scratch defects. For each type of defect, a corresponding defect detection method is employed to effectively separate the normal background from the defect area, specifically including: Analyze the characteristics of various defects and the relationships and properties between them; Detect manufacturing defects related to line width or line spacing dimensional parameters, and manufacturing defects related to hole parameters; Based on Jardon's curve theorem, region segmentation is performed for trauma-type scratches and manufacturing-type non-uniform defects, specifically including: The surface image of the pre-processed integrated circuit packaging process is converted into an HSV color space image, and the brightness V component of the color space image is obtained. Segmentation curves for surface images are constructed based on Jordan's curve theorem; Region segmentation of trauma-type scratches and manufacturing-type uneven defects based on segmentation curves; Based on the segmentation of the region, determine whether there are trauma-type scratches and manufacturing-type unevenness defects; Region segmentation for pollution-type color defects based on the Riemannian manifold method, specifically including: Obtain image background samples and map them to the Riemannian manifold space; Construct a similarity model between the image background sample and the image to be tested, and calculate the similarity between the image background sample and the image to be tested; The optimal threshold is obtained by utilizing the principle of low-probability events; Regions larger than the optimal threshold are classified as background, and regions smaller than the optimal threshold are classified as defects, thus completing the region segmentation of pollution-type color defects; Optimize the line edges of the preprocessed surface image, calculate the constant cross-sectional curvature, and determine whether there are manufacturing irregularities based on the constant cross-sectional curvature; Identify the defect area for each defect type; Determine whether the defects meet the requirements of industrial production and output the surface defect detection results.

2. The surface defect detection method for integrated circuit packaging process according to claim 1, characterized in that, The feature extraction of the surface image of the preprocessed integrated circuit packaging process is specifically performed by using a regularization method to optimize and improve the low-rank embedding algorithm to extract features from the surface image of the preprocessed integrated circuit packaging process.

3. The surface defect detection method for integrated circuit packaging process according to claim 1, characterized in that, The defect type classification operation of the image after feature extraction based on defect features is specifically based on the symptotic group classifier.

4. The surface defect detection method for integrated circuit packaging process according to claim 3, characterized in that, The defect type classification operation of the feature-extracted image is based on the symplectic group classifier, specifically including: Map the image dataset to Sp(2n), and take the n-dimensional row vectors of Sp(2n) to form a set. Specifically, it is expressed as: ； Wherein, the vector (I1, I2,…, I…) n ,) represents any image sample; Select Q i Convert to the corresponding symplectic matrix and find its singular values, specifically including: Q i When applied to an image dataset, it is represented as: ； Perform singular value decomposition on the training image samples; Take the first k largest singular values ​​and construct a discriminant function; Solve the discriminant function to output the category of the image dataset.

5. The surface defect detection method for integrated circuit packaging process according to claim 1, characterized in that, Optimizing the line edges of the preprocessed surface image specifically includes: The preprocessed surface image is converted from RGB color space to Lab color space, and image processing is performed based on morphological methods to extract the line edges in the L component of Lab color space. Fitting operations are performed based on the principle of numerical approximation to optimize the line edges extracted from the L component.

6. A surface defect detection system for integrated circuit packaging processes, characterized in that, To implement the surface defect detection method for integrated circuit packaging process according to any one of claims 1-5, the system includes: a microscopic imaging acquisition device, an image preprocessing module, a defect detection module, and a surface defect detection result output module; The microscopic imaging acquisition device is used to acquire surface digital images of the integrated circuit packaging process; The image preprocessing module is used to perform image preprocessing operations on the acquired surface digital images; The defect detection module is used to detect different types of defect images after preprocessing and determine the defect region, specifically including: Feature extraction is performed on the surface image of the preprocessed integrated circuit packaging process; The defect type classification operation is performed on the feature-extracted image based on the defect characteristics; For each type of defect, a corresponding defect detection method is used to achieve effective separation between the normal background and the defect area; Identify the defect area for each defect type; The surface defect detection result output module is used to determine whether the defect meets the requirements of industrial production and output the surface defect detection result.

Images