Intelligent defect analysis method and system based on chip external visual inspection

Abstract

The present invention provides an intelligent defect detection system and method based on external visual inspection of a chip. The system provided by the present invention includes a material transmission component that transmits a to-be-detected part from a first position to a second position; a material control component that moves the to-be-detected part from a first sampling direction to an Nth sampling direction; a light source that emits light to each detection surface formed by the to-be-detected part in the corresponding first sampling direction to an Nth sampling direction; an imaging module that obtains image data of the detection surface formed by the to-be-detected part from the first sampling direction to the Nth sampling direction; a data processing module that stores a pre-trained neural network model, obtains image data of the imaging module, and outputs a recognition result through the neural network model; a sorting module that is provided with a first sorting area to an Nth sorting area; and a control module that sends a signal to the sorting module according to the recognition result, and controls the sorting module to sort the to-be-detected part to any area in the first sorting area to the Nth sorting area. The present invention determines the defect area and the recognition result through a neural network.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition neural networks, and particularly relates to an intelligent defect analysis method and system based on external visual inspection of chips. Background Art

[0002] With the rapid development of the electronics industry, semiconductor chips are widely used in various fields. During the chip manufacturing process, surface defects are inevitable and need to be visually inspected to control product quality and ensure the normal progress of subsequent production processes.

[0003] Traditional chip visual inspection mainly relies on manual methods. The inspector manually places the chip to be inspected on a platform and conducts manual inspection by visual observation. After detecting a defect, the unqualified product is removed. During this inspection process, due to the visual fatigue that easily occurs after long hours of work by workers, misdetection and missed detection may occur; in addition, the traditional manual method has a low degree of automation, low efficiency, and cannot process abnormal parts in a timely manner, nor can it archive the corresponding measurement results, and thus cannot meet the work requirements of rising labor costs, high detection accuracy, and high speed in today's era.

[0004] In view of this, there is an urgent need to propose an intelligent defect analysis method and system based on external visual inspection of chips. Summary of the Invention

[0005] Therefore, the present invention provides an intelligent defect analysis method and system based on external visual inspection of chips, which uses a neural network model to perform the judgment and recognition of chip surface defects.

[0006] The first aspect of the present invention provides an intelligent defect analysis system based on external visual inspection of chips, including:

[0007] A material transfer component configured to transfer the part to be inspected from a first position to a second position;

[0008] A material control component configured to move the part to be inspected from a first sampling direction to an Nth sampling direction;

[0009] A light source configured to emit light to each detection surface formed by the part to be inspected in the corresponding first sampling direction to the Nth sampling direction;

[0010] An imaging module configured to acquire image data of the detection surface formed by the part to be inspected from the first sampling direction to the Nth sampling direction;

[0011] A data processing module configured to store a pre-trained neural network model and obtain an identification result output by the neural network model from the image data of the imaging module;

[0012] The sorting module is configured to be provided with a first sorting area to an Nth sorting area;

[0013] The control module is configured to send a signal to the sorting module according to the recognition result, and control the sorting module to sort the part to be detected to the corresponding area of the first sorting area to the Nth sorting area.

[0014] Further, the material transmission component includes

[0015] The material conveying tray is controlled by a control motor to transmit the part to be detected from the first position to the second position;

[0016] The picking manipulator is configured to pick up the part to be detected on the material conveying tray.

[0017] The second aspect of the present invention provides an intelligent defect analysis method based on external visual inspection of chips, including the following steps:

[0018] Including the following steps:

[0019] S1. Obtain a reference data set;

[0020] S2. Obtain a training data set;

[0021] S3. Determine the defect domain and determine the defect threshold for each defect domain;

[0022] S4. Iteratively train through the training data set to obtain the optimal defect threshold;

[0023] S5. Input the image to be detected and output the recognition result.

[0024] Further, the determination of the defect domain includes the following steps:

[0025] Determining the defect domain includes the following steps:

[0026] Divide the detected surface into multiple sample instances according to a preset grid size;

[0027] For each sample example, after determining the distribution area of the gray-scale mean value, determine at least the first gray scale and the second gray scale included in the sample instance, and fit the fitting boundary line between any first gray scale and the second gray scale;

[0028] Determine the first defect domain with the first fitting boundary line;

[0029] Determine the second defect domain with the second fitting boundary line;

[0030] Judge the defect type of the defect domain, and perform maximum boundary fitting on the images in the reference data set according to the defect area type to determine the defect threshold.

[0031] Further, step S4 is specifically as follows:

[0032] Iterative training;

[0033] In each iteration, the maximum fitting boundary obtained from the previous training is used as the fitting boundary limit line;

[0034] According to a preset decreasing training segmentation rate, the fitting boundary is divided into multiple boundary instances;

[0035] For each boundary instance, a new continuity threshold is determined, and an inference extension line is determined;

[0036] Taking the fitting boundary limit line as the reference value, the connection point of the inference extension line is fitted so that it does not exceed the fitting boundary limit line, obtaining a new maximum fitting boundary;

[0037] The area range between the fitting boundary line and the maximum fitting boundary is the new defect threshold;

[0038] Repeat the above steps until the decreasing training segmentation rate reaches a predetermined value to obtain the optimal defect threshold.

[0039] The above technical solution of the present invention has the following advantages compared with the prior art:

[0040] The present invention realizes defect recognition for the external image detection of the chip through training with a neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a structural block diagram of an intelligent defect analysis system based on external visual inspection of a chip provided in Embodiment 1 of the present invention;

[0042] Figure 2 is a flowchart of an intelligent defect analysis method based on external visual inspection of a chip provided in Embodiment 1 of the present invention;

[0043] Figure 3 is a structural diagram of an intelligent defect analysis system based on external visual inspection of a chip provided in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of explanation, numerous details are provided to give a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices may be shown in a simplified manner to simplify the drawings. Embodiment 1

[0045] An embodiment of the present disclosure provides an intelligent defect analysis system based on external visual inspection of chips, such as Figure 1 and Figure 3 shown. A first aspect of the present invention provides an intelligent defect analysis system based on external visual inspection of chips, including:

[0046] A material transfer component configured to transfer a workpiece to be detected from a first position to a second position;

[0047] A material control component configured to move the workpiece to be detected from a first sampling direction to an Nth sampling direction;

[0048] A light source configured to emit light to each detection surface formed by the workpiece to be detected in the corresponding first sampling direction to the Nth sampling direction;

[0049] An imaging module configured to acquire image data of the detection surface formed by the workpiece to be detected from the first sampling direction to the Nth sampling direction;

[0050] A data processing module configured to store a pre-trained neural network model and obtain an identification result output by the neural network model from the image data of the imaging module;

[0051] A sorting module configured to be provided with a first sorting area to an Nth sorting area;

[0052] A control module configured to send a signal to the sorting module according to the identification result and control the sorting module to sort the workpiece to be detected to the corresponding areas of the first sorting area to the Nth sorting area.

[0053] Further, the material transfer component includes

[0054] A material conveying tray controlled by a control motor to transfer a workpiece to be detected from a first position to a second position;

[0055] A material taking manipulator configured to grasp the workpiece to be detected on the material conveying tray.

[0056] According to the appendix of this embodiment Figure 1 , specifically, it includes:

[0057] A microscopic imaging device: used to collect high-definition images of chips;

[0058] A material transfer component: including a material taking manipulator, a material distributing manipulator, a material conveying tray, a material sorting tray, and a positioning sensor;

[0059] A material control component for fixing the chip to be measured and rotating the chip to be measured;

[0060] Illumination structure for providing illumination to the chip under test; the illumination structure can set different illumination effects according to image acquisition requirements;

[0061] Defect analysis and processing center, that is, the data processing module is divided into different defect domains. Parameters are set before detection, and corresponding images are selected to enter different defect domains for defect analysis, and a defect analysis report is generated;

[0062] Control system: Place one side of the chip to be detected on the material conveying tray, control the pick-up manipulator to grab the chip to be tested from the material conveying tray and put it into the material control device; the material control device controls the rotation of the chip to enable the microscopic imaging device to sequentially collect high-definition images of different chips; the microscopic imaging device sends the collected images to the defect analysis and processing center for defect analysis to generate a defect analysis report; the sorting manipulator puts the detected chips into the qualified area or the corresponding defect area of the material sorting tray; put the other side to be inspected of the once qualified chips into the material conveying tray, repeat the above steps, and finally the sorting manipulator puts the detected chips into the qualified area or the corresponding defect area of the material sorting tray. Embodiment 2

[0063] The present disclosure embodiment provides an intelligent defect analysis method based on external visual inspection of chips, as Figure 2 shown, including the following steps:

[0064] S1. Obtain a reference data set;

[0065] S2. Obtain a training data set;

[0066] S3. Determine the defect domain and determine the defect threshold of each defect domain;

[0067] S4. Iteratively train through the training data set to obtain the optimal defect threshold;

[0068] S5. Input the image to be detected and output the recognition result.

[0069] Further, the determination of the defect domain includes the following steps:

[0070] Divide the detected surface into multiple sample instances according to a preset grid size;

[0071] For each sample example, after determining the distribution area of the gray mean value, determine at least the first gray value and the second gray value included in the sample instance, and fit the fitting boundary line between any first gray value and the second gray value; the fitting boundary line is a boundary fitting performed with gray value pixels according to the distribution of the gray mean value.

[0072] Determine the first defect domain with the first fitting boundary line;

[0073] Determine a second defect domain with a second fitting boundary line;

[0074] Judge the defect type of the defect domain, perform maximum boundary fitting on the images in the reference dataset according to the defect region type, and determine the defect threshold.

[0075] Further, the specific step S4 is as follows:

[0076] Iterative training;

[0077] In each iteration, use the maximum fitting boundary of the previous training as the fitting boundary limit line;

[0078] According to a preset decreasing training segmentation rate, divide the fitting boundary into multiple boundary instances;

[0079] For each boundary instance, determine a new continuity threshold and determine the inference extension line;

[0080] Taking the fitting boundary limit line as the reference value, set the connection point fitting of the inference extension line to not exceed the fitting boundary limit line to obtain a new maximum fitting boundary;

[0081] The area range between the fitting boundary line and the maximum fitting boundary is the new defect threshold;

[0082] Repeat the above steps until the decreasing training segmentation rate reaches a predetermined value.

[0083] In the embodiment of the present disclosure, the decreasing training segmentation rate is a preset learning rate. After the fitting boundary in the model is divided according to the preset initial segmentation length, each time it is divided decrementally, and the segmentation length of each training is shorter than that of the previous training. In the embodiment of the present disclosure, the training segmentation rate is divided to the predetermined value.

[0084] The continuity threshold defined in the embodiment of the present disclosure is a threshold set when the curvature change continuity rate and the curvature change tolerance of each pixel in the boundary instance are both within a preset range. And through the continuity threshold, the curvature median value and the curvature change continuity rate median value of each pixel are determined, and then the inference extension line of this section of the boundary can be obtained. By calculating the maximum fitting boundary fitted between the connection points of the inference extension lines of each boundary instance, a new boundary image is obtained, and this boundary image is compared with the reference dataset, and its similarity rate is calculated.

[0085] Taking the occluded pattern as an example in the embodiment of the present disclosure, if a pattern is occluded and forms a first gray area and a second gray area with two sets of gray values, after fitting the first gray area, the maximum fitting boundary of the fitting is greater than the original fitting boundary. By gradually dividing the boundary, a more accurate original boundary can be obtained to obtain more accurate data by comparing with the reference dataset.

[0086] It should be noted that the foregoing description in the embodiments of the present disclosure is about the curvature boundary.

[0087] If there is no curvature at this boundary, fitting is performed through the separation degree of pixels. Referring to the setting for the decreasing training segmentation rate, there is a setting similar to the decreasing training segmentation rate in this setting.

[0088] Set the decreasing training separation degree, and divide the fitting boundary into multiple separation degree boundary instances.

[0089] For each boundary instance, determine the separation degree threshold and determine the extended line of the separation inference.

[0090] The extended line of the separation inference is the extended line connecting from the training decreasing separation degree region of one boundary instance to the system training decreasing separation degree region of the closest boundary instance.

[0091] Repeat the above steps until the decreasing training separation degree reaches a predetermined value.

[0092] It should be noted that the extended line of the separation inference obtained through each decreasing training separation degree in the embodiments of the present disclosure needs to be fitted to all the maximum fitting boundaries to obtain multiple sets of images, and then determine the optimal setting of the separation degree threshold.

[0093] Such as Figure 2 shown, the embodiments of the present disclosure use a CNN model for training, and its model structure will not be elaborated here.

[0094] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure, enabling those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments merely represent possible variations. Unless explicitly required, the individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or substituted for parts and features of other embodiments. Moreover, the terms used in this application are only for describing the embodiments and do not limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to also include the plural forms. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations including one or more of the associated listed items. Additionally, when used in this application, the term "comprise" and its variants "comprises" and / or "comprising" etc. mean the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, or apparatus comprising the element. In this document, each embodiment may focus on the differences from other embodiments, and the same or similar parts among the embodiments may be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, the relevant parts may refer to the description of the method part.

Claims

1. Intelligent defect analysis system based on chip external visual inspection, characterized by: include: A material transfer assembly configured to transfer the part to be inspected from a first position to a second position; A material control component, configured to move the to-be-detected part from a first sampling direction to an Nth sampling direction; A light source configured to emit light to each detection surface of the to-be-detected component formed corresponding to the first sampling direction to the Nth sampling direction; An imaging module is configured to obtain image data of a detection surface formed by the component to be detected from the first sampling direction to the Nth sampling direction; A data processing module is configured to store a pre-trained neural network model, obtain image data of the imaging module, and output a recognition result through the neural network model; The sorting module is configured to be provided with a first sorting area to an Nth sorting area; A control module, configured to send a signal to the sorting module according to the recognition result, and control the sorting module to sort the parts to be detected to corresponding areas of the first sorting area to the Nth sorting area; The system is implemented by the following method, comprising the following steps: S1. Obtain a reference data set; S2, obtain training data set; S3, determining a defect definition domain, and determining a defect threshold value of each defect definition domain; Determining the defect definition domain includes the following steps: Dividing the detection surface into a plurality of sample instances according to a preset grid size; For each sample example, after determining the distribution area of ​​the grayscale mean, determine at least the first grayscale and the second grayscale included in the sample example, and fit a fitting boundary line between any first grayscale and the second grayscale; determining a first defect definition domain having a first fitting boundary line; determining a second defect definition domain having a second fitting boundary line; Determine the defect type of the defect definition domain, perform maximum boundary fitting on the image in the reference data set according to the defect type, and determine the defect threshold; S4, iteratively train the training data set to obtain the optimal defect threshold; S5, input the image to be detected and output the recognition result; Before executing step S3, the following steps are also included: Perform grayscale processing on the images in the reference data and sample data sets; A differential image set is obtained; Before the step S4, the method further includes performing binarization processing on the corresponding defects in the differential image to obtain a binarized image.

2. The intelligent defect analysis system based on chip external visual inspection according to claim 1 is characterized in that: The material transmission component includes The material conveying plate is controlled by a control motor to transfer the part to be inspected from a first position to a second position; The material picking robot is configured to grab the parts to be inspected on the material conveying tray.

3. The intelligent defect analysis system based on chip external visual inspection according to claim 1 is characterized in that: The specific steps of the maximum boundary fitting are: Split the fitted boundary into multiple boundary instances, For each boundary instance, determining its continuity threshold, and calculating the inference extension line of the boundary instance according to the continuity threshold; The reasoning extension lines of all boundary instances intersect to form multiple connection points, and the connection lines formed by multiple connection points are fitted to obtain the maximum fitting boundary. The area between the fitting boundary line and the maximum fitting boundary is the defect threshold.

4. The intelligent defect analysis system based on chip external visual inspection according to claim 3 is characterized in that: The step S4 is specifically as follows: Iterative training; In each iteration, the maximum fitting boundary of the last training is used as the fitting boundary limit line; According to a preset decreasing training splitting rate, the fitted boundary is split into multiple boundary instances; For each boundary instance, a new continuity threshold is determined and the inference extension line is determined; Taking the fitting boundary limit line as the reference value, the connection point fitting of the inference extension line is set to not exceed the fitting boundary limit line, so as to obtain a new maximum fitting boundary; The area between the fitting boundary line and the maximum fitting boundary is the new defect threshold; Repeat the above steps until the training segmentation rate is reduced to a predetermined value and the optimal defect threshold is obtained.

Images