Chip defect detection method and system based on neural network

Abstract

The invention discloses a chip defect detection method and system based on a neural network, and the method comprises the steps: carrying out the photographing of a chip on a production line through employing a high-definition photographing device, and obtaining the real-time image data of the chip. According to the method, an image acquisition and recognition technology is adopted, firstly, acquired images are subjected to noise reduction processing, so that data are in the same standard, have reference and can be studied, a defect feature matching model meeting requirements is constructed by using a neural network model through a large amount of historical image data, and due to the fact that the types of defects are different, the defect feature matching accuracy is improved. According to the method, real-time image data are divided into multiple degrees, such as large appearance defect depth, large length, large width and the like, and the real-time image data are input into a defect feature matching model to obtain a matching value, so that the traditional mode of qualitatively checking chip defects by depending on the experience of professionals in the prior art is eliminated; and powerful technical support is provided for batch detection of chip defects.

Description

Technical Field

[0001] The present invention relates to the technical field of chip defect detection, and in particular to a chip defect detection method and system based on neural network. Background Art

[0002] In the chip production process, the wafer processing process is complex and closely linked, and the production cost is high. Affected by various factors such as process and raw materials, atoms do not absolutely and strictly follow the periodic arrangement of the lattice. Different types of defects may exist in the process of crystal growth and chemical vapor deposition. Chip defects are divided into appearance defects and internal defects. Appearance defects are detected by the depth, length and width of the defects to determine whether they meet the production standards.

[0003] The traditional method is for workers to inspect the chips on the production line one by one. This method relies on the workers' inspection experience, is time-consuming and labor-intensive, and is prone to missed inspections, false inspections, etc. It is also difficult to guarantee accuracy and it is difficult to meet the needs of workers. Summary of the invention

[0004] In order to solve the above technical problems, a neural network-based chip defect detection method and system are provided. This technical solution solves the problem that the traditional method proposed in the above background technology is that the staff detects the chips on the production line one by one. This method needs to rely on the staff's detection experience, is time-consuming and labor-intensive, and is prone to missed detections, false detections, etc., and the accuracy is difficult to guarantee.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A chip defect detection method based on neural network, comprising:

[0007] Use high-definition camera equipment to shoot the chips on the production line and obtain real-time image data of the chips;

[0008] Retrieving historical image data of the chip from the database, and preprocessing the historical image data, wherein the preprocessing is image noise reduction processing;

[0009] Performing rectangular positioning and tilt angle correction on the preprocessed image to obtain rectangular chip image data;

[0010] Perform defect recognition on the rectangular chip image data to obtain defect set data, wherein the defect set data includes length information, width information and depth information of the defect;

[0011] Based on the length, width and depth information of the defect, a neural network is used to build a defect feature matching model;

[0012] Input the real-time chip image data into the feature matching model and calculate the matching value;

[0013] Issue an early warning for chip image data with a large matching value.

[0014] Preferably, the preprocessing of the historical image data specifically comprises the following steps:

[0015] Decompose the high-frequency and low-frequency information of noisy images;

[0016] The pixels of low-frequency information are processed by local pixel grouping, and the unbiased estimation of the error is used to approximate the similarity between the local pixel block and the target pixel block, and a sample set of similar local pixel blocks is obtained;

[0017] Traverse each sample set obtained, use the principal component analysis algorithm to denoise it in turn, calculate the covariance matrix, obtain the orthogonal transformation matrix, and combine it with the eigenvalue matrix to remove the dimensions containing a small amount of information in the sample set to obtain the reconstructed low-frequency components;

[0018] Decompose high-frequency information into overlapping blocks of the same size, calculate the Euclidean distance to construct similar blocks into groups, use singular value decomposition to learn the adaptive learning dictionary of each group, calculate sparse coding through the split Bregman iterative algorithm combined with convex optimization algorithm, and reconstruct the high-frequency components using sparse coding and adaptive learning dictionary;

[0019] The inverse wavelet transform aggregates the high-frequency components and the low-frequency components to obtain the denoised image.

[0020] Preferably, the rectangular positioning and tilt angle correction of the pre-processed image specifically comprises the following steps:

[0021] Perform edge detection on the image in the horizontal and vertical directions, and sum the detection results in the horizontal and vertical directions;

[0022] Based on the first morphological closing operation, the defective part of the image is processed to obtain the path area, and then the second morphological closing operation is used to remove the single line interference on the image;

[0023] Filling holes in the image processed by the first morphological closing operation;

[0024] Calculate the minimum bounding rectangle according to each white area in the image after the holes are filled, and obtain information of each bounding rectangle, wherein the rectangle information includes coordinates of the four vertices of the rectangle, an inclination angle, and the number of white pixels in the rectangular area;

[0025] According to the calculated rectangular tilt angles, their distribution within the right angle range is counted, and the angles with more concentrated distribution are used to perform preliminary tilt angle correction.

[0026] Preferably, the calculation formula of the minimum enclosing rectangle is:

[0027] S min =min(S1,S2,.....S n )

[0028] In the formula, S min is the minimum enclosing rectangle;

[0029] S1,S2,.....S n is the area of ​​several groups of bounding rectangles.

[0030] Preferably, the step of obtaining rectangular chip image data specifically comprises the following steps:

[0031] In the image after the initial tilt angle correction, the defect area is located in the zero degree direction or the ninety degree direction, and the error is within the range of plus or minus ten degrees;

[0032] The candidate rectangles of the defect area are screened at the zero-degree and ninety-degree directions respectively to determine the final position and final angle of the defect area;

[0033] Perform a final rotation to obtain rectangular chip image data.

[0034] Preferably, the method of constructing a defect feature matching model using a neural network based on the length information, width information and depth information of the defect specifically comprises the following steps:

[0035] Based on the feedforward neural network, a defect feature matching model is constructed;

[0036] Based on machine learning, the plurality of defect length information, the plurality of width information and the plurality of depth information are labeled and divided to obtain a training set, a validation set and a test set;

[0037] The training set, validation set and test set are used to perform supervised training, validation and testing on the defect feature matching model to obtain the defect feature matching model whose accuracy meets the preset accuracy requirement.

[0038] Preferably, the specific calculation formula of the feature matching model is:

[0039]

[0040] Among them, P j is the matching degree between the input defect set data and the jth defect type, N j is the total number of historical defect set data corresponding to the jth defect type, N0 is the total number of features of the input defect set data, N 同is the total number of identical features between the feature data of the input defect set data and the historical defect set data corresponding to the j-th defect type, α' l is the correlation weight between the feature data of the j-th defect type and the l-th input defect set data and the same feature in the historical defect set data corresponding to the j-th defect type, and max() is the maximum value function.

[0041] Preferably, the step of issuing an early warning for chip image data with a larger matching value specifically includes the following steps:

[0042] Compare the calculated feature matching value with the preset threshold;

[0043] If the feature matching value is greater than or equal to the preset threshold, the produced chip has a major appearance defect, an early warning is issued, and the parameters in the production line are readjusted;

[0044] If the feature matching value is less than the preset threshold, the produced chip has minor appearance defects and no processing is performed.

[0045] A chip defect detection system based on neural network, comprising:

[0046] A shooting module, which uses high-definition shooting equipment to shoot chips on the production line to obtain real-time image data of the chips;

[0047] An image data processing module, the image data processing module is used to retrieve the historical image data of the chip from the database, pre-process the historical image data, and obtain the chip image data in a regular rectangular shape;

[0048] A feature matching model building module, wherein the feature matching model building module is used to build a defect feature matching model using a neural network based on length information, width information and depth information of the defect;

[0049] A calculation module, which is used to input real-time chip image data into a feature matching model to calculate a matching value;

[0050] An early warning module, which is used to issue an early warning for chip image data with a large matching value;

[0051] The image data processing module internally integrates a preprocessing unit and an image correction unit. The preprocessing unit is used to perform noise reduction processing on historical image data, and the image correction unit is used to perform rectangular positioning and tilt angle correction on the image after noise reduction processing.

[0052] Compared with the prior art, the present invention provides a chip defect detection method and system based on neural network, which has the following beneficial effects:

[0053] In real life, chip defect parameters are important data that describe the characteristics of internal or surface defects of chips. These parameters are crucial for evaluating the quality, performance and reliability of chips. The present invention detects the appearance defects of chips, that is, the physical factors of their appearance. The appearance defects that affect the chip include the length, width and depth of the defects. The present invention adopts image acquisition and recognition technology. The acquired images are first subjected to noise reduction processing so that the data are in the same standard, which is referenceable and researchable. A defect feature matching model that meets the requirements is constructed using a neural network model through a large amount of historical image data. The historical image data in the present invention can be chips on the same and same batch production line or chips on different batches of production lines. Since the types of defects are also different, they are divided into multiple degrees. For example, some appearance defects have a large depth, some have a large length, and some have a large width, etc. The real-time image data is input into the defect feature matching model to obtain the matching value, which gets rid of the traditional mode of qualitatively checking chip defects by relying on the experience of professionals in the prior art, and provides strong technical support for batch detection of chip defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of the chip defect detection method of the present invention;

[0055] Figure 2 A schematic diagram of a method for preprocessing historical image data in the present invention;

[0056] Figure 3 It is a schematic diagram of a method for performing rectangle positioning and tilt angle correction on a pre-processed image in the present invention;

[0057] Figure 4 A schematic diagram of a method for obtaining rectangular chip image data in the present invention;

[0058] Figure 5 A schematic diagram of a method for constructing a defect feature matching model using a neural network in the present invention;

[0059] Figure 6 This is a schematic diagram of a method for providing early warning for chip image data with a large matching value in the present invention. DETAILED DESCRIPTION

[0060] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0061] Example 1

[0062] Please refer to Figure 1-Figure 6 As shown, a chip defect detection method based on neural network includes:

[0063] Use high-definition camera equipment to shoot the chips on the production line and obtain real-time image data of the chips;

[0064] Retrieving the historical image data of the chip from the database and preprocessing the historical image data, wherein the preprocessing is image noise reduction processing;

[0065] Performing rectangular positioning and tilt angle correction on the preprocessed image to obtain rectangular chip image data;

[0066] Perform defect recognition on rectangular chip image data to obtain defect set data, which includes length information, width information and depth information of the defect;

[0067] Based on the length, width and depth information of the defect, a neural network is used to build a defect feature matching model;

[0068] Input the real-time chip image data into the feature matching model and calculate the matching value;

[0069] Issue an early warning for chip image data with a large matching value.

[0070] It can be understood by those skilled in the art that chip defect parameters are important data describing the characteristics of internal or surface defects of a chip. These parameters are crucial for evaluating the quality, performance and reliability of a chip. The present invention detects the appearance defects of a chip, that is, the physical factors of its appearance. The appearance defects that affect the chip include the length, width and depth of the defect. The present invention adopts image acquisition and recognition technology. The acquired image is first subjected to noise reduction processing so that the data is in the same standard and has reference and research value. A defect feature matching model that meets the requirements is constructed using a neural network model through a large amount of historical image data. The historical image data in the present invention can be chips from the same and same batch of production lines or chips from different batches of production lines. Since the types of defects are also different, they are divided into multiple degrees. For example, some appearance defects have a large depth, some have a large length, and some have a large width, etc. The real-time image data is input into the defect feature matching model to obtain a matching value, thereby getting rid of the traditional mode of qualitatively inspecting chip defects by relying on the experience of professionals in the prior art, and providing strong technical support for batch detection of chip defects.

[0071] The preprocessing of historical image data specifically includes the following steps:

[0072] Decompose the high-frequency and low-frequency information of noisy images;

[0073] The pixels of low-frequency information are processed by local pixel grouping, and the unbiased estimation of the error is used to approximate the similarity between the local pixel block and the target pixel block, and a sample set of similar local pixel blocks is obtained;

[0074] Traverse each sample set obtained, use the principal component analysis algorithm to denoise it in turn, calculate the covariance matrix, obtain the orthogonal transformation matrix, and combine it with the eigenvalue matrix to remove the dimensions containing a small amount of information in the sample set to obtain the reconstructed low-frequency components;

[0075] Decompose high-frequency information into overlapping blocks of the same size, calculate the Euclidean distance to construct similar blocks into groups, use singular value decomposition to learn the adaptive learning dictionary of each group, calculate sparse coding through the split Bregman iterative algorithm combined with convex optimization algorithm, and reconstruct the high-frequency components using sparse coding and adaptive learning dictionary;

[0076] The inverse wavelet transform aggregates the high-frequency components and the low-frequency components to obtain the denoised image.

[0077] It can be understood by those skilled in the art that, through wavelet transform, the high-frequency information and low-frequency information of the image are processed separately, and the low-frequency components are restored using the LPG-PCA method. In the process of restoring the high-frequency components, the natural image is sparsely represented in the group domain, and the inherent local sparsity and non-local self-similarity of the natural image are clearly and effectively characterized in a unified manner, and the high-frequency part is extracted from the image without being affected by noise. While effectively removing noise, the image details are retained, high sparsity and high restoration quality are achieved, and the visual performance of the image as well as the peak signal-to-noise ratio and structural similarity are improved.

[0078] The steps of performing rectangle positioning and tilt angle correction on the preprocessed image are as follows:

[0079] Perform edge detection on the image in the horizontal and vertical directions, and sum the detection results in the horizontal and vertical directions;

[0080] Based on the first morphological closing operation, the defective part of the image is processed to obtain the path area, and then the second morphological closing operation is used to remove the single line interference on the image;

[0081] Filling holes in the image processed by the first morphological closing operation;

[0082] Calculate the minimum bounding rectangle according to each white area in the image after the holes are filled, and obtain the information of each bounding rectangle, which includes the coordinates of the four vertices of the rectangle, the tilt angle, and the number of white pixels in the rectangular area;

[0083] According to the calculated rectangular tilt angles, their distribution within the right angle range is counted, and the angles with more concentrated distribution are used to perform preliminary tilt angle correction.

[0084] The calculation formula for the minimum enclosing rectangle is:

[0085] S min =min(S1,S2,.....S n )

[0086] In the formula, S min is the minimum enclosing rectangle;

[0087] S1,S2,.....S n is the area of ​​several groups of bounding rectangles.

[0088] The steps of obtaining rectangular chip image data specifically include the following steps:

[0089] In the image after the initial tilt angle correction, the defect area is located in the zero degree direction or the ninety degree direction, and the error is within the range of plus or minus ten degrees;

[0090] The candidate rectangles of the defect area are screened at the zero-degree and ninety-degree directions respectively to determine the final position and final angle of the defect area;

[0091] Perform a final rotation to obtain rectangular chip image data.

[0092] Based on the length information, width information and depth information of the defect, the defect feature matching model is constructed using a neural network, which specifically includes the following steps:

[0093] Based on the feedforward neural network, a defect feature matching model is constructed;

[0094] Based on machine learning, multiple defect length information, multiple width information, and multiple depth information are labeled and divided to obtain training sets, verification sets, and test sets;

[0095] The defect feature matching model is supervised trained, verified and tested using the training set, validation set and test set to obtain a defect feature matching model whose accuracy meets the preset accuracy requirements.

[0096] The specific calculation formula of the feature matching model is:

[0097]

[0098] Among them, P j is the matching degree between the input defect set data and the jth defect type, N j is the total number of historical defect set data corresponding to the jth defect type, N0 is the total number of features of the input defect set data, N同 is the total number of identical features between the feature data of the input defect set data and the historical defect set data corresponding to the j-th defect type, α' l is the correlation weight between the feature data of the j-th defect type and the l-th input defect set data and the same feature in the historical defect set data corresponding to the j-th defect type, and max() is the maximum value function.

[0099] The specific steps of issuing an early warning for chip image data with a large matching value include the following:

[0100] Compare the calculated feature matching value with the preset threshold;

[0101] If the feature matching value is greater than or equal to the preset threshold, the produced chip has a major appearance defect, an early warning is issued, and the parameters in the production line are readjusted;

[0102] If the feature matching value is less than the preset threshold, the produced chip has minor appearance defects and no processing is performed.

[0103] A chip defect detection system based on neural network, comprising:

[0104] The shooting module uses high-definition shooting equipment to shoot the chips on the production line and obtain real-time image data of the chips;

[0105] An image data processing module, the image data processing module is used to retrieve the historical image data of the chip from the database, pre-process the historical image data, and obtain the chip image data in a regular rectangular shape;

[0106] A feature matching model building module, wherein the feature matching model building module is used to build a defect feature matching model using a neural network based on length information, width information and depth information of the defect;

[0107] A calculation module, which is used to input real-time chip image data into a feature matching model to calculate a matching value;

[0108] An early warning module, which is used to issue an early warning for chip image data with a large matching value;

[0109] The image data processing module internally integrates a preprocessing unit and an image correction unit. The preprocessing unit is used to perform noise reduction processing on historical image data, and the image correction unit is used to perform rectangular positioning and tilt angle correction on the image after noise reduction processing.

[0110] In summary, chip defect parameters are important data that describe the characteristics of internal or surface defects of chips. These parameters are crucial for evaluating the quality, performance and reliability of chips. The present invention detects the appearance defects of chips, that is, the physical factors of their appearance. The appearance defects that affect the chip include the length, width and depth of the defects. The present invention adopts image acquisition and recognition technology. The collected images are first subjected to noise reduction processing so that the data are in the same standard, which is referenceable and researchable. A defect feature matching model that meets the requirements is constructed using a neural network model through a large amount of historical image data. The historical image data in the present invention can be chips on the same and same batch production line or chips on different batches of production lines. Since the types of defects are also different, they are divided into multiple degrees. For example, some appearance defects have a large depth, some have a large length, and some have a large width, etc. The real-time image data is input into the defect feature matching model to obtain the matching value, which gets rid of the traditional mode of qualitatively checking chip defects by relying on the experience of professionals in the prior art, and provides strong technical support for batch detection of chip defects.

[0111] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A chip defect detection method based on neural network, characterized in that: include: Use high-definition camera equipment to shoot the chips on the production line and obtain real-time image data of the chips; Retrieving historical image data of the chip from the database, and preprocessing the historical image data, wherein the preprocessing is image noise reduction processing; Performing rectangular positioning and tilt angle correction on the preprocessed image to obtain rectangular chip image data; Perform defect recognition on the rectangular chip image data to obtain defect set data, wherein the defect set data includes length information, width information and depth information of the defect; Based on the length, width and depth information of the defect, a neural network is used to build a defect feature matching model; Input the real-time chip image data into the feature matching model and calculate the matching value; Issue an early warning for chip image data with a large matching value.

2. The chip defect detection method based on neural network according to claim 1, characterized in that: The preprocessing of the historical image data specifically comprises the following steps: Decompose the high-frequency and low-frequency information of noisy images; The pixels of low-frequency information are processed by local pixel grouping, and the unbiased estimation of the error is used to approximate the similarity between the local pixel block and the target pixel block, and a sample set of similar local pixel blocks is obtained; Traverse each sample set obtained, use the principal component analysis algorithm to denoise it in turn, calculate the covariance matrix, obtain the orthogonal transformation matrix, and combine it with the eigenvalue matrix to remove the dimensions containing a small amount of information in the sample set to obtain the reconstructed low-frequency components; Decompose high-frequency information into overlapping blocks of the same size, calculate the Euclidean distance to construct similar blocks into groups, use singular value decomposition to learn the adaptive learning dictionary of each group, calculate sparse coding through the split Bregman iterative algorithm combined with convex optimization algorithm, and reconstruct the high-frequency components using sparse coding and adaptive learning dictionary; The inverse wavelet transform aggregates the high-frequency components and the low-frequency components to obtain the denoised image.

3. The chip defect detection method based on neural network according to claim 2, characterized in that: The rectangular positioning and tilt angle correction of the pre-processed image specifically comprises the following steps: Perform edge detection on the image in the horizontal and vertical directions, and sum the detection results in the horizontal and vertical directions; Based on the first morphological closing operation, the defective part of the image is processed to obtain the path area, and then the second morphological closing operation is used to remove the single line interference on the image; Filling holes in the image processed by the first morphological closing operation; Calculate the minimum bounding rectangle according to each white area in the image after the holes are filled, and obtain information of each bounding rectangle, wherein the rectangle information includes coordinates of the four vertices of the rectangle, an inclination angle, and the number of white pixels in the rectangular area; According to the calculated rectangular tilt angles, their distribution within the right angle range is counted, and the angles with more concentrated distribution are used to perform preliminary tilt angle correction.

4. The chip defect detection method based on neural network according to claim 3 is characterized in that: The calculation formula of the minimum enclosing rectangle is: S min =min(S1,S2,.....S n ) In the formula, S min is the minimum enclosing rectangle; S1,S2,.....S n is the area of ​​several groups of bounding rectangles.

5. The chip defect detection method based on neural network according to claim 4 is characterized in that: The obtaining of rectangular chip image data specifically comprises the following steps: In the image after the initial tilt angle correction, the defect area is located in the zero degree direction or the ninety degree direction, and the error is within the range of plus or minus ten degrees; The candidate rectangles of the defect area are screened at the zero-degree and ninety-degree directions respectively to determine the final position and final angle of the defect area; Perform a final rotation to obtain rectangular chip image data.

6. The chip defect detection method based on neural network according to claim 5, characterized in that: The defect feature matching model constructed by using a neural network based on the length information, width information and depth information of the defect specifically includes the following steps: Based on the feedforward neural network, a defect feature matching model is constructed; Based on machine learning, the plurality of defect length information, the plurality of width information and the plurality of depth information are labeled and divided to obtain a training set, a validation set and a test set; The training set, validation set and test set are used to perform supervised training, validation and testing on the defect feature matching model to obtain the defect feature matching model whose accuracy meets the preset accuracy requirement.

7. The chip defect detection method based on neural network according to claim 6, characterized in that: The specific calculation formula of the feature matching model is: Among them, P j is the matching degree between the input defect set data and the jth defect type, N j is the total number of historical defect set data corresponding to the jth defect type, N0 is the total number of features of the input defect set data, N 同 is the total number of identical features between the feature data of the input defect set data and the historical defect set data corresponding to the j-th defect type, α' l is the correlation weight between the feature data of the j-th defect type and the l-th input defect set data and the same feature in the historical defect set data corresponding to the j-th defect type, and max() is the maximum value function.

8. The chip defect detection method based on neural network according to claim 7, characterized in that: The step of issuing an early warning for chip image data with a large matching value specifically includes the following steps: Compare the calculated feature matching value with the preset threshold; If the feature matching value is greater than or equal to the preset threshold, the produced chip has a major appearance defect, an early warning is issued, and the parameters in the production line are readjusted; If the feature matching value is less than the preset threshold, the produced chip has minor appearance defects and no processing is performed.

9. A chip defect detection system based on a neural network, used to implement a chip defect detection method based on a neural network as claimed in any one of claims 1 to 8, characterized in that: include: A shooting module, which uses high-definition shooting equipment to shoot chips on the production line to obtain real-time image data of the chips; An image data processing module, the image data processing module is used to retrieve the historical image data of the chip from the database, pre-process the historical image data, and obtain the chip image data in a regular rectangular shape; A feature matching model building module, wherein the feature matching model building module is used to build a defect feature matching model using a neural network based on length information, width information and depth information of the defect; A calculation module, which is used to input real-time chip image data into a feature matching model to calculate a matching value; The early warning module is used to issue an early warning for chip image data with a large matching value.

10. The chip defect detection system based on neural network according to claim 9, characterized in that: The image data processing module internally integrates a preprocessing unit and an image correction unit. The preprocessing unit is used to perform noise reduction processing on historical image data, and the image correction unit is used to perform rectangular positioning and tilt angle correction on the image after noise reduction processing.

Images