Wafer edge contamination detection method, device, equipment and storage medium

Abstract

The application discloses a wafer edge contamination detection method, device and equipment and a storage medium, relates to the field of image processing, generates a wafer pad image by detecting a collected image, and determines a suspicious image that cannot be classified; the suspicious image is subjected to gray scale processing and erosion operation to obtain an original edge image; the original edge image is subjected to connectivity analysis, filtering, breakpoint filling and repair to obtain a repaired pad image; the edge contour of the repaired pad image is extracted, the frame of the repaired pad image is determined based on the edge contour and the surrounding area formed by the contour, and edge comparison analysis is performed on the wafer pad image to determine the contamination condition. According to the scheme, the suspicious image is analyzed by combining deep learning prediction and digital image processing, the contamination condition is determined by comparing and analyzing the frame repair result and the original image, and compared with the prediction result of the common image recognition detection model, the accuracy is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a wafer edge contamination detection method, device, equipment and storage medium. BACKGROUND

[0002] In the production process of semiconductor chips, wafer pads may form flaws, defects or contamination due to various factors, especially wafer pad edge wear and deformation. In the industrial field, digital image processing technology can usually be used for identification and detection.

[0003] In related technologies, in the factory quality inspection scene, there are problems such as light and environment changes, wafer pad deformation and damage, large-scale data processing and diversity, complexity, etc. Poor light environment, wafer pad wear, dirt, etc. can all cause image quality to decline, affecting the accuracy of image processing algorithms; and different shapes, sizes, colors and textures greatly increase the complexity of the algorithm and the real-time nature of the judgment. Ordinary detection models cannot guarantee high flexibility in large-scale data processing scenarios and real-time scenarios, and there is a high rate of missed detection and false detection. SUMMARY

[0004] The embodiments of the present application provide a wafer edge contamination detection method, device, equipment and storage medium, which improves the accuracy and real-time performance of image recognition in large-scale wafer pad data scenarios.

[0005] In one aspect, the present application provides a wafer edge contamination detection method, the method comprising:

[0006] Target detection and identification are performed on the collected images in the database, wafer pad images are generated and extracted, and pad images in doubt that cannot be classified are determined through classification and screening;

[0007] The doubtful pad images are subjected to gray scale processing and erosion operation, the center pixels of the set structure elements are retained as foreground pixels of the image, and the image edge data is extracted to obtain the original edge image;

[0008] The original edge image is subjected to connectivity analysis and filtering, and the breakpoints in the filtered edge image are filled and repaired according to the original edge image to obtain a repaired pad image;

[0009] The edge contour of the repaired pad image is extracted, the pad frame of the repaired pad image is determined based on the edge contour and the enclosed area formed by the contour, and edge comparison analysis is performed based on the wafer pad image to determine the contamination of the doubtful pad.

[0010] Specifically, the target detection takes the wafer pad as the target for identification, and the coordinates of the outer rectangular frame of the pad target are taken as the detection result, and the wafer pad image is intercepted and output.

[0011] Specifically, the gray processing and erosion operation on the suspicious pad image comprises:

[0012] Converting the wafer pad image from an RGB three-channel image to a gray image;

[0013] Performing Gaussian filtering on the gray image, and filtering image noise by convolving the gray image with a Gaussian convolution kernel;

[0014] Arranging the pixels of the noise-filtered gray image according to the brightness ratio, and filtering out high-brightness pixels and low-brightness pixels according to the set image brightness range;

[0015] Performing a binarization operation on the pixel-filtered gray image to analyze the foreground pixel content and background pixel content of the gray image;

[0016] Performing an erosion operation on the binarized image, and traversing the image by sliding a structure element placed at the edge of the image, and when the structure element is completely covered by foreground pixels, the center pixel of the structure element is retained as a new foreground pixel.

[0017] Specifically, the binarization operation on the pixel-filtered gray image comprises:

[0018] Calculating the gray image by Otsu binarization algorithm to obtain a binarization threshold value that can maximize the inter-class variance of foreground and background pixels;

[0019] Performing a binarization operation on the gray image based on the binarization threshold value to analyze the foreground pixel content and background pixel content of the gray image.

[0020] Specifically, the connectivity analysis and filtering of the original edge image comprises:

[0021] Performing a closing operation on the original edge image, and based on the dilation operation and erosion operation therein, closing and connecting the unclosed and broken edge pixels, and smoothing the edges;

[0022] Calculating the connected components and assigning the connected domain type labels based on the smoothed edge image, classifying the pixel points of the edge image according to the type labels, and calculating the scale of each connected domain;

[0023] Comparing the scale values of each connected domain with a preset minimum scale value min_length, filtering out the connected domains smaller than min_length, and obtaining the edge image filtered of false edges.

[0024] Specifically, the filling and repairing of the breakpoints in the filtered edge image according to the original edge image comprises:

[0025] The pixel points corresponding to the filtered connected domain labels are set to white, and the rest are set to black, and then the AND operation is performed with the original edge image pixel by pixel to obtain an intermediate pad image;

[0026] The closing operation is performed on the intermediate pad image to connect the broken edges and fill the image holes to obtain the repaired pad image.

[0027] Specifically, the pad frame of the repaired pad image is determined based on the edge contour and the surrounding area formed by the contour, comprising:

[0028] The surrounding contour of the maximum surrounding area formed by the edge contour is compared with the circumscribed rectangular frame of the target detection and recognition, and when the area difference between the surrounding contour and the circumscribed rectangular frame does not exceed the set threshold, the surrounding contour of the maximum surrounding area is determined as the pad frame.

[0029] Specifically, the damage condition of the suspicious pad is determined based on the edge comparison analysis of the wafer pad image, comprising:

[0030] Respectively taking the four sides of the pad frame as the center, a rectangular image of a set width is cut from the binary operation image as a pad edge damage analysis image;

[0031] The erosion operation is performed on the cut damage analysis image, and the overall variance of the pixel value fluctuation after the erosion operation is calculated, and when the overall variance value is greater than the corresponding fluctuation threshold, it is determined that the suspicious pad is contaminated.

[0032] Specifically, the sliding window variance of the pixel value fluctuation after the erosion operation is also calculated, and when the sliding window variance value and / or the overall variance value is greater than the corresponding fluctuation threshold, it is determined that the suspicious pad is contaminated.

[0033] On the other hand, the application also provides a wafer edge contamination detection device, the device comprises:

[0034] An image determination module is configured to perform target detection and recognition on the collected images in the database, extract the wafer pad image, and determine the suspicious pad image that cannot be classified through classification screening;

[0035] An image acquisition module is configured to perform gray scale processing and erosion operation on the suspicious pad image, retain the center pixel of the set structure element as the foreground pixel of the image, and extract the image edge data to obtain an original edge image;

[0036] an image repairing module configured to perform connectivity analysis and filtering on the original edge image, and fill and repair breakpoints in the filtered edge image according to the original edge image, to obtain a repaired pad image;

[0037] a contamination determination module configured to extract an edge contour of the repaired pad image, determine a pad frame of the repaired pad image based on the edge contour and an enclosed area formed by the edge contour, and perform edge comparison analysis based on the wafer pad image to determine a contamination condition of the suspicious pad.

[0038] In another aspect, the present application also provides a computer device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the wafer edge contamination detection method according to any one of the above aspects.

[0039] In another aspect, the present application also provides a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by a processor to implement the wafer edge contamination detection method according to any one of the above aspects.

[0040] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects: the present application extracts a wafer pad image and classifies it, for a suspicious wafer pad for which it is not determined whether there is pad edge contamination, uses gray-scale image processing and erosion operation to distinguish image foreground and background pixels, then extracts image edge data to obtain an original edge image; further, connectivity analysis and filtering and repairing of the original edge image can close the pad edge contour of the breakpoints to obtain a repaired pad image; by comparing and analyzing the shape and area that can be enclosed by the edge contour with the original wafer pad image, the actual contamination condition can be determined.

[0041] The scheme combines deep learning model prediction and digital image processing technology to perform image prediction and image repairing processing on the wafer pad image, compares and analyzes the frame repairing result and the original image to determine the contamination condition, which can greatly improve the detection efficiency compared with artificial sequential analysis, and the accuracy is greatly improved compared with the prediction result of the ordinary image recognition detection model, and is suitable for large-scale data processing and analysis under an industrial database. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of a wafer edge contamination detection method provided by the present application;

[0043] Figure 2 Fig. 1 is a schematic diagram of a wafer pad image under a possible embodiment, which is subjected to grayscale processing, binarization operation and extraction to obtain an original edge image;

[0044] Figure 3 Fig. 2 is a schematic diagram of a process of performing a closing operation and connectivity analysis on the original edge image;

[0045] Figure 4 Fig. 3 is a schematic diagram of performing a contamination detection analysis on a damage analysis image intercepted in four directions;

[0046] Figure 5 Fig. 4 is a schematic diagram of a process of wafer pad edge contamination analysis;

[0047] Figure 6 Fig. 5 is a structural schematic diagram of a wafer edge contamination detection device provided by an embodiment of the present application;

[0048] Figure 7 Fig. 6 is a structural block diagram of a computer device provided by an example embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0050] In this article, "multiple" refers to two or more. "And / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship.

[0051] Figure 1 Fig. 7 is a flowchart of a wafer edge contamination detection method provided by the present application, which includes the following steps:

[0052] Step 101, target detection and recognition are performed on the collected images in the database, wafer pad images are extracted and generated, and suspicious pad images that cannot be classified are determined through classification screening.

[0053] This scheme needs to collect a large number of collected images containing wafer pad content from an industrial environment and store them in a database. After data labeling, a target detection model such as yolox using a supervised learning algorithm is used to perform target detection on the collected images, that is, the wafer pad is detected as a target, and the coordinates of the outer rectangle of the pad target are returned as the detection result. The model intercepts and outputs the rectangular wafer pad image according to the coordinates.

[0054] The classification screening is to classify the identified wafer pad images according to actual needs, for example, according to wafer defect classification. The application is to identify the pad edge contamination type. For other types of clear edge contamination and other defect types, the identification result can be directly output, and the size of the confidence value can be used for confirmation. For pad edge contamination images with a confidence value less than a set value, it is considered that the suspicious pad image cannot be classified, that is, it needs to be processed subsequently.

[0055] In some embodiments, the classification screening can use a ResNet image classification model, and the intercepted wafer pad image is input as a classification image to output the classification result of the image through the model.

[0056] Step 102, the suspicious pad image is subjected to gray scale processing and erosion operation, the center pixel of the set structure element is retained as the foreground pixel of the image, and the image edge data is extracted to obtain the original edge image.

[0057] Because the photographed wafer photo is colored, in order to facilitate the accuracy of subsequent processing and reduce the amount and complexity of processing data, gray scale processing of the image is selected. After gray scale processing, filtering and erosion operations are performed.

[0058] Erosion operation is also called corrosion operation, which is an image processing operation that can thin or shorten small or elongated objects in an image. It can reduce the area of bright regions by corroding them with black regions through convolution operation of structure elements and images. In many image processing software, the corrosion operation is provided as a common image processing function. In the field of image morphology processing, the corrosion operation is implemented by convolving a specified shape and size structure element with a binary image. Therefore, in order to ensure effective identification of image pixel content, gray scale image binarization processing and image filtering operations are also required.

[0059] After performing the erosion operation on the gray scale image, the wafer pad edge in the image can be extracted using an edge detection algorithm (such as Canny algorithm), that is, the original edge image is obtained.

[0060] Figure 2 It is a schematic diagram of wafer pad image gray scale processing, binarization operation and original edge image extraction under a possible embodiment. The left side is a gray scale image formed after gray scale processing, the middle is a black and white pixel image formed after binarization operation, and the right side is an edge contour image obtained by erosion and extraction according to the binarized image.

[0061] Step 103, the original edge image is subjected to connectivity analysis and filtering, and the breakpoints in the filtered edge image are filled and repaired according to the original edge image to obtain a repaired pad image;

[0062] The connectivity analysis and filtering is a technique for detecting whether the edge contour of the image is complete. Since the acquisition image obtained in the industrial environment may not have good clarity, the contour of the obtained original edge image may not be completely closed, and therefore connectivity detection and repair need to be performed through a closing operation. The closing operation is an important morphological operation, and mainly for connecting the broken edge breakpoints of the edge image and filling and repairing the holes, and performing a certain degree of smoothing repair on the edge. The repaired image is a repair pad image, which is shown in detail in Figure 2 .

[0063] In step 104, the edge contour of the repair pad image is extracted, the pad frame of the repair pad image is determined based on the edge contour and the enclosed area formed by the contour, and the edge comparison analysis is performed based on the wafer pad image to determine the contamination condition of the suspicious pad.

[0064] The edge contour of the repair pad image is complete closure detection, and at this time the shape and area enclosed by the edge contour can be determined and compared with the original wafer pad image. The wafer pad image here refers to a rectangular image framed from the acquisition image, and the shape and area of the two images are compared to determine the contamination condition of the suspicious pad. The reference threshold can be determined according to the size and shape of the wafer pad.

[0065] In summary, the wafer pad image is extracted and classified, and for the suspicious wafer pad for which it is not determined whether there is pad edge contamination, the gray-scale image processing and erosion operation are used to distinguish the image foreground and background pixels, and then the image edge data is extracted to obtain the original edge image. Further, the connectivity analysis and filtering of the original edge image can close the broken pad edge contour to obtain the repair pad image. By comparing the shape and area enclosed by the edge contour with the original wafer pad image, the actual contamination condition can be determined.

[0066] The scheme combines the deep learning model prediction and the digital image processing technology to perform image prediction and image repair processing on the wafer pad image, compares the edge repair result with the original image to determine the contamination condition, which greatly improves the detection efficiency compared with artificial analysis, and the accuracy is greatly improved compared with the prediction result of the ordinary image recognition detection model, and is suitable for large-scale data processing and analysis in the industrial database.

[0067] In some embodiments, the gray-scale processing and erosion operation on the suspicious pad image includes the following detailed steps:

[0068] S1, convert the wafer pad image from RGB three-channel graph to grayscale graph;

[0069] This step performs the operation of converting from RGB three-channel graph to grayscale graph, converting the color image of three channels of red, green and blue into a single-channel grayscale image, so that the subsequent image processing is simpler and the complexity is reduced. At the same time, since the grayscale image has only one channel, the amount of calculation and storage space required during processing is smaller, thereby accelerating the subsequent image processing and analysis tasks. When converting the three-channel RGB image to the grayscale image, the following weighted sum formula is used:

[0070] Gray(R,G,B) = 0.299 x R + 0.587 x G + 0.114 x B

[0071] Where R, G, and B are the pixel values of the red, green, and blue channels.

[0072] This weight setting is based on the sensitivity of the human eye to different colors. The human eye is most sensitive to green, followed by red, and finally blue. Therefore, when converting a color image to a grayscale image, the weight of green is the largest, followed by red, and the weight of blue is the smallest.

[0073] S2, perform Gaussian filtering on the grayscale graph, and filter image noise by convolution kernel on the grayscale image;

[0074] Gaussian filtering is a linear filter, mainly used to reduce image noise. Its working principle is to use a Gaussian kernel to convolve the image, which means that the new value of each pixel is the weighted sum of the pixel values in its neighborhood and the Gaussian kernel value. Because the weight of the center pixel is the largest, and the weight gradually decreases with the distance. Gaussian filtering can effectively reduce random noise in the image, because noise is usually high-frequency part, and Gaussian filter as a low-pass filter, allows low-frequency components to pass through, while reducing high-frequency part, so that in the subsequent edge extraction step, noise will not be mistaken for edge.

[0075] S3, arrange the pixels of the noise-filtered grayscale image according to the brightness ratio, and filter out high-brightness pixels and low-brightness pixels according to the set image brightness range;

[0076] The image brightness range of this step is set according to empirical values, and the setting of this range for different types of wafer pads and different environments is different. This step can discard a certain proportion of the lowest brightness part and a certain proportion of the highest brightness part in the grayscale image, leaving the remaining pixels with a tendency to flatten the brightness change, which can help adapt to some extreme brightness pictures in the subsequent edge extraction results, and reduce a certain amount of calculation.

[0077] S4, the pixel filtered gray scale image is binarized, and the foreground pixel content and the background pixel content of the gray scale image are parsed;

[0078] The binarization operation is the key to the post-image erosion operation, but the difference lies in the specific parameters of the binarization operation, how to effectively distinguish the foreground pixel content and the background pixel content. Therefore, the present application provides two binarization operation steps, specifically:

[0079] 1) Calculate the gray scale image by Otsu binarization algorithm, and obtain a binarization threshold value that can maximize the inter-class variance of foreground and background pixels;

[0080] 2) Perform a binarization operation on the gray scale image based on the binarization threshold value, and parse the foreground pixel content and the background pixel content of the gray scale image;

[0081] The first binarization is to select an appropriate binarization threshold value, and the second binarization is performed with the adaptive binarization threshold value obtained above and a certain proportion of the coefficient as the threshold value.

[0082] S5, the binarized image is subjected to an erosion operation, and the structure element placed on the edge of the image is slid to traverse the image, and when the structure element is completely covered by the foreground pixels, the center pixel of the structure element is retained as a new foreground pixel.

[0083] The erosion operation mainly aims to reduce the foreground pixels in the image. A structure element (a small two-dimensional array used to define the operation mode of the erosion operation) can be set on the image and slid to traverse the entire image. Only when the structure element is completely covered by the foreground pixels, the center pixel is retained as a foreground pixel, otherwise it will become a background. The result is that the edge pixels are "eaten up", thereby reducing the foreground object.

[0084] For the image after the erosion operation, the Canny algorithm is used to extract the edge, and the specific operation is as follows:

[0085] 1) Calculate the gradient in the horizontal and vertical directions of the image by using the Sobel operator, and calculate the overall gradient strength and direction by using the two gradients;

[0086] The Sobel operator is a 3*3 matrix, which is represented as Sx and Sy. The former is used to calculate the pixel gradient matrix Gx in the x-axis direction of the image, and the latter is used to calculate the pixel gradient matrix Gy in the x direction of the image. The specific form is:

[0087]

[0088]

[0089] where I is the grayscale image matrix, and * here represents the cross-correlation operation (convolution operation can be regarded as the cross-correlation operation after rotating the convolution kernel by 180 degrees). It should be noted that the origin of the image matrix coordinate system is at the top left corner, and the positive direction of the x-axis is from left to right, and the positive direction of the y-axis is from top to bottom.

[0090] 2). Non-maximum suppression is performed; this is a refinement step that aims to ensure that edges are only one pixel wide. For each pixel, its neighbors in the current gradient direction are checked. If the gradient strength of the current pixel is not the maximum in its neighborhood, it is set to 0, which ensures that only the "sharpest" edges are preserved.

[0091] 3). Two high and low thresholds are set according to the image features. Pixels with intensities higher than the high threshold are considered "strong" edges, pixels with intensities lower than the low threshold are directly discarded, and pixels between the two thresholds are marked as "weak" edges. In subsequent edge tracking, "weak" edge pixels that are directly or indirectly connected to "strong" edge pixels are retained, while all other "weak" edge pixels are discarded. This ensures that only potential edges connected to clear edges are preserved.

[0092] These steps together ensure that Canny edge extraction can detect real and continuous image edges while eliminating most noise and false edges.

[0093] As mentioned earlier, in the actual industrial environment, the original edge image obtained from the picture may have breakpoints and holes in the edge profile, i.e., the profile is not connected, which requires connectivity analysis and filtering, and image repair is needed in the case of determining that the profile is not connected.

[0094] In some embodiments, the connectivity analysis and filtering of the original edge image include the following steps:

[0095] A. Perform a closing operation on the original edge image, and based on the dilation operation and erosion operation therein, close and connect the non-closed and broken edge pixels, and smooth the edges;

[0096] In this step, the dilation operation expands the foreground pixels in the image to their adjacent background pixels. Specifically, for a binary image such as an edge image, the edge pixel points are considered foreground objects, and the other pixel points are background objects. We place a structure element on the image and slide the structure element to traverse the entire image. If the structure element has at least one overlapping point with the foreground pixel (e.g., pixel value 255), all regions covered by the structure element are set to foreground pixels. The result is that the edge pixels are expanded, thereby increasing the foreground object.

[0097] The erosion operation reduces the foreground pixels in the image. We place the structuring element on the edge image and slide it to traverse the entire image, similar to the previous operation.

[0098] B. Calculate connected components based on the smoothed edge image and assign connected component type labels. According to the type labels, classify the pixels of the edge image and calculate the size of each connected component.

[0099] Due to the complex background of the wafer pad, after completing the above steps, there will still be a part of the false edge, so it is still necessary to filter. When calculating the connected components of the edge image, the connected component type can be selected and the 4 directions above and below or the 8 directions after adding the diagonal line of the pixel point are considered. The calculation of the connected component will assign a unique label to each connected component. Each pixel point of the edge image will get the label of which connected component it belongs to. In this way, the length of each connected component can be obtained.

[0100] C. Compare the size of each connected component with the preset minimum size min_length. Remove the connected components smaller than min_length to obtain the edge image filtered of false edges.

[0101] The process of obtaining the repaired pad image also includes the following steps:

[0102] D. Set the pixel points corresponding to the filtered connected component labels to white, and the rest to black. Then perform a bitwise AND operation with the original edge image pixel by pixel to obtain an intermediate pad image.

[0103] This process sets the pixel points corresponding to the filtered connected components to white, and the rest of the pixel points to black. Then perform a bitwise AND operation with the original edge image pixel by pixel to obtain an intermediate pad image.

[0104] E. Perform a closing operation on the intermediate pad image to connect broken edges and fill image holes to obtain a repaired pad image.

[0105] Perform a closing operation on the intermediate pad image again. Since the shorter false edges have been filtered in the previous step, the structuring element used in this closing operation can be set to a larger and more regular structure to fill the holes and connect the broken edges to obtain a repaired pad image.

[0106] Figure 3is the flowchart of the process of performing closing operation and connectivity analysis on the original edge image. The left side is the original edge image obtained for the first time, the middle is the image obtained after performing closing operation for the first time, and the right side is the repaired pad image obtained after performing connectivity analysis and closing operation again. From the right side of the image, it can be seen that after connectivity analysis and second closing operation, the edge profile can be well repaired, and the shorter edges inside the edge profile are filtered out.

[0107] After obtaining the repaired pad image, it can be compared and analyzed with the previously intercepted wafer pad image. The process can include the following steps:

[0108] SA, compare the maximum enclosing area of the edge profile with the target detection and recognition bounding box, and when the area difference between the enclosing profile and the bounding box is less than the set threshold, determine the maximum enclosing area of the enclosing profile as the pad bounding box.

[0109] This step is to extract the profile first, and then compare the profile with the maximum enclosing area with the target bounding box obtained by the yolox model. It should be noted that the determined pad bounding box is a rectangular bounding box selected by the bounding shape. The subsequent comparison is also based on the area of the bounding rectangle. If the area difference does not exceed the set threshold, the profile with the maximum enclosing area is used as the final high-precision pad bounding box, because the accuracy of the pad bounding box is crucial for edge contamination detection.

[0110] If the area difference after comparison is too large, the yolox model is used to identify the intercepted bounding box as a fallback safety measure, which is mainly for a small part of extreme scenarios. The result of deep learning is more stable. In summary, a more accurate pad bounding box is obtained in this step.

[0111] SB, respectively, with the four sides of the pad bounding box as the center, respectively, intercept a rectangular image with a set width from the binary image as the pad edge damage analysis image;

[0112] The four sides correspond to four small rectangular blocks. Erosion operation is performed on the images of the four small rectangular blocks, and different structure elements are used for horizontal and vertical directions to achieve appropriate smoothing effect.

[0113] SC, erosion operation is performed on the intercepted damage analysis image, and the overall variance of pixel value fluctuation after erosion operation is calculated. When the overall variance value is greater than the corresponding fluctuation threshold, it is determined that the suspicious pad is contaminated.

[0114] Figure 4An example of a schematic diagram of a smudge detection analysis on a damage analysis image taken in four directions is listed. This step takes a small rectangular region containing the left edge as an example, assuming that an array is maintained, the elements of which are in order: the number of consecutive black pixels from the first pixel in the first row of the rectangular region, the number of consecutive black pixels from the first pixel in the second row of the rectangular region,..., until the number of consecutive black pixels from the first pixel in the last row.

[0115] Taking a simple example, assuming that the image is a 4*4 rectangular region image, the following table lists a possible pixel form, and the array maintained is [2, 4, 1, 2]. Of course, the actual image data contains much more pixel content, which is only used here for convenience of illustration.

[0116] Table 1: Rectangular region pixel distribution table

[0117] Black Black White White Black Black Black Black Black White White White Black Black White White

[0118] Taking a small rectangular region containing the left edge as an example, we calculate the overall variance of the array maintained, which represents the fluctuation degree of the left edge pixel of the pad. Once the fluctuation degree is greater than a certain predefined threshold, it means that there is an edge pollution phenomenon on the left edge of the pad.

[0119] In actual situations, since the rectangular region can be large, there can be a small part of the rectangular region that has relatively severe pixel value fluctuations, and the fluctuation degree of the whole can not be so large. Taking an example, assuming that the array maintained is [2, 10, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], it can be seen that a small part of the region has severe fluctuations, and the overall variance is pulled down by other parts of the region so that it does not reach the predefined threshold. Therefore, we can also use the method of calculating the variance of a sliding window to more sensitively capture the region with severe fluctuations.

[0120] Specifically, assuming in the above example of [2, 10, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], we define the size of the sliding window to be 4 and the sliding step to be 1, then we can get a series of arrays representing smaller parts of the rectangular area [2, 10, 2, 2], [10, 2, 2, 2], [2, 2, 2, 2]… Calculate the variance of the smaller part of the array respectively, as a series of values of the sliding window variance, we can additionally predefine a threshold to determine whether the sliding window variance has reached this threshold to determine whether the edge of the pad exists edge pollution phenomenon. The final judgment, as long as any one of the pad's edge frame exists any kind of edge pollution (whether the overall variance reaches the threshold or the variance of a certain sliding window reaches the threshold), it is judged that this pad exists edge pollution.

[0121] Figure 5 The flowchart of wafer pad edge damage analysis is shown, which sequentially goes through gray processing, twice binaryzation operation, edge extraction, first closed operation, connectivity analysis after second closed operation, then finds the largest area contour of the repaired pad image, takes its circumscribed rectangle as the found pad frame, and determines the pad edge damage according to the comparison with the first wafer pad image. When the damage is confirmed, the NG label is framed and selected.

[0122] In summary, in order to be able to detect image data in large scale and maintain a certain real-time in the industrial quality inspection scene, the technology of combining AI prediction and digital image processing is adopted, which takes the advantages of each other to achieve 1+1>2 effect, and improves the speed and accuracy of detection. The first round of detection by AI model can fully exert the parallelism of AI model, and quickly perform a round of preliminary screening and help the subsequent digital image processing steps to locate (if the digital image processing is used for positioning, it is relatively slow). In view of the problems of uncontrollable, inflexible adjustment and lack of interpretability of the pad frame predicted by the AI model, a set of digital image processing scheme suitable for the scene is proposed to obtain higher positioning accuracy of the pad frame, and each step is flexible and controllable, and has interpretability. Finally, a set of effective and comprehensive quantitative standards for judging edge pollution are given.

[0123] In view of the problems of missed detection and false detection of a certain scale case by using only AI model for prediction, the advantages of AI model are retained, and its responsibility is refined to directly output the result (whether edge pollution) for certain cases, and to output the positioning result of the pad target for doubtful cases, which assists the subsequent digital image processing process. In the digital image processing process, this scheme establishes a set of quantitative standards for judging edge pollution, which has considerable flexibility and can be flexibly adjusted according to difficult cases to adapt to all cases, achieving the minimum missed detection rate and false detection rate.

[0124] Figure 6 A structure schematic diagram of a wafer edge contamination detection device provided by an embodiment of the present application is shown, and the device comprises:

[0125] An image determination module 610 is configured to perform target detection and identification on the collected images in the database, extract wafer pad images, and determine suspicious pad images that cannot be classified through classification screening;

[0126] An image acquisition module 620 is configured to perform gray scale processing and erosion operation on the suspicious pad images, retain the center pixels of the set structure elements as foreground pixels of the images, and extract image edge data to obtain original edge images;

[0127] An image repair module 630 is configured to perform connectivity analysis and filtering on the original edge images, fill and repair breakpoints in the filtered edge images according to the original edge images, and obtain repaired pad images;

[0128] A contamination determination module 640 is configured to extract the edge contour of the repaired pad images, determine the pad frame of the repaired pad images based on the edge contour and the enclosed area formed by the contour, and perform edge comparison analysis based on the wafer pad images to determine the contamination condition of the suspicious pad.

[0129] In addition, the present application also provides a computer device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the wafer edge contamination detection method in the above aspect.

[0130] In addition, the present application also provides a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the wafer edge contamination detection method in the above aspect.

[0131] The wafer edge contamination detection device provided by the embodiments of the present application can be applied to the wafer edge contamination detection method provided in the above embodiments, and the related details are referred to the above method embodiments, which have similar implementation principles and technical effects, and will not be described here.

[0132] It should be noted that the wafer edge contamination detection device provided in the embodiments of the present application is only exemplified by the above division of each functional module / functional unit when performing the caliper operation, and in actual application, the above functions can be completed by different functional modules / functional units according to needs, that is, the internal structure of the wafer edge contamination detection device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation of the wafer edge contamination detection method provided by the method embodiments belongs to the same concept as the implementation of the wafer edge contamination detection device provided by the present embodiment, and the specific implementation process of the wafer edge contamination detection device provided by the present embodiment is described above. The method embodiments, and will not be repeated here.

[0133] Figure 7 A structural block diagram of a computer device provided by an example embodiment of the present application is shown. The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer device can include, but is not limited to, a processor and a memory. The processor and the memory can be connected by a bus or other means. The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, graphics processing units (GPUs), embedded neural network processing units (NPUs) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components, and the like chips, or combinations of the above-mentioned chips.

[0134] The processor can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 1701 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor can also include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0135] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, i.e. implements the methods in the above method embodiments. The memory can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0136] In some embodiments, the computer device can also optionally include a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit, a display screen, a keyboard.

[0137] The peripheral interface can be used to connect at least one I / O (Input / Output) related peripheral device to the processor and the memory. In some embodiments, the processor, the memory and the peripheral interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory and the peripheral interface can be implemented on a separate chip or circuit board, to which the present embodiments are not limited.

[0138] The display screen is used to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen is a touch display screen, the display screen also has the ability to collect touch signals on or above the surface of the display screen. The touch signals can be input as control signals to the processor for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in some other embodiments, the display screen can be at least two, arranged on different surfaces of the computer device or in a folding design; in some other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can also be arranged in an irregular shape other than a rectangle, i.e. a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0139] The power supply is used to supply power to various components in the computer device. The power supply can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0140] Those skilled in the art can understand that, Figure 7 The structure shown in the above figure does not constitute a limitation on the computer device, and can include more or fewer components than the figure, or combine certain components, or use different component arrangements.

[0141] The embodiments of the present application further disclose a computer readable storage medium. Specifically, the computer readable storage medium is used to store a computer program, and the computer program is executed by a processor to implement the method in the above method embodiments. Those skilled in the art can understand that all or part of the processes in the above method embodiments of the present application can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above method embodiments. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0142] The specific embodiments are only an explanation of the present application, and are not a limitation of the present application. Those skilled in the art can make modifications to the embodiments according to the needs after reading the present specification, and the modifications are not creative contributions, but are protected by the patent law as long as the modifications are within the scope of the claims of the present application.

Claims

1. A wafer edge contamination detection method, characterized by, The method comprises: target detection and recognition are performed on the collected images in the database to extract the coordinates of the pad target contour rectangle as a detection result to generate a wafer pad image, and a suspicious pad image that cannot be classified is determined through classification screening; gray scale processing and erosion operation are performed on the suspicious pad image, the center pixel of a set structure element is retained as a foreground pixel of the image, and edge data of the image is extracted to obtain an original edge image; connectedness analysis and filtering are performed on the original edge image, and breakpoints in the filtered edge image are filled and repaired according to the original edge image to obtain a repaired pad image; an edge contour of the repaired pad image is extracted, a surrounding contour of a maximum surrounding area formed by the edge contour is compared with a contour rectangle of target detection and recognition, and when an area difference between the surrounding contour and the contour rectangle does not exceed a set threshold, the surrounding contour is determined as a pad contour; and a rectangular image of a set width is cut from a binary image as a pad edge damage analysis image with four sides of the pad contour as centers respectively; erosion operation is performed on the cut damage analysis image respectively, and a total variance of pixel value fluctuation after the erosion operation is calculated, and when the total variance is greater than a corresponding fluctuation threshold, it is determined that the suspicious pad is contaminated.

2. The method of claim 1, wherein, The gray scale processing and erosion operation on the suspicious pad image comprise: the wafer pad image is converted from an RGB three-channel image to a gray scale image; Gaussian filtering is performed on the gray scale image, and a Gaussian convolution kernel is used to convolve the gray scale image to filter image noise; pixels of the noise-filtered gray scale image are arranged according to brightness proportion, and high-brightness pixels and low-brightness pixels are filtered according to a set image brightness range; the pixel-filtered gray scale image is subjected to binaryzation operation to analyze foreground pixel content and background pixel content of the gray scale image; erosion operation is performed on the binaryzation image, and a structure element placed at an image edge is used for sliding traversal of the image, and when the structure element is completely covered by foreground pixels, the center pixel of the structure element is retained as a new foreground pixel.

3. The method of claim 2, wherein, The binaryzation operation on the pixel-filtered gray scale image comprises: Otsu binaryzation algorithm is used to calculate the gray scale image to obtain a binaryzation threshold value that can maximize the inter-class variance of foreground and background pixels; binaryzation operation is performed on the gray scale image based on the binaryzation threshold value to analyze the foreground pixel content and the background pixel content of the gray scale image.

4. The method of claim 1, wherein, The connectedness analysis and filtering of the original edge image comprise: a closing operation is performed on the original edge image, and un-closed and broken edge pixels are closed and connected based on the dilation operation and the erosion operation in the closing operation to smooth the edge; connected components are calculated based on the smoothed edge image, and a connected domain type label is assigned, and pixels of the edge image are classified according to the type label to calculate the scale of each connected domain. The various connected domain scale values are compared with a preset minimum scale value min_length, and connected domains smaller than min_length are filtered out to obtain an edge image with false edges filtered out.

5. The method of claim 4, wherein, The filling and repairing of the breakpoints in the filtered edge image according to the original edge image is performed to obtain a repaired pad image. The pixel points corresponding to the filtered connected domain labels are set to white, and the rest are set to black, and then an AND operation is performed with the original edge image pixel by pixel to obtain an intermediate pad image. A closing operation is performed on the intermediate pad image to connect the broken edges and fill in the image holes to obtain the repaired pad image.

6. The method of claim 1, wherein, The sliding window variance of the pixel value fluctuation after the erosion operation is also calculated, and when the sliding window variance value and / or the overall variance value is greater than the corresponding fluctuation threshold, it is determined that the suspicious pad is contaminated.

7. A wafer edge contamination detection apparatus characterized by comprising: The device comprises: An image determination module is configured to perform target detection and recognition on the collected images in the database, take the coordinates of the outer rectangular frame of the pad target as the detection result, extract and generate a wafer pad image, and determine a suspicious pad image that cannot be classified through classification screening; An image acquisition module is configured to perform grayscale processing and erosion operation on the suspicious pad image, retain the center pixels of the set structure elements as foreground pixels of the image, and extract image edge data to obtain an original edge image; An image repair module is configured to perform connectivity analysis and filtering on the original edge image, and fill and repair the breakpoints in the filtered edge image according to the original edge image to obtain a repaired pad image; A contamination determination module is configured to extract the edge contour of the repaired pad image, compare the enclosing contour formed by the maximum enclosing area of the edge contour with the outer rectangular frame of the target detection and recognition, and determine the enclosing contour as the pad frame when the area difference between the enclosing contour and the outer rectangular frame does not exceed a set threshold. The four sides of the pad frame are respectively taken as the center, and rectangular images with a set width are respectively cut from the binary operation image as pad edge damage analysis images. The erosion operation is performed on the cut damage analysis images, and the overall variance of the pixel value fluctuation after the erosion operation is calculated. When the overall variance value is greater than the corresponding fluctuation threshold, it is determined that the suspicious pad is contaminated.

8. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize the wafer edge contamination detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize the wafer edge contamination detection method according to any one of claims 1 to 6.

Images