Wafer defect detection methods, defect detection model training methods and devices

Abstract

This invention provides a wafer defect detection method, a defect detection model training method, and an apparatus. The defect detection model is trained using a fused image of a training defect image and a training reference image. The fused image contains difference information between the training defect image and the training reference image, thereby incorporating reference image information into the defect detection model. In this configuration, the defect detection model can combine the defect information of the defect image itself and its difference information from the reference image to perform defect detection, rather than being limited to the defect image itself. Therefore, even with a small amount of training defect image data, accurate wafer defect detection results can be obtained.

Description

Technical Field

[0001] This invention relates to the field of semiconductor technology, and specifically to a wafer defect detection method, a defect detection model training method, and an apparatus. Background Technology

[0002] In related technologies, the growing demand for interconnected, intelligent, compact, and long-standby electronic devices is a major driver of the trend towards smaller critical dimensions in integrated circuits. However, as mass production below 10 nanometers (nm) becomes mainstream, it is increasingly recognized that defects introduced by device manufacturers' components can significantly impact yield and manufacturing costs. Identifying, locating, and classifying these random or systematic defects is becoming increasingly challenging.

[0003] In the wafer inspection process provided by related technologies, an initial inspection of wafer defects is first performed to obtain the approximate location coordinates of the defects, which are then output to a results file. Then, defect images are acquired based on the approximate location coordinates for a second inspection. However, existing methods suffer from a high false detection rate. Summary of the Invention

[0004] To address the problems in the prior art, the present invention aims to provide a wafer defect detection method, a defect detection model training method, and an apparatus to solve the problem of low accuracy in related wafer defect detection methods.

[0005] This invention provides a wafer defect detection method, comprising:

[0006] Acquire an image of the defect to be inspected and a corresponding reference image, wherein the reference image does not contain wafer defects;

[0007] The defect image to be inspected and the reference image are registered, and the registered defect image to be inspected and the reference image are fused to obtain a fused image, which contains information about the defect image to be inspected and information about the reference image.

[0008] The fused image is input into the defect detection model, which outputs the defect detection results on the wafer.

[0009] The defect detection model is trained from multiple fused training images, which contain information about the training defect images and training reference images.

[0010] In some embodiments, acquiring an image of the defect to be inspected and a corresponding reference image includes:

[0011] The target wafer containing the defect to be inspected and its target location on the target wafer are obtained, and the target wafer is imaged based on the target location to obtain an image of the defect to be inspected.

[0012] On the wafer, the adjacent wafers of the target wafer are determined, and the corresponding positions of the target positions on the adjacent wafers are also determined. Based on the corresponding positions, the adjacent wafers are imaged to obtain a reference image.

[0013] In some embodiments, the registered image of the defect to be inspected is fused with a reference image to obtain a fused image, including:

[0014] Determine the overlapping area between the registered defect image and the reference image;

[0015] The registered image of the defect to be inspected and the reference image are merged to obtain a fused image, wherein the fused image includes at least a portion of the overlapping region, which is determined based on the target location of the defect to be inspected.

[0016] In some embodiments, registration is performed between the image of the defect to be inspected and a reference image, including:

[0017] The defect image to be inspected is deformed at least once using a transformation matrix until the correlation coefficient between the deformed defect image and the reference image is not lower than a threshold. The deformed defect image is then used as the defect image to be inspected and registered with the reference image.

[0018] This disclosure also provides a method for training a defect detection model, comprising:

[0019] Obtain a training set, which includes unlabeled training fusion images and labeled training fusion images. The training fusion images include registered training defect image information and training reference image information. The training reference images do not contain wafer defects, and the labels are defect location labels.

[0020] The defect detection model is trained using the training set.

[0021] In some embodiments, obtaining the training set includes:

[0022] Obtain the training defect image and at least one corresponding training reference image;

[0023] The training defect images are registered with the corresponding training reference images;

[0024] The registered training defect image is fused with the training reference image to obtain the training fused image;

[0025] The training set is constructed using unlabeled training fusion images and labeled training fusion images.

[0026] In some embodiments, obtaining a training defect image and at least one corresponding training reference image includes:

[0027] The defect is located on the first wafer and its first position on the first wafer is obtained, and the first wafer is imaged based on the first position to obtain a training defect image;

[0028] A second wafer adjacent to the first wafer and a second position on the second wafer corresponding to the first position are determined on the wafer, and the second wafer is imaged based on the second position to obtain a training reference image.

[0029] In some embodiments, the registered training defect image is fused with the training reference image to obtain a training fused image, including:

[0030] Determine the overlapping region between the registered training defect image and the training reference image;

[0031] The registered training defect image and the training reference image are channel-merged to obtain a training fused image, wherein the training fused image includes at least a portion of the overlapping region, and the at least a portion of the region is determined based on the target location of the defect.

[0032] In some embodiments, registration of the defect image and the corresponding training reference image includes:

[0033] The defect image is deformed using a transformation matrix until the correlation coefficient between the deformed image and the training reference image is not lower than a threshold. The deformed image is then used as the training defect image registered with the training reference image.

[0034] In some embodiments, training the defect detection model using the training set includes processing the unlabeled training fusion image and the labeled training fusion image as follows:

[0035] Slice the training fused image;

[0036] Perform multi-feature scale transformation on each slice image;

[0037] Convolution is performed on the multi-scale transformed image of each slice to obtain the feature vector;

[0038] Defect detection is performed based on feature vectors to obtain defect detection prediction results;

[0039] The loss function is calculated based on the defect detection results and the corresponding real defect information, and the model parameters of the defect detection model are adjusted according to the loss value until the training stopping condition is met; wherein, the loss function includes at least the channel difference between the training defect image and the training reference image.

[0040] This disclosure also provides a wafer defect detection device, which includes:

[0041] The acquisition module acquires an image of the defect to be inspected and a corresponding reference image, wherein the reference image does not contain wafer defects;

[0042] The fusion module registers the defect image to be inspected with a reference image, and then fuses the registered defect image to be inspected with the reference image to obtain a fused image. The fused image contains information about the defect image to be inspected and information about the reference image.

[0043] The prediction module integrates the image input defect detection model and outputs the defect detection results on the wafer.

[0044] The wafer defect detection method, defect detection model training method, and apparatus provided by this invention have the following advantages:

[0045] A defect detection model is trained using a fused image of the training defect image and a training reference image. The fused image incorporates the difference information between the training defect image and the training reference image, thus introducing reference image information into the defect detection model. In this approach, the defect detection model can combine the defect information of the defect image itself and its difference information from the reference image for defect detection, rather than being limited to the defect image itself. Therefore, even with a small amount of training defect image data, accurate wafer defect detection results can be obtained. Furthermore, the defect detection accuracy is high and stable under various application scenarios and operating conditions. Attached Figure Description

[0046] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description with reference to the accompanying drawings.

[0047] Figure 1 A flowchart illustrating the wafer defect detection method provided in this disclosure embodiment is shown.

[0048] Figure 2 A partially enlarged view of a wafer with defects to be inspected, according to an embodiment of this disclosure, is shown;

[0049] Figure 3 This illustration shows a schematic diagram of an image of a defect to be inspected, provided by an embodiment of this disclosure.

[0050] Figure 4 A schematic diagram of a reference image provided by an embodiment of this disclosure;

[0051] Figure 5 A flowchart illustrating an image registration method for defects to be inspected provided by an embodiment of this disclosure;

[0052] Figure 6 This illustration shows a deformed image of a defect to be inspected according to an embodiment of the present disclosure.

[0053] Figure 7This illustration shows an overlaid image of a defect to be inspected, provided by an embodiment of this disclosure.

[0054] Figure 8 This illustration shows an overlaid reference image provided by an embodiment of the present disclosure;

[0055] Figure 9 This illustration shows a fused image provided by an embodiment of the present disclosure;

[0056] Figure 10 A schematic diagram illustrating the inference structure of a defect detection model provided in an embodiment of this disclosure is shown.

[0057] Figure 11 A flowchart illustrating a training method for a defect detection model provided in an embodiment of this disclosure;

[0058] Figure 12 This illustration shows a fused image after manual annotation, provided by an embodiment of the present disclosure.

[0059] Figure 13 A schematic diagram of a module of one embodiment of the wafer defect detection device provided in this disclosure is shown.

[0060] Figure 14 A schematic diagram of a module illustrating an embodiment of a training apparatus for the defect detection model provided in this disclosure;

[0061] Figure 15 This is a schematic diagram of the operation of the electronic device of the present invention;

[0062] Figure 16 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0063] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0064] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0065] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.

[0066] Among related technologies, the emerging technology of deep learning provides a new method for defect detection. Wafer defect detection schemes based on deep learning can simultaneously focus on both low-level and high-level semantic information of defect images, thus enabling defect identification even without a reference image. Deep learning possesses high-level semantic information, allowing for human-like judgment and making it less susceptible to noise and residual electrons.

[0067] However, the inventors in this case noted that, due to the lack of comparative information with reference images, a large amount of training defect image data covering the entire scene is often required to achieve the desired results. Therefore, a massive amount of defect image data is needed, and new defect scenes require new training defect image data to train the model. Consequently, the wafer defect detection scheme based on deep learning technology has certain application limitations, resulting in a low accuracy rate in actual wafer defect detection.

[0068] This disclosure provides an improved wafer defect detection method. A defect detection model is trained using a fused image of a defect image and a reference image. The fused image contains difference information between the defect image and the reference image, thereby incorporating reference image information into the defect detection model. In this case, the defect detection model can combine the defect information of the defect image itself and its difference information with the reference image for defect detection, rather than being limited to the defect image itself. Therefore, even with a small amount of training defect image data, accurate wafer defect detection results can be obtained, exhibiting better transferability / universality and robustness.

[0069] Figure 1 A flowchart illustrating the wafer defect detection method provided in this disclosure embodiment is shown, such as... Figure 1 The method shown includes the following steps:

[0070] Step 110: Obtain the image of the defect to be inspected and the corresponding reference image, wherein the reference image does not contain wafer defects;

[0071] Step 120: Register the defect image to be inspected and the reference image, and fuse the registered defect image to be inspected with the reference image to obtain a fused image. The fused image contains the defect image information and the reference image information.

[0072] Step 130: Input the fused image into the defect detection model and output the defect detection results on the wafer;

[0073] The defect detection model is trained from multiple fused training images, which contain information about the training defect images and training reference images.

[0074] In this embodiment of the disclosure, the image of the defect to be inspected can be a wafer image that has been preliminarily detected as potentially containing defects and requires further re-inspection. Therefore, the defect detection result includes one or more of the following: a judgment result on whether a defect is present and information on the location of the defect; these are not specifically limited herein.

[0075] In this embodiment of the disclosure, acquiring the image of the defect to be inspected and the corresponding reference image includes:

[0076] The target wafer containing the defect to be inspected and its target location on the target wafer are obtained, and the target wafer is imaged based on the target location to obtain an image of the defect to be inspected.

[0077] On the wafer, the adjacent wafers of the target wafer are determined, and the corresponding positions of the target positions on the adjacent wafers are also determined. Based on the corresponding positions, the adjacent wafers are imaged to obtain a reference image.

[0078] In one embodiment, the image of the defect to be inspected is taken from a target wafer on a wafer. It can be an image of a complete target wafer, an image of a partial area on the target wafer, or an image of at least two target wafers, and is not limited thereto. For high-magnification images of defects to be inspected, they are typically partial areas on the target wafer.

[0079] In this embodiment, the target location is the approximate location of the defect that is initially detected. Based on the target location, the area of ​​the defect to be inspected can be roughly located for imaging.

[0080] In one feasible embodiment, firstly, a preliminary inspection device, such as an optical inspection system, outputs a result file. By parsing this file, the wafer number / serial number (Wafer ID), coordinate information, etc., can be obtained. The coordinate information includes at least the coordinate information of the defect to be inspected, and the coordinate information of the defect to be inspected can at least determine the wafer in which the defect to be inspected is located. The position of the defect to be inspected on the wafer can be represented as the specific position of the defect to be inspected relative to the center or vertex of the wafer.

[0081] For example, Figure 2This is a magnified view of a portion of the wafer, including six wafer dies. The asterisks in the magnified view represent defects to be inspected. A defect marked with an asterisk is located at die_(m,n), indicating that it is located on the die in the m-th row and n-th column of the wafer. die_(m-1,n), die_(m,n-1), die_(m+1,n), and die_(m,n+1) are all dies adjacent to die_(m,n). Furthermore, the position (dx, dy) of the asterisk defect on die_(m,n) indicates the specific position of the defect relative to the lower left corner vertex of die_(m,n).

[0082] Taking an adjacent die (m+1, n) as an example, the offset coordinates corresponding to the asterisk defect on that adjacent die can be determined by offsetting the lower left corner vertex of that die (the die in the (m+1)th row and the nth column) by dx in the x-direction and dy in the y-direction. Here, for example, only one coordinate position (dx, dy) is used to represent the location of the defect to be inspected.

[0083] Typically, a defect corresponds to a region consisting of multiple pixels on the image of the defect to be inspected. The location of the defect can be represented by the coordinates of the upper left corner (lp) and the lower right corner (rb) of the region.

[0084] Based on the defect coordinate information in the above results file, the wafer is moved to that position, and then image data of the defect is obtained through electron beam imaging. This image is called the defect image to be inspected. Figure 3 The defect is located in region S.

[0085] Next, based on the coordinates of the die containing the defect, its adjacent dies are determined. The machine is then moved to the offset coordinate position corresponding to one of the adjacent dies containing the defect, and an electron beam is used to acquire a reference image. Figure 4 .

[0086] In this embodiment of the disclosure, the registered image of the defect to be inspected is fused with a reference image to obtain a fused image, including:

[0087] Determine the overlapping area between the registered defect image and the reference image;

[0088] The registered image of the defect to be inspected and the reference image are merged to obtain a fused image, wherein the fused image includes at least a portion of the overlapping region, which is determined based on the target location of the defect to be inspected.

[0089] In this embodiment, image fusion is achieved through channel merging.

[0090] The fused image is a portion of the image to be defected, which can save computational load on subsequent defect detection models and improve defect detection efficiency.

[0091] In this embodiment of the disclosure, registration is performed between the image of the defect to be inspected and the reference image, including:

[0092] The defect image to be inspected is deformed at least once using a transformation matrix until the correlation coefficient between the deformed defect image and the reference image is not lower than a threshold. The deformed defect image is then used as the defect image to be inspected and registered with the reference image.

[0093] In practice, factors such as mechanical structure, wafer loading / unloading errors, and variations in the electron beam magnetic field can cause inconsistencies between the background of the defect image and the reference image. Issues such as translation, rotation, scaling, and electron residue can occur, leading to background transformations in the two images. For example... Figure 3 and Figure 4 Therefore, the two images above are image registered and aligned.

[0094] Image registration refers to mapping one image (such as a defect image) onto another image (such as a reference image) through spatial transformation, that is, matching corresponding points in the set of images one-to-one.

[0095] In this embodiment, the core idea of ​​image registration is to achieve image alignment by comparing the regions with the smallest differences between the defect image to be inspected and the reference image. Specifically, the difference between the images is calculated using the correlation coefficient formula (3):

[0096]

[0097] Where Cov(X,Y) is the covariance of X and Y, Var[X] is the variance of X, and Var[Y] is the variance of Y. ρ xy It can be used to measure the linear relationship between X and Y. |ρ xy The larger the value, the greater the correlation; conversely, the smaller the value, the smaller the correlation.

[0098] Formula (1) mainly consists of the variance formula (2) and the covariance formula (3):

[0099] Var[X]=δ 2 =∑(X-μ)2 / N (2)

[0100] Where, δ 2 X is the population variance, μ is the population mean, and N is the population count (number of pixels).

[0101] Cov(X, Y)=E[(XE[X])(YE[Y])] (3)

[0102] Cov(X,Y) is the covariance between two real random variables X and Y with expected values ​​E[X] and E[Y], respectively, where variable Y contains pixel information from a reference image.

[0103] Then, the image of the defect to be inspected is deformed, and the problem is transformed into finding the optimal transformation matrix. The transformation matrix is ​​composed of parameters, so the problem is described by formula (4):

[0104]

[0105] Where Co represents a set of coordinates, I r (Co) is a given reference image, I w (Y) represents a warped image. This is a set of coordinates corresponding to parameter P (the parameter of the transformation matrix).

[0106] In the embodiments disclosed herein, such as Figure 5 As shown, registering the defect image to be inspected and the reference image may include the following steps:

[0107] Step 510: Apply Gaussian blur (denoising) to the defect image and the reference image respectively; other methods can also be used for denoising.

[0108] Step 520: Use a transformation matrix to deform the defect image to be inspected. The transformation matrix can be either an affine transformation matrix or a projective transformation matrix. The initialization of the transformation matrix keeps the original image unchanged.

[0109] Step 530: Calculate the correlation coefficient between the deformed defect image and the reference image, as shown in formula (5):

[0110] ρ xy =|i r / |i r |- i w (P) / |i w (P)|| 2 (5)

[0111] Among them, i rThe vector i represents the reference image. w (P) represents the vector of the curved image.

[0112] Step 540: Determine whether the correlation coefficient is less than the set threshold;

[0113] Step 550: If yes, modify the parameters of the transformation matrix using gradient descent, and return to step 520 using the modified transformation matrix. Recalculate the correlation coefficient between the deformed defect image and the reference image, and then compare the correlation coefficient with the threshold. Repeat the above process until the correlation coefficient is greater than the threshold, and finally obtain and output the transformation matrix. This matrix can be used to deform the defect image, and the deformed defect image can be registered and aligned with the reference image.

[0114] Using the calculated transformation matrix, Figure 3 The image of the defect to be inspected is deformed to obtain the warped image 6. Obviously, Figure 6 relatively Figure 3 It is offset, that is, after mapping the defect image 3 onto the reference image 4, we get... Figure 6 , Figure 6 The image is the image of the defect to be inspected (image 3) registered and aligned with the reference image (image 4).

[0115] The deformed image of the defect to be inspected is obtained using the method described above. The deformed image of the defect to be inspected is then fused with the reference image. This process includes two steps: cropping and overlaying, which are described in detail below.

[0116] Cropping process: Cropping out the overlapping area between the deformed defect image and the reference image. Specifically, this involves cropping out the overlapping area between the deformed defect image and the reference image. Figure 6 The transformed image of the defect to be inspected and Figure 4 The reference image shown is cropped, and the cropping areas are all the same size. The maximum cropping area can be... Figure 6 and Figure 4 The overlapping portion can also be smaller than the overlapping portion, such as a region cropped from the middle. For example, from... Figure 6 The transformed image of the defect to be inspected was cropped from the middle. Figure 7 ,from Figure 4 The reference image shown was cropped from the middle. Figure 8 Then the result is Figure 7 and Figure 8 They overlap. Figure 7 Represents overlapping images of defects to be inspected. Figure 8 Reference images indicating overlap.

[0117] Overlay processing / channel fusion: Overlay enables the fusion of information from different scenarios, thereby identifying areas of difference caused by defects and achieving the goal of defect detection. Figure 7 The overlapping images of the defects to be inspected are placed into two channels: the red channel, the green channel, and the blue channel. Figure 8 The overlapping reference image shown is placed into another channel, and after overlaying / channel merging, the result is... Figure 9 The image shown is a fused image. Alternatively, an overlapping image of the defect to be inspected can be placed in one of the three channels, and an overlapping reference image can be placed in the other two channels. The channels are then fused to obtain a fused image.

[0118] Alternatively, multiple reference images of the defect to be inspected can be obtained, and the above steps can be repeated for each reference image to obtain multiple fused images corresponding to one defect image. Figure 2 Taking the enlarged partial schematic diagram as an example, the image of the defect to be inspected marked with an asterisk has a maximum of four reference images, from which four corresponding fused images can be obtained.

[0119] In this embodiment of the disclosure, the aforementioned fused image is input into the defect detection model, which processes the fused image and outputs the defect detection result. The fused image integrates information from the defect image to be detected and information from the reference image, containing richer information. Based on the differences between the defect image to be detected and the reference image, the defect detection model can accurately determine the type, location, and other information of the defect, and is applicable to more application scenarios and different working conditions.

[0120] In this embodiment, the defect detection model can be constructed using a neural network, such as... Figure 10 The inference structure of the defect detection model based on the neural network is explained below.

[0121] First, the fused image is sliced ​​into N*M grid slices, where N and M are integers;

[0122] Next, each grid slice undergoes multi-scale feature transformation, including large-scale, medium-scale, and small-scale transformations, without specifying a particular feature scale. The transformed grid slices are then used for feature extraction through multiple convolutional layers, such as... Figure 10 Small-scale, medium-scale, and large-scale features are extracted to obtain feature vectors.

[0123] Finally, the feature vector is used to predict the defect detection result.

[0124] In a corresponding embodiment, the defect detection result includes the defect judgment result and defect location information.

[0125] Specifically, in a neural network, the defect detection results obtained through forward inference can be a list, where each item in the list contains information about a defect, including the coordinates of the top-left corner, the coordinates of the bottom-right corner, and the probability of whether it is a defect.

[0126] This disclosure, through the introduction of the difference between the defect image to be detected and the reference image contained in the fused image, organically combines the defect detection method of related technologies with deep learning technology, enriches the defect detection model's judgment of defects, thereby improving the defect detection accuracy, enhancing the robustness of the defect detection model, and exhibiting better transferability / universality.

[0127] The defect detection model disclosed herein can be used as follows: Figure 11 The training method shown in the embodiment can also be obtained through other methods, which are not limited here.

[0128] like Figure 11 As shown, the defect detection model training method proposed in this disclosure includes, but is not limited to, the following steps:

[0129] Step 1110: Obtain the training set, which includes unlabeled training fusion images and labeled training fusion images. The training fusion images include registered training defect image information and training reference image information. The training reference images do not contain wafer defects and are labeled as defect location markers.

[0130] Step 1120: Train the defect detection model using the training set.

[0131] By using this method to train a defect detection model and incorporating reference image information into the model training process, the deep learning's ability to identify defects can be enriched, thereby achieving higher accuracy.

[0132] This disclosure uses a supervised method to train a defect detection model. Specifically, the training data for the defect detection model includes a large number of fused training images, with defect locations labeled for each fused image. Figure 9 Taking the fused image shown as an example, it is manually annotated, with the location of defects represented by rectangular boxes, resulting in labeled fused image 12. After obtaining the fused image, defects in the fused image are labeled to obtain the labeled training fused image. Figure 9 The fused image shown is labeled to obtain the following: Figure 12 The labeled training fusion images shown are... Figure 12The boxes in the diagram indicate defects within their respective regions. All unlabeled training fusion images and their corresponding labeled training fusion images constitute the training set. After labeling each training fusion image, the training set of training fusion images is obtained. Each unlabeled training fusion image and its corresponding labeled training fusion image in the training set form a training set. The defect detection model is trained using the training data from these training sets.

[0133] In this embodiment of the disclosure, obtaining multiple training fusion images may specifically include the following steps:

[0134] Obtain the training defect image and at least one corresponding training reference image;

[0135] The training defect images are registered with the corresponding training reference images;

[0136] The registered training defect image is fused with the training reference image to obtain the training fused image;

[0137] The training set is trained using unlabeled training fused images and labeled training fused images.

[0138] In this embodiment of the disclosure, obtaining the training defect image and at least one corresponding training reference image may specifically include the following steps:

[0139] The defect is located on the first wafer and its first position on the first wafer is obtained, and the first wafer is imaged based on the first position to obtain a training defect image;

[0140] A second wafer adjacent to the first wafer and a second position on the second wafer corresponding to the first position are determined on the wafer, and the second wafer is imaged based on the second position to obtain a training reference image.

[0141] The process of obtaining the training reference image can be referred to the process of obtaining the reference image above, and will not be repeated here.

[0142] In this embodiment of the disclosure, the registered training defect image is fused with the training reference image to obtain a training fused image, which may include, but is not limited to, the following steps:

[0143] Determine the overlapping region between the registered training defect image and the training reference image;

[0144] The registered training defect image and the training reference image are channel-merged to obtain a training fused image, wherein the training fused image includes at least a portion of the overlapping region, and the at least a portion of the region is determined based on the target location of the defect.

[0145] The image fusion process can be referred to the image fusion process described above, and is not limited here. The fusion of image information is achieved through channel merging, which can completely integrate the information of the training defect image and the training reference image. The fused image contains information from both the training defect image and the training reference image; that is, it includes both defect information and contrast information from the reference image. The neural network model can achieve higher precision defect detection based on the differences between the two, resulting in a defect detection model with better universality / transferability. Furthermore, the scheme provided in this disclosure can achieve good training results with less training data.

[0146] In this embodiment of the disclosure, the registration of the defective image and the corresponding training reference image may include, but is not limited to, the following steps:

[0147] The defect image is deformed using a transformation matrix until the correlation coefficient between the deformed image and the training reference image is not lower than a threshold. The deformed image is then used as the training defect image registered with the training reference image.

[0148] The image registration process for training defective images can be referenced from the image registration process described above, and is not limited here.

[0149] In this embodiment of the disclosure, the defect detection model includes convolutional layers. Training the defect detection model using a training set may include, but is not limited to, the following steps:

[0150] Slice the training fused image;

[0151] Perform multi-feature scale transformation on each slice image;

[0152] The multi-scale transformed image of each slice image is convolved using a convolutional layer to obtain a feature vector;

[0153] Defect detection is performed based on feature vectors to obtain defect detection prediction results;

[0154] The loss function is calculated based on the defect detection results and the corresponding real defect information, and the model parameters of the defect detection model are adjusted according to the loss value until the training stopping condition is met. The loss function includes at least the channel difference between the training defect image and the training reference image.

[0155] In this embodiment, the feature vector and the real defect information are used to construct a loss function, as shown in formula (6):

[0156] L=k1MSE(lt)+k2MSE(rb)+k3MSE(d)+k4MSE(c) (6)

[0157] Where, k iThese are hyperparameters; lt and rb represent the coordinates of the top-left and bottom-right corners, respectively.

[0158] d indicates whether it is a defect, and c is the channel containing training defect image information and training reference image information.

[0159] MSE(lt) and MSE(rb) are used for target location regression;

[0160] MSE(d) is used to determine the confidence level for defect assessment;

[0161] MSE(c) is used for channel difference confidence, which can be used to represent the channel difference between the training defective image and the training reference image.

[0162] Therefore, the loss function mainly consists of a classification network for defects (defects and non-defects), differences between channels, and the location information of the defects. Introducing the differences between channels organically combines defect detection methods from related technologies with deep learning methods. Backpropagation is performed through this loss function to update network parameters until the minimum value of the loss function is obtained. By incorporating information from the reference image, the deep learning's judgment of defects is enriched, resulting in higher accuracy and improved robustness, transferability, and universality of the defect detection model.

[0163] In this embodiment, the training stopping condition can be reaching a preset number of iterations, the performance index of the defect detection model after parameter adjustment reaching a preset index, or the minimum difference between the defect detection prediction result and the labeled information in a training set. The model obtained in this process can then be used for subsequent online inference to predict the location information of defects in the image in real time.

[0164] Figure 13 This is a schematic diagram of a module of one embodiment of the wafer defect detection device provided in this disclosure, as shown below. Figure 13 As shown, the wafer defect detection device 1300 includes, but is not limited to, the following modules:

[0165] The acquisition module 1310 acquires the image of the defect to be inspected and the corresponding reference image, wherein the reference image does not contain wafer defects;

[0166] The fusion module 1320 registers the defect image to be inspected and the reference image, and then fuses the registered defect image to be inspected with the reference image to obtain a fused image. The fused image contains information about the defect image to be inspected and information about the reference image.

[0167] The prediction module 1330 will integrate the image input to the defect detection model and output the defect detection results on the wafer.

[0168] The implementation principle of the above modules can be found in [link to documentation]. Figure 1 The relevant information regarding the wafer defect detection method shown will not be repeated here.

[0169] Optionally, the acquisition module 1310 is specifically used for:

[0170] The target wafer containing the defect to be inspected and its target location on the target wafer are obtained, and the target wafer is imaged based on the target location to obtain an image of the defect to be inspected.

[0171] On the wafer, the adjacent wafers of the target wafer are determined, and the corresponding positions of the target positions on the adjacent wafers are also determined. Based on the corresponding positions, the adjacent wafers are imaged to obtain a reference image.

[0172] Optionally, the fusion module 1320 is specifically used for:

[0173] Determine the overlapping area between the registered defect image and the reference image;

[0174] The registered image of the defect to be inspected and the reference image are merged to obtain a fused image, wherein the fused image includes at least a portion of the overlapping region, which is determined based on the target location of the defect to be inspected.

[0175] Optionally, the fusion module 1320 is also specifically used for:

[0176] Determine the overlapping area between the registered defect image and the reference image;

[0177] The registered image of the defect to be inspected and the reference image are merged to obtain a fused image, wherein the fused image includes at least a portion of the overlapping region, which is determined based on the target location of the defect to be inspected.

[0178] Optionally, the fusion module 1320 is also specifically used for:

[0179] The defect image to be inspected is deformed at least once using a transformation matrix until the correlation coefficient between the deformed defect image and the reference image is not lower than a threshold. The deformed defect image is then used as the defect image to be inspected and registered with the reference image.

[0180] Figure 14 A block diagram of the training apparatus for the defect detection model provided in the embodiments of this disclosure, as shown below. Figure 14 As shown, the training device 1400 for the defect detection model includes, but is not limited to, the following modules:

[0181] The acquisition module 1410 acquires a training set, which includes unlabeled training fusion images and labeled training fusion images. The training fusion images include registered training defect image information and training reference image information. The training reference images do not contain wafer defects, and the labels are defect location labels.

[0182] Training module 1420 uses the training set to train the defect detection model.

[0183] In an optional embodiment, the acquisition module 1410 is specifically used for:

[0184] Obtain the training defect image and at least one corresponding training reference image;

[0185] The training defect images are registered with the corresponding training reference images;

[0186] The registered training defect image is fused with the training reference image to obtain the training fused image;

[0187] A training set is constructed using unlabeled and labeled training fusion images.

[0188] In an optional embodiment, the acquisition module 1410 is further configured to:

[0189] The defect is located on the first wafer and its first position on the first wafer is obtained, and the first wafer is imaged based on the first position to obtain a training defect image;

[0190] A second wafer adjacent to the first wafer and a second position on the second wafer corresponding to the first position are determined on the wafer, and the second wafer is imaged based on the second position to obtain a training reference image.

[0191] In an optional embodiment, the acquisition module 1410 is further configured to:

[0192] Determine the overlapping region between the registered training defect image and the training reference image;

[0193] The registered training defect image and the training reference image are channel-merged to obtain a training fused image, wherein the training fused image includes at least a portion of the overlapping region, and the at least a portion of the region is determined based on the target location of the defect.

[0194] In an optional embodiment, the acquisition module 1410 is specifically used for:

[0195] The defect image is deformed using a transformation matrix until the correlation coefficient between the deformed image and the training reference image is not lower than a threshold. The deformed image is then used as the training defect image registered with the training reference image.

[0196] In an optional embodiment, the training module 1420 is specifically used to perform the following processing on the unlabeled training fusion image and the labeled training fusion image respectively:

[0197] Slice the training fused image;

[0198] Perform multi-feature scale transformation on each slice image;

[0199] Convolution is performed on the multi-scale transformed image of each slice to obtain the feature vector;

[0200] Defect detection is performed based on feature vectors to obtain defect detection prediction results;

[0201] The loss function is calculated based on the defect detection results and the corresponding real defect information, and the model parameters of the defect detection model are adjusted according to the loss value until the training stopping condition is met; wherein, the loss function includes at least the channel difference between the training defect image and the training reference image.

[0202] This invention also provides an electronic device, including a processor and a memory storing executable instructions for the processor. The processor is configured to execute steps of a wafer defect detection method or a defect detection model training method via executing the executable instructions.

[0203] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0204] Figure 15 A schematic diagram of the electronic device of the present invention is shown below. (Refer to the following...) Figure 15 To describe an electronic device 1500 according to this embodiment of the present invention. Figure 15 The electronic device 1500 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0205] like Figure 15 As shown, the electronic device 1500 is presented in the form of a general-purpose computing device. The components of the electronic device 1500 may include, but are not limited to: at least one processing unit 1510, at least one storage unit 1520, a bus 1530 connecting different platform components (including storage unit 1520 and processing unit 1510), a display unit 1540, etc.

[0206] The storage unit stores program code, which can be executed by the processing unit 1510 to perform the steps described in the section on the training method for wafer defect detection or defect detection model according to various exemplary embodiments of the present invention. For example, the processing unit 1510 can execute... Figure 1 Or the steps shown in 10.

[0207] Storage unit 1520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 1521 and / or cache memory 1522, and may further include read-only memory (ROM) 1523.

[0208] Storage unit 1520 may also include a program / utility 1524 having a set (at least one) of program modules 1525, such program modules 1525 including but not limited to: processing system, one or more application programs, other program modules and program data, each of these examples or some combination of these may include an implementation of a network environment.

[0209] Bus 1530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0210] Electronic device 1500 can also communicate with one or more external devices 150 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1500, and / or with any device that enables electronic device 1500 to communicate with one or more other computing devices (e.g., router, modem, etc.). Such communication can be performed through input / output (I / O) interface 1550.

[0211] Furthermore, electronic device 1500 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 1560. Network adapter 1560 can communicate with other modules of electronic device 1500 via bus 1530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0212] This invention also provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of a wafer defect detection method or a defect detection model training method. In some possible implementations, various aspects of this invention can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to execute the steps described in the above-described wafer defect detection method or defect detection model training method section of this specification, according to various exemplary embodiments of the invention.

[0213] refer to Figure 16 As shown, a program product 1600 for implementing the above-described method according to an embodiment of the present disclosure is described. The program product for implementing the above-described method according to an embodiment of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0214] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0215] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0216] Program code for performing the processing of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0217] In summary, the purpose of this invention is to provide a wafer defect detection method, a defect detection model training method, and an apparatus. The defect detection model is trained using a fused image of a training defect image and a training reference image. The fused image contains difference information between the training defect image and the training reference image, thereby incorporating reference image information into the defect detection model. In this configuration, the defect detection model can perform defect detection by combining the defect information of the defect image itself and its difference information with the reference image, rather than being limited to the defect image itself. Therefore, even with a small amount of training defect image data, accurate wafer defect detection results can be obtained.

[0218] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting wafer defects, characterized in that, include: Acquiring an image of a defect to be inspected and a corresponding reference image, wherein the reference image does not contain wafer defects, the acquisition of the image of the defect to be inspected and the corresponding reference image includes: acquiring the target wafer where the defect to be inspected is located and the target position on the target wafer, and performing electron beam imaging on the target wafer based on the target position to obtain the image of the defect to be inspected; determining the adjacent wafers of the target wafer on the wafer, and the corresponding positions on the adjacent wafers corresponding to the target position, and performing electron beam imaging on the adjacent wafers based on the corresponding positions to obtain the reference image; The image of the defect to be inspected and the reference image are registered, and the registered image of the defect to be inspected and the reference image are fused to obtain a fused image. The fused image contains information of the image of the defect to be inspected and information of the reference image. The fused image includes at least a portion of the overlapping area of ​​the image of the defect to be inspected and the reference image. The fused image is input into the defect detection model, and the defect detection results on the wafer are output. The defect detection model is trained from multiple fused training images, which contain training defect image information and training reference image information.

2. The wafer defect detection method according to claim 1, characterized in that, The registered image of the defect to be inspected is fused with the reference image to obtain a fused image, including: Determine the overlapping area between the registered defect image to be inspected and the reference image; The registered image of the defect to be inspected and the reference image are merged by channel merging to obtain the fused image, wherein at least a portion of the region is determined based on the target location of the defect to be inspected.

3. The wafer defect detection method according to claim 1 or 2, characterized in that, The registration of the defect image to be inspected and the reference image includes: The defect image to be inspected is subjected to at least one deformation process using a transformation matrix until the correlation coefficient between the deformed defect image and the reference image is not lower than a threshold. The deformed defect image is then used as the defect image to be inspected and registered with the reference image.

4. A training method for a defect detection model, characterized in that, include: Obtain a training set, which includes unlabeled training fusion images and labeled training fusion images. The training fusion images include registered training defect image information and training reference image information. The training reference images do not contain wafer defects, and the labels are defect location labels. The defect detection model is trained using the training set. Obtaining the training set includes: Obtain a training defect image and at least one corresponding training reference image; register the training defect image and the corresponding training reference image; fuse the registered training defect image and the training reference image to obtain the training fused image, the training fused image including at least a portion of the overlapping region of the training defect image and the training reference image; construct the training set using the unlabeled training fused image and the labeled training fused image; The acquisition of the training defect image and at least one corresponding training reference image includes: The defect is located on a first wafer and its first position on the first wafer is obtained, and electron beam imaging is performed on the first wafer based on the first position to obtain the training defect image; A second wafer adjacent to the first wafer and a second position on the second wafer corresponding to the first position are determined on the wafer, and electron beam imaging is performed on the second wafer based on the second position to obtain the training reference image.

5. The training method for the defect detection model according to claim 4, characterized in that, The process of fusing the registered training defect image with the training reference image to obtain the fused training image includes: Determine the overlapping region between the registered training defect image and the training reference image; The registered training defect image and training reference image are channel-merged to obtain the training fused image, wherein at least a portion of the region is determined based on the target location of the defect.

6. The training method for the defect detection model according to claim 4, characterized in that, The registration of the defective image and the corresponding training reference image includes: The defect image is deformed using a transformation matrix until the correlation coefficient between the deformed image and the training reference image is not lower than a threshold. The deformed image is then used as the training defect image registered with the training reference image.

7. The training method for the defect detection model according to claim 4, characterized in that, The step of training the defect detection model using the training set includes performing the following processing on the unlabeled training fusion image and the labeled training fusion image respectively: Slice the trained fused image; Perform multi-feature scale transformation on each slice image; Convolution is performed on the multi-scale transformed image of each slice to obtain the feature vector; Defect detection is performed based on the feature vector to obtain defect detection prediction results; The loss function is calculated based on the defect detection results and the corresponding real defect information, and the model parameters of the defect detection model are adjusted according to the loss value until the training stopping condition is met; wherein, the loss function includes at least the channel difference between the training defect image and the training reference image.

8. A wafer defect detection device, characterized in that, include: An acquisition module acquires an image of a defect to be inspected and a corresponding reference image, wherein the reference image does not contain wafer defects. Acquiring the image of the defect to be inspected and the corresponding reference image includes: acquiring the target wafer where the defect to be inspected is located and its target position on the target wafer, and performing electron beam imaging on the target wafer based on the target position to obtain the image of the defect to be inspected; determining the adjacent wafers of the target wafer on the wafer, and the corresponding positions on the adjacent wafers corresponding to the target position, and performing electron beam imaging on the adjacent wafers based on the corresponding positions to obtain the reference image. The fusion module registers the defect image to be inspected and the reference image, and fuses the registered defect image to be inspected with the reference image to obtain a fused image. The fused image contains defect image information and reference image information, and includes at least a portion of the overlapping area between the defect image to be inspected and the reference image. The prediction module inputs the fused image into the defect detection model and outputs the defect detection results on the wafer.

Images