Mask-based wafer defect classification system and method

Abstract

The present invention discloses a mask-based wafer defect classification system and method, which relates to the field of wafer defect classification technology. The system assigns a corresponding category number to the pixels in each defective area to form a label image. A 1x1 convolutional layer is added to convert a four-channel input image into a three-channel feature map. The label image and the three-channel feature map are then input into an image semantic segmentation model for training. When a defective wafer image is input into the trained image semantic segmentation model, the model performs inference and outputs the wafer's defect location and classification results. Based on the defect location and classification results, the wafer is evaluated for defects and a defect report is generated. The classification system splices the wafer image with the defect mask in the channel direction, allowing all defective areas to be fully classified through a single segmentation inference. This not only optimizes image segmentation accuracy but also improves processing efficiency, significantly enhancing the performance and applicability of the wafer defect classification system.

Description

Technical Field

[0001] The present invention relates to the technical field of wafer defect classification, and in particular to a mask-based wafer defect classification system and method. Background Art

[0002] In semiconductor manufacturing, wafers are the basic materials of integrated circuits. Any tiny defects in the manufacturing process may affect the performance of the final product. The classification system is a technology used to detect and analyze wafer surface defects during the semiconductor manufacturing process. This system combines image processing, machine learning, and computer vision technologies to effectively identify and classify different types of defects on the wafer surface to ensure quality control during the production process.

[0003] The existing technology has the following deficiencies:

[0004] Existing classification systems typically process images and defect masks separately (i.e., all defects in an image are cut out and then classified one by one). This results in high complexity and insufficient information fusion, affecting the accuracy of defect location and classification. Furthermore, the results in the model inference stage have marginal errors and cannot meet the needs of different application scenarios. Furthermore, the post-processing steps are cumbersome, which increases the response time of the classification system and reduces the performance and applicability of the wafer defect classification system.

[0005] Based on this, this application proposes a mask-based wafer defect classification system and method, which splices the wafer image and the defect mask in the channel direction. All defect areas can be classified through one segmentation reasoning, which not only optimizes the image segmentation accuracy, but also improves the processing efficiency, significantly improving the performance and applicability of the wafer defect classification system. Summary of the Invention

[0006] The object of the present invention is to provide a mask-based wafer defect classification system and method to address the deficiencies in the background technology.

[0007] In order to achieve the above object, the present invention provides the following technical solution: a mask-based wafer defect classification method, the classification method comprising the following steps:

[0008] The classification system splices the wafer image containing defects and its corresponding defect mask in the channel direction to form a four-channel input image;

[0009] Label each defect area in the defect mask by its category, assign the pixels in each defect area a corresponding category number, and form a label image;

[0010] Convert the four-channel input image into a three-channel feature map by adding a 1x1 convolutional layer;

[0011] Input the label image and the three-channel feature map into the image semantic segmentation model for training;

[0012] When a defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference and outputs the wafer defect location and classification results;

[0013] Based on the defect location and classification results, the wafer is defect evaluated and a defect report is generated.

[0014] In a preferred embodiment, the four-channel input image is converted into a three-channel feature map by adding a 1x1 convolutional layer, comprising the following steps:

[0015] Assume the input image ,in and are the height and width of the image respectively. The fourth channel represents the defect mask information. The first three channels are standard RGB image data. The 1x1 convolution layer converts its output into a three-channel feature map. ;

[0016] For each pixel Perform convolution to perform a weighted sum on the four input channels and generate a new output channel.

[0017] In a preferred embodiment, for each pixel Perform convolution, perform weighted summation on the four input channels, and generate new output channels. The convolution operation expression is:

[0018] ,in: Represents the output feature map In position Hedi The pixel values ​​of the output channels, is the weight of the 1x1 convolution kernel, which represents the conversion matrix that compresses four input channels into three output channels. represents the index of the input channel, represents the index of the output channel, is the input image In position Hedi The pixel values ​​of the input channels, is the bias term used to adjust the output.

[0019] In a preferred embodiment, the label image and the three-channel feature map are input into the image semantic segmentation model for training, comprising the following steps:

[0020] The generated label image And the three-channel feature map obtained by converting the 1x1 convolution layer Input into the U-Net model for training;

[0021] The U-Net model consists of an encoder and a decoder. The encoder extracts features from the input image through multiple convolutional layers and pooling layers. The decoder gradually restores the spatial resolution of the image through upsampling layers and convolutional layers, and outputs a classification probability map for each pixel.

[0022] The encoder consists of multiple convolutional layers and maximum pooling layers. Each convolution operation extracts spatial features from the image, and the pooling operation gradually reduces the spatial resolution of the image.

[0023] The decoder consists of upsampling layers and convolutional layers to restore the spatial resolution of the image and generate per-pixel classification results.

[0024] In a preferred embodiment, each defective area in the defect mask is labeled by category, and pixels in each defective area are assigned corresponding category numbers to form a label image, which includes the following steps:

[0025] set up Indicates the current wafer image, is an ideal template image, then the defect mask Expressed as: ,in, is the pixel position of the image, is the deviation threshold;

[0026] After generating the preliminary defect mask, each defect area in the defect mask is marked. The label number starts from 1 and increases, and the background area is marked as 0. Each defect area Label image Pixels are represented as: ,in, is the pixel value in the label image, Indicates the sth defect area, where s is the number of the defect area.

[0027] In a preferred embodiment, the classification system splices a wafer image containing defects and its corresponding defect mask in the channel direction to form a four-channel input image, including the following steps:

[0028] Assume the original wafer image is ,in and Represent the height and width of the image respectively, and the third dimension is 3, then the defect mask is expressed as , where 1 represents the defect area and 0 represents the background area;

[0029] The four-channel input image formed after stitching is: ,in, Represents a four-channel input image.

[0030] In a preferred embodiment, the image semantic segmentation model performs inference and outputs wafer defect location and classification results, including the following steps:

[0031] For application scenarios that focus on accuracy requirements, a mask correction strategy is introduced to correct the segmentation results output by the image semantic segmentation model by using the original defect mask as the positioning basis;

[0032] The area beyond the original mask range is set to zero so that the segmentation result coincides with the original mask. The category with the highest proportion of pixels in each defect area is counted to determine the defect category of the defect area, and the defect location and classification results are obtained.

[0033] In a preferred embodiment, a mask correction strategy is introduced to correct the segmentation results output by the image semantic segmentation model by using the original defect mask as a positioning basis, including the following steps:

[0034] Set the area beyond the original mask range to zero, expressed as:

[0035] ;in, is the corrected segmentation result, is the segmentation result output by the image semantic segmentation model, is the position in the original mask Pixel value of

[0036] For each defect area , count the number of pixels of each category in the defect area , select the category with the largest proportion The defect category of the defect area is expressed as:

[0037] ,in, Indicates area Medium Category The number of pixels, It is the category with the largest number of categories.

[0038] In a preferred embodiment, the image semantic segmentation model performs inference and outputs wafer defect location and classification results, further comprising the following steps:

[0039] For application scenarios that focus on efficiency requirements, the output results of the image semantic segmentation model are directly used as the final results of defect localization and classification.

[0040] This application also provides a mask-based wafer defect classification system, including an image stitching module, a region annotation module, an image conversion module, a model training module, a defect reasoning module, and a report generation module;

[0041] Image stitching module: stitches the wafer image containing defects with its corresponding defect mask in the channel direction to form a four-channel input image;

[0042] Region labeling module: labels each defect region in the defect mask by category, assigns the pixels in each defect region a corresponding category number, and forms a label image;

[0043] Image conversion module: converts the four-channel input image into a three-channel feature map by adding a 1x1 convolutional layer;

[0044] Model training module: inputs the label image and three-channel feature map into the image semantic segmentation model for training;

[0045] Defect reasoning module: When a defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference and outputs the wafer defect location and classification results;

[0046] Report generation module: Based on the defect location and classification results, the module performs defect assessment on the wafer and generates a defect report.

[0047] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0048] The present invention uses a classification system to splice a wafer image containing defects with its corresponding defect mask in the channel direction to form a four-channel input image, classify each defect area in the defect mask, assign the pixels of each defect area to the corresponding category number to form a label image, convert the four-channel input image into a three-channel feature map by adding a 1x1 convolutional layer, input the label image and the three-channel feature map into an image semantic segmentation model for training, and when the defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference and outputs the defect location and classification results of the wafer. Based on the defect location and classification results, the wafer is defect assessed and a defect report is generated. The classification system splices the wafer image and the defect mask in the channel direction, and all defect areas can be fully classified through a single segmentation reasoning, which not only optimizes the image segmentation accuracy, but also improves the processing efficiency, significantly improving the performance and applicability of the wafer defect classification system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0050] Figure 1 Flowchart of the classification method of the present invention.

[0051] Figure 2 This is an architectural diagram of the classification system of the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] Example: See Figure 1 As shown, this embodiment proposes a mask-based wafer defect classification method, which includes the following steps:

[0054] The classification system concatenates defective wafer images and their corresponding defect masks in the channel direction to form a four-channel input image. The first three channels of the image are standard RGB channels, while the fourth channel comes from the defect mask and indicates the location of potential defect areas in the wafer image.

[0055] Defect masks can be generated using existing techniques (such as die-to-golden and die-to-die algorithms) to identify the location and shape of each defect region. Segmentation and annotation tools are then used to label each defect region within the generated defect mask. The pixels in each defect region are assigned a corresponding category number, which is then used to form a label image. The numbering starts at 1 and increases incrementally (0 is reserved for background). This labeled defect mask provides a label image for training the image semantic segmentation model, clearly indicating the type of each defect region.

[0056] Since most image semantic segmentation models (such as the U-Net segmentation model) only support three-channel RGB image input, a custom 1x1 convolutional layer is added (this 1x1 convolutional layer randomly initializes parameters and sets the bias to false to achieve seamless integration of the four-channel image conversion and the image semantic segmentation model). This converts the four-channel input image into a three-channel feature map suitable for the existing image semantic segmentation model (the U-Net segmentation model).

[0057] The label image and the converted three-channel feature map (the three-channel feature map is obtained by converting the four-channel input image) are input into the image semantic segmentation model (U-Net segmentation model) for training. The trained image semantic segmentation model can output the location and classification information of defects in the wafer image.

[0058] When the defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference:

[0059] During the inference phase, the segmentation results output by the image semantic segmentation model may contain certain errors, especially in defect edge areas. Therefore, a precision-prioritized mask correction strategy is introduced for applications requiring high precision. The segmentation results output by the image semantic segmentation model are corrected using the original defect mask as the basis for positioning. Specifically, regions outside the original mask range are set to zero, ensuring that the final segmentation result coincides with the original mask. Subsequently, the category with the highest pixel proportion within each defect area is counted to determine the defect category of the defect area, resulting in accurate defect location and classification results.

[0060] For applications requiring high efficiency, the output of the image semantic segmentation model can be directly used as the final result for defect localization and classification, without further post-processing. This solution can improve processing speed and is suitable for time-sensitive tasks such as real-time monitoring.

[0061] Based on the final positioning and classification results, the wafer defects are evaluated and a defect report is generated. The report will include information such as defect type, location, and size for further quality control and decision support.

[0062] This application uses a classification system to splice a wafer image containing defects with its corresponding defect mask in the channel direction to form a four-channel input image, classify each defect area in the defect mask, assign the pixels of each defect area to the corresponding category number to form a label image, convert the four-channel input image into a three-channel feature map by adding a 1x1 convolution layer, input the label image and the three-channel feature map into the image semantic segmentation model for training, and when the defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference and outputs the defect location and classification results of the wafer. Based on the defect location and classification results, the wafer is defect-evaluated and a defect report is generated. This classification system splices the wafer image and the defect mask in the channel direction, and all defect areas can be classified through a single segmentation inference. This not only optimizes the image segmentation accuracy, but also improves the processing efficiency, significantly improving the performance and applicability of the wafer defect classification system.

[0063] The classification system concatenates defective wafer images and their corresponding defect masks in the channel direction to form a four-channel input image. The first three channels of the image are standard RGB channels, while the fourth channel comes from the defect mask and indicates the location of potential defect areas in the wafer image.

[0064] In a mask-based wafer defect classification system, the classification system forms a four-channel input image by splicing a defective wafer image with the corresponding defect mask in the channel direction. The first three channels of this input image are standard RGB channels, containing the color and brightness information of the wafer image, while the fourth channel comes from the defect mask and is used to indicate the location and shape of potential defect areas in the image. The mask is generated using existing technologies (such as die-to-golden and die-to-die algorithms) to identify defective areas on the wafer surface. In this way, the four-channel input image not only contains visual information about the wafer but also carries the identification of the defect location, allowing the classification system to simultaneously process the image content and the location of the defect area, further improving classification accuracy.

[0065] The wafer image containing defects and its corresponding defect mask are spliced ​​in the channel direction to form a four-channel input image. Traditional image semantic segmentation models usually process three-channel RGB images, but in this method, additional defect mask information is added as the fourth channel to ensure that the model can simultaneously obtain visual features and defect area information. Assume that the original wafer image is ,in and Represent the height and width of the image respectively, and the third dimension is 3 (representing the RGB channels), then the defect mask can be expressed as , which is a single-channel binary image, where 1 represents the defect area and 0 represents the background area. After splicing, the four-channel input image formed is: ,in, represents a four-channel input image, Usually represents a set of real numbers (real-numbers), including RGB images and defect mask information.

[0066] Defect masks can be generated using existing techniques (such as die-to-golden and die-to-die algorithms) to identify the location and shape of each defect region. Segmentation and annotation tools are then used to label each defect region within the generated defect mask. The pixels in each defect region are assigned a corresponding category number, which is then used to form a label image. The numbering starts at 1 and increases incrementally (0 is reserved for background). This labeled defect mask provides a label image for training the image semantic segmentation model, clearly indicating the type of each defect region.

[0067] In mask-based wafer defect classification systems, defect mask generation is a critical step. Defect masks are typically generated using existing technologies, such as die-to-golden and die-to-die algorithms, which accurately identify the location and shape of defect areas within wafer images. In these generated masks, the boundaries and shape of the defect areas are typically determined by comparing the differences between a standard template (golden die) and the current image (die). The defect mask is typically a binary image, with a value of 1 representing the defect area and a value of 0 representing the background area. The mask generation process must account for image noise, deformation, and various possible process errors, making accuracy crucial.

[0068] First, a preliminary defect mask is generated using existing techniques (e.g., die-to-golden, die-to-die, etc.). These methods use the pixel differences between the standard wafer template (golden-die) and the current image (die) to generate the defect area. Specifically, the die-to-golden algorithm compares the current wafer image with the ideal template at the pixel level to find the deviation between the two and then generates a mask for the defect area. Assume Indicates the current wafer image, is an ideal template image, then the defect mask It can be expressed as:

[0069] ,in, is the pixel position of the image, The result of this process is a rough defect mask that may contain some noise and errors.

[0070] After generating the preliminary defect mask, the next step is to label each defect area in the mask. The goal of this step is to assign a unique category number to each defect area and generate a corresponding label image. Specifically, through the segmentation and annotation tool, each independent defect area in the mask can be labeled and a category label can be assigned to each area. The label number starts at 1 and increases, and the background area (the area without defects) is marked as 0. For each defect area (s is the number of the defect area), its label image The pixels can be expressed as:

[0071] ,in, is the pixel value in the label image, Represents the sth defect area. Through this labeling process, the label image The category of each defect area is clearly pointed out, and accurate label data is provided for model training.

[0072] After assigning a category label to each defective region using the annotation tool, the resulting labeled images (LLL) are stored as part of the training data for subsequent model training. These labeled images are mapped one-to-one to the corresponding input images (wafer images containing defects) and are learned by the image semantic segmentation model. The model's goal is to learn how to predict the category of each pixel based on the pixel values ​​in the input image, thereby locating and classifying defective regions on the wafer surface.

[0073] Since most image semantic segmentation models (such as the U-Net segmentation model) only support three-channel RGB image input, a custom 1x1 convolutional layer is added (this 1x1 convolutional layer randomly initializes parameters and sets the bias to false to achieve seamless integration of the four-channel image conversion and the image semantic segmentation model). This converts the four-channel input image into a three-channel feature map suitable for the existing image semantic segmentation model (the U-Net segmentation model).

[0074] Image semantic segmentation models, such as the U-Net segmentation model, typically expect a three-channel RGB image as input. However, mask-based wafer defect classification tasks require input images with four channels: the first three channels are standard RGB channels, and the fourth channel is a defect mask, which indicates the location of the defective area in the image. This four-channel input is incompatible with the traditional three-channel RGB input format. Directly passing the four-channel input to existing image segmentation models results in the model being unable to process and learn correctly. Therefore, the key to addressing this issue lies in adapting the input to the model.

[0075] To achieve seamless integration between four-channel input and existing three-channel image semantic segmentation models (such as U-Net), this solution proposes adding a custom 1x1 convolutional layer to convert the four-channel input image into a three-channel feature map suitable for model input. The purpose of this 1x1 convolutional layer is to compress the four-channel information into a three-channel feature map through learning, while preserving the spatial information in the image and the defect location information provided by the mask.

[0076] The 1x1 convolution layer is a special convolution operation whose convolution kernel size is 1x1, which means that it does not change the spatial dimensions of the input image (the height and width remain unchanged), but can change the number of channels. In this design, we use the 1x1 convolution layer to compress the four-channel input image into a three-channel feature map. Assume that the input image ,in and The height and width of the image are respectively, the fourth channel represents the defect mask information, and the first three channels are standard RGB image data. The 1x1 convolution layer converts its output into a three-channel feature map .

[0077] The core operation of the 1x1 convolutional layer is to perform a multiplication of each pixel. Perform convolution, which performs a weighted sum on the four input channels and generates a new output channel. The mathematical formula for the convolution operation is as follows: ,in: Represents the output feature map In position Hedi The pixel values ​​of the output channels, is the weight of the 1x1 convolution kernel, which represents the conversion matrix that compresses four input channels into three output channels. Here, represents the index of the input channel, represents the index of the output channel, is the input image In position Hedi The pixel values ​​of the input channels, is the bias term, which is used to perform translation adjustment on the output (in this solution, the bias term is set to false, that is, the bias term is not used).

[0078] By introducing a custom 1x1 convolutional layer, a four-channel input image (including RGB images and defect mask information) can be successfully converted into a three-channel feature map suitable for existing image semantic segmentation models (such as U-Net). This method allows four-channel images to be seamlessly integrated with traditional three-channel image semantic segmentation models, not only preserving the image's visual information but also effectively incorporating defect location information, thereby improving defect classification accuracy.

[0079] The label image and the converted three-channel feature map (the three-channel feature map is obtained by converting the four-channel input image) are input into the image semantic segmentation model (U-Net segmentation model) for training. The trained image semantic segmentation model can output the location and classification information of defects in the wafer image.

[0080] In the image semantic segmentation task, the three-channel feature map transformed by the 1x1 convolutional layer, along with the label image, is fed into an image semantic segmentation model (such as the U-Net segmentation model) for training. U-Net is a typical convolutional neural network (CNN) specifically designed for image segmentation. Its core concept is to extract image features and restore spatial resolution using a symmetrical encoder-decoder architecture. After training, the U-Net model can accurately locate and classify defects in wafer images.

[0081] First, the generated label image And the three-channel feature map obtained by converting the 1x1 convolution layer Input them together into the U-Net model for training. Label image Contains the category label of each pixel, where the pixel value of each defect area is the corresponding category number, and the pixel value of the background area is 0. Feature map It contains the visual features of the image and the location of the defect. By combining these two, the U-Net model can learn how to predict the category of each pixel based on the features in the input image.

[0082] The U-Net model consists of two main parts: an encoder and a decoder. The encoder extracts features from the input image through multiple convolutional and pooling layers, while the decoder gradually restores the spatial resolution of the image through upsampling and convolutional layers, ultimately outputting a classification probability map for each pixel.

[0083] The encoder part is usually composed of multiple convolutional layers and max-pooling layers. Each convolution operation extracts spatial features from the image, and the pooling operation gradually reduces the spatial resolution of the image, allowing the model to learn higher-level abstract features. Each convolutional layer of the encoder outputs It can be expressed as: ,in, is the feature map input to the convolutional layer, is the convolution kernel, is the bias term, Represents the activation function (such as ReLU). This process forms the feature extraction part by stacking multiple layers.

[0084] The decoder consists of upsampling layers (such as transposed convolution or upsampling layer operations) and convolutional layers, gradually restoring the spatial resolution of the image and ultimately generating a classification result for each pixel. During the decoding process, U-Net uses skip connections to pass features from the encoder layer directly to the decoder, preserving detailed information while facilitating accurate boundary prediction.

[0085] During training, the U-Net model updates its network weights by minimizing the loss function, usually using cross-entropy loss to evaluate the difference between the model output and the true label. The cross-entropy loss function is defined as:

[0086] ;

[0087] in, is the cross entropy loss, and are the height and width of the image, is the number of categories, is the position in the label image The true category, is the class probability output by the model. During the training process, the U-Net model optimizes its parameters through the back-propagation algorithm, reducing the loss value, thereby improving the ability to identify and classify defects.

[0088] During training, the U-Net model encodes and decodes a four-channel input image (containing RGB and defect mask information) to learn how to predict the defect category and location based on image features. Using a cross-entropy loss function, the model continuously optimizes weights to reduce prediction error, ultimately achieving efficient defect classification in wafer images. After model training, inference and post-processing are performed to output the precise location and category of each defect, effectively supporting quality control in semiconductor manufacturing.

[0089] When the defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference:

[0090] During the inference phase, the segmentation results output by the image semantic segmentation model may contain certain errors, especially in defect edge areas. Therefore, a precision-prioritized mask correction strategy is introduced for applications requiring high precision. The segmentation results output by the image semantic segmentation model are corrected using the original defect mask as the basis for positioning. Specifically, regions outside the original mask range are set to zero, ensuring that the final segmentation result coincides with the original mask. Subsequently, the category with the highest pixel proportion within each defect area is counted to determine the defect category of the defect area, resulting in accurate defect location and classification results.

[0091] During the inference phase, the image semantic segmentation model outputs a category prediction for each pixel in the wafer image based on the learning results from the training process. However, the model's segmentation results may contain certain errors, especially in the defect edge area. This can lead to loss of some details or misclassification, affecting the precise location of defects. Therefore, in applications with high precision requirements, a precision-prioritized mask correction strategy is introduced to optimize the model's output and ensure more accurate final defect classification and location.

[0092] First, the image semantic segmentation model generates preliminary segmentation results will be consistent with the original defect mask The original defect mask is generated based on the actual image and typically contains the exact location and shape of the defect area. This mask information provides a reference for the segmentation results and helps determine which areas require correction. Defect areas in the original mask are marked as 1, and background areas are marked as 0.

[0093] After comparing the inference result with the original mask, the segmentation result output by the model is , we perform mask correction. The core idea of ​​correction is to set the area beyond the original mask range to zero, ensuring that the final segmentation result is consistent with the original defect mask and avoiding the error of the segmentation result exceeding the defect area. The specific operation is as follows:

[0094] ;in, is the corrected segmentation result, is the position in the original mask If the location belongs to the defect area (i.e., the mask value is 1), the model's prediction value is retained, otherwise the segmentation result of the location is set to zero. This process ensures that the output segmentation area strictly conforms to the boundary of the original defect mask, thereby reducing the possibility of misclassification, especially in edge areas.

[0095] The corrected segmentation result may still contain multiple defective areas, and each area may contain pixels of multiple categories. In this case, we need to determine the final category of each defective area. In order to improve the accuracy of classification, we use the voting method, which counts the category with the highest proportion of pixels in each defective area to determine the final category of the area. Specifically, for each defective area , we count the number of pixels of each category in the area Then select the category with the largest proportion As the defect category of the area. That is:

[0096] ,in, Indicates area Medium Category The number of pixels, The category with the largest number of categories is considered to be the region In this way, we can further improve the accuracy of the segmentation results and ensure that the category identification of each defect area is the most representative.

[0097] The corrected segmentation results will accurately reflect the location and category information of each defect area. The final classification label for each pixel is included, and the category of each defect area is determined by voting as the defect type in that area. Ultimately, the system can generate a detailed defect report based on the output classification map, including information such as the type, location, and size of each defect, providing an accurate basis for subsequent quality control and decision support.

[0098] The precision-first mask correction strategy modifies the model output so that the segmentation results strictly conform to the boundaries of the original defect mask, avoiding misclassification of areas outside the bounds. A voting method is combined to count the pixel categories within each defect area and ultimately determine the defect category, ensuring accurate defect location and classification. This process improves the performance of the image semantic segmentation model in high-precision applications and provides reliable support for subsequent quality control and decision-making.

[0099] For applications requiring high efficiency, the output of the image semantic segmentation model can be directly used as the final result for defect localization and classification, without further post-processing. This solution can improve processing speed and is suitable for time-sensitive tasks such as real-time monitoring.

[0100] In certain application scenarios, especially real-time monitoring and high-frequency tasks, processing speed requirements are extremely high. These applications often require rapid response and immediate feedback, such as the rapid detection and classification of wafer surface defects in semiconductor manufacturing. To meet these high-efficiency requirements, the output of the image semantic segmentation model can be directly used as the final result of defect localization and classification without further post-processing. This solution not only saves post-processing time but also improves the response speed of the entire inspection system, making it suitable for time-sensitive tasks.

[0101] The following factors are mainly used to determine whether an application scenario has high efficiency requirements:

[0102] Real-time feedback requirements: Tasks such as online inspection and real-time monitoring require rapid response to ensure that defects can be immediately identified and corrected during the manufacturing process.

[0103] High processing frequency: In some production environments, the processing frequency of the equipment is very high, and the processing time of each image must be controlled within an extremely short range to avoid affecting the entire production process.

[0104] Tolerance and accuracy balance: In some scenarios, the system can tolerate a certain degree of error and allow slight inaccuracies in the segmentation results as long as the detection speed and efficiency can be guaranteed. However, for tasks that require high accuracy (such as medical imaging diagnosis), this approach is not suitable.

[0105] Parallel processing capability: In large-scale production processes, multiple images or regions may need to be processed simultaneously, and the system's processing speed becomes a bottleneck. To address this, it is often necessary to choose a solution that directly outputs the segmentation results to improve the overall efficiency of the system.

[0106] For application scenarios with high efficiency requirements, you can choose to directly use the output of the image semantic segmentation model as the final result of defect location and classification. Specifically, in the inference stage, the image semantic segmentation model directly generates a category prediction for each pixel based on the input image. Assume that the output of the segmentation model is ,in and are the height and width of the image, is the number of categories, and outputs the classification probability of each pixel. The output of the model is for each pixel In each category The predicted probability on The calculation is as follows:

[0107] ,in, Is the logits output by the model, indicating the position Belongs to category By normalizing the category probability of each pixel, we can finally get the probability that each pixel belongs to each category. The category predicted by the model is the category with the highest probability, that is: , the output of the model That is, the category prediction for each pixel. , the predicted category of the pixel Indicates the defect category to which the location belongs, or the background area. ,We can further mark its location according to the predicted category label to obtain the precise location and category of the defect area.,This process does not require additional post-processing, and the output classification results are directly used as the final defect classification and location information.

[0108] By skipping the post-processing step, the system can quickly generate a class label for each pixel, significantly reducing computation time. In applications such as real-time monitoring, the processing time of each image becomes critical to system responsiveness. Without additional post-processing such as mask correction, class voting, or edge smoothing, the model output can be used directly for defect detection and classification tasks. This approach can significantly improve processing efficiency, especially in large-scale production lines or environments requiring rapid response, ensuring that more images are processed per second, thereby increasing production efficiency and inspection coverage.

[0109] Based on the final positioning and classification results, the wafer defects are evaluated and a defect report is generated. The report will include information such as defect type, location, and size for further quality control and decision support.

[0110] In wafer defect inspection systems, the final localization and classification results provide crucial information for defect assessment. Subsequent processing and analysis of the segmentation model output allows for the generation of a detailed defect report that includes not only the defect type but also key parameters such as its precise location and size, providing a robust basis for quality control and decision support.

[0111] First, the system is based on the output of the image semantic segmentation model , determine the category of each pixel and classify all defect areas. In , the category label is determined by the classification result predicted by the model. Specifically, for each region ,The final defect category of this area Calculated by the following formula: ,in, Indicates area Internal category The number of pixels, is the final defect category of the area. Through this process, all defect areas are assigned clear category labels to ensure accurate identification of defect types.

[0112] Next, the system will accurately calculate the location of each defect area. It is composed of multiple pixels. By extracting the spatial coordinates of these pixels, the specific location of the defect can be determined. Assume that the pixel coordinates of the defect area are , then the region The spatial position of the defect can be represented by the average coordinate value of all pixels in the area, that is, the center of gravity of the defect area. for:

[0113] ,in, Is the defect area The total number of pixels in is the center coordinate of the defect area, indicating the center position of the area.

[0114] The size of the defect area is an important parameter in quality control, and the relative size of the defect can usually be evaluated by counting the number of pixels within the defect area. , its size It can be calculated by the following formula: ,in, Is the defect area The number of pixels represents the "area" or size of the region. This value provides a quantitative indicator of defect severity for subsequent decision support. Furthermore, for certain defect types (such as circular or elliptical defects), their size can be estimated by fitting a shape (e.g., minimum enclosing rectangle or minimum enclosing circle). This approach provides a more precise description of geometric dimensions and is suitable for morphological analysis.

[0115] After calculating the defect type, location, and size, the system will generate a defect report that summarizes the detailed information of all defects and typically contains the following:

[0116] 1. Defect type: The category label of each defect area.

[0117] 2. Defect location: the coordinates of the center of gravity of each defect area , to describe the spatial location of the defect.

[0118] 3. Defect size: the number of pixels in each defect area , used to describe the relative size of defects.

[0119] 4. Defect distribution: The distribution of defects on the wafer surface helps determine whether the defects are regular or random.

[0120] 5. Number of defects: The total number of defects detected, as well as the number of defects in each category, facilitate statistical analysis. This information can provide the quality control team with detailed data support on wafer quality and help the team determine whether adjustments to the production process are needed.

[0121] Based on defect reports, quality control personnel can further analyze the severity of defects. For example, defects covering a larger area may require special attention, or a certain type of defect may indicate a systemic problem in the production process. Furthermore, through statistical analysis of different defect types, production lines can identify potential process issues and implement targeted improvement measures. For example, if a certain type of defect frequently occurs in a certain location, it may indicate that equipment or process parameters need adjustment.

[0122] In terms of decision support, defect reports provide quantitative evidence for production management, helping the production team make data-based decisions, such as:

[0123] Production stop decision: Decide whether to stop the production line for inspection or adjustment based on the number and type of defects.

[0124] Product screening: By setting a threshold for defect size or quantity, unqualified wafers are screened out for removal or rework.

[0125] Process optimization: Through statistical analysis of defect types and locations, we can identify possible systematic problems in production and optimize the production process.

[0126] To improve production efficiency and reduce manual intervention, defect assessment and reporting can be integrated into automated production monitoring systems. When a defect is detected, a defect report is automatically generated and fed directly to the production team or quality control system. Based on this automated feedback, production processes can be adjusted in real time to ensure that production quality meets standards.

[0127] By evaluating defects based on the final positioning and classification results, the system generates a detailed defect report, including key indicators such as defect type, location, and size, providing an important basis for quality control and decision support. This report not only helps quality control personnel determine whether adjustments are necessary, but also provides production management with a quantitative analysis tool to promote continuous optimization and improvement of the production process.

[0128] See also Figure 2 As shown, this embodiment also proposes a mask-based wafer defect classification system, including an image stitching module, a region annotation module, an image conversion module, a model training module, a defect reasoning module, and a report generation module;

[0129] Image stitching module: This module stitches the defective wafer image and its corresponding defect mask channel-wise to form a four-channel input image. The first three channels of the image are standard RGB channels, while the fourth channel, derived from the defect mask, indicates the location of potential defect areas in the wafer image. This four-channel input image is sent to the region annotation module and the image conversion module.

[0130] Region Annotation Module: Defect masks can be generated using existing technologies (such as die-to-golden and die-to-die algorithms) to identify the location and shape of each defect region. Segmentation and annotation tools are then used to label each defect region within the generated defect mask. Pixels in each defect region are assigned a corresponding category number, which is then used to form a label image. The numbering starts at 1 and increases incrementally (0 is reserved for background). This labeled defect mask provides a label image for training the image semantic segmentation model, clearly indicating the type of each defect region. The labeled image is then sent to the model training module.

[0131] Image conversion module: Since most image semantic segmentation models (such as the U-Net segmentation model) only support three-channel RGB image input, this module adds a custom 1x1 convolutional layer (this 1x1 convolutional layer uses random initialization parameters and sets the bias to false to achieve seamless integration of four-channel image conversion and the image semantic segmentation model). This converts the four-channel input image into a three-channel feature map suitable for the existing image semantic segmentation model (the U-Net segmentation model). The three-channel feature map is then sent to the model training module.

[0132] Model training module: The label image and the converted three-channel feature map (the three-channel feature map is obtained by converting the four-channel input image) are input into the image semantic segmentation model (U-Net segmentation model) for training. The trained image semantic segmentation model can output the location and classification information of defects in the wafer image. The trained image semantic segmentation model is sent to the defect reasoning module.

[0133] Defect reasoning module: When a defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs reasoning, including:

[0134] During the inference phase, the segmentation results output by the image semantic segmentation model may have certain errors, especially in the defect edge area. For this reason, a precision-prioritized mask correction strategy is introduced for application scenarios with high precision requirements. The segmentation results output by the image semantic segmentation model are corrected by using the original defect mask as the positioning basis. The specific operation is: the area beyond the original mask range is set to zero to ensure that the final segmentation result coincides with the original mask. Subsequently, the category with the highest pixel category ratio in each defect area is counted to determine the defect category of the defect area, thereby obtaining accurate defect positioning and classification results;

[0135] For applications requiring high efficiency, the output of the image semantic segmentation model can be directly used as the final result for defect localization and classification, without further post-processing. This solution can improve processing speed and is suitable for time-sensitive tasks such as real-time monitoring. The localization and classification results are sent to the report generation module.

[0136] Report Generation Module: Based on the final positioning and classification results, the wafer defects are evaluated and a defect report is generated. The report will include information such as defect type, location, and size for further quality control and decision support.

[0137] It should be understood that the term "and / or" as used herein simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the related objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0138] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0140] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A mask-based wafer defect classification method, characterized by: The classification method comprises the following steps: The classification system splices the wafer image containing defects and its corresponding defect mask in the channel direction to form a four-channel input image; Label each defect area in the defect mask by its category, assign the pixels in each defect area a corresponding category number, and form a label image; Convert the four-channel input image into a three-channel feature map by adding a 1x1 convolutional layer; Input the label image and the three-channel feature map into the image semantic segmentation model for training; When a defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference and outputs the wafer defect location and classification results; Based on the defect location and classification results, the wafer is defect evaluated and a defect report is generated.

2. The mask-based wafer defect classification method according to claim 1, wherein: The four-channel input image is converted into a three-channel feature map by adding a 1x1 convolutional layer, The following steps are involved: Assume the input image ,in and are the height and width of the image respectively. The fourth channel represents the defect mask information. The first three channels are standard RGB image data. The 1x1 convolution layer converts its output into a three-channel feature map. , Represents a set of real numbers, including RGB image and defect mask information; For each pixel Perform convolution to perform a weighted sum on the four input channels and generate a new output channel.

3. The mask-based wafer defect classification method according to claim 2, wherein: For each pixel Perform convolution, perform weighted summation on the four input channels, and generate new output channels. The convolution operation expression is: ,in: Represents the output feature map In position Hedi The pixel values ​​of the output channels, is the weight of the 1x1 convolution kernel, which represents the conversion matrix that compresses four input channels into three output channels. represents the index of the input channel, represents the index of the output channel, is the input image In position Hedi The pixel values ​​of the input channels, is the bias term used to adjust the output.

4. The mask-based wafer defect classification method according to claim 3, wherein: The label image and the three-channel feature map are input into the image semantic segmentation model for training, which includes the following steps: The generated label image And the three-channel feature map obtained by converting the 1x1 convolution layer Input into the U-Net model for training; The U-Net model consists of an encoder and a decoder. The encoder extracts features from the input image through multiple convolutional layers and pooling layers. The decoder gradually restores the spatial resolution of the image through upsampling layers and convolutional layers, and outputs a classification probability map for each pixel. The encoder consists of multiple convolutional layers and maximum pooling layers. Each convolution operation extracts spatial features from the image, and the pooling operation gradually reduces the spatial resolution of the image. The decoder consists of upsampling layers and convolutional layers to restore the spatial resolution of the image and generate per-pixel classification results.

5. The mask-based wafer defect classification method according to claim 4, wherein: Each defect area in the defect mask is labeled by category, and the pixels in each defect area are assigned a corresponding category number to form a label image, which includes the following steps: set up Indicates the current wafer image, is an ideal template image, then the defect mask Expressed as: ,in, is the pixel position of the image, is the deviation threshold; After generating the preliminary defect mask, each defect area in the defect mask is marked. The label number starts from 1 and increases, and the background area is marked as 0. Each defect area Label image Pixels are represented as: ,in, is the pixel value in the label image, Indicates the sth defect area, where s is the number of the defect area.

6. The mask-based wafer defect classification method according to claim 5, wherein: The classification system stitches the defective wafer image and its corresponding defect mask in the channel direction to form a four-channel input image. The system includes the following steps: Assume the original wafer image is ,in and Represent the height and width of the image respectively, and the third dimension is 3, then the defect mask is expressed as , where 1 represents the defect area and 0 represents the background area; The four-channel input image formed after stitching is: ,in, Represents a four-channel input image.

7. The mask-based wafer defect classification method according to claim 1, wherein: The image semantic segmentation model performs inference and outputs wafer defect location and classification results, including the following steps: For application scenarios that focus on accuracy requirements, a mask correction strategy is introduced to correct the segmentation results output by the image semantic segmentation model by using the original defect mask as the positioning basis; The area beyond the original mask range is set to zero so that the segmentation result coincides with the original mask. The category with the highest proportion of pixels in each defect area is counted to determine the defect category of the defect area, and the defect location and classification results are obtained.

8. The mask-based wafer defect classification method according to claim 7, wherein: A mask correction strategy is introduced to correct the segmentation results output by the image semantic segmentation model by using the original defect mask as the positioning basis. The strategy includes the following steps: Set the area beyond the original mask range to zero, expressed as: ;in, is the corrected segmentation result, is the segmentation result output by the image semantic segmentation model, is the position in the original mask Pixel value of For each defect area , count the number of pixels of each category in the defect area , select the category with the largest proportion The defect category of the defect area is expressed as: ,in, Indicates area Medium Category The number of pixels, It is the category with the largest number of categories.

9. The mask-based wafer defect classification method according to claim 1, wherein: The image semantic segmentation model performs inference and outputs wafer defect location and classification results, which also includes the following steps: For application scenarios that focus on efficiency requirements, the output results of the image semantic segmentation model are directly used as the final results of defect localization and classification.

10. A mask-based wafer defect classification system for implementing the classification method according to any one of claims 1 to 9, characterized in that: It includes image stitching module, region annotation module, image conversion module, model training module, defect reasoning module and report generation module; Image stitching module: stitches the wafer image containing defects with its corresponding defect mask in the channel direction to form a four-channel input image; Region labeling module: labels each defect region in the defect mask by category, assigns the pixels in each defect region a corresponding category number, and forms a label image; Image conversion module: converts the four-channel input image into a three-channel feature map by adding a 1x1 convolutional layer; Model training module: inputs the label image and three-channel feature map into the image semantic segmentation model for training; Defect reasoning module: When a defective wafer image is input into the trained image semantic segmentation model, the image semantic segmentation model performs inference and outputs the wafer defect location and classification results; Report generation module: Based on the defect location and classification results, the module performs defect assessment on the wafer and generates a defect report.

Images