Wafer defect segmentation method and system based on adaptive instance-aware loss

Abstract

The application discloses a wafer defect segmentation method and system based on adaptive instance-aware loss, relates to the technical field of wafer defect recognition, and calculates cross-entropy loss by taking the UNet segmentation model output and corresponding labels as input, obtains a cross-entropy loss graph, obtains the annotation instance contour in each label by using an open-source computer vision library according to the label corresponding to each cross-entropy loss graph, retains the pixel loss of a pixel whose pixel loss is greater than or equal to a loss threshold, calculates the average value of all retained pixel losses, obtains the perception loss corresponding to the cross-entropy loss graph, takes the average of the perception losses of N cross-entropy loss graphs, and completes the construction of an adaptive instance-aware loss function. The segmentation method introduces an instance-based perception loss mechanism, overcomes the problems that a traditional loss function cannot distinguish between difficult and easy samples, pays insufficient attention to defect edges, and has low segmentation precision, and improves the precision and adaptability of wafer defect segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wafer defect recognition, and in particular to a wafer defect segmentation method and system based on adaptive instance-aware loss. BACKGROUND

[0002] In the wafer manufacturing process, defect detection is a key link to ensure product quality. Traditional wafer defect detection methods mainly rely on manual inspection or threshold-based image processing technology. Manual inspection is inefficient and easily affected by subjective factors, and it is difficult to meet the growing demand for large-scale production. Although threshold-based image processing technology achieves a certain degree of automation, its performance is easily affected by factors such as light changes and noise interference, making it difficult to accurately detect small defects with low contrast.

[0003] In recent years, deep learning technology has made breakthroughs in image segmentation, providing a new approach to wafer defect detection. Semantic segmentation methods based on deep learning can accurately divide each pixel in an image into different categories (such as scratches, particles, and background), thereby achieving automatic detection and precise positioning of defects.

[0004] However, in actual wafer images, defects are usually small in size and few in number, with defect pixels accounting for a very small proportion of the entire image, while background pixels account for the vast majority. This serious class imbalance problem, when using traditional cross-entropy loss function for training, will cause the model to pay excessive attention to the background area, ignoring the learning of defects, ultimately leading to a high defect miss rate.

[0005] Therefore, the present application proposes a wafer defect segmentation method and system based on adaptive instance-aware loss, which introduces an instance-based perception loss mechanism to overcome the problems of traditional loss functions, such as inability to distinguish between difficult and easy samples, insufficient attention to defect edges, and low segmentation accuracy. The present application improves the precision and adaptability of wafer defect segmentation and reduces the defect miss rate. SUMMARY

[0006] The present application aims to provide a wafer defect segmentation method and system based on adaptive instance-aware loss to solve the problems in the background art.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solution: a wafer defect segmentation method based on adaptive instance-aware loss, the segmentation method comprising the following steps:

[0008] S1: Calculate the cross-entropy loss by outputting the UNet segmentation model and the corresponding label, and obtain the cross-entropy loss map;

[0009] S2: Using the open-source computer vision library, the contour of the labeled instance in each label is obtained according to the label corresponding to each cross-entropy loss map;

[0010] S3: Based on each labeled instance contour, the minimum bounding rectangle of each labeled instance is obtained, and the minimum bounding rectangle of each labeled instance is expanded by several pixels to the four sides to obtain an expanded bounding rectangle;

[0011] S4: On the cross-entropy loss map, the average loss in the expanded bounding rectangle of each labeled instance is calculated, and the maximum average loss is selected as the loss threshold;

[0012] S5: Retain the pixel loss greater than or equal to the loss threshold, and calculate the average of all retained pixel losses to obtain the perception loss corresponding to the cross-entropy loss map, and average the perception losses of the N cross-entropy loss maps to complete the construction of the adaptive instance perception loss function;

[0013] S6: The UNet segmentation model is trained using the adaptive instance perception loss function, and the wafer defects of the image to be processed are segmented by the trained UNet segmentation model.

[0014] In a preferred embodiment, step S5: retaining the pixel loss greater than or equal to the loss threshold, and calculating the average of all retained pixel losses to obtain the perception loss corresponding to the cross-entropy loss map, comprises the following steps:

[0015] For each training image, obtain its cross-entropy loss map and the corresponding loss threshold loss_thresh;

[0016] Iterate through each pixel in the loss map and extract the pixel loss value loss_pixel;

[0017] Determine whether the condition that the pixel loss value loss_pixel is greater than or equal to the loss threshold loss_thresh is met;

[0018] If the condition is met, the loss value of the pixel is included in the retained set;

[0019] The loss values of all retained pixels are accumulated and averaged to obtain the perception loss loss_aware of the current image;

[0020] If there are no retained pixels in the image, the average loss of all pixels in the entire image is used as the perception loss loss_aware.

[0021] In a preferred embodiment, step S5: averaging the perception losses of the N cross-entropy loss maps to complete the construction of the adaptive instance perception loss function, comprises the following steps:

[0022] An awareness loss loss aware of N images is obtained, and the N awareness losses loss aware are arithmetically averaged to obtain a training loss value loss total of the current batch;

[0023] The training loss value loss total is used as an adaptive instance-aware loss function for back propagation and parameter optimization of the model.

[0024] In a preferred embodiment, step S4: calculating the average loss in the extended bounding rectangle of each labeled instance on the cross-entropy loss map, and selecting the maximum average loss as the loss threshold, comprises the following steps:

[0025] For each defect instance, the extended bounding rectangle coordinates thereof are obtained, and a corresponding local region is extracted in the cross-entropy loss map based on the bounding rectangle coordinates;

[0026] All pixel positions in the local region are traversed, and the cross-entropy loss values are read one by one and accumulated to form a total loss value sum_loss, and the total pixel count count is counted;

[0027] On the premise that the total pixel count count is not zero, the average loss value loss inst of the instance is calculated;

[0028] For all valid defect instances in the image, the corresponding average loss values loss inst are calculated respectively, and the maximum value thereof is selected as the loss threshold loss thresh of the current image;

[0029] If all instances have no valid pixels or are all skipped, the average cross-entropy loss of all pixels in the current image is used as the loss threshold loss thresh.

[0030] In a preferred embodiment, step S3: obtaining the minimum bounding rectangle of each labeled instance based on the contour of each labeled instance, comprises the following steps:

[0031] Based on the extracted defect instance contour, the minimum bounding rectangle of each instance is obtained, which is generated by an image processing library function according to the two-dimensional coordinate point set of the instance contour, and contains the left upper corner coordinates and size information of the rectangle.

[0032] In a preferred embodiment, step S3: extending the minimum bounding rectangle of each labeled instance to four sides by a number of pixels to obtain an extended bounding rectangle, comprises the following steps:

[0033] The minimum bounding rectangle is extended by a preset number of pixels padding in the up, down, left and right four directions to obtain the extended bounding rectangle;

[0034] determine whether the extended bounding rectangle exceeds the image boundary, and if the boundary exceeds, perform cropping to make the extended bounding rectangle fall within the image width W and height H;

[0035] if the size of the extended rectangle is illegal, including the height or width after cropping is negative, skip the instance;

[0036] output the extended bounding rectangle coordinates and the corresponding instance class number and contour number.

[0037] In a preferred embodiment, step S2: according to the label corresponding to each cross-entropy loss map, use the open source computer vision library to obtain the labeled instance contour in each label, including the following steps:

[0038] For each of the N training images, obtain the label map and cross-entropy loss map of the image;

[0039] Iterate through each class number in the label map except the background, for each class number, construct a binary mask image;

[0040] input the binary image to the contour extraction function of the open source computer vision library Open-CV, perform contour detection operation, obtain all independent contour instances with closed boundary in each class, each contour is represented by a set of two-dimensional boundary point coordinates;

[0041] bind each contour with its corresponding class number, and output as the instance level structure information of the current image.

[0042] In a preferred embodiment, the label map and the loss map are both two-dimensional images with a dimension of 1xHxW, where the value of each pixel is an integer code representing a defect class;

[0043] the pixel value of the binary mask image is equal to the position value of the current class number, which is assigned as 255, and the rest of the positions are assigned as 0.

[0044] In a preferred embodiment, step S1: output the cross-entropy loss calculated by the UNet segmentation model and the corresponding label to obtain the cross-entropy loss map, including the following steps:

[0045] process the input image by the UNet segmentation model to obtain a tensor with a dimension of NxCxHxW output by the UNet segmentation model, where N is the number of images in a batch, C is the number of defect classes, H is the image height, and W is the image width; and obtain the corresponding label, the label has a dimension of NxHxW, and each pixel in the label is an integer value representing the true class;

[0046] For each pixel point in each image sample, the predicted probability value corresponding to the class indicated by the label is extracted from the C-dimensional class probability vector output by the UNet segmentation model, the cross-entropy loss value of the position is calculated, and the loss value is recorded in the cross-entropy loss graph of the pixel point;

[0047] A cross-entropy loss graph with a dimension of N*1*H*W is constructed for subsequent difficult example region extraction and instance-aware loss function construction.

[0048] The application also provides a wafer defect segmentation system based on adaptive instance-aware loss, which includes a contour labeling module, a loss threshold calculation module, a loss function construction module, and a defect segmentation module.

[0049] The contour labeling module: the cross-entropy loss is calculated by the UNet segmentation model output and the corresponding label, the cross-entropy loss graph is obtained, and the labeled instance contour in each label is obtained using the open source computer vision library according to the label corresponding to each cross-entropy loss graph.

[0050] The loss threshold calculation module: based on each labeled instance contour, the minimum bounding rectangle of each labeled instance is obtained, and the minimum bounding rectangle of each labeled instance is expanded by several pixels to the four sides to obtain an expanded bounding rectangle. On the cross-entropy loss graph, the average loss in the expanded bounding rectangle of each labeled instance is calculated, and the maximum average loss is selected as the loss threshold.

[0051] The loss function construction module: the pixel loss greater than or equal to the loss threshold is retained, and the average value of all retained pixel losses is calculated to obtain the perception loss corresponding to the cross-entropy loss graph. The perception loss of N cross-entropy loss graphs is averaged to complete the construction of the adaptive instance-aware loss function.

[0052] The defect segmentation module: the UNet segmentation model is trained using the adaptive instance-aware loss function, and the wafer defects of the image to be processed are segmented by the trained UNet segmentation model.

[0053] In the above technical solution, the technical effects and advantages provided by the application are:

[0054] The application calculates cross-entropy loss by outputting the UNet segmentation model and corresponding labels, obtains a cross-entropy loss graph, obtains the contour of the labeled instance in each label according to the label corresponding to each cross-entropy loss graph using an open source computer vision library, retains the pixel loss of the pixel loss greater than or equal to the loss threshold, and calculates the average value of all retained pixel losses, obtains the perception loss corresponding to the cross-entropy loss graph, and averages the perception losses of the N cross-entropy loss graphs to complete the construction of the adaptive instance perception loss function. The segmentation method overcomes the problems of the traditional loss function, such as being unable to distinguish between difficult and easy samples, paying insufficient attention to defect edges, and low segmentation precision, and improves the precision and adaptability of wafer defect segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0056] Figure 1 The flowchart of the segmentation method of the present application.

[0057] Figure 2 The framework diagram of the segmentation system of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme of the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0059] Embodiment 1: Please refer to Figure 1 As shown in the figure, the present embodiment provides a wafer defect segmentation method based on adaptive instance perception loss, which includes the following steps:

[0060] S1: First, the cross-entropy loss is calculated by the UNet segmentation model output (dimension NxCxHxW) and the corresponding label (dimension NxHxW), at this time it is not averaged, we call it the cross-entropy loss map, its dimension is 1xHxW, where N is the number of batch images, C is the number of defect categories, H is the image height, and W is the image width, UNet is a convolutional neural network architecture commonly used for image segmentation, cross-entropy loss (Cross-Entropy-Loss) is a commonly used classification loss function to measure the difference between the predicted distribution and the true distribution, and the cross-entropy loss map (Cross-Entropy-Loss-Map) is a two-dimensional image formed by calculating the cross-entropy loss of each pixel point.

[0061] S2: For each cross-entropy loss map (dimension 1xHxW) in N images, according to its corresponding label (dimension 1xHxW), use the open source computer vision library Open-CV to obtain the contour of each annotated instance in each label, where Open-CV is an open source computer vision library for image processing and contour extraction, Label is the true value image in the segmentation task, which labels the class of each pixel, Annotated-Instance is each independent target or defect region in the label image, and Contour is the connected curve of the boundary of the object in the image, which is used to accurately locate the target region.

[0062] S3: Based on the contour of each annotated instance, the minimum bounding rectangle of each annotated instance is obtained, and the minimum bounding rectangle of each annotated instance is expanded by several pixels to the four sides to obtain the extended bounding rectangle, here it is defaulted to 5 pixels, where Minimum-Bounding-Rectangle is the smallest area rectangle that can just enclose a target, Expand is to increase several pixels around the original rectangle, and Extended-Bounding-Rectangle is the rectangle region obtained by expanding the minimum bounding rectangle outward.

[0063] S4: On the cross-entropy loss map, the average loss loss_inst of each annotated instance in the extended bounding rectangle is calculated, and the maximum average loss loss_inst is selected as the loss threshold loss_thresh, where the average loss (loss_inst) is the average value of the loss values of all pixels in a certain annotated instance region, and the loss threshold (loss_thresh) is the maximum average loss value in multiple instances, which is used as a threshold for selecting high-loss pixels.

[0064] S5: retain the pixel loss of the pixel loss greater than or equal to the loss threshold loss_thresh, and calculate the average value of all retained pixel losses to obtain the perception loss loss_aware corresponding to the cross-entropy loss map, which is equivalent to only focusing on the difficult example pixels with high loss in the instance, thereby forcing the network to pay more attention to the key areas of the edges and interiors of the defects, and the perception loss loss_aware of the N cross-entropy loss maps is averaged to complete the construction of the adaptive instance perception loss function, wherein the perception loss (loss_aware) refers to the average loss calculated only on the "difficult learning area" (high loss pixel), which is used to guide the model to pay more attention to the defect boundary or key area, and the difficult example pixel (Hard-Pixels / Difficult-Pixels) represents a pixel point with high difficulty in segmentation and high prediction error of the model.

[0065] S6: training the UNet segmentation model using the adaptive instance perception loss function, wherein the optimizer adopts the stochastic gradient descent SGD, the learning rate decay strategy adopts the polynomial function poly strategy, the initial learning rate is set to 0.001, and the wafer defects of the to-be-processed image are segmented by the UNet segmentation model completed by training, wherein the adaptive instance perception loss function (Adaptive-Instance-Aware-Loss-Function) is a loss function that dynamically focuses on the loss of difficult pixels in each instance, the stochastic gradient descent (SGD, Stochastic-Gradient-Descent) is a basic algorithm for optimizing neural network parameters, the polynomial function decay strategy (poly-learning-rate-schedule) is a method for reducing the learning rate in the form of a polynomial according to the training progress, the initial learning rate (initial-learning-rate) is the learning rate size set at the beginning of training, which generally gradually decreases during the training process, and the wafer defect (Wafer-Defects) is a defect area on the wafer surface in semiconductor manufacturing.

[0066] The present application obtains the cross-entropy loss map by outputting the UNet segmentation model and calculating the cross-entropy loss with the corresponding label, obtains the annotation instance contour in each label using the open source computer vision library according to the label corresponding to each cross-entropy loss map, retains the pixel loss of the pixel loss greater than or equal to the loss threshold, and calculates the average value of all retained pixel losses to obtain the perception loss corresponding to the cross-entropy loss map, averages the perception loss of the N cross-entropy loss maps to complete the construction of the adaptive instance perception loss function. The segmentation method overcomes the problems of traditional loss functions, such as inability to distinguish between difficult and easy samples, insufficient attention to defect edges, and low segmentation precision, and improves the precision and adaptability of wafer defect segmentation.

[0067] Embodiment 2: Please refer to Figure 2 As shown in the figure, the embodiment provides a wafer defect segmentation system based on adaptive instance-aware loss, which comprises a contour labeling module, a loss threshold calculation module, a loss function construction module and a defect segmentation module.

[0068] The contour labeling module: the cross-entropy loss is calculated by the UNet segmentation model output and the corresponding label, the cross-entropy loss graph is obtained, and according to the label corresponding to each cross-entropy loss graph, the open source computer vision library is used to obtain the labeled instance contour in each label. The labeled instance contour is sent to the loss threshold calculation module, and the cross-entropy loss graph is sent to the loss threshold calculation module and the loss function construction module.

[0069] The loss threshold calculation module: based on each labeled instance contour, the minimum bounding rectangle of each labeled instance is obtained, and each labeled instance minimum bounding rectangle is expanded by several pixels to the four sides to obtain the expanded bounding rectangle. On the cross-entropy loss graph, the average loss in each labeled instance expanded bounding rectangle is calculated, and the maximum average loss is selected as the loss threshold. The loss threshold is sent to the loss function construction module.

[0070] The loss function construction module: the pixel loss greater than or equal to the loss threshold is retained, and the average value of all retained pixel losses is calculated to obtain the perception loss corresponding to the cross-entropy loss graph. The perception losses of N cross-entropy loss graphs are averaged to complete the construction of the adaptive instance-aware loss function. The adaptive instance-aware loss function is sent to the defect segmentation module.

[0071] The defect segmentation module: the UNet segmentation model is trained using the adaptive instance-aware loss function, and the wafer defects of the image to be processed are segmented by the trained UNet segmentation model.

[0072] Embodiment 3: S1: first, the cross-entropy loss is calculated by the UNet segmentation model output (dimension NxCxHxW) and the corresponding label (dimension NxHxW). At this time, it is not averaged, and we call it cross-entropy loss graph. Its dimension is 1xHxW, where N is the number of batch images, C is the number of defect categories, H is the image height, and W is the image width.

[0073] The loss distribution graph of each training image at the pixel level is obtained, which provides basic data support for subsequent difficult example region identification and instance-aware loss construction. The specific process is as follows:

[0074] In the semantic segmentation task of wafer defect images, the input image is first processed by the UNet segmentation model to obtain the model output result, which has a dimension of NxCxHxW, where N represents the number of images in the batch (batchsize), C represents the number of classes predicted by the model, usually including a background class and several defect classes, and H and W represent the height and width of the image, respectively.

[0075] At the same time, the dimension of the label data is NxHxW, where the value of each pixel represents the real class to which the pixel belongs (integer encoding, such as 0 for background and 1, 2, etc. for different defect types).

[0076] Next, the cross-entropy loss is calculated for each pixel point of each image. Instead of averaging the loss value, the independent loss value of each pixel is retained to generate a loss map (Cross-Entropy-Loss-Map) that represents the prediction error of the model at each pixel. The loss map has a dimension of NxlxHxW, i.e. each image corresponds to a single-channel loss map. The processing logic is as follows:

[0077] For each pixel (h, w) in each image sample i∈[0, N-1]:

[0078] The model prediction output is a probability distribution vector p_i, h, w, which has a length of C and represents the prediction probability of the pixel in each class (usually obtained through Softmax activation).

[0079] The real label index provided by the label is t_i, h, w, which takes an integer value between 0 and C-1, representing the real classification of the pixel.

[0080] The cross-entropy loss calculation logic for this pixel is:

[0081] The prediction probability value p_correct corresponding to the real label index t_i, h, w is taken from the probability distribution vector p_i, h, w, and -log(p_correct) is calculated as the cross-entropy loss value of the pixel.

[0082] For batch input images, the Cross-Entropy-Loss-Map is calculated based on the UNet output and the real label, which retains the high spatial resolution error information and provides a basis for subsequent difficult example region identification and perception loss function construction. For each image in the batch , the following steps are performed:

[0083] Model output tensor: where, is the number of batch images; is the total number of defect classes (including background). For image space size, The image The pixel point Belongs to the category The prediction probability (normalized by Softmax).

[0084] Label tensor: , where: Indicates the true class label of the pixel.

[0085] For each image And each pixel position , extract the true label , the corresponding prediction probability is The cross-entropy loss of this position is defined as:

[0086] Note: Ensure If the numerical value is close to 0, set the lower limit (such as To avoid numerical instability. Fill all pixel loss values into a single-channel tensor , namely the cross-entropy loss map: The loss map retains the spatial structure information, and has stronger pixel distribution perception ability compared with the traditional average loss.

[0087] Suppose an image input dimension is , the number of categories (0 is background, 1 and 2 are two types of defects), the model output probability example (pixel position (1, 1)): (Corresponding categories 0, 1, 2), the label value is: Then the cross-entropy loss of this position is: The corresponding pixel point of the final cross-entropy loss map : , for all Positions are traversed in turn, and the complete cross-entropy loss map .

[0088] By constructing a pixel-by-pixel cross-entropy loss map, the spatial distribution information of the model prediction error is retained. The loss map has the same size as the label map, and each pixel value represents the prediction error intensity of the position, which is suitable for subsequent difficult example pixel recognition and instance-level loss analysis. Compared with the traditional average loss function, the cross-entropy loss map can provide more fine-grained supervision signals, effectively improving the model's ability to recognize complex edge structures and fine-grained defects.

[0089] This way of calculating pixel-by-pixel loss, unlike the strategy of averaging the entire image or batch in regular classification tasks, preserves the high spatial resolution error information, which is crucial for subsequent difficult example pixel screening, area analysis and perception loss modeling.

[0090] Suppose there is an input image (N = 1) with a size of 256 x 256, containing three types of labels (C = 3, background, defect 1, and defect 2), and the model's output result is 1 x 3 x 256 x 256, and the label is 1 x 256 x 256. At coordinate (120, 200), the label value is 2 (defect 2), and the model's output Softmax result at this position is [0.1, 0.2, 0.7], then the cross-entropy loss of this pixel is -log(0.7) ≈ 0.357. This value will be saved in the corresponding position of the loss map.

[0091] By traversing all the pixels of the entire image, the cross-entropy loss map of this sample is completed. Finally, a single-channel image with the same size (1 x 256 x 256) is obtained to represent the prediction error intensity distribution of the model at each pixel.

[0092] This step focuses on preserving pixel-by-pixel loss without batch averaging or image-level averaging. The generated cross-entropy loss map serves as the basis for difficult example region mining, with good positioning and resolution. The cross-entropy loss function used should support the 'none' option to ensure that the loss value is output by pixel. With label information, subsequent instance-level region division and local error statistics can be achieved, providing fine-grained data support for instance perception.

[0093] S2: For each cross-entropy loss map (dimension 1 x H x W) in N images, according to its corresponding label (dimension 1 x H x W), use the open-source computer vision library Open-CV to obtain the contour of each labeled instance in each label.

[0094] After completing the construction of the cross-entropy loss map for each image, this step aims to identify the contour regions of all labeled instances with clear semantics based on the label map corresponding to the image. These contour information will be used for subsequent loss statistics and perception modeling of instance-level regions.

[0095] The label map has the same dimension as the loss map (1 x H x W), where each pixel's value is its real class number (usually an integer), such as 0 for background and 1, 2, 3 for different defect types. Since the label map is a semantic segmentation label map, there may be multiple discontinuous regions (i.e., multiple instances) in the same class, so further image processing is needed to distinguish different instances in the same class.

[0096] To this end, the contour detection function (such as find-Contours function) provided by the open source image processing library Open-CV is used to extract the contours of the label image by class and instance. The specific processing logic is as follows:

[0097] Iterate through each image in the batch, a total of N images, and process the corresponding label image (dimension 1xHxW) of each image.

[0098] Extract all class numbers (excluding background class number 0) in the current label image, for example, class 1 and class 2 appear in the label.

[0099] For each class number c, convert the original label image to a binary mask image:

[0100] Assign a value of 255 to all pixels with a label value equal to c, and assign a value of 0 to the remaining positions.

[0101] Get a binary image mask_c that only contains pixels of this class, and the size is still HxW.

[0102] Use Open-CV's find-Contours function to perform contour detection on the binary image mask_c:

[0103] This function takes a binary image as input and returns the edge curves (i.e. contours) of all connected regions in the image;

[0104] Each contour represents a set of boundary points of a continuous region, represented by a polygon approximation;

[0105] These contour sets are used to identify each independent instance in the class.

[0106] For the label image of each training image, extract all closed annotation instance contours under each class to provide a spatial structure basis for subsequent instance-based loss focusing. For each image , perform the following steps:

[0107] Input label image: , where , represents the class number of each pixel; input cross-entropy loss image: , which is used in subsequent steps and does not participate in contour extraction.

[0108] For each non-background class number that appears in the label image, construct the binary mask image corresponding to this class , defined as follows:

[0109] , the binary image The contour extraction function (such as find-Contours) in the open-source image processing library Open-CV is input to perform the contour detection operation. The mathematical function of the extraction function is: wherein,

[0110] Each contour is a set of boundary points:

[0111] Each contour is bound to its corresponding class number and recorded as a set of structured instance information: .

[0112] Let the training image be , the image size is , and the label image is as follows:

[0113] The class set is , and the binary mask image is constructed:

[0114] The contour extraction is performed: the contour extraction function outputs a contour set , which contains a closed contour: , and the final output instance structure is: .

[0115] In this embodiment, for the label image of each training image, all non-background class numbers are traversed, the specified class position is set to 255, and the rest is set to 0 by constructing a binary mask image. Then, the contour extraction function in the open-source image processing library Open-CV is used to perform contour detection on the binary image to obtain all closed boundary independent labeled instances in each class. Each contour is represented by a set of boundary point coordinates and is bound to the class number, and the output is the instance-level structure information, which is used for subsequent bounding rectangle construction and local loss statistics processing.

[0116] Contour filtering (optional): for each extracted contour, filtering can be performed according to the area, shape complexity, or number of boundary points: for example, contours with an area smaller than 5 pixels are removed to exclude possible noise; contours with broken or non-closed boundaries are reconstructed or filled to ensure the stability of subsequent bounding rectangle processing.

[0117] Finally, each image can obtain a multi-level structure: class number→corresponding multiple contours (instances), each contour is composed of a set of two-dimensional boundary points (x, y), and these contour data will be uniformly transmitted to the loss threshold calculation module for subsequent processing.

[0118] Suppose that the label map of an image contains class 1 (defect A) and class 2 (defect B): after masking class 1, three independent regions (corresponding to three different defect instances) are found; after extraction using find-Contours, three contours are obtained, each of which consists of multiple boundary points; similarly, class 2 is processed to obtain two instance contours; all five contours will be used as the instance-level structure information of the current image to participate in the loss calculation in the next stage.

[0119] When using find-Contours of Open-CV, a suitable mode parameter (such as RETR_EXTERNAL or RETR_TREE) should be set to ensure that all required contours are extracted; if there are connected targets or blurred boundaries in the label map, morphological operations (such as erosion, dilation, connected component labeling) can be used for preprocessing; ensure that the pixel value of the label map is an integer class code, and cannot be a floating-point type or a non-discrete value, otherwise the contour extraction will fail; all contour data should be attached with its class number when output, so that the category of each instance can be identified in the subsequent process.

[0120] S3: Based on the contour of each labeled instance, the minimum bounding rectangle of each labeled instance is obtained, and each labeled instance is expanded by several pixels in four directions to obtain an expanded bounding rectangle. Here, the default is 5 pixels.

[0121] After the extraction of the labeled instance contour is completed, the core task of this step is to obtain the minimum bounding rectangle (Minimum-Bounding-Rectangle) based on the contour of each instance, and to expand the rectangle in the up, down, left and right directions by several pixels to construct a more tolerant local analysis window that covers the potential fuzzy boundary area, providing spatial support for subsequent high-loss pixel focusing and local perception loss calculation.

[0122] The purpose is to enhance the sensitivity to the edge region of the defect. Actual wafer defects often have problems such as fuzzy boundary and irregular distribution of predicted fuzzy interval. By expanding the bounding rectangle, the area outside the contour but related to the defect semantics can be covered, thereby avoiding the risk of ignoring the high-error area based on the strict contour. The processing logic is as follows: minimum bounding rectangle acquisition: for each instance contour (i.e. a set of boundary points) extracted in step S2, use an image processing library (such as the boundingRect function in Open-CV) to obtain its minimum bounding rectangle. The function logic is as follows:

[0123] Input: a set of contour boundary points, usually in the format of a list of two-dimensional coordinate points, such as [(x1, y1), (x2, y2),...];

[0124] Output: the top-left corner coordinate (x_min, y_min) and the rectangle size (width, height);

[0125] The resulting rectangle is the minimum rectangle that covers the whole contour region, and the rectangle is axis-aligned (non-rotated rectangle).

[0126] After obtaining the original bounding rectangle (x_min, y_min, width, height), the following direction expansion operations are performed:

[0127] Left side: expand padding pixels to the left: x_min_new = x_min - padding;

[0128] Top side: expand padding pixels upwards: y_min_new = y_min - padding;

[0129] Right side: expand to the right: width_new = width + 2 x padding;

[0130] Height: similarly expand: height_new = height + 2 x padding;

[0131] The default padding = 5 pixels (can be set as a hyperparameter or dynamically adjusted), forming a new expanded bounding rectangle region.

[0132] Since the expansion operation may cause the bounding rectangle to exceed the image boundary (for example, the top-left corner is negative, or the bottom-right corner exceeds the image width and height), the validity of the expanded coordinates must be judged:

[0133] x_min_new and y_min_new must be greater than or equal to 0;

[0134] x_max_new = x_min_new + width_new cannot exceed the image width W;

[0135] y_max_new = y_min_new + height_new cannot exceed the image height H;

[0136] If it exceeds the boundary, it is automatically cropped to the image boundary range.

[0137] For each expanded bounding rectangle, the corresponding instance class number, original contour number, and final expanded region coordinates (x_min, y_min, x_max, y_max) should be retained; this structure data will be used in the next step to extract the corresponding local region from the cross-entropy loss map for loss analysis.

[0138] Suppose there is a defect instance contour, the original rectangle returned by boundingRect is (x = 100, y = 150, width = 20, height = 10), and the padding is set to 5, then after expansion:

[0139] x_min_new = 95, y_min_new = 145;

[0140] width_new = 30, height_new = 20;

[0141] x_max_new = 125, y_max_new = 165;

[0142] If the image size is 128x128, the coordinates x_max_new = 125 and y_max_new = 165 are out of range, and y_max_new needs to be truncated to 128, and height_new is updated to 128-y_min_new = 128-145 = -17. Here it is indicated that the expansion should be judged as illegal (because the expansion area exceeds the image size resulting in negative height), and a threshold should be set to skip this instance or limit the expansion range within the image.

[0143] Specifically, mainstream image libraries such as Open-CV provide stable rectangle detection and expansion support, which is beneficial to actual engineering deployment; the number of expanded pixels can be dynamically adjusted according to the defect scale features, prediction fuzziness, etc. in the data set; this step is particularly important for improving instance perception ability, as it introduces a spatial context tolerance band, avoiding the overfitting effect of model training caused by contour edge truncation; the constructed expanded bounding rectangle area will be used as a subsequent regional loss analysis window, forming the spatial basis for instance-level loss perception.

[0144] S4: On the cross-entropy loss map, calculate the average loss loss_inst within the expanded bounding rectangle of each labeled instance, and select the maximum average loss loss_inst as the loss threshold loss_thresh.

[0145] After extracting the expanded bounding rectangle of each instance, the goal of this step is to perform local statistics on the cross-entropy loss map to obtain the average loss value loss_inst of each instance region, and select the maximum average loss value as the unified loss threshold loss_thresh. This threshold will be used as a standard for subsequent high-loss pixel screening, focusing the model's learning ability on difficult example regions. By introducing instance-level average loss metrics, the model can avoid excessive attention to low-value regions of the overall loss, effectively identifying key regions with large prediction errors. The processing logic is as follows:

[0146] For each annotated instance's extended bounding box (x_min, y_min, x_max, y_max), extract the corresponding rectangular region's pixel loss values from the cross-entropy loss map (dimension 1xHxW). This local loss region contains all the pixel loss values of the instance's contour and its surrounding extended region. Make sure the boundary clipping is done during the cropping process (see the boundary clipping logic in S3) to avoid invalid pixels or out-of-bound access.

[0147] Sum up all the pixel loss values in the cropped region one by one, then divide by the number of pixels in the region to get the average loss value loss_inst for the instance. Logical description: initialize the cumulative value sum_loss = 0 and the counter count = 0; for each pixel (h, w) in the rectangular region, perform: read the corresponding cross-entropy loss value loss_pixel; perform sum_loss += loss_pixel and count += 1; after the loop, calculate loss_inst = sum_loss / count; By this calculation method, the overall error level of each instance in its local region is obtained.

[0148] In an image, there may be multiple defect instances, each instance corresponding to a loss_inst. Select the maximum value max(loss_inst) among all instance average loss values as the loss threshold loss_thresh for the image. This strategy ensures that the most difficult instance region can be focused on during subsequent pixel screening, thereby improving the network's ability to distinguish difficult samples.

[0149] If an instance region has zero pixels due to invalid contours or too small size, skip the instance and record it in the log. If all instances in an image cannot calculate valid loss_inst (e.g., all are noise or empty instances), you can degrade to use the global loss average as loss_thresh to ensure the normal operation of the subsequent process.

[0150] Suppose there are three instances in an image, whose extended bounding boxes cover 200, 350, and 150 pixels respectively. Through accumulation and average calculation, we get: the average loss of instance 1 is 0.12; the average loss of instance 2 is 0.18; the average loss of instance 3 is 0.09; then loss_thresh = 0.18, and only the pixels with loss value greater than or equal to 0.18 will be retained for perceptual loss construction in the future.

[0151] The average loss value of the example region measures the difficulty of the model's prediction in the example region, which can avoid simple pixels (low loss value) dominating the training, so that the model focuses more on complex boundaries, ambiguous regions and small defects. Using the maximum loss_inst as the threshold can ensure that the model always focuses on the most challenging example region, forming a more targeted training signal. This step is one of the core parts of the adaptive instance-aware loss, which directly affects the subsequent high-loss pixel screening and the calculation accuracy of loss_aware.

[0152] S5: Retain the pixel loss of the pixel whose pixel loss is greater than or equal to the loss threshold loss_thresh, and calculate the average value of all retained pixel losses to obtain the perception loss loss_aware corresponding to the cross-entropy loss map. This is equivalent to only focusing on difficult example pixels with high loss in the instance, thereby forcing the network to pay more attention to the key areas of the edges and interiors of the defects. The perception loss loss_aware of the N cross-entropy loss maps is averaged to complete the construction of the adaptive instance-aware loss function.

[0153] After obtaining the loss threshold loss_thresh of each image, the goal of this step is to further screen and focus on the cross-entropy loss map, only retaining "difficult example pixels" (i.e. pixels with the largest prediction error) whose loss value is higher than or equal to the threshold, and calculating the perception loss loss_aware of each image on this basis. Through this pixel-level screening mechanism, the model's learning of difficult-to-identify areas can be effectively enhanced, and dependence on "easy-to-predict areas" can be reduced, thereby improving the model's perception ability of "key pixels" such as defect boundaries, small structures, and inter-class confusion areas.

[0154] By normalizing and averaging the perception loss of all training images, the adaptive instance-aware loss function is finally formed for the backward propagation and parameter update of the model. The processing logic is as follows:

[0155] The input is the cross-entropy loss map of the current image (dimension 1xHxW) and the instance loss threshold loss_thresh; traverse each pixel position (h, w) of the loss map, extract its loss value loss_pixel; judge whether the condition loss_pixel>=loss_thresh is met, if met, the pixel is included in the retained set. At the same time, in order to achieve high efficiency, this process can usually be completed at the matrix level at one time by means of Boolean masking operation (masking).

[0156] Suppose the remaining high-loss pixels are a total of M, the loss values of these pixels are accumulated and averaged: initialize the variable sum_loss_aware = 0, count_aware = 0; for all pixels that meet the conditions: sum_loss_aware += loss_pixel; count_aware += 1;

[0157] Finally, calculate loss_aware = sum_loss_aware / count_aware;

[0158] If the remaining pixels are zero (extreme case, such as the model predicts too well or the data is abnormal), a default fallback mechanism should be set, such as using the average loss of the whole image as loss_aware, or skipping this sample from participating in the average.

[0159] For each image in a batch (a total of N images), repeat the above operation to obtain N loss_aware values respectively; calculate the arithmetic mean of the N perception loss values as the overall loss function value of the current batch: initialize the variable sum_batch_loss = 0; for each image's loss_aware_i: sum_batch_loss += loss_aware_i; calculate loss_total = sum_batch_loss / N; finally, the loss_total value is the adaptive instance perception loss function value of the model in the current batch, used for back propagation and optimizer gradient update.

[0160] Suppose a cross-entropy loss map of an image contains 256x256 = 65536 pixels, after filtering by threshold loss_thresh = 0.18, 900 pixels are retained, and the average loss value of these pixels is 0.26, then the loss_aware of the image is 0.26. If a batch contains 4 images, their perception losses are 0.26, 0.21, 0.30, and 0.18 respectively, then the overall adaptive instance perception loss function value is (0.26 + 0.21 + 0.30 + 0.18) / 4 = 0.2375.

[0161] This step constructs a loss function highly related to the actual prediction difficulty, which can dynamically focus on training according to instance complexity; the high-loss pixel screening mechanism makes the model focus on the most challenging areas in each training, improving the edge learning ability; through "per-image adaptive threshold + whole-image average strategy", local perception and global stability are combined, which helps to avoid training shock or overfitting; compared with the traditional whole-image average loss, this strategy significantly enhances the robustness and generalization ability of the model in the wafer defect segmentation task.

[0162] S6: The UNet segmentation model is trained using the adaptive instance-aware loss function, wherein the optimizer is a stochastic gradient descent (SGD), the learning rate decay strategy is a polynomial function (poly) strategy, the initial learning rate is set to 0.001, and the wafer defects of the to-be-processed image are segmented by the trained UNet segmentation model.

[0163] After the adaptive instance-aware loss function is constructed, this step aims to iteratively optimize and train the UNet segmentation model using the loss function, and finally obtain a wafer image segmentation model with stronger defect awareness and boundary recognition capabilities. In order to ensure the convergence and generalization performance of the model during training, this step adopts standard optimization strategies, including a parameter update mechanism based on stochastic gradient descent (SGD) and a learning rate dynamic adjustment strategy based on a polynomial function decay (poly).

[0164] After training, the obtained model will be used for automatic defect segmentation tasks of actual wafer images, forming an integrated closed-loop process from training to inference. The processing logic is as follows:

[0165] The segmentation model used is a UNet structure, which has typical features such as encoding-decoding symmetric architecture, cross-layer skip connection, and strong local texture and edge structure preservation ability; the adaptive instance-aware loss function constructed in the previous stage is used as the main optimization objective function during training; in each training batch, the average perception loss (i.e. loss_total) within the batch is calculated as the loss value of the current iteration.

[0166] Stochastic gradient descent (Stochastic-Gradient-Descent, SGD) is used as the model parameter update optimizer; the SGD optimizer adjusts the network weights based on the loss gradient information of the current mini-batch in each iteration to minimize the perception loss; to improve training stability and generalization effect, a momentum parameter (such as 0.9) can be selected to accelerate convergence and avoid falling into local optimum; in addition, a weight decay (such as 1e-4) can be set for regularization to suppress overfitting.

[0167] A polynomial decay strategy (poly) is used to dynamically adjust the learning rate during training to achieve more reasonable training progress control. The basic logic of the poly strategy is as follows: the initial learning rate is set to lr_init=0.001; it gradually decreases with the training round until it tends to 0 at the end of training; the current learning rate is calculated every epoch or every iteration: let lr_current=lr_init×(1-iter / max_iter)power, where power is usually set to 0.9 (adjustable), iter represents the current iteration number, and max_iter represents the maximum total number of iterations; this strategy has the characteristics of fast learning in the early stage and fine tuning in the later stage, and is suitable for the demand for detail accuracy in segmentation tasks.

[0168] According to the conventional deep learning process, data loading, forward propagation, loss calculation, back propagation and parameter updating are performed in turn; model verification and evaluation are performed after each epoch or a certain number of steps, and indicators such as IoU, Dice, Precision, etc. are calculated; the best model parameter weight during training is saved (which can be saved according to the optimal verification loss or indicator);

[0169] After training, the saved best model parameter weight is used to load the UNet network; the wafer image to be detected is input, and the class prediction result of each pixel is obtained through the forward inference process; the output is a segmentation map indicating the position and shape of each defect region on the wafer surface; further superimposed on the original image to form a visual overlay for defect detection and judgment in the production process.

[0170] Suppose the training is performed for a total of 10,000 steps, the initial learning rate is 0.001, and the current is at step 5000, using the poly decay strategy with power=0.9, then the current learning rate will be reduced to about half of the original value, and the training process will gradually converge. At the same time, the loss value optimized by the model at each step is not the average pixel loss, but the loss_aware after instance screening and difficult example focusing, thereby strengthening the model's ability to recognize defect boundaries.

[0171] The adaptive instance-aware loss combines the instance region and difficult example pixel double focusing strategy, which is a significant improvement over the traditional pixel average loss; the SGD optimizer has good convergence performance and generalization ability, and is suitable for training of medium-sized semantic segmentation models; the poly learning rate decay takes into account the learning speed and accuracy tuning, avoiding too fast jumping in the early stage of training and falling into oscillation in the later stage; the trained UNet model not only has high segmentation accuracy, but also has strong adaptability to complex edge structures, and has high practical value for wafer defect detection.

[0172] Embodiment 4: In this example, UNet network is used as the segmentation model, and the adaptive instance-aware loss function is used for model parameter updating. The specific detection steps are as follows:

[0173] Step one: data preparation, first, the collected wafer defect images are pixel-level labeled, different values are used for different categories, and finally the label image corresponding to the defect image is obtained, wherein the background pixel value is 0, and the pixel value corresponding to each defect starts from 1 and increases continuously;

[0174] Step two: data preprocessing, in order to improve the generalization ability of the model, a variety of data enhancement techniques can be used to preprocess the wafer defect image, including image scaling, horizontal flipping, random cropping and normalization;

[0175] Step four: model construction, ResNet18 network model is used as the feature extraction backbone network of UNet segmentation model, and ImageNet pre-training weight is used for parameter initialization, and the rest of the parameters are randomly initialized;

[0176] Step five: model training, the adaptive instance-aware loss function is used to train the UNet segmentation model, wherein the optimizer adopts the stochastic gradient descent SGD, the learning rate decay strategy adopts the polynomial function poly strategy, and the initial learning rate is set to 0.001;

[0177] Step six: model inference and evaluation, the trained UNet segmentation model is used to infer the wafer defect image, and the mIoU segmentation precision is used as the measurement index to evaluate the consistency of the segmentation result and the true label of the label image. The actual test shows that compared with the cross-entropy loss function, the loss function of the present application can improve the mIoU segmentation precision by more than 5%.

[0178] The hardware platform used in the system implementation of this embodiment is a computer equipped with a GTx3080 graphics card, and the software platform uses the pytorch deep learning framework. When deploying, tensor-RT is used for quantization acceleration.

[0179] Specifically, the above only describes the preferred embodiments of the present application and is not intended to limit the present application.

[0180] In the description of the present specification, the description of the terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0181] The preferred embodiments of the application disclosed above are only to help explain the present application. The preferred embodiments are not intended to be exhaustive or to limit the application to the specific form disclosed. Many modifications and variations are possible in light of this teaching. It is intended that the specification and examples be considered as exemplary only, with the fact that over time, the preferred embodiments can be modified in many ways within the scope and spirit of the application. The application is defined by the claims and their full scope of equivalents.

Claims

1. A wafer defect segmentation method based on adaptive instance-aware loss, characterized in that: The segmentation method includes the following steps: S1: Calculate the cross-entropy loss using the UNet segmentation model output and corresponding labels, and obtain the cross-entropy loss map; S2: Based on the label corresponding to each cross-entropy loss map, use an open-source computer vision library to obtain the outline of the labeled instance in each label; S3: Based on the outline of each annotation instance, obtain the minimum bounding rectangle of each annotation instance, and expand the minimum bounding rectangle of each annotation instance to the four sides by a certain number of pixels to obtain the expanded bounding rectangle. S4: On the cross-entropy loss map, calculate the average loss within the bounding rectangle of each labeled instance after expansion, and select the maximum average loss as the loss threshold. S5: Retain the pixel loss that is greater than or equal to the loss threshold, calculate the average value of all retained pixel losses, obtain the perceptual loss corresponding to the cross-entropy loss map, calculate the average value of the perceptual loss of N cross-entropy loss maps, and complete the construction of the adaptive instance perceptual loss function. S6: The UNet segmentation model is trained using an adaptive instance-aware loss function, and the trained UNet segmentation model is used to segment wafer defects in the image to be processed.

2. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 1, characterized in that: Step S5: Retain the pixel losses that are greater than or equal to the loss threshold, and calculate the average of all retained pixel losses to obtain the perceptual loss corresponding to the cross-entropy loss map, including the following steps: For each training image, obtain its cross-entropy loss map and the corresponding loss threshold loss_thresh; Iterate through each pixel in the loss graph and extract the pixel loss value loss_pixel; Determine whether the condition that the pixel loss value loss_pixel is greater than or equal to the loss threshold loss_thresh is met; If the condition is met, the loss value of that pixel is included in the retention set; The loss values ​​of all retained pixels are summed and averaged to obtain the perceptual loss loss_aware of the current image; If no pixels are retained in the image, the average loss of all pixels in the entire image is used as the perceptual loss loss_aware.

3. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 2, characterized in that: Step S5: Calculate the mean of the perceptual loss for the N cross-entropy loss maps to complete the construction of the adaptive instance perceptual loss function, including the following steps: Obtain the perceptual loss_aware for N images, and perform an arithmetic average of the N perceptual losses_aware to obtain the training loss value loss_total for the current batch. The training loss value loss_total is used as an adaptive instance-aware loss function for backpropagation and parameter optimization of the model.

4. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 3, characterized in that: Step S4: On the cross-entropy loss map, calculate the average loss within the expanded bounding rectangle of each labeled instance, and select the maximum average loss as the loss threshold. This includes the following steps: For each defect instance, obtain its expanded bounding rectangle coordinates, and extract the corresponding local region in the cross-entropy loss map based on the bounding rectangle coordinates; Traverse all pixel positions within the local region, read the cross-entropy loss value one by one and sum them to form the total loss value sum_loss, and count the total number of pixels count. Calculate the average loss value loss_inst for this instance, assuming the total number of pixels count is not zero; For all valid defect instances in the image, calculate the corresponding average loss value loss_inst, and select the maximum value as the loss threshold loss_thresh for the current image; If all instances have no valid pixels or are all skipped, the loss degenerates to using the average cross-entropy loss of all pixels in the current image as the loss threshold loss_thresh.

5. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 4, characterized in that: Step S3: Based on the outline of each annotation instance, obtain the minimum bounding rectangle of each annotation instance, including the following steps: Based on the extracted defect instance contours, the minimum bounding rectangle of each instance is obtained. The minimum bounding rectangle is generated by an image processing library function based on the set of two-dimensional coordinate points of the instance contour, including the coordinates of the upper left corner of the rectangle and its size information.

6. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 5, characterized in that: Step S3: Expand the minimum bounding rectangle of each labeled instance by a certain number of pixels on all four sides to obtain the expanded bounding rectangle, including the following steps: The minimum bounding rectangle is expanded by a preset number of pixels in the four directions (up, down, left, and right) to obtain the expanded bounding rectangle. Determine whether the expanded bounding rectangle exceeds the image boundary. If the boundary exceeds the boundary, crop the bounding rectangle so that it falls within the image width W and height H. If the expanded rectangle's dimensions are unacceptable, including if the height or width after cropping is negative, then skip that instance. Output the coordinates of the expanded bounding rectangle and its corresponding instance class number and outline number.

7. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 1, characterized in that: Step S2: Based on the label corresponding to each cross-entropy loss map, use an open-source computer vision library to obtain the outline of the labeled instance in each label, including the following steps: For each of the N training images, obtain the label map and cross-entropy loss map for that image; Iterate through each category number in the label image except for the background, and construct a binary mask image for each category number; The binary image is input into the contour extraction function of the open-source computer vision library Open-CV to perform contour detection operations and obtain all independent contour instances with closed boundaries in each category. Each contour is represented by a set of two-dimensional boundary point coordinates. Each contour is bound to its corresponding category number and output as instance-level structural information of the current image.

8. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 7, characterized in that: Both the label map and the cross-entropy loss map are two-dimensional images with dimensions of 1×H×W, where the value of each pixel is an integer code representing the defect category; The pixel value of the binary mask image is assigned a value of 255 at the position where the current category number is equal, and 0 at the other positions.

9. The wafer defect segmentation method based on adaptive instance-aware loss according to claim 1, characterized in that: Step S1: Calculate the cross-entropy loss using the UNet segmentation model output and corresponding labels, and obtain the cross-entropy loss map, including the following steps: The input image is processed using the UNet segmentation model to obtain a tensor with output dimensions of N×C×H×W, where N is the number of images in a batch, C is the number of defect categories, H is the image height, and W is the image width; and the corresponding labels are obtained, with the label dimensions being N×H×W, and each pixel in the label being an integer value representing the true category. For each pixel in each image sample, the predicted probability value corresponding to the category indicated by the label is extracted from the C1-dimensional category probability vector output by the UNet segmentation model. The cross-entropy loss value at that location is calculated and recorded in the cross-entropy loss map of that pixel. A cross-entropy loss map with dimensions N×1×H×W is constructed for subsequent hard example region extraction and instance-aware loss function construction.

10. A wafer defect segmentation system based on adaptive instance-aware loss, used to implement the segmentation method according to any one of claims 1-9, characterized in that: It includes a contour annotation module, a loss threshold calculation module, a loss function construction module, and a defect segmentation module; Contour annotation module: Calculates the cross-entropy loss by matching the output of the UNet segmentation model with the corresponding label, obtains the cross-entropy loss map, and uses an open-source computer vision library to obtain the outline of the annotated instance in each label based on the label corresponding to each cross-entropy loss map. Loss threshold calculation module: Based on the outline of each labeled instance, obtain the minimum bounding rectangle of each labeled instance, and expand the minimum bounding rectangle of each labeled instance to four sides by a certain number of pixels to obtain the expanded bounding rectangle. On the cross-entropy loss map, calculate the average loss within the expanded bounding rectangle of each labeled instance, and select the maximum average loss as the loss threshold. Loss function construction module: retain the pixel loss that is greater than or equal to the loss threshold, calculate the average value of all retained pixel losses, obtain the perceptual loss corresponding to the cross-entropy loss map, calculate the average value of the perceptual loss of N cross-entropy loss maps, and complete the construction of the adaptive instance perceptual loss function. Defect segmentation module: The UNet segmentation model is trained using an adaptive instance-aware loss function, and the trained UNet segmentation model is used to segment wafer defects in the image to be processed.

Images