ATE wafer graph defect identification method and system based on Transform network

Abstract

The invention provides an ATE wafer graph defect identification method and system based on a Transform network, the ATE wafer graph defect identification method based on the Transform network integrates data preprocessing, hybrid network modeling and end-to-end training, the system can make full use of local and global features of a wafer graph, and the defect identification accuracy of the wafer graph is improved. The synchronous and accurate execution of macroscopic defect classification and microdefect positioning is realized, the automation degree and the result reliability of defect detection are improved, and a comprehensive and efficient analysis tool is provided for semiconductor process optimization.

Description

Technical Field

[0001] This invention relates to the field of chip identification technology, and in particular to a method and system for identifying defects in ATE wafer images based on Transformer networks. Background Technology

[0002] With the rapid development of my country's semiconductor industry, chip manufacturing processes are becoming increasingly complex, placing higher demands on product yield control. To identify and eliminate defective die units as early as possible before chip packaging, avoiding the need to invest in expensive subsequent packaging and testing stages, the industry commonly employs a technical solution called wafer probe testing (CP testing). This solution, after wafer manufacturing is complete, uses automated test equipment (ATE) to test the electrical performance and functionality of each individual die on the wafer, thereby screening out defective products before packaging and providing crucial data for monitoring and optimizing the front-end manufacturing process. Specifically, the ATE testing system, consisting of a tester, probe station, and probe cards, performs automated testing on each die of the wafer, recording the test results (multiple types of pass or fail) for each die. This results in a two-dimensional matrix wafer map, which visually displays the spatial distribution of defective dies on the wafer, providing a basis for subsequent defect analysis.

[0003] Currently, there are two main types of defect analysis methods based on wafer images. On the one hand, traditional defect analysis methods rely primarily on human experience or simple image processing rules, with engineers judging potential process problems based on the spatial distribution patterns of defects presented on the wafer image. On the other hand, although some research has attempted to apply convolutional neural networks (CNNs) for defect detection to achieve automated analysis, most of this research focuses on processing optical inspection (AOI) images of wafers. Optical images typically contain rich texture, color, and detail information, making them suitable for feature extraction by CNNs.

[0004] Currently, existing wafer map-based defect analysis methods, both domestically and internationally, can serve as supporting tools for determining whether anomalies exist in the front-end process. However, these methods have significant limitations. Traditional defect analysis mainly relies on human experience or simple image processing rules. However, due to the increasing complexity of semiconductor manufacturing processes, defect patterns are not only diverse but also frequently involve mixed-type defects. This subjective judgment-based approach is not only inefficient and inconsistent but also struggles to effectively extract valuable information from wafer maps containing massive amounts of data, and it is difficult to accurately classify and locate complex defects with uncertainty and diverse morphologies.

[0005] On the other hand, although some studies have attempted to apply convolutional neural networks (CNNs) for defect detection, these approaches also suffer from fundamental design flaws. Firstly, there is a mismatch between data format and model design. Existing research largely focuses on processing optical inspection (AOI) images of wafers, which contain rich texture and color information. However, the wafer images generated from ATE test data processed in this invention are essentially sparse, categorical two-dimensional logistic matrices, with completely different data formats and features. This means that CNN models designed for optical images cannot be directly and effectively applied to analyze ATE wafer images. More fundamentally, the inherent locality of the CNN architecture limits its performance. CNNs use small-sized convolutional kernels for sliding window operations, making them adept at capturing local features, but they struggle to effectively model the long-range dependencies crucial for defect pattern recognition. For defect patterns requiring a global field of view for accurate identification, such as "scratches" traversing the wafer or "ring-shaped" defects precisely distributed along the edges, CNNs, due to their limited receptive field, struggle to capture their complete spatial morphology. This localized processing nature also makes it impossible to form an effective association of scattered defect information at different locations on the entire wafer map, making it difficult to establish a comprehensive global contextual understanding. Therefore, it is particularly ineffective when dealing with complex mixed defects composed of multiple single defects superimposed on each other. Summary of the Invention

[0006] In view of this, the present invention provides a method for ATE wafer pattern defect identification based on Transformer networks. One or more embodiments of this specification also relate to an ATE wafer pattern defect identification system based on Transformer networks, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0007] According to a first aspect of the present invention, a method for identifying defects in ATE wafer patterns based on Transformer networks is provided, comprising: The raw wafer image data is converted into a standardized tensor format through data preprocessing, and a classification label and a segmentation label are constructed for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, which is used to mark the defect location. The normalized tensor is processed through a hybrid network model, including: extracting multi-scale local features through a shared encoder, establishing long-distance dependencies and outputting global feature representations through a global context modeling module, outputting defect category probabilities based on global feature representations through a classification task branch, and reconstructing pixel-level segmentation masks based on global feature representations through a segmentation task branch. By using an end-to-end joint training approach, the parameters of the hybrid network model are optimized using a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

[0008] In some implementations, extracting multi-scale local features via a shared encoder includes: The input tensor is processed by multiple cascaded convolutional modules, each of which includes a standard convolutional layer, a batch normalization layer, a ReLU activation function, and a pooling layer. The input feature map is weighted and optimized using a pixel-level attention module to generate an optimized feature map.

[0009] In some implementations, weighted optimization of the input feature map using a pixel-level attention module includes: The input feature map is compressed using the first convolutional layer; Expand the effective receptive field and capture medium- to long-range spatial context information by using hollow convolutional layers; The number of channels is restored to the original dimension through the second convolutional layer; Pixel-level weight matrices are generated using the Sigmoid activation function; Feature recalibration is achieved by performing element-wise multiplication between the pixel-level weight matrix and the original input feature map; The recalibrated features are added to the original input features through residual connections to form the final output.

[0010] In some implementations, establishing long-distance dependencies through a global context modeling module includes: The two-dimensional feature map is flattened and segmented into a one-dimensional image patch sequence by the image patch sequence generation unit, and a learnable category token is concatenated at the beginning of the sequence. Learnable positional codes are added to each element of the image patch sequence using positional coding units; The sequence after position encoding is processed by a Transformer encoder, which consists of multiple identical layers stacked together. Each layer includes a multi-head self-attention sublayer and a position feedforward network sublayer. Residual connections and layer normalization are applied after each sublayer.

[0011] In some implementations, processing sequences via multi-head self-attention sublayers includes: The input sequence is linearly projected into multiple independent sets of query vectors, key vectors, and value vectors; The output of each head is calculated by scaling the dot product attention, and the corresponding first calculation formula includes:

[0012] Where Q represents the query matrix, derived from a linear projection of the input sequence; K represents the key matrix, derived from a linear projection of the input sequence; V represents the value matrix, derived from a linear projection of the input sequence; d k The dimension of the key vector is represented by T; T represents the matrix transpose operation. The outputs of all heads are concatenated and then linearly projected to obtain the final output of the self-attention sublayer.

[0013] In some implementations, processing the sequence via a location-feedforward network sublayer includes: Two linear transformation layers and the GeLU activation function are applied independently to each position in the sequence; The second calculation formula in the output is:

[0014] in, The input at position i represents the result of the output of the multi-head self-attention sub-layer after residual connection and layer normalization; Linear() represents linear transformation; GeLU() represents Gaussian error linear unit activation function; This represents the output at the i-th position.

[0015] In some implementations, the GeLU activation function takes the form of:

[0016] Here, x represents the input value, which comes from the output of the first linear transformation layer in the position feedforward network sublayer.

[0017] In some implementations, the output of defect category probabilities through the classification task branch includes: Extract the feature vector corresponding to the category token output by the Transformer encoder; The feature vectors are processed by fully connected layers and the Softmax activation function to output the probability of various defects.

[0018] In some implementations, reconstructing pixel-level segmentation masks by splitting task branches includes: The global feature representation is upsampled using a decoder; Multi-scale features are fused using skip connections with the shared encoder; A high-resolution pixel-level segmentation mask is reconstructed.

[0019] According to a second aspect of the present invention, an ATE wafer pattern defect identification system based on Transformer networks is provided, comprising: The data preprocessing module is used to convert the raw wafer image data into a standardized tensor format and to construct a classification label and a segmentation label for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, used to mark the defect location. The hybrid network model includes a shared encoder, a global context modeling module, a classification task branch, and a segmentation task branch. The shared encoder receives the normalized tensor output by the data preprocessing module and extracts multi-scale local features. The global context modeling module receives the multi-scale local features output by the shared encoder, establishes long-distance dependencies, and outputs a global feature representation. The classification task branch outputs the defect category probability based on the global feature representation, and the segmentation task branch reconstructs a pixel-level segmentation mask based on the global feature representation. The model training module is used to optimize the parameters of the hybrid network model through an end-to-end joint training method and a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

[0020] The raw wafer image data is converted into a standardized tensor format through data preprocessing, and a classification label and a segmentation label are constructed for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, which is used to mark the defect location. The normalized tensor is processed through a hybrid network model, including: extracting multi-scale local features through a shared encoder, establishing long-distance dependencies and outputting global feature representations through a global context modeling module, outputting defect category probabilities based on global feature representations through a classification task branch, and reconstructing pixel-level segmentation masks based on global feature representations through a segmentation task branch. By using an end-to-end joint training approach, the parameters of the hybrid network model are optimized using a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

[0021] In some implementations, extracting multi-scale local features via a shared encoder includes: The input tensor is processed by multiple cascaded convolutional modules, each of which includes a standard convolutional layer, a batch normalization layer, a ReLU activation function, and a pooling layer. The input feature map is weighted and optimized using a pixel-level attention module to generate an optimized feature map.

[0022] In some implementations, weighted optimization of the input feature map using a pixel-level attention module includes: The input feature map is compressed using the first convolutional layer; Expand the effective receptive field and capture medium- to long-range spatial context information by using hollow convolutional layers; The number of channels is restored to the original dimension through the second convolutional layer; Pixel-level weight matrices are generated using the Sigmoid activation function; Feature recalibration is achieved by performing element-wise multiplication between the pixel-level weight matrix and the original input feature map; The recalibrated features are added to the original input features through residual connections to form the final output.

[0023] In some implementations, establishing long-distance dependencies through a global context modeling module includes: The two-dimensional feature map is flattened and segmented into a one-dimensional image patch sequence by the image patch sequence generation unit, and a learnable category token is concatenated at the beginning of the sequence. Learnable positional codes are added to each element of the image patch sequence using positional coding units; The sequence after position encoding is processed by a Transformer encoder, which consists of multiple identical layers stacked together. Each layer includes a multi-head self-attention sublayer and a position feedforward network sublayer. Residual connections and layer normalization are applied after each sublayer.

[0024] In some implementations, processing sequences via multi-head self-attention sublayers includes: The input sequence is linearly projected into multiple independent sets of query vectors, key vectors, and value vectors; The output of each head is calculated by scaling the dot product attention, and the corresponding first calculation formula includes:

[0025] Where Q represents the query matrix, derived from a linear projection of the input sequence; K represents the key matrix, derived from a linear projection of the input sequence; V represents the value matrix, derived from a linear projection of the input sequence; d k The dimension of the key vector is represented by T; T represents the matrix transpose operation. The outputs of all heads are concatenated and then linearly projected to obtain the final output of the self-attention sublayer.

[0026] In some implementations, processing the sequence via a location-feedforward network sublayer includes: Two linear transformation layers and the GeLU activation function are applied independently to each position in the sequence; The second calculation formula in the output is:

[0027] in, The input at position i represents the result of the output of the multi-head self-attention sub-layer after residual connection and layer normalization; Linear() represents linear transformation; GeLU() represents Gaussian error linear unit activation function; This represents the output at the i-th position.

[0028] In some implementations, the GeLU activation function takes the form of:

[0029] Here, x represents the input value, which comes from the output of the first linear transformation layer in the position feedforward network sublayer.

[0030] In some implementations, the output of defect category probabilities through the classification task branch includes: Extract the feature vector corresponding to the category token output by the Transformer encoder; The feature vectors are processed by fully connected layers and the Softmax activation function to output the probability of various defects.

[0031] In some implementations, reconstructing pixel-level segmentation masks by splitting task branches includes: The global feature representation is upsampled using a decoder; Multi-scale features are fused using skip connections with the shared encoder; A high-resolution pixel-level segmentation mask is reconstructed.

[0032] According to a third aspect of the present invention, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-described ATE wafer image defect identification method based on Transformer networks.

[0033] According to a fourth aspect of the present invention, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described ATE wafer pattern defect identification method based on Transformer networks.

[0034] According to a fifth aspect of the present invention, a computer program is provided, wherein when the computer program is executed in a computer, the computer is instructed to perform the steps of the above-described ATE wafer pattern defect identification method based on Transformer networks.

[0035] At least one embodiment of the present invention integrates data preprocessing, hybrid network modeling and end-to-end training. This system can make full use of the local and global features of the wafer map to achieve simultaneous and accurate execution of macroscopic defect classification and microscopic defect localization, thereby improving the automation level and reliability of defect detection and providing a comprehensive and efficient analysis tool for semiconductor process optimization. Attached Figure Description

[0036] Figure 1 This is a flowchart of an ATE wafer image defect identification method based on Transformer network provided by the present invention; Figure 2 This is a simplified structural diagram of the pixel-level attention module in the ATE wafer image defect identification method based on Transformer network provided by the present invention; Figure 3 This is a simplified structural diagram of the multi-head attention mechanism in an ATE wafer image defect identification method based on Transformer network provided by the present invention. Figure 4 This is a simplified structural diagram of the convolutional-Transformer network integrated structure in the ATE wafer image defect identification method based on Transformer network provided by the present invention. Figure 5 This is a simplified structural diagram of an ATE wafer image defect identification system based on a Transformer network provided by the present invention; Figure 6 This is a structural block diagram of a computing device provided by the present invention. Detailed Implementation

[0037] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0038] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items. The modifications “a” and “a plurality” as used in this disclosure are illustrative and not restrictive, and those skilled in the art will understand that they should be understood as “one or more” unless the context clearly indicates otherwise.

[0039] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0040] To address the limitation of traditional convolutional neural networks (CNNs) in their inherent local receptive field, which hinders their ability to effectively model long-range dependencies and global spatial distribution features of defect patterns in wafer images, this invention proposes a novel method for automatic, accurate, and efficient ATE wafer image defect detection. This invention aims to fully leverage the complementary advantages of both technologies by introducing an innovative convolutional-Transformer hybrid network architecture. Specifically, this invention first utilizes a convolutional network to efficiently extract local detail features from the wafer image. Then, fundamentally different from existing technologies, this invention innovatively uses a Transformer architecture to model the global context of these features.

[0041] The core advantage of Transformer lies in its self-attention mechanism. This mechanism overcomes the local window limitations of convolutional operations by calculating the correlation weights between features at each location on the wafer image and features at all other locations, thus establishing a complete global receptive field within a single-layer network. This capability enables the model of this invention to truly understand the spatial distribution logic of defects. For example, it can directly correlate anomalies at one end of the wafer with anomalies at the other end, identifying that they together constitute a specific, global defect pattern, such as a "scratch-type" or a complex "hybrid" defect. Therefore, the ultimate goal of this invention is to provide an intelligent detection scheme that can deeply understand the global contextual information of the wafer image, achieving simultaneous classification and segmentation of multiple single and hybrid defect patterns, thereby significantly improving the accuracy, robustness, and automation level of defect detection.

[0042] See Figure 1 , Figure 1 A flowchart of an ATE wafer image defect identification method based on a Transformer network, according to some embodiments of this specification, is shown, specifically including the following steps.

[0043] Step 101: Convert the original wafer image data into a normalized tensor format through data preprocessing, and construct a classification label and a segmentation label for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, used to mark the defect location.

[0044] Step 102: Process the normalized tensor through a hybrid network model, including: extracting multi-scale local features through a shared encoder, establishing long-distance dependencies and outputting global feature representations through a global context modeling module, outputting defect category probabilities based on global feature representations through a classification task branch, and reconstructing pixel-level segmentation masks based on global feature representations through a segmentation task branch.

[0045] Step 103: Optimize the parameters of the hybrid network model using an end-to-end joint training method and a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss. The classification loss includes a threshold loss function with a learnable classification threshold.

[0046] Raw wafer map data can refer to the unprocessed set of wafer test results obtained from automated test equipment (ATE). For example, it can be a two-dimensional logical matrix containing Bin Codes read in batches from the ATE test equipment database through the Semiconductor Manufacturing Execution System (MES) interface, providing raw information for defect identification and localization. Standardized tensor format refers to a data structure suitable for neural network processing that is numerically normalized and dimensionally uniform. For example, mapping the symbolic Bin Codes of the raw wafer map to numerical values ​​and adjusting their size to a fixed value ensures data input consistency and improves model training stability. Classification labels can refer to macroscopic category identifiers assigned to wafer maps by engineers based on defect spatial distribution patterns. For example, assigning categories such as "center-clustered" or "edge-ringed" to wafer maps based on a predefined defect pattern library guides the model in learning macroscopic defect patterns. Segmentation labels can refer to binary masks that are the same size as the wafer image and are used to locate defects at the pixel level. For example, all die locations in the wafer image that are identified as defects are marked as 1, while good products and invalid areas are marked as 0. This can provide accurate ground truth values ​​of defect locations for supervising the segmentation task.

[0047] The data preprocessing module can refer to the functional unit that performs data transformation and label construction. For example, it processes raw ATE data through numerical mapping, size normalization, and binarization mask generation to construct standardized training samples containing classification and segmentation labels. A hybrid network model can refer to a composite model integrating multiple neural network architectures, such as a deep learning network containing a shared encoder, a Transformer module, and parallel task branches, used to simultaneously extract features from wafer images and complete classification and segmentation tasks. A shared encoder can refer to the common backbone responsible for feature extraction in a hybrid network model, such as a network composed of multiple cascaded convolutional modules that receives standardized tensors and outputs multi-scale local features, providing shared basic feature representations for subsequent tasks. Multi-scale local features can refer to the set of feature maps with different receptive fields captured by the encoder at different network layers. For example, it extracts wafer image structure information from fine to macroscopic levels through convolution and pooling operations to comprehensively describe the local spatial patterns of defects.

[0048] The global context modeling module can refer to a component based on the Transformer architecture used to establish long-distance dependencies. For example, it can serialize feature maps and calculate global relationships using a multi-head self-attention mechanism, integrating the contextual information of the entire wafer image to enhance feature representation. Global feature representation can refer to the features output after global context modeling that contain overall spatial relationships, such as the serialized features output by the Transformer encoder, which include global semantic information corresponding to category tokens, used to support classification and segmentation decisions. The classification task branch can refer to a network branch specifically handling macroscopic defect category prediction. For example, it can process the category token features in the global feature representation using fully connected layers and a Softmax function to output the probability distribution of each defect category. The defect category probability can refer to the model's confidence score for the wafer image belonging to each predefined defect category. For example, it can convert the logits output by the classification branch into probability values ​​using a Softmax function, used for the final determination of macroscopic defect patterns. The segmentation task branch can refer to a network branch responsible for generating pixel-level localization results. For example, it can use transposed convolutions and skip connections to upsample and fuse global feature representations, reconstructing high-resolution segmentation masks for accurate defect localization. A pixel-level segmentation mask can refer to a binary image with the same size as the input wafer image, where each pixel is labeled with the probability of a defect. For example, it can be obtained by thresholding the tensor output by the segmentation branch and can be used to visualize the specific distribution location of defects.

[0049] The model training module can refer to the functional unit that performs network parameter optimization. For example, it might use backpropagation to jointly optimize classification and segmentation losses, enabling the model to adaptively learn defect features through end-to-end training. End-to-end joint training can refer to a training strategy that simultaneously optimizes multiple tasks. For example, it might simultaneously calculate classification and segmentation losses in a single forward propagation and update shared parameters through weighted summation backpropagation, improving overall model performance and training efficiency. The total loss function can refer to the overall objective function used to guide parameter optimization during model training. For example, it might be composed of a weighted sum of segmentation losses (such as cross-entropy) and classification losses (such as cross-entropy and threshold loss) with preset weights, used to balance multi-task learning. Segmentation loss can refer to a function that measures the difference between the model's segmentation output and the ground truth mask. For example, it might use binary cross-entropy loss to calculate the prediction error for each pixel, driving the segmentation branch to learn accurate defect localization. Classification loss can refer to a function that measures the difference between the model's classification prediction and the true class. For example, it might be composed of standard cross-entropy loss and a learnable threshold loss, used to optimize classification accuracy and adaptively determine decision boundaries. A learnable classification threshold can refer to a decision boundary parameter dynamically determined during model training. For example, by optimizing a threshold loss function, categories with predicted scores exceeding the threshold are classified as positive, thus adapting to different data distributions and improving classification robustness. A threshold loss function can also refer to an auxiliary loss function specifically used to train the classification threshold. For instance, optimizing the threshold by calculating the difference between the number of predicted positives and the label cardinality can make the model output match the true data distribution.

[0050] As a concrete example: In a defect analysis scenario on a semiconductor manufacturing line, the data preprocessing module retrieves raw Bin Code data from a batch of wafer images from the ATE database via the MES system API. Each wafer image is a 256x256 two-dimensional logical matrix. The module maps the symbolic Bin Code to normalized values ​​(e.g., mapping defective Bin Code to 1.0 and good Bin Code to 0), and generates a classification label (e.g., "hybrid") and a corresponding 256x256 binary segmentation mask for each wafer image. The shared encoder of the hybrid network model (consisting of 5 convolutional modules, each containing 3x3 convolution, batch normalization, and ReLU) receives this normalized tensor and outputs a 512-dimensional multi-scale local feature map. The global context modeling module segments the feature map into a 16x16 image patch sequence, concatenates learnable class tokens, adds positional encoding, and then processes the input through a 6-layer Transformer encoder (each layer containing 8 self-attention heads with a hidden dimension of 512), outputting a global feature representation. The classification task branch extracts the feature vectors corresponding to the category tokens. Through two fully connected layers (dimensions 256 and the number of categories, respectively) and a Softmax function, it outputs a probability distribution such as "center-clustered: 0.85, edge-ringed: 0.10, mixed: 0.05". The segmentation task branch upsamples the global feature representation using four transposed convolutional modules (4x4 kernels, stride 2) and fuses it with features from the corresponding encoder layers via skip connections, ultimately outputting a 256x256 segmentation mask. The model training module uses the Adam optimizer (learning rate 0.001). The total loss function is a weighted sum of the segmentation loss (binary cross-entropy, weight 1.0) and the classification loss (cross-entropy weight 1.0 + threshold loss weight 0.5). The label cardinality LC in the threshold loss is preset to 1.2, and the batch size B=32, performing end-to-end joint training.

[0051] The beneficial effects of one of the embodiments in this specification include at least the following: by integrating data preprocessing, hybrid network modeling and end-to-end training, the system can make full use of the local and global features of the wafer map to achieve simultaneous and accurate execution of macroscopic defect classification and microscopic defect localization, thereby improving the automation level and reliability of defect detection and providing a comprehensive and efficient analysis tool for semiconductor process optimization.

[0052] In some implementations, extracting multi-scale local features through a shared encoder includes: processing the input tensor through multiple cascaded convolutional modules, each convolutional module including a standard convolutional layer, a batch normalization layer, a ReLU activation function, and a pooling layer; and performing weighted optimization on the input feature map through a pixel-level attention module to generate an optimized feature map.

[0053] A shared encoder can refer to the common network part in a hybrid network model responsible for extracting basic features from the input data. For example, it may consist of multiple cascaded convolutional modules, each containing standard convolutional layers, batch normalization layers, ReLU activation functions, and pooling layers, providing a unified multi-scale feature representation for subsequent classification and segmentation tasks. A standard convolutional layer can refer to a network layer that performs linear convolution operations to extract spatial features. For example, it may use a 3x3 learnable convolutional kernel to perform a sliding window calculation on the input feature map to capture local patterns and spatial correlations of defects in a wafer image. A batch normalization layer can refer to a technique that standardizes the input data of the network layer to accelerate training. For example, it may calculate the mean and variance of the current batch of data after each convolutional layer and then normalize it, which can alleviate internal covariate shift problems and improve model training stability. A ReLU activation function can refer to an activation function that introduces a non-linear transformation. For example, it may set all negative values ​​of the convolutional layer output to zero while retaining positive values, introducing non-linear expressive power into the neural network, enabling it to fit complex functions.

[0054] like Figure 2 As shown, a pixel-level attention module can refer to a component that spatially weights feature maps to enhance key regions. For example, it can generate an attention map by using a convolutional sequence containing dilated convolutions and multiplying it with the original feature map, which can adaptively improve the response of defect-related features and suppress background noise. The input feature map can refer to a two-dimensional feature tensor input to the attention module, such as a feature map with a specific number of channels and spatial size output by the previous convolutional module, serving as the basis for attention calculation to generate weighted calibration. The optimized feature map can refer to the output features processed by the attention mechanism, such as the feature tensor obtained by residually concatenating the original input features with the attention weights, which can preserve the original information while enhancing the expression of key features. Global modeling can refer to the process of establishing long-distance dependencies between features in a neural network, such as inputting the local feature sequence output by the encoder into the Transformer module for self-attention calculation, which can integrate the contextual information of the entire wafer map. High-quality feature input can refer to feature representations with rich semantic information and spatial details, such as multi-scale feature maps enhanced by pixel-level attention modules, which can provide a more focused feature foundation for subsequent Transformer modules targeting defect regions.

[0055] As a concrete example: In a wafer image defect detection system, a shared encoder receives a 256x256 normalized tensor generated by a data preprocessing module. This encoder consists of five cascaded convolutional modules, each containing: a standard convolutional layer (using a 3x3 kernel, stride 1, padding 1), a batch normalization layer, a ReLU activation function, and a max-pooling layer (2x2 window, stride 2). A pixel-level attention module follows the third convolutional module. This module first compresses the 256 channels to 128 channels using a 1x1 convolution, then expands the receptive field using a 3x3 dilated convolution (dilation rate 2), and then restores it to 256 channels using another 1x1 convolution. Finally, an attention map is generated using a sigmoid function. This attention map is element-wise multiplied with the original input feature map, and then an optimized feature map is generated through residual connections. This feature map serves as a high-quality feature input, fed into a subsequent global context modeling module for global modeling, providing enhanced feature representations for classification and segmentation tasks.

[0056] By working in concert with a cascaded convolutional module and a pixel-level attention module, this shared encoder can effectively extract multi-scale local features from the wafer image and enhance defect-related features while preserving detailed information. This provides a high-quality feature base for subsequent global context modeling and task branching, thereby improving the defect recognition and localization performance of the entire system.

[0057] In some implementations, weighted optimization of the input feature map using a pixel-level attention module includes: channel compression of the input feature map through a first convolutional layer; expansion of the effective receptive field and capture of medium- to long-range spatial context information through a dilated convolutional layer; restoration of the number of channels to the original dimension through a second convolutional layer; generation of a pixel-level weight matrix through a sigmoid activation function; recalibration of the features by performing element-wise multiplication between the pixel-level weight matrix and the original input feature map; and addition of the recalibrated features to the original input features through a residual connection to form the final output.

[0058] A pixel-level attention module can refer to a neural network component that spatially adaptively weights a feature map, for example, by generating attention weights through a sequence of convolutions containing dilated convolutions, to enhance the importance of defect-related regions in the feature map and suppress irrelevant background.

[0059] The first convolutional layer can refer to the initial convolutional operation in the attention module that performs channel dimension transformation. For example, using a 1x1 convolutional kernel to compress the number of channels in the input feature map reduces computational complexity and provides dimensionality-reduced features for subsequent processing. The input feature map can refer to the raw feature data fed into the attention module for processing, such as the multi-channel feature tensor output by the previous convolutional module in the shared encoder, which serves as the basis for attention calculations to generate spatial weights. A dilated convolutional layer can refer to a special convolutional operation that expands the receptive field. For example, using a 3x3 convolutional kernel with a dilation rate greater than 1 to insert gaps between kernel elements can capture a wider range of contextual information without increasing the number of parameters. The effective receptive field can refer to the range of the input region that each output unit in a convolutional operation can influence. For example, setting a dilation rate of 2 makes a 3x3 convolutional kernel have a receptive field equivalent to a standard 5x5 convolution, capturing longer-distance spatial dependencies. Medium- to long-distance spatial contextual information can refer to the semantic relationships between non-adjacent locations in the feature map. For example, establishing spatial relationships between feature points across multiple pixels through dilated convolutions can better understand the distribution patterns of defect regions. The second convolutional layer can refer to the subsequent convolutional operation in the attention module that restores the channel dimension. For example, using a 1x1 convolutional kernel to restore the number of feature channels from the compressed state to the original number can maintain the consistency of the feature dimension and prepare for subsequent calculations. The original dimension can refer to the initial number of channels in the input feature map. For example, after channel compression, a 1x1 convolution can restore 128 channels to 256 channels, ensuring that the output feature dimension matches the input feature dimension. The Sigmoid activation function can refer to a non-linear function that maps real numbers to the interval between 0 and 1. For example, applying a Sigmoid transformation to the output of the second convolutional layer can generate an attention weight matrix with values ​​between 0 and 1. The pixel-level weight matrix can refer to an attention distribution map with the same spatial size as the input feature map. For example, each position output by the Sigmoid function corresponds to a weight value between 0 and 1, used to indicate the importance of each spatial position. The attention map can refer to a spatial weight mapping representing the distribution of feature importance. For example, a two-dimensional matrix with the same size as the input feature map generated by the Sigmoid function can guide the network to focus on key regions related to defects. Element-wise multiplication refers to the operation of multiplying corresponding values ​​between two matrices of the same size. For example, multiplying the pixel-level weight matrix point-by-point with the original input feature map can recalibrate the feature values. Important regions refer to parts of the feature map that are highly relevant to the current task. For instance, the wafer image region containing defects is given higher weights in the attention map, allowing the network to focus on these key locations for feature learning. Less important regions refer to parts of the feature map that are less relevant to the current task. For example, the background or normal regions in the wafer image are given lower weights in the attention map, reducing the interference of these regions on the model's decisions.Residual connections can refer to short-circuit connections that directly add the module's input and output. For example, adding the original input features to attention-weighted features can preserve original information and alleviate the vanishing gradient problem. The original input features can refer to the initial feature data before the attention module's processing, such as multi-scale feature maps passed from the previous layer of the shared encoder, serving as the baseline input for residual connections to maintain information integrity. The final output can refer to the feature result generated after the attention module's processing, such as the optimized feature map obtained by adding the weighted features to the original input features through residual connections, providing enhanced feature representations for subsequent network layers.

[0060] As a concrete example: In a wafer image defect detection system, the pixel-level attention module receives a 256-channel feature map as input from the output of the third convolutional module of the shared encoder. The first convolutional layer uses a 1x1 convolutional kernel to compress the channels, halving the 256 channels to 128 channels. The dilated convolutional layer uses a 3x3 convolutional kernel with a dilation rate of 2 to expand the effective receptive field to capture medium- to long-range spatial context information. The second convolutional layer then restores the number of channels to the original 256 channels using a 1x1 convolution. Subsequently, a pixel-level weight matrix, i.e., the attention map, is generated through a sigmoid activation function. This matrix is ​​then element-wise multiplied with the original input feature map to recalibrate the features, highlighting important regions (such as areas with dense defects) and suppressing secondary regions (such as the background). Finally, the recalibrated features are added to the original input features through residual connections to form the final output of this module, which is then passed to subsequent network layers for further processing.

[0061] By using a series of convolutional layers and dilated convolutions, this pixel-level attention module can effectively capture the global contextual information of the feature map and generate accurate spatial attention weights. This enables feature enhancement of critical defect regions and suppression of non-critical regions, thereby improving the accuracy and robustness of subsequent defect identification and localization.

[0062] like Figure 4 As shown, in some implementations, establishing long-distance dependencies through the global context modeling module includes: flattening and segmenting the two-dimensional feature map into a one-dimensional image patch sequence through the image patch sequence generation unit, and concatenating a learnable class token at the beginning of the sequence; adding a learnable positional code to each element of the image patch sequence through the positional encoding unit; and processing the sequence with the added positional code through a Transformer encoder, which consists of multiple identical layers stacked together, each layer including a multi-head self-attention sublayer and a positional feedforward network sublayer, with residual connections and layer normalization applied after each sublayer.

[0063] The global context modeling module can refer to a network component based on the Transformer architecture used to establish long-range dependencies between features. For example, it can process serialized features through a self-attention mechanism, integrating global information from the entire wafer image to enhance feature representation. The Transformer architecture can refer to a deep learning model structure based on a self-attention mechanism, such as a stacked encoder layer, each layer containing a multi-head self-attention sublayer and a feedforward network sublayer, used to process sequential data and model global dependencies. The image patch sequence generation unit can refer to a preprocessing component that converts a two-dimensional feature map into a one-dimensional sequence. For example, it can segment the feature map into fixed-size image patches and flatten them into a vector sequence, providing serialized input data for the Transformer module. A two-dimensional feature map can refer to a feature tensor with spatial and channel dimensions, such as a feature map with height H, width W, and number of channels C output by a shared encoder, serving as the input source for the image patch sequence generation unit. A one-dimensional image patch sequence can refer to a vector sequence obtained by segmenting and flattening a two-dimensional feature map. For example, it can divide the feature map into PxP blocks, each block flattened into a vector sequence of length P. 2 A vector of size ×C arranged in order can preserve local structural information and adapt to the input requirements of the Transformer. A learnable category token can refer to a special trainable vector, such as adding an extra token with the same dimension as the image patch vector at the beginning of an image patch sequence, used to aggregate global information and serve as input features for classification tasks. A positional encoding unit can refer to a functional module that adds positional information to a sequence, such as generating a learnable positional vector with the same dimension as the sequence length and adding it to the input sequence, enabling the Transformer to perceive the relative or absolute position of each element in the sequence. Learnable positional encoding can refer to positional representation vectors obtained through training, such as initializing an embedding matrix corresponding to the maximum sequence length and optimizing these positional vectors during training, providing the model with flexible positional awareness. A Transformer encoder can refer to a feature transformation module composed of multiple stacked identical layers, such as each encoder layer containing a multi-head self-attention mechanism and a feedforward neural network, capable of deep feature transformation and context modeling of the input sequence. Identical layers can refer to network layers with the same structure but independent parameters, such as 12 identical Transformer encoder layers stacked sequentially, allowing for the extraction of more complex feature representations through deep network structures.

[0064] like Figure 3As shown, a multi-head self-attention sublayer can refer to a feature processing layer that performs multiple self-attention calculations in parallel. For example, it linearly projects the input sequence into eight independent query, key, and value vectors and calculates attention for each, capturing dependencies from different representation subspaces. A position-forward network sublayer can refer to a fully connected network applied independently to each position, such as one composed of two linear transformation layers and a GeLU activation function, providing non-linear feature transformation capabilities for each sequence position. Residual connections can refer to short-circuit connections that directly add the layer input and output, such as adding the input and output features after each sublayer, mitigating the gradient vanishing problem in deep network training. Layer normalization can refer to techniques that standardize the feature dimensions of each sample, such as calculating the mean and variance of all channels at each sequence position and normalizing them, stabilizing the training process and accelerating convergence.

[0065] As a concrete example: In a wafer image defect detection system, the global context modeling module receives a feature map (16x16x512 in size) output by the shared encoder. The image patch sequence generation unit divides the feature map into 256 1x1 image patches (since the feature map is already small), each patch is flattened into a 512-dimensional vector, forming a 256-dimensional image patch sequence. A learnable class token (512-dimensional) is added to the beginning of the sequence, resulting in a sequence of 257 in total length. The position encoding unit generates 257 512-dimensional learnable position vectors and adds them to the input sequence. The Transformer encoder consists of 6 identical layers stacked together, each layer containing: a multi-head self-attention sublayer (8 heads, each with a dimension of 64) and a position feedforward network sublayer (with a middle dimension of 2048, using GeLU activation). Residual connections and layer normalization are applied after each sublayer. The feature vector corresponding to the class token in the final output sequence is passed to the classification task branch, and the remaining image patch features are passed to the segmentation task branch.

[0066] Leveraging the global modeling capabilities of the Transformer architecture, this module can establish long-distance dependencies between different regions of the wafer map, integrate global contextual information, and provide semantically rich feature representations for subsequent classification and segmentation tasks, significantly improving the ability to identify complex defect patterns.

[0067] In some implementations, processing the sequence via a multi-head self-attention sublayer includes: linearly projecting the input sequence into multiple independent sets of query vectors, key vectors, and value vectors; and calculating the output of each head using scaled dot-product attention, with the corresponding first calculation formula including:

[0068] Where Q represents the query matrix, derived from a linear projection of the input sequence; K represents the key matrix, derived from a linear projection of the input sequence; V represents the value matrix, derived from a linear projection of the input sequence; dk The dimension of the key vector is represented by T; T represents the matrix transpose operation. The outputs of all heads are concatenated and then linearly projected to obtain the final output of the self-attention sublayer.

[0069] The input sequence can refer to the vector sequence fed into the attention mechanism for processing, such as the feature sequence output by the positional encoding unit plus positional encoding, serving as the input source for self-attention computation. Linear projection can refer to the operation of transforming the input vector to different spaces through fully connected layers. For example, using a learnable weight matrix to map the input vector into query, key, and value vectors respectively can provide different feature representations for the attention mechanism. Multiple independent query vectors can refer to multiple query representations obtained through linear projection. For example, passing the input sequence through eight different linear layers yields eight query vectors, which can be used to compute attention weights in different feature subspaces. Key vectors can refer to the key information representation used to compute attention weights. For example, converting the input sequence into a key vector with the same dimension as the query vector through linear projection allows for similarity matching with the query vector during attention computation. Value vectors can refer to the representation vector storing actual feature information. For example, converting the input sequence into a value vector through linear projection is used for weighted summation of feature information according to attention weights. Scaling dot product attention can refer to a mechanism for calculating attention weights. For example, dividing the dot product of the query vector and the key vector by the square root of their dimensions and then applying the Softmax function produces normalized attention weights, which are then weighted and summed over the value vectors. The query matrix can refer to the matrix representation of all query vectors, such as a matrix formed by arranging the query vectors at each position in the sequence row-wise. This matrix is ​​used for matrix multiplication with the key matrix to calculate attention weights. The key matrix can refer to the matrix representation of all key vectors, such as a matrix formed by arranging the key vectors at each position in the sequence row-wise. This matrix can be used to calculate similarity scores with the query matrix. The value matrix can refer to the matrix representation of all value vectors, such as a matrix formed by arranging the value vectors at each position in the sequence row-wise. This matrix is ​​then weighted and summed according to the attention weights to obtain the output. The dimension of the key vector can refer to the feature length of each key vector, for example, set to 64 dimensions, which is used as the dimension value in the denominator of the scaling factor to stabilize gradient calculation. Matrix transpose can refer to a linear algebraic operation that interchanges the rows and columns of a matrix. For example, transposing the key matrix to match its dimensions with the query matrix facilitates matrix multiplication to obtain the attention score. Attention output can refer to the weighted summation result after the attention mechanism, such as the weighted feature representation obtained by multiplying the attention weights by the value matrix, which can capture the dependencies between different positions in the sequence. The output of each head can refer to the feature representation calculated by a single attention head; for example, eight attention heads can each calculate eight different feature matrices, representing the dependencies of the sequence from different perspectives. The total number of heads can refer to the number of attention heads computed in parallel; for example, setting it to eight heads, each focusing on a different feature subspace, can enhance the model's ability to capture multiple dependencies. The learnable output weight matrix can refer to the parameter matrix that maps the concatenated result to the target dimension; for example, using a 768x512 weight matrix to project the concatenated features to a specified dimension can fuse information from multiple heads and control the output dimension.The final output of the self-attention sublayer can refer to the comprehensive feature representation after multi-head attention and linear projection. For example, the feature obtained by concatenating the outputs of multiple heads and transforming them through the output weight matrix can provide features rich in contextual information for subsequent layers.

[0070] As a concrete example: In a layer of the Transformer encoder, a multi-head self-attention sublayer receives an input sequence of length 257 and dimension 512. This sublayer first transforms the input sequence into eight independent query vectors, key vectors, and value vectors (each with dimension 64) through linear projection. The query matrix, key matrix, and value matrix are each composed of these vectors. In the scaled dot product attention calculation, the query matrix is ​​multiplied by the transpose of the key matrix and divided by 8 (the square root of the dimension 64 of the key vector). Then, a normalized exponential function is applied to obtain the attention weights, which are multiplied by the value matrix to obtain the output of each head. The outputs of the eight heads are concatenated along the feature dimension and then linearly transformed through a learnable output weight matrix (size 512x512) to produce the final output of the self-attention sublayer. This output is then passed to the location feedforward network sublayer after residual connections and layer normalization.

[0071] Through multi-head parallel computation mechanism, this self-attention sublayer can capture rich dependencies from different feature subspaces, integrate global contextual information, and provide semantically rich feature representations for each position in the sequence, significantly enhancing the model's ability to understand wafer map defect patterns.

[0072] In some implementations, processing the sequence through a position-fedforward network sublayer includes: independently applying two linear transform layers and the GeLU activation function to each position in the sequence; and calculating the second formula for the output as follows:

[0073] in, The input at position i represents the result of the output of the multi-head self-attention sub-layer after residual connection and layer normalization; Linear() represents linear transformation; GeLU() represents Gaussian error linear unit activation function; This represents the output at the i-th position.

[0074] In some implementations, the GeLU activation function takes the form of:

[0075] Here, x represents the input value, which comes from the output of the first linear transformation layer in the position feedforward network sublayer.

[0076] In some implementations, outputting the probability of a defect category through the classification task branch includes: extracting the feature vector corresponding to the category token output by the Transformer encoder; processing the feature vector through a fully connected layer and a Softmax activation function to output the probability of each type of defect.

[0077] Category tokens can refer to special trainable vectors added to the beginning of a sequence, such as the 512-dimensional vector added to the beginning of a Transformer input sequence. These vectors are used to aggregate global information and serve as input features for classification tasks. The probabilities of different defect types can refer to the model's confidence score for a wafer image belonging to each predefined defect category. For example, probability values ​​such as "center-clustered: 0.85, edge-ringed: 0.10" output by the Softmax function are used to determine the final macroscopic defect pattern.

[0078] As a concrete example: In a wafer image defect detection system, the classification task branch receives the global feature representation output from the global context modeling module. This branch first extracts the 512-dimensional feature vectors corresponding to the category tokens in the sequence, then performs feature transformation through two fully connected layers (the first layer from 512 to 256 dimensions, the second layer from 256 to the number of categories), and finally uses the Softmax activation function to convert the output into a probability distribution for various types of defects, such as "center-clustered type: 0.85, edge-ringed type: 0.10, mixed type: 0.05". These probability values ​​are output as the final classification result, used to determine the macroscopic defect patterns of the wafer image.

[0079] Through a specially designed classification branch structure, the system can effectively utilize the semantic information in the global feature representation to accurately identify the macroscopic defect patterns of the wafer image, providing a reliable classification basis for subsequent process analysis and quality control.

[0080] In some implementations, reconstructing a pixel-level segmentation mask by segmenting task branches includes: upsampling the global feature representation using a decoder; fusing multi-scale features using skip connections with the shared encoder; and reconstructing a high-resolution pixel-level segmentation mask.

[0081] A transposed convolution module can refer to a deconvolutional unit that performs upsampling operations. For example, a transposed convolutional layer using a 4x4 kernel and a stride of 2 can double the spatial size of the feature map and gradually restore detailed information. Upsampling can refer to the operation of increasing the spatial size of the feature map. For example, using transposed convolution to gradually upsample a 16x16 feature map to 256x256 can restore spatial details to generate a high-resolution output. Skip connections can refer to short-circuit paths that connect encoder features to decoder features. For example, concatenating feature maps of the same spatial size in the encoder with corresponding layer features in the decoder can preserve low-level detailed information. A high-resolution pixel-level segmentation mask can refer to a binarized output image with the same size as the input wafer image. For example, a 256x256 matrix output by the decoder, where each pixel value represents the probability of a defect at that location.

[0082] As a concrete example: In a wafer image defect detection system, the decoder of the segmentation task branch receives a 16x16x512 feature map output from the global context modeling module. This decoder consists of four cascaded transposed convolutional modules, each containing a 4x4 transposed convolution (stride 2), batch normalization, and ReLU activation. Through skip connections, each layer of the decoder fuses feature maps of corresponding spatial sizes from the encoder (e.g., 64x64, 32x32, 16x16). After a series of upsampling and feature fusion operations, a high-resolution 256x256 pixel-level segmentation mask is finally reconstructed, where each pixel value, after Sigmoid activation, represents the probability that the location is a defect, completing the reconstruction process from abstract features to specific localization results.

[0083] Through the decoder structure and skip connection mechanism, this segmentation task branch can effectively fuse multi-scale features, restore spatial details while maintaining semantic information, and achieve accurate mapping from global features to pixel-level localization, providing reliable microscopic location information for defect analysis.

[0084] In some implementations, the threshold loss function is calculated as follows:

[0085] in, The threshold loss is represented by LC; the label cardinality is represented by B; the batch size is represented by L; the output sequence length is represented by N; and the total number of samples is represented by N. The model outputs the predicted score of the i-th sample in the j-th category, which comes from the output of the classification task branch; t represents the learnable classification threshold; l() represents the indicator function; the subscript i iterates through each sample in the batch from 1 to B, and the subscript j iterates through each category from 1 to L.

[0086] In some implementations, the basic convolutional unit operation of the convolutional module is represented as:

[0087] in, The input features are derived from the output of the previous convolutional module or the initial input tensor; K represents the learnable convolutional kernel; Conv() represents the convolution operation; BN() represents batch normalization. Represents the ReLU activation function; This indicates the output features.

[0088] In some implementations, the optimized feature map generation process is represented as follows:

[0089] in, This represents the input feature map, which originates from the output of the previous convolutional module in the shared encoder. represents the pixel-level weight matrix, which is generated internally by convolution and the sigmoid activation function within the pixel-level attention module; \otimes represents element-wise multiplication. This represents the optimized feature map.

[0090] In some implementations, the output sequence of the image patch sequence generation unit is represented as:

[0091] Where F represents the sequence of image patches obtained by flattening and segmenting the two-dimensional feature map output by the shared encoder; T represents the learnable class token; concat() represents the concatenation operation; and Y represents the concatenated sequence.

[0092] In some implementations, the output sequence of the position coding unit is represented as:

[0093] Where Y represents the sequence output by the image patch sequence generation unit; PE represents the learnable positional code; and A represents the sequence after adding the positional code.

[0094] In some implementations, the final output of the self-attention sublayer is represented as:

[0095] in, This represents the output of the nth head, derived from the scaling dot product attention calculation formula; n represents the total number of heads. represents the learnable output weight matrix; S represents the final output of the self-attention sublayer.

[0096] In some implementations, the output of residual connections and layer normalization after the self-attention sublayer is represented as:

[0097] in, This represents the input sequence of the i-th sample, which originates from the output of the position coding unit; LN() represents the self-attention output of the i-th sample, which comes from the output of the multi-head self-attention sub-layer; LN() represents the layer normalization operation. This represents the output of the residual connection and layer normalization of the i-th sample.

[0098] In some implementations, the layer normalization operation is calculated as follows:

[0099] Where t represents the input vector, which comes from the output of the residual connection; ave(t) represents the mean of the input vector t; This represents the variance of the input vector t; This represents a minimal constant to prevent the denominator from being zero.

[0100] In some implementations, the output of residual connections and layer normalization after the location feedforward network sublayer is represented as:

[0101] in, This represents the position of the i-th sample and is the output of the feedforward network. This represents the final output of the i-th sample.

[0102] Corresponding to the above method embodiments, this specification also provides an embodiment of an ATE wafer image defect identification system based on Transformer networks. Figure 5 This specification illustrates a schematic diagram of an ATE wafer image defect identification system based on a Transformer network, provided in some embodiments of this specification. For example... Figure 5 As shown, the system includes: The data preprocessing module 501 is used to convert the original wafer image data into a standardized tensor format and construct a classification label and a segmentation label for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, used to mark the defect location. The hybrid network model 502 includes a shared encoder, a global context modeling module, a classification task branch, and a segmentation task branch. The shared encoder receives the normalized tensor output by the data preprocessing module and extracts multi-scale local features. The global context modeling module receives the multi-scale local features output by the shared encoder, establishes long-distance dependencies, and outputs a global feature representation. The classification task branch outputs the defect category probability based on the global feature representation. The segmentation task branch reconstructs a pixel-level segmentation mask based on the global feature representation. The model training module 503 is used to optimize the parameters of the hybrid network model by adopting an end-to-end joint training method and through a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

[0103] The raw wafer image data is converted into a standardized tensor format through data preprocessing, and a classification label and a segmentation label are constructed for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, which is used to mark the defect location. The normalized tensor is processed through a hybrid network model, including: extracting multi-scale local features through a shared encoder, establishing long-distance dependencies and outputting global feature representations through a global context modeling module, outputting defect category probabilities based on global feature representations through a classification task branch, and reconstructing pixel-level segmentation masks based on global feature representations through a segmentation task branch. By using an end-to-end joint training approach, the parameters of the hybrid network model are optimized using a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

[0104] In some implementations, extracting multi-scale local features via a shared encoder includes: The input tensor is processed by multiple cascaded convolutional modules, each of which includes a standard convolutional layer, a batch normalization layer, a ReLU activation function, and a pooling layer. The input feature map is weighted and optimized using a pixel-level attention module to generate an optimized feature map.

[0105] In some implementations, weighted optimization of the input feature map using a pixel-level attention module includes: The input feature map is compressed using the first convolutional layer; Expand the effective receptive field and capture medium- to long-range spatial context information by using hollow convolutional layers; The number of channels is restored to the original dimension through the second convolutional layer; Pixel-level weight matrices are generated using the Sigmoid activation function; Feature recalibration is achieved by performing element-wise multiplication between the pixel-level weight matrix and the original input feature map; The recalibrated features are added to the original input features through residual connections to form the final output.

[0106] In some implementations, establishing long-distance dependencies through a global context modeling module includes: The two-dimensional feature map is flattened and segmented into a one-dimensional image patch sequence by the image patch sequence generation unit, and a learnable category token is concatenated at the beginning of the sequence. Learnable positional codes are added to each element of the image patch sequence using positional coding units; The sequence after position encoding is processed by a Transformer encoder, which consists of multiple identical layers stacked together. Each layer includes a multi-head self-attention sublayer and a position feedforward network sublayer. Residual connections and layer normalization are applied after each sublayer.

[0107] In some implementations, processing sequences via multi-head self-attention sublayers includes: The input sequence is linearly projected into multiple independent sets of query vectors, key vectors, and value vectors; The output of each head is calculated by scaling the dot product attention, and the corresponding first calculation formula includes:

[0108] Where Q represents the query matrix, derived from a linear projection of the input sequence; K represents the key matrix, derived from a linear projection of the input sequence; V represents the value matrix, derived from a linear projection of the input sequence; d k The dimension of the key vector is represented by T; T represents the matrix transpose operation. The outputs of all heads are concatenated and then linearly projected to obtain the final output of the self-attention sublayer.

[0109] In some implementations, processing the sequence via a location-feedforward network sublayer includes: Two linear transformation layers and the GeLU activation function are applied independently to each position in the sequence; The second calculation formula in the output is:

[0110] in, The input at position i represents the result of the output of the multi-head self-attention sub-layer after residual connection and layer normalization; Linear() represents linear transformation; GeLU() represents Gaussian error linear unit activation function; This represents the output at the i-th position.

[0111] In some implementations, the GeLU activation function takes the form of:

[0112] Here, x represents the input value, which comes from the output of the first linear transformation layer in the position feedforward network sublayer.

[0113] In some implementations, the output of defect category probabilities through the classification task branch includes: Extract the feature vector corresponding to the category token output by the Transformer encoder; The feature vectors are processed by fully connected layers and the Softmax activation function to output the probability of various defects.

[0114] In some implementations, reconstructing pixel-level segmentation masks by splitting task branches includes: The global feature representation is upsampled using a decoder; Multi-scale features are fused using skip connections with the shared encoder; A high-resolution pixel-level segmentation mask is reconstructed.

[0115] In some implementations, the threshold loss function is calculated as follows:

[0116] in, The threshold loss is represented by LC; the label cardinality is represented by B; the batch size is represented by L; the output sequence length is represented by N; and the total number of samples is represented by N. The model outputs the predicted score of the i-th sample in the j-th category, which comes from the output of the classification task branch; t represents the learnable classification threshold; l() represents the indicator function; the subscript i iterates through each sample in the batch from 1 to B, and the subscript j iterates through each category from 1 to L.

[0117] In some implementations, the basic convolutional unit operation of the convolutional module is represented as:

[0118] in, The input features are derived from the output of the previous convolutional module or the initial input tensor; K represents the learnable convolutional kernel; Conv() represents the convolution operation; BN() represents batch normalization. Represents the ReLU activation function; This indicates the output features.

[0119] In some implementations, the optimized feature map generation process is represented as follows:

[0120] in, This represents the input feature map, which originates from the output of the previous convolutional module in the shared encoder. represents the pixel-level weight matrix, which is generated internally by convolution and the sigmoid activation function within the pixel-level attention module; \otimes represents element-wise multiplication. This represents the optimized feature map.

[0121] In some implementations, the output sequence of the image patch sequence generation unit is represented as:

[0122] Where F represents the sequence of image patches obtained by flattening and segmenting the two-dimensional feature map output by the shared encoder; T represents the learnable class token; concat() represents the concatenation operation; and Y represents the concatenated sequence.

[0123] In some implementations, the output sequence of the position coding unit is represented as:

[0124] Where Y represents the sequence output by the image patch sequence generation unit; PE represents the learnable positional code; and A represents the sequence after adding the positional code.

[0125] In some implementations, the final output of the self-attention sublayer is represented as:

[0126] in, This represents the output of the nth head, derived from the scaling dot product attention calculation formula; n represents the total number of heads. represents the learnable output weight matrix; S represents the final output of the self-attention sublayer.

[0127] In some implementations, the output of residual connections and layer normalization after the self-attention sublayer is represented as:

[0128] in, This represents the input sequence of the i-th sample, which originates from the output of the position coding unit; LN() represents the self-attention output of the i-th sample, which comes from the output of the multi-head self-attention sub-layer; LN() represents the layer normalization operation. This represents the output of the residual connection and layer normalization of the i-th sample.

[0129] In some implementations, the layer normalization operation is calculated as follows:

[0130] Where t represents the input vector, which comes from the output of the residual connection; ave(t) represents the mean of the input vector t; This represents the variance of the input vector t; This represents a minimal constant to prevent the denominator from being zero.

[0131] In some implementations, the output of residual connections and layer normalization after the location feedforward network sublayer is represented as:

[0132] in, This represents the position of the i-th sample and is the output of the feedforward network. This represents the final output of the i-th sample.

[0133] The above is an illustrative scheme of an ATE wafer pattern defect identification system based on a Transformer network according to this embodiment. It should be noted that the technical solution of this ATE wafer pattern defect identification system based on a Transformer network belongs to the same concept as the technical solution of the ATE wafer pattern defect identification method based on a Transformer network described above. Details not described in detail in the technical solution of the ATE wafer pattern defect identification system based on a Transformer network can be found in the description of the technical solution of the ATE wafer pattern defect identification method based on a Transformer network described above.

[0134] Figure 6 A structural block diagram of a computing device 600 according to some embodiments of this specification is shown. The components of the computing device 600 include, but are not limited to, a memory 601 and a processor 602. The processor 602 is connected to the memory 601 via a bus 603, and a database 605 is used to store data.

[0135] The computing device 600 also includes an access device 604 that enables the computing device 600 to communicate via one or more networks 606. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 604 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0136] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0137] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0138] The processor 602 executes the following computer-executable instructions, which, when executed by the processor, implement the steps of the ATE wafer pattern defect identification method based on Transformer networks described above. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the ATE wafer pattern defect identification method based on Transformer networks described above belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the ATE wafer pattern defect identification method based on Transformer networks described above.

[0139] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described ATE wafer image defect identification method based on Transformer networks.

[0140] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the ATE wafer image defect identification method based on Transformer network described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the ATE wafer image defect identification method based on Transformer network described above.

[0141] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is instructed to perform the steps of the above-described ATE wafer image defect identification method based on Transformer networks.

[0142] The above is an illustrative example of a computer program in this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the ATE wafer image defect identification method based on Transformer networks described above. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the ATE wafer image defect identification method based on Transformer networks described above.

[0143] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0144] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0145] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0146] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0147] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this invention. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for defect identification in ATE wafer images based on Transformer networks, characterized in that, include: The raw wafer image data is converted into a standardized tensor format through data preprocessing, and a classification label and a segmentation label are constructed for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, used to mark the defect location. The normalized tensor is processed by a hybrid network model, including: extracting multi-scale local features through a shared encoder, establishing long-distance dependencies and outputting global feature representations through a global context modeling module, outputting defect category probabilities based on the global feature representations through a classification task branch, and reconstructing pixel-level segmentation masks based on the global feature representations through a segmentation task branch. The parameters of the hybrid network model are optimized by using an end-to-end joint training method and a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

2. The method according to claim 1, characterized in that, The extraction of multi-scale local features via a shared encoder includes: The input tensor is processed by multiple cascaded convolutional modules, each of which includes a standard convolutional layer, a batch normalization layer, a ReLU activation function, and a pooling layer. The input feature map is weighted and optimized using a pixel-level attention module to generate an optimized feature map.

3. The method according to claim 2, characterized in that, The step of weighted optimization of the input feature map through a pixel-level attention module includes: The input feature map is compressed using the first convolutional layer; Expand the effective receptive field and capture medium- to long-range spatial context information by using hollow convolutional layers; The number of channels is restored to the original dimension through the second convolutional layer; Pixel-level weight matrices are generated using the Sigmoid activation function; The pixel-level weight matrix is ​​element-wise multiplied with the original input feature map to achieve feature recalibration. The recalibrated features are added to the original input features through residual connections to form the final output.

4. The method according to claim 1, characterized in that, The establishment of long-distance dependencies through the global context modeling module includes: The two-dimensional feature map is flattened and segmented into a one-dimensional image patch sequence by the image patch sequence generation unit, and a learnable category token is concatenated at the beginning of the sequence. A learnable positional code is added to each element of the image patch sequence using a positional coding unit; The sequence after position encoding is processed by a Transformer encoder, which consists of multiple identical layers stacked together. Each layer includes a multi-head self-attention sublayer and a position feedforward network sublayer. Residual connections and layer normalization are applied after each sublayer.

5. The method according to claim 4, characterized in that, The sequence processed by the multi-head self-attention sublayer includes: The input sequence is linearly projected into multiple independent sets of query vectors, key vectors, and value vectors; The output of each head is calculated by scaling the dot product attention, and the corresponding first calculation formula includes: Where Q represents the query matrix, derived from a linear projection of the input sequence; K represents the key matrix, derived from a linear projection of the input sequence; V represents the value matrix, derived from a linear projection of the input sequence; d k The dimension of the key vector is represented by T; T represents the matrix transpose operation. The outputs of all heads are concatenated and then linearly projected to obtain the final output of the self-attention sublayer.

6. The method according to claim 5, characterized in that, The sequence processed through the position feedforward network sublayer includes: Two linear transformation layers and the GeLU activation function are applied independently to each position in the sequence; The second calculation formula in the output is: in, The input at position i represents the result of the output of the multi-head self-attention sub-layer after residual connection and layer normalization; Linear() represents linear transformation; GeLU() represents Gaussian error linear unit activation function; This represents the output at the i-th position.

7. The method according to claim 6, characterized in that, The GeLU activation function takes the following form: Where x represents the input value, which comes from the output of the first linear transformation layer in the position feedforward network sublayer.

8. The method according to claim 4, characterized in that, The probability of the defect category output through the classification task branch includes: Extract the feature vector corresponding to the category token output by the Transformer encoder; The feature vector is processed by a fully connected layer and a Softmax activation function to output the probability of various defects.

9. The method according to claim 4, characterized in that, The process of reconstructing the pixel-level segmentation mask by segmenting task branches includes: The global feature representation is upsampled using a decoder; Multi-scale features are fused using skip connections with the shared encoder; A high-resolution pixel-level segmentation mask is reconstructed.

10. A defect identification system for ATE wafers based on Transformer networks, characterized in that, include: The data preprocessing module is used to convert the raw wafer image data into a standardized tensor format and to construct a classification label and a segmentation label for each wafer image. The classification label represents the macroscopic defect category of the wafer image, and the segmentation label is a binary mask with the same size as the wafer image, used to mark the defect location. The hybrid network model includes a shared encoder, a global context modeling module, a classification task branch, and a segmentation task branch. The shared encoder receives the normalized tensor output by the data preprocessing module and extracts multi-scale local features. The global context modeling module receives the multi-scale local features output by the shared encoder, establishes long-distance dependencies, and outputs a global feature representation. The classification task branch outputs the defect category probability based on the global feature representation, and the segmentation task branch reconstructs a pixel-level segmentation mask based on the global feature representation. The model training module is used to optimize the parameters of the hybrid network model through an end-to-end joint training method and a total loss function. The total loss function is composed of a weighted average of segmentation loss and classification loss, and the classification loss includes a threshold loss function with a learnable classification threshold.

Images