Quantitative model construction method and device for semiconductor chip defects, equipment and medium

Abstract

The invention discloses a quantitative model construction method and device for semiconductor chip defects, equipment and a medium, and the method comprises the steps: carrying out the preprocessing and marking of an obtained microscopic image, and obtaining training data; constructing a multi-task network model comprising a shared encoder, a segmentation decoder and a regression branch; constructing a loss function including a segmentation task and a regression task; and inputting the training data into the multi-task network model, and training by adopting a preset training strategy based on the loss function to obtain a quantitative model. According to the method and the device, a quantitative model for synchronously optimizing defect instance segmentation and physical parameter regression can be provided, so that the efficiency and the precision of defect quantitative calculation are improved.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor manufacturing detection technology, and in particular to a method, device, equipment and medium for constructing a quantitative model of semiconductor chip defects. Background Art

[0002] In the semiconductor manufacturing process, the detection and quantitative analysis of chip surface or internal defects (such as cracks, holes, metal residues, etc.) are crucial to product quality control.

[0003] Traditional inspection methods often rely on manual visual inspection or single image processing algorithms, resulting in low efficiency, high missed detection rates, and an inability to quantify physical parameters of defects. While deep learning technology has been applied to defect detection in recent years, existing models typically separate defect segmentation and parameter prediction into independent tasks, leading to computational redundancy and low accuracy in the inspection process. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a method, device, equipment and medium for constructing a quantization model of semiconductor chip defects, providing a quantization model that simultaneously optimizes defect instance segmentation and physical parameter regression, thereby improving the efficiency and accuracy of defect quantification calculation.

[0005] An embodiment of the present invention provides a method for constructing a quantitative model of semiconductor chip defects, the method comprising:

[0006] Preprocess and label the acquired microscopic images to obtain training data;

[0007] Build a multi-task network model that includes a shared encoder, a segmentation decoder, and a regression branch;

[0008] Construct a loss function that includes segmentation tasks and regression tasks;

[0009] The training data is input into the multi-task network model, and training is performed using a preset training strategy based on the loss function to obtain a quantized model.

[0010] Preferably, the method further comprises:

[0011] Input the acquired test data into the quantization model, and output the instance segmentation results obtained by the segmentation decoder in the quantization model and the defect physical parameter prediction values obtained by the regression branch;

[0012] The instance segmentation results and the predicted values of the defect physical parameters are compared with the corresponding label data in the test data, and a preset post-processing algorithm is used to optimize the segmentation boundaries of the segmentation decoder in the quantization model, and a quantization report is generated.

[0013] Preferably, the preprocessing and labeling of the acquired microscopic images to obtain training data includes:

[0014] Scale the acquired microscopic image to a preset size and perform normalization;

[0015] Label the standard defect instance mask and the corresponding defect physical parameter of the processed image;

[0016] Divide the standardized image according to the preset ratio to determine the training data and test data;

[0017] The defect physical parameter labels include area, depth and conductivity offset.

[0018] Preferably, the construction includes a multi-task network model comprising a shared encoder, a segmentation decoder, and a regression branch, including:

[0019] ResNet-50 is used as the backbone network for extracting multi-scale features to construct the shared encoder;

[0020] The segmentation encoder is constructed by outputting instance segmentation results based on a feature pyramid network and a mask prediction head;

[0021] A global average pooling layer and a fully connected layer are connected after the high-level features of the shared encoder to perform defect quantification as regression branches, output the defect physical parameter prediction results, and complete the construction of the multi-task network model.

[0022] Preferably, the construction includes loss functions for the segmentation task and the regression task, including:

[0023] Determine the loss of the segmentation task of the segmentation decoder by jointly optimizing the Dice loss and the cross entropy loss;

[0024] The prediction results of the regression branch are weighted summed using mean square error loss to obtain the loss of the regression task;

[0025] The loss function is obtained by taking the weighted sum of the loss of the segmentation task and the loss of the regression task as the total loss.

[0026] Preferably, the training strategy includes using an Adam optimizer, a preset initial learning rate, and a cosine annealing schedule.

[0027] Preferably, the method further comprises:

[0028] Obtain a microscopic image of the chip to be tested and perform preprocessing;

[0029] The preprocessed image is input into the quantization model, and the instance segmentation results obtained by the segmentation decoder in the output model and the defect physical parameter prediction results obtained by the regression branch are output.

[0030] An embodiment of the present invention further provides a device for constructing a quantitative model of semiconductor chip defects, the device comprising:

[0031] A preprocessing module is used to preprocess and annotate the acquired microscopic images to obtain training data;

[0032] Model building module, used to build a multi-task network model containing a shared encoder, a segmentation decoder, and a regression branch;

[0033] Loss determination module, used to construct loss functions including segmentation tasks and regression tasks;

[0034] The training module is used to input the training data into the multi-task network model, and perform training based on the loss function using a preset training strategy to obtain a quantized model.

[0035] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for constructing a quantitative model of semiconductor chip defects as described in any one of the above embodiments.

[0036] An embodiment of the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method for constructing a quantitative model of semiconductor chip defects as described in any one of the above embodiments.

[0037] The present invention provides a method, apparatus, device, and medium for constructing a quantization model for semiconductor chip defects. The method preprocesses and annotates captured microscopic images to obtain training data; constructs a multi-task network model comprising a shared encoder, a segmentation decoder, and a regression branch; constructs a loss function for both the segmentation task and the regression task; inputs the training data into the multi-task network model, and trains using a preset training strategy based on the loss function to obtain a quantization model. This application can provide a quantization model that simultaneously optimizes defect instance segmentation and physical parameter regression, thereby improving the efficiency and accuracy of defect quantization calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a method for constructing a quantitative model of semiconductor chip defects provided by an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the principle of a multi-task network model provided by an embodiment of the present invention;

[0040] Figure 3 1 is a schematic structural diagram of a device for constructing a quantitative model of semiconductor chip defects provided by an embodiment of the present invention;

[0041] Figure 4 It is a structural diagram of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] The embodiment of the present invention provides a method for constructing a quantitative model of semiconductor chip defects. Figure 1 , is a flow chart of a method for constructing a quantitative model of semiconductor chip defects provided by an embodiment of the present invention, wherein steps S1 to S4 of the method are as follows:

[0044] S1, preprocessing and labeling the acquired microscopic images to obtain training data;

[0045] S2, build a multi-task network model including a shared encoder, a segmentation decoder, and a regression branch;

[0046] S3, constructs a loss function including segmentation and regression tasks;

[0047] S4: input the training data into the multi-task network model, and perform training based on the loss function using a preset training strategy to obtain a quantized model.

[0048] In the specific implementation of this embodiment, the acquired microscopic image is preprocessed. Exemplarily, the preprocessing process includes operations such as noise reduction and contrast enhancement;

[0049] During labeling, regions of interest such as defects in the image are segmented and labeled, and relevant quantitative parameters are also labeled to obtain a data set for training, providing high-quality data for subsequent model training and ensuring that the model can learn accurate features and patterns.

[0050] It should be noted that the training data may include a training set and a validation set.

[0051] A multi-task network model is constructed, consisting of a shared encoder, a segmentation decoder, and a regression branch. The shared encoder extracts common features from the image, which are then shared and utilized by the segmentation decoder and regression branches. The segmentation decoder performs defect segmentation, identifying defective areas in the image. The regression branch performs parameter prediction, predicting defect-related parameters (such as size and location). This structural design allows for simultaneous segmentation and regression tasks within a single network, reducing model redundancy.

[0052] Construct a loss function that encompasses both the segmentation and regression tasks. By properly designing the loss function, you can optimize the results of both the segmentation and regression tasks simultaneously, allowing the model to achieve good performance on both tasks. For example, you can combine the loss of the segmentation task (such as cross entropy loss) and the loss of the regression task (such as mean squared error loss) with certain weights to form the final loss function.

[0053] The preprocessed and annotated training data is fed into the multi-task network model. Based on the constructed loss function, training is performed using a preset training strategy (such as stochastic gradient descent, Adam optimizer, etc.). By continuously adjusting the model parameters, the loss function is minimized, ultimately resulting in a quantized model.

[0054] Existing methods separate defect segmentation and parameter prediction into independent tasks, which means that when processing images, each task requires separate feature extraction and other operations, resulting in a waste of computing resources. However, this solution constructs a multi-task network model, and the shared encoder can extract common features, which are shared by the segmentation and decoding tasks, reducing repeated feature extraction and other computational processes, and effectively solving the problem of computational redundancy. By completing the defect segmentation and parameter prediction tasks simultaneously in one network model, there is no need to train two independent models to handle different tasks separately, thus achieving end-to-end efficient analysis. From inputting microscopic images to outputting segmentation results and parameter prediction results, the entire process can be completed in one model, which improves the efficiency and convenience of analysis. By constructing a loss function that includes segmentation tasks and regression tasks, and optimizing both tasks at the same time, the model can learn more accurate features and patterns in both segmentation and regression, thereby improving the performance of the model in defect detection and parameter prediction, and improving computational accuracy and prediction accuracy.

[0055] In another embodiment of the present invention, the method further includes:

[0056] Input the acquired test data into the quantization model, and output the instance segmentation results obtained by the segmentation decoder in the quantization model and the defect physical parameter prediction values obtained by the regression branch;

[0057] The instance segmentation results and the predicted values of the defect physical parameters are compared with the corresponding label data in the test data, and a preset post-processing algorithm is used to optimize the segmentation boundaries of the segmentation decoder in the quantization model, and a quantization report is generated.

[0058] During the specific implementation of this embodiment, the test set images are input into the trained quantization model.

[0059] It should be noted that the microscopic images used for testing are collected and organized to ensure that the images in the test set are representative and can cover defects or targets of different types and characteristics.

[0060] The quantized model is a trained multi-task network model that contains a shared encoder, a segmentation decoder, and a regression branch. It can simultaneously process image instance segmentation tasks and physical parameter prediction tasks.

[0061] The model processes the input test images and inputs the test set images into the model one by one. The model performs feature extraction, segmentation, and decoding on the images according to its network structure, and simultaneously outputs instance segmentation results (i.e., identifying the segmented areas of each defect or target in the image) and physical parameter prediction values (such as the size, position, angle, and other related physical parameters of the defect).

[0062] Apply post-processing algorithms such as non-maximum suppression to the instance segmentation results output by the model. Specifically, for each segmented region, calculate its confidence score, compare the confidence scores of adjacent segmented regions, suppress overlapping regions with low confidence, and retain the segmentation boundaries with the highest confidence. Post-processing algorithms such as non-maximum suppression are used to optimize the instance segmentation results. Non-maximum suppression can remove overlapping segmented regions and retain the segmentation boundaries with the highest confidence, making the segmentation results more accurate and clear, and the boundaries more reasonable.

[0063] Generate a quantitative report based on the optimized segmentation results and predicted physical parameters. This report can include detailed information such as the number, size, location, and type of defects, as well as a quantitative analysis of this information, providing a basis for subsequent decision-making and evaluation.

[0064] Post-processing algorithms such as non-maximum suppression effectively optimize segmentation boundaries, reduce overlap and inaccuracies in segmented regions, and make segmentation results more realistic, improving segmentation quality and accuracy. The generated quantitative report provides detailed defect information and quantitative analysis results, providing users with intuitive and accurate data support, facilitating decision-making and evaluation. For example, in areas such as quality control and fault diagnosis, timely measures can be taken based on the report results.

[0065] In another embodiment provided by the present invention, the above step S1 specifically includes:

[0066] Scale the acquired microscopic image to a preset size and perform normalization;

[0067] Label the standard defect instance mask and the corresponding defect physical parameter of the processed image;

[0068] Divide the standardized image into a preset proportion to determine the training data and test data;

[0069] The defect physical parameter labels include area, depth and conductivity offset.

[0070] During the specific implementation of this embodiment, a microscopic image of the chip is obtained.

[0071] Since the original images of semiconductor chips may vary in size, all images need to be scaled to 512×512 pixels for ease of subsequent model training and processing. This step is achieved using image scaling algorithms, such as bilinear interpolation and bicubic interpolation. For example, bilinear interpolation calculates the grayscale value of each pixel in the scaled image by taking a weighted average of the grayscale values of adjacent pixels in the original image. This resizes the image to the specified size of 512×512 pixels while minimizing image quality.

[0072] After scaling the image to a uniform size, normalize it. The goal of normalization is to adjust the image's pixel values to a fixed range, typically [0, 1] or [-1, 1]. A common normalization method is linear normalization. Mapping pixel values to the [0, 1] range helps improve model training efficiency and stability, making the model more consistent when processing different images.

[0073] The scaled and normalized images are divided into training, validation, and test sets. The ratio of these divisions can usually be adjusted based on the specific situation. For example, a common ratio is 70% for training, 15% for validation, and 15% for testing. The training set is used to train the model, allowing it to learn the features and patterns in the images; the validation set is used to evaluate the model's performance during training and adjust the model's hyperparameters (such as the learning rate and number of iterations) to prevent overfitting. The test set is used to ultimately evaluate the trained model's ability to generalize to unseen data, thereby obtaining the model's true performance metrics.

[0074] Each semiconductor chip microscopic image is annotated with a defect mask and corresponding physical parameter labels (area, depth, and conductivity offset). Specifically, the defect depth is measured using a confocal microscope, and the resistivity of the defect area is measured using a four-probe method using electrical testing equipment. The conductivity offset is calculated based on the geometric dimensions.

[0075] When annotating defect instance masks, use specialized image annotation tools (such as LabelMe, CVAT, etc.) to manually or semi-automatically outline the defect area in the image and generate a binary mask image, where the defect area is 1 and the background area is 0.

[0076] When labeling physical parameters, it's necessary to determine the area, depth, and conductivity offset corresponding to each defect instance based on actual measurement or analysis results, and then associate these parameters with the corresponding defect instance. For example, a microscope can be used to measure the defect depth, an image analysis algorithm can be used to calculate the defect area, and electrical testing equipment can be used to measure the conductivity offset.

[0077] Preprocessing operations provide high-quality, uniformly formatted, and accurately annotated datasets, facilitating accurate defect detection and physical parameter prediction.

[0078] In another embodiment provided by the present invention, the above step S2 specifically includes the following steps:

[0079] ResNet-50 is used as the backbone network for extracting multi-scale features to construct the shared encoder;

[0080] The segmentation encoder is constructed by outputting instance segmentation results based on a feature pyramid network and a mask prediction head;

[0081] A global average pooling layer and a fully connected layer are connected after the high-level features of the shared encoder to perform defect quantification as regression branches, output the defect physical parameter prediction results, and complete the construction of the multi-task network model.

[0082] In the specific implementation of this embodiment, the multi-task network model includes three main parts: a shared encoder, a segmentation decoder, and a regression branch. Different tasks are processed simultaneously in one network. The features extracted by the shared encoder can be shared by the segmentation and decoding tasks, thereby reducing computational redundancy and improving the efficiency and performance of the model.

[0083] The shared encoder uses ResNet-50 as the backbone network to extract multi-scale features. Specifically:

[0084] ResNet (residual network) is a classic deep convolutional neural network architecture. ResNet-50 indicates that the network contains 50 layers. Its core feature is the introduction of residual connections, which solves the gradient vanishing and gradient exploding problems that occur in deep neural networks as the number of network layers increases, allowing the network to learn features at a deeper level. In this solution, ResNet-50 serves as the backbone network of the shared encoder, performing layer-by-layer convolution operations on the input semiconductor chip microscopic image to continuously extract image features.

[0085] ResNet-50 is used as the backbone network for extracting multi-scale features. During the forward propagation of the ResNet-50 network, different layers output feature maps with different receptive fields and semantic levels. Feature maps contain information about the image at different scales. For example, shallow feature maps may contain more detailed information (such as edges and textures), while deep feature maps have higher semantic information (such as object categories). The shared encoder obtains multi-scale features from the outputs of these different layers, providing rich feature representations for subsequent segmentation and decoding tasks.

[0086] When building the segmentation decoder, the defect instance segmentation results are output based on the Feature Pyramid Network (FPN) and the mask prediction head. Specifically:

[0087] The main function of the Feature Pyramid Network (FPN) is to fuse the multi-scale features extracted by the shared encoder to construct a feature pyramid with different scales. It combines high-level semantic information with low-level detail information through a top-down and lateral connection approach. Specifically, the high-level semantic feature maps are upsampled and fused (usually by addition or concatenation) with the corresponding low-level feature maps, so that the feature maps at each scale contain rich semantic and detail information. This allows FPN to detect and segment objects in images at different scales, improving segmentation accuracy and robustness.

[0088] Based on the feature map fused by FPN, the mask prediction head further classifies each pixel to determine whether it belongs to a defect instance. Through a series of convolution operations and activation functions, it outputs a binary mask of the same size as the input image, where 1 represents the defect area and 0 represents the background area, thus achieving defect instance segmentation.

[0089] The regression branch uses a global average pooling layer and a fully connected layer after the encoder's high-level features to simultaneously predict the defect area, depth, and electrical parameters. Specifically:

[0090] Starting from the high-level feature maps of the shared encoder (which already contain high-level semantic information and an abstract representation of the entire image), a global average pooling layer is applied. This layer performs a global average operation on each feature map, compressing each feature map into a single value. This converts the spatial information of the high-level feature map into a fixed-length feature vector, reducing the number of parameters while preserving the key information of the feature.

[0091] Based on the feature vector output by the global average pooling layer, multiple fully connected layers are connected. By performing linear transformations and activation operations on the input feature vector, the fully connected layers learn the mapping relationship between the features and the defect area, depth, and electrical parameters. Ultimately, the fully connected layers output the predicted defect area, depth, and electrical parameter values, achieving simultaneous prediction of these physical parameters.

[0092] In another embodiment of the present invention, the step S3 includes the following steps:

[0093] Determine the loss of the segmentation task of the segmentation decoder by jointly optimizing the Dice loss and the cross entropy loss;

[0094] The prediction results of the regression branch are weighted summed using mean square error loss to obtain the loss of the regression task;

[0095] The loss function is obtained by taking the weighted sum of the loss of the segmentation task and the loss of the regression task as the total loss.

[0096] In the specific implementation of this embodiment, the Dice coefficient is used to measure the similarity between two sets. In image segmentation, it can measure the degree of overlap between the predicted segmentation mask and the true mask. For each input training image, the defect instance segmentation mask predicted by the model and the corresponding true mask are substituted into the above formula to calculate the Dice loss. The cross entropy loss is used to measure the difference between the predicted probability distribution and the true distribution. In the segmentation task, the predicted probability of each pixel belonging to the defect category is usually compared with the true label (0 or 1). In actual operation, the cross entropy loss of each training image is calculated based on the probability value of each pixel output by the model belonging to the defect category and the true mask label.

[0097] Combining Dice loss and cross entropy loss, we get the loss function for the segmentation task:

[0098] Joint loss L seg =α×L Dice +(1-α)×L CE ;

[0099] Among them, L Dice is the Dice loss, L CEis the cross entropy loss, and α is a hyperparameter used to balance the weights of the two losses, usually ranging from [0, 1]. During training, the backpropagation algorithm is used to update the parameters of the segmentation decoder based on the joint loss, so that the predicted segmentation mask is closer to the true mask.

[0100] The mean squared error loss is used to measure the average squared error between the predicted value and the true value. For the defect area, depth, and electrical parameters predicted in the regression task, their mean squared error losses are calculated separately.

[0101] The regression loss function is derived by weighting the mean squared error losses of area, depth, and electrical parameters. Different weight coefficients are used to adjust the importance of the losses of different parameters. Backpropagation is used to update the parameters of the regression branch based on the total weight, bringing the predicted physical parameters closer to the true values.

[0102] The segmentation task loss and the regression task loss are weighted together to obtain the total loss function, and the weights are determined through grid search.

[0103] In another embodiment provided by the present invention, the training strategy includes using an Adam optimizer, a preset initial learning rate, and a cosine annealing schedule.

[0104] Specifically, the Adam optimizer is used for training. Adam optimizer is an optimization algorithm widely used in deep learning training. It combines the advantages of the adaptive gradient algorithm (Adagrad) and the root mean square propagation algorithm (RMSProp) to adaptively adjust the learning rate of each parameter.

[0105] In this solution, the Adam optimizer is selected to update the parameters of the multi-task network model (including the shared encoder, segmentation decoder, and regression branch). Specifically, the Adam optimizer calculates the gradient of each parameter in each training iteration and adjusts the parameter update step size based on the first-order moment estimate (i.e., the average gradient) and the second-order moment estimate (i.e., the average of the square of the gradient) of the gradient. The core idea is to calculate appropriate learning rates for different parameters separately, so that the model can converge to a better solution faster during training. In this way, for parameters of different properties in the model (such as parameters of different layers in the shared encoder, or parameters in the segmentation decoder and regression branch), the Adam optimizer can dynamically adjust the learning rate according to the changes in their gradients, thereby improving the efficiency and stability of training.

[0106] The initial learning rate is set to 1e-4 (i.e., 0.0001). The learning rate is a critical hyperparameter in optimization algorithms, determining the step size of each parameter update. A suitable initial learning rate is crucial for model training. If the learning rate is set too high, the parameter update step size will be too large, potentially preventing the model from converging during training or even causing parameter explosion. If the learning rate is set too low, the parameter update step size will be too small, resulting in very slow model convergence and requiring more training time and computing resources. In this solution, after extensive experimentation and empirical selection, the initial learning rate is set to 1e-4. This allows the model to update parameters at an appropriate rate at the beginning of training while ensuring model stability, gradually learning the features in the image and task-related patterns.

[0107] Cosine annealing scheduling is a learning rate adjustment strategy. Its basic idea is to dynamically adjust the learning rate based on the shape of the cosine function. In the early stages of training, the learning rate is maintained at a high level, allowing the model to quickly explore the parameter space and find possible optimal solutions. As training progresses, the learning rate gradually decreases, similar to the annealing process, allowing the model to make more refined adjustments near the optimal solution and avoid missing the optimal solution. Specifically, in each training epoch, the current learning rate is calculated according to the formula of the cosine function. As training epochs increase, the learning rate starts at an initial value of 1e-4 and gradually decreases according to the law of the cosine function until it reaches a pre-set minimum value (usually non-zero). After the learning rate reaches the minimum value, a new cosine annealing cycle can be restarted, causing the learning rate to increase and then decrease again. This can help the model escape the local optimal solution and further improve its performance and generalization ability. By using cosine annealing scheduling, the learning rate can be dynamically adjusted during training to meet the needs of the model at different stages, improving training effectiveness and final model performance.

[0108] The training strategy provided in this application provides an effective parameter update and learning rate adjustment mechanism for the training of multi-task network models by reasonably selecting the Adam optimizer, setting the initial learning rate, and adopting cosine annealing scheduling, which helps the model achieve better training results and performance in segmentation and regression tasks.

[0109] In another embodiment of the present invention, the method further includes:

[0110] Obtain a microscopic image of the chip to be tested and perform preprocessing;

[0111] The preprocessed image is input into the quantization model, and the instance segmentation results obtained by the segmentation decoder in the output model and the defect physical parameter prediction results obtained by the regression branch are output.

[0112] In the specific implementation of this embodiment, a specialized microscopic imaging device (such as an electron microscope or optical microscope) is used to image the semiconductor chip to be inspected, obtaining microscopic images of the chip surface or internal structure. These images contain various detailed information about the chip, including possible defects, flaws, or other features that require detection and analysis.

[0113] Preprocessing: A series of preprocessing operations are performed on the acquired microscopic images to improve image quality and make them more suitable for input into the quantitative model. Preprocessing operations include:

[0114] The images are uniformly scaled to the same size as during training.

[0115] Normalize the pixel values of the image and adjust the pixel value range to a fixed interval.

[0116] Perform noise reduction. Common noise reduction methods include Gaussian filtering and median filtering. Gaussian filtering removes Gaussian noise by taking a weighted average of image pixels, while median filtering replaces the pixel value with the median value of the pixel's neighborhood, effectively removing noise.

[0117] The preprocessed microscopic image of the chip to be inspected is fed into a trained and quantized multi-task network model. This model consists of three main components: a shared encoder, a segmentation decoder, and a regression branch. The shared encoder first performs feature extraction on the input image, extracting multi-scale feature information.

[0118] The segmentation decoder uses the features extracted by the shared encoder to segment defect instances in the image through a feature pyramid network (FPN) and a mask prediction head. The final output is a binary mask image of the same size as the input image, where defect areas are marked as 1 and background areas are marked as 0. This results in instance segmentation of defects in the chip image, clearly defining the location and shape of the defects in the image.

[0119] The regression branch uses the high-level features of the shared encoder, processed by a global average pooling layer and a fully connected layer, to predict the physical parameters of the defect (such as area, depth, and conductivity offset). The predicted physical parameter values of the defect are ultimately output, providing quantitative metrics for further analysis and evaluation of chip quality and performance.

[0120] By effectively processing and analyzing the microscopic images of the semiconductor chip to be inspected, and using the trained quantification model to simultaneously output the instance segmentation results of the defects and the physical parameter prediction results, accurate detection and quantification of chip defects can be achieved, providing accurate evaluation for chip quality control, fault diagnosis, etc.

[0121] In another embodiment provided by the present invention, taking a semiconductor manufacturer's wafer defect dataset as an example, it includes 5,000 512×512 SEM images, and the defect types include metal residues, cracks and holes. Each image is annotated with an instance mask and physical parameters.

[0122] The dataset was divided into training set, validation set and test set according to the ratio of 8:1:1. The bicubic interpolation method was used to unify the image size and perform normalization (mean = 0.5, standard deviation = 0.2).

[0123] Build a multi-task network model, see Figure 2 , which is a schematic diagram of the principle of the multi-task network model provided by an embodiment of the present invention. The encoder uses a pre-trained ResNet-50 to perform feature extraction and obtain Stage 1 to Stage 4 features respectively;

[0124] The extracted features are input into the segmentation decoder and regression branch respectively.

[0125] The segmentation decoder adopts the FPN structure and obtains a 512×512 mask prediction output.

[0126] The regression branch is connected to the SE attention module and the fully connected layer after the Stage 4 feature of ResNet-50, that is, after GAP and FCs, the regression output is obtained.

[0127] Preferably, during training, the segmentation loss weight is set to 0.7, the regression loss weight is set to 0.3, the batch size is set to 8, and the iterations are 100 cycles.

[0128] The model obtained in this application is used to quantify defects and compared with different schemes to obtain performance comparison results, see Table 1.

[0129] Table 1 Performance comparison table

[0130] method Segmentation IoU (%) Area error (%) Depth error (%) This program 92.7 3.2 4.8 U-Net+Independent Regression 89.1 6.5 7.9 Mask R-CNN + regression 90.4 5.1 6.3

[0131] Experiments show that the proposed method achieves an IoU of 92.7% in the defect segmentation task, and the average relative error of parameter regression is less than 5%, which is better than the separately trained U-Net (IoU 89.1%) and ResNet regression models (error 8.3%).

[0132] This application solution uses a multi-task learning framework and attention mechanism to simultaneously optimize segmentation and parameter regression tasks, reduce computational redundancy, and significantly improve the efficiency and accuracy of semiconductor defect detection under the conditions of segmentation IoU reaching 92.7% and parameter error less than 5%.

[0133] Another embodiment of the present invention provides a device for constructing a quantitative model of semiconductor chip defects, see Figure 3, is a schematic structural diagram of a device for constructing a quantitative model of semiconductor chip defects provided by an embodiment of the present invention, the device comprising:

[0134] A preprocessing module is used to preprocess and annotate the acquired microscopic images to obtain training data;

[0135] Model building module, used to build a multi-task network model containing a shared encoder, a segmentation decoder, and a regression branch;

[0136] Loss determination module, used to construct loss functions including segmentation tasks and regression tasks;

[0137] The training module is used to input the training data into the multi-task network model, and perform training based on the loss function using a preset training strategy to obtain a quantized model.

[0138] It should be noted that the device for constructing a quantitative model of semiconductor chip defects provided in an embodiment of the present invention is used to execute all the process steps of the method for constructing a quantitative model of semiconductor chip defects in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0139] See also Figure 4 , is a schematic diagram of the structure of a terminal device provided by an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a program for constructing a quantitative model of semiconductor chip defects. When the processor executes the computer program, the steps of the above-mentioned embodiments of the method for constructing a quantitative model of semiconductor chip defects are implemented, such as Figure 1 Alternatively, the processor implements the functions of the modules in the above-mentioned device embodiments when executing the computer program.

[0140] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device. For example, the computer program may be divided into various modules, and the specific functions of each module are not described in detail here.

[0141] The terminal device may be a computing device such as a desktop computer, laptop, PDA, or cloud server. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a terminal device and does not limit the terminal device. The terminal device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0142] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0143] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0144] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0145] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0146] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for constructing a quantitative model of semiconductor chip defects, characterized in that: The method comprises: Preprocess and label the acquired microscopic images to obtain training data; Build a multi-task network model that includes a shared encoder, a segmentation decoder, and a regression branch; Construct a loss function that includes segmentation tasks and regression tasks; The training data is input into the multi-task network model, and training is performed using a preset training strategy based on the loss function to obtain a quantized model.

2. The method for constructing a quantitative model of semiconductor chip defects according to claim 1, wherein: The method further comprises: Input the acquired test data into the quantization model, and output the instance segmentation results obtained by the segmentation decoder in the quantization model and the defect physical parameter prediction values obtained by the regression branch; The instance segmentation results and the predicted values of the defect physical parameters are compared with the corresponding label data in the test data, and a preset post-processing algorithm is used to optimize the segmentation boundaries of the segmentation decoder in the quantization model, and a quantization report is generated.

3. The method for constructing a quantitative model of semiconductor chip defects according to claim 1, wherein: The preprocessing and labeling of the acquired microscopic images to obtain training data includes: Scale the acquired microscopic image to a preset size and perform normalization; Label the standard defect instance mask and the corresponding defect physical parameter of the processed image; Divide the standardized image according to the preset ratio to determine the training data and test data; The defect physical parameter labels include area, depth and conductivity offset.

4. The method for constructing a quantitative model of semiconductor chip defects according to claim 1, wherein: The multi-task network model constructed with a shared encoder, a segmentation decoder, and a regression branch includes: ResNet-50 is used as the backbone network for extracting multi-scale features to construct the shared encoder; The segmentation encoder is constructed by outputting instance segmentation results based on a feature pyramid network and a mask prediction head; A global average pooling layer and a fully connected layer are connected after the high-level features of the shared encoder to perform defect quantification as regression branches, output the defect physical parameter prediction results, and complete the construction of the multi-task network model.

5. The method for constructing a quantitative model of semiconductor chip defects according to claim 1, wherein: The construction includes loss functions for segmentation and regression tasks, including: Determine the loss of the segmentation task of the segmentation decoder by jointly optimizing the Dice loss and the cross entropy loss; The prediction results of the regression branch are weighted summed using mean square error loss to obtain the loss of the regression task; The loss function is obtained by taking the weighted sum of the loss of the segmentation task and the loss of the regression task as the total loss.

6. The method for constructing a quantitative model of semiconductor chip defects according to claim 1, wherein: The training strategy includes using the Adam optimizer, a preset initial learning rate, and a cosine annealing schedule.

7. The method for constructing a quantitative model of semiconductor chip defects according to claim 1, wherein: The method further comprises: Obtain a microscopic image of the chip to be tested and perform preprocessing; The preprocessed image is input into the quantization model, and the instance segmentation results obtained by the segmentation decoder in the output model and the defect physical parameter prediction results obtained by the regression branch are output.

8. A device for constructing a quantitative model of semiconductor chip defects, characterized in that: The device comprises: A preprocessing module is used to preprocess and annotate the acquired microscopic images to obtain training data; Model building module, used to build a multi-task network model containing a shared encoder, a segmentation decoder, and a regression branch; Loss determination module, used to construct loss functions including segmentation tasks and regression tasks; The training module is used to input the training data into the multi-task network model, and perform training based on the loss function using a preset training strategy to obtain a quantized model.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for constructing a quantitative model of semiconductor chip defects according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for constructing a quantitative model of semiconductor chip defects according to any one of claims 1 to 7.

Images