A semiconductor manufacturing defect detection method and electronic device

Abstract

The application discloses a semiconductor manufacturing defect detection method, an electronic device and a storage medium, the method comprising: using a trained defect detection model to detect a scanning electron microscope (SEM) image to obtain a detection result; wherein the defect detection model is based on YOLOv8, and the overall loss function is a mixed loss function combining a classification loss and a regression loss; when the defect detection model performs feature extraction, frequency domain decoupling feature extraction is performed and spatial pixel-level feature fusion is performed, and a specially designed mixed loss function is combined to solve the problems of class imbalance and accurate positioning. Compared with the detection method of the prior art, the application has higher detection accuracy, stronger model robustness, and meets real-time requirements. While ensuring accuracy improvement, the inference speed that meets the production line beat is maintained, and finally, more robust and accurate automated detection of defects in semiconductor manufacturing, especially subtle and low-contrast defects, is realized.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology in semiconductor manufacturing processes, and more particularly to a semiconductor manufacturing defect detection method and electronic device. Background Technology

[0002] As semiconductor process technology nodes continue to shrink and device structures become increasingly complex and integrated, nanoscale defect detection in the manufacturing process has become a critical factor affecting product yield and reliability. Scanning electron microscopy, with its nanoscale resolution, has become the primary means of observing and identifying these defects. However, traditional detection methods relying on manual interpretation or based on classic image processing algorithms such as thresholding and morphology generally suffer from low detection efficiency, high subjectivity, and insufficient sensitivity to weak contrast and sub-resolution defects, making it difficult to meet the dual demands of modern mass production lines for both speed and accuracy.

[0003] In recent years, object detection models based on convolutional neural networks have provided a new paradigm for automated visual inspection. Nevertheless, directly applying a general object detection framework to semiconductor SEM image analysis still faces several unique challenges: First, defect targets are small, often at the pixel or sub-pixel level, and their morphology is highly variable; second, SEM images are affected by various factors such as electron beam scattering, sample charging, and noise, resulting in significant fluctuations in the signal-to-noise ratio; third, the distribution of defect categories is highly uneven, with critical defects such as random particles and bridging potentially appearing at extremely low frequencies. Furthermore, in actual semiconductor production lines, SEM images of different batches of wafers exhibit significant variations in background grayscale and contrast, which manifests as low-frequency background drift in the image frequency domain; while true nanoscale micro-defects (such as line collapses and microbridges) and shot noise from electron counting primarily manifest as high-frequency details. Existing general-purpose target detection models often extract high- and low-frequency features in a mixed manner. This leads to gradient oscillations caused by background drift or noise interference in extremely low-contrast environments, resulting in severe missed and false detections. Fourth, industrial environments require detection systems to meet real-time or near-real-time processing capabilities specified by production lines, in addition to extremely high accuracy. Therefore, how to apply the target detection framework to semiconductor manufacturing defect detection to meet the detection needs of this application scenario has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a semiconductor manufacturing defect detection method and electronic device to solve the problem of insufficient precision and accuracy in defect detection during the semiconductor manufacturing process in the prior art. Furthermore, the method of this invention makes it possible to apply the target detection framework in semiconductor manufacturing defect detection, thereby effectively improving the efficiency and accuracy of detection.

[0005] According to a first aspect of the present invention, a method for detecting defects in semiconductor manufacturing is provided, comprising: The image to be detected is input into a pre-trained defect detection model, and the defect detection result of the image to be detected is obtained based on the output result of the defect detection model. The image to be detected includes a SEM image. The defect detection model is a target detection model based on YOLOv8, which uses a hybrid loss function as the loss function. The hybrid loss function is constructed based on classification loss and regression loss.

[0006] The defect detection model employed in the semiconductor manufacturing defect detection method of this invention employs a hybrid loss function that combines classification loss and regression loss. The classification loss can alleviate the problem of class imbalance, while the regression loss can improve the bounding box regression accuracy by comprehensively considering overlapping areas, center point distance, and aspect ratio similarity. This enables the overall defect detection model based on the YOLOv8 target detection framework to effectively detect defects such as bridging, line collapse, microbridges, and gaps, demonstrating good adaptability and stability. Compared with existing solutions, this method maintains high inference efficiency while significantly improving detection accuracy, generalization ability, and deployment flexibility, realizing the effective application of the target detection framework in semiconductor manufacturing defect detection.

[0007] In some implementations, the dataset is constructed using the following methods: Collect a number of labeled defect types and their calibration ranges from different batches and under different noise conditions, covering the target defect types; Enhancement preprocessing of SEM images; The processed SEM images are divided into training set, validation set and independent test set according to a preset ratio.

[0008] This invention enhances the model's ability to extract edge features, improves the multi-scale feature fusion method, and designs a hybrid loss function for imbalanced data and precise localization, ultimately achieving more robust and accurate automated detection of defects in semiconductor manufacturing, especially subtle and low-contrast defects.

[0009] In some implementations, the enhanced preprocessing includes: Apply one or more combinations of random-triggered geometric transformations, physical model noise injection, and illumination and contrast perturbations to SEM images. in, The geometric transformations include one or more combinations of random rotation (±90°) and horizontal / vertical flipping; The physical model noise injection includes injecting Gaussian noise into the training set images with a 50% probability. The Gaussian noise has a mean μ=0 and a variance σ that is randomly sampled in the interval [0.01, 0.03]. At the same time, Poisson noise is introduced to simulate the shot noise effect of photon / electron counting. The illumination and contrast disturbances include using HSV color space transformation to randomly adjust brightness and contrast, with the adjustment factor range set to ±15%.

[0010] Therefore, this setting ensures maximum sample diversity and prevents overfitting.

[0011] In some implementations, the defect detection model includes the following features during feature extraction: The input feature map is separated into low-frequency feature maps and high-frequency feature maps; The high-frequency feature map is edge-processed to obtain an enhanced high-frequency feature map; The low-frequency feature map is calibrated to obtain a standard low-frequency feature map; The enhanced high-frequency feature map and the standard low-frequency feature map are spliced ​​together to obtain the output feature map.

[0012] In some implementations, separating the input feature map into low-frequency feature maps and high-frequency feature maps includes: Pooling is performed on the input feature map to obtain low-frequency feature maps; The low-frequency feature map is separated from the input feature map to obtain the high-frequency feature map; The edge-mapping process for the high-frequency feature map includes: Asymmetric strip convolution is applied to the high-frequency feature map to enhance its edges, resulting in an enhanced high-frequency feature map. The standard processing of the low-frequency feature map includes: The low-frequency feature map is processed sequentially by global pooling, multilayer sensing, and activation function to obtain the first weight coefficients. The standard low-frequency feature map is obtained by multiplying each element of the low-frequency feature map by the first weight coefficient. The process of stitching together the enhanced high-frequency feature map and the standard low-frequency feature map includes: The enhanced high-frequency feature map and the standard low-frequency feature map are concatenated, and a two-dimensional spatial weight matrix is ​​generated through a spatial attention network. This matrix is ​​then multiplied with the concatenated feature map and added to the original input feature map by residual addition to obtain the output feature map.

[0013] Therefore, by adopting this configuration, the defect detection model of this invention creatively introduces a frequency domain decoupling edge enhancement and background gray suppression module in the feature extraction stage to address the complex physical imaging interference unique to SEM images. Existing technologies extract features by mixing them in the spatial domain, which easily leads to aliasing of defect signals and noise. This module first decouples the image in the frequency domain: in the high-frequency branch, it utilizes the prior geometric features that defects such as semiconductor line collapse and microbridges are mostly long and thin strips, and uses 1×K and K×1 asymmetric strip convolutions to effectively filter out non-directional shot noise while enhancing the edges of long and thin defects; in the low-frequency branch, it designs an adaptive gating mechanism for background gray drift to address the background brightness drift caused by the charging effect in different batches of wafers, and calibrates the background through dynamic gating weights; finally, it introduces a spatial pixel-level attention mechanism to fuse high and low frequency reconstructed features. This design fundamentally improves the model's cross-batch noise resistance and spatial absolute positioning accuracy for low-contrast, nanoscale micro-defects.

[0014] According to a second aspect of the present invention, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method described in the first aspect above.

[0015] According to a third aspect of the present invention, a storage medium is provided having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0016] Compared with existing detection methods, this invention offers higher detection accuracy, stronger model robustness, and meets real-time requirements. It effectively enhances the model's ability to identify subtle and low-contrast defects through edge enhancement and feature fusion mechanisms, which is particularly important in SEM images with complex backgrounds. Furthermore, its data augmentation strategy combined with noise simulation and targeted loss function design enables the model to adapt to different SEM imaging conditions and noise levels. The defect detection model used in this invention is based on the efficient architecture of YOLOv8, maintaining an inference speed that meets production line cycle times while ensuring improved accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1This is an overall flowchart of a semiconductor manufacturing defect detection method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the method for constructing a dataset used when training a defect detection model in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. Figure 3 This is a schematic diagram of a SEM image showing the labeled defect type and its calibration range in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. Figure 4 This is a schematic diagram of an enhanced SEM image in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. Figure 5 This is a schematic diagram illustrating the principle of the regression loss function in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. Figure 6 This is a schematic diagram illustrating the principle of the bounding box regression loss function in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. Figure 7 This is a flowchart illustrating the steps of feature extraction in a defect detection model within a semiconductor manufacturing defect detection method according to an embodiment of the present invention. Figure 8 This is a schematic diagram of the detection results of a semiconductor manufacturing defect detection method according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the composition of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0021] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, elements, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0022] In this invention, terms such as "module," "device," and "system" refer to relevant entities applied to a computer, such as hardware, combinations of hardware and software, software, or software in execution. More specifically, for example, an element can be, but is not limited to, a process running on a processor, a processor, an object, an executable element, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be an element. One or more elements may be in an execution process and / or thread, and elements may be localized on a single computer and / or distributed across two or more computers, and may be run on various computer-readable media. Elements can also communicate via local and / or remote processes based on signals having one or more data packets, for example, signals from data interacting with another element in a local system, a distributed system, and / or interacting with other systems via signals over a network of the Internet.

[0023] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] The present invention will now be described in further detail with reference to the accompanying drawings.

[0025] The semiconductor manufacturing defect detection method in this invention can be applied to semiconductor manufacturing defect detection devices, enabling users to achieve intelligent detection of semiconductor manufacturing defects, ensuring detection accuracy and precision, and improving detection efficiency. These semiconductor manufacturing defect detection devices include, but are not limited to, smartphones, tablets, personal PCs, computers, and cloud servers.

[0026] Figure 1 The flowchart of a semiconductor manufacturing defect detection method according to an embodiment of the present invention is illustrated schematically. (Refer to...) Figure 1 As shown, the semiconductor manufacturing defect detection method of the present invention specifically includes the following steps: Step S11: Input the image to be detected into the pre-trained defect detection model, and obtain the defect detection result of the image to be detected based on the output result of the defect detection model, wherein the image to be detected includes an SEM image.

[0027] The defect detection model used in this step is based on YOLOv8 and has a hybrid loss function that combines classification loss and regression loss. It is trained on a pre-built dataset.

[0028] When training a defect detection model, it is mainly trained based on the training set and validation set in the dataset.

[0029] For example, refer to Figure 2 As shown, Figure 2 This illustration schematically demonstrates the training method of the defect detection model in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. (Refer to...) Figure 2 As shown, this method can be specifically implemented by including the following steps: Step S21: Obtain the dataset, which includes a training set, a validation set, and an independent test set; Step S22: Train the defect detection model using the training set within a preset training period, and at the end of each training period, use the validation set to perform forward inference to calculate the intersection-union ratio of the predicted bounding boxes and the true bounding boxes at the end of each training period, and count the number of true positives, false positives and false negatives. Step S23: Determine the validation set loss, precision, and recall for each training period based on the intersection-union ratio of the predicted bounding boxes and the true bounding boxes, the number of true positives, false positives, and false negatives at the end of each training period; Step S24: Determine the optimal model as the trained defect detection model based on the validation set loss, precision, and recall corresponding to each training cycle.

[0030] Step S21 is the step of obtaining the dataset, while steps S22 to S24 are the steps of training the defect detection model based on the obtained dataset.

[0031] In step S21, the acquired dataset is constructed from SEM images with labeled defect types and their corresponding calibration ranges, including a training set, a validation set, and an independent test set. Specifically, the dataset used is an SEM image dataset, where each SEM image is labeled with a defect type and its corresponding calibration range. This calibration range can be indicated by a bounding box indicating the location of the corresponding defect. These defect types and their corresponding calibration ranges can be manually calibrated, or obtained through detection by an existing detection model followed by manual verification; this embodiment does not impose any restrictions. The constructed dataset includes a training set, a validation set, and an independent test set. The training set is used to train the defect detection model, the validation set is used to verify the accuracy and precision of the trained defect detection model, and the independent test set is used to comprehensively evaluate the finally selected model. Specifically, the independent test set remains invisible throughout the entire defect detection model training and parameter tuning process. The core purpose of the independent test set is to provide an objective and unbiased comprehensive evaluation of the final model's generalization ability, detection accuracy, and recall rate after the optimal model has been selected based on the training and validation sets. This method effectively verifies whether the model can maintain stable defect recognition performance when faced with entirely new and unknown SEM images in actual production lines, thereby eliminating the risk of overfitting and ensuring the reliability of its actual deployment.

[0032] For example, refer to Figure 3 As shown, Figure 3 The diagram illustrates the process of constructing a dataset in a semiconductor manufacturing defect detection method according to an embodiment of the present invention. (Refer to...) Figure 3 As shown, the dataset can be constructed using the following methods: Step S31: Collect a number of original SEM images of labeled defect types and their calibration ranges from different batches and under different noise conditions, covering the target defect types; Step S32: Perform enhancement preprocessing on the original SEM image to obtain an enhanced SEM image; Step S33: Divide the original SEM image and the enhanced SEM image into a training set, a validation set, and an independent test set according to a preset ratio.

[0033] In step S31, SEM images from different batches and under different noise conditions, covering the target defect type, are collected. These SEM images can be those with pre-labeled defect types and their calibrated ranges, or they can be manually labeled after collection. The labeled SEM images are shown below. Figure 4As shown, target defect types can include bridging defects, line collapse defects, microbridge defects, gap defects, and potential gap defects. Different colors can be used to mark different defect types; for example, gap defects are marked with a red box, bridging defects with a yellow box, microbridge defects with a pinkish-purple box, and potential gap defects with a bright red box. The specific color can be chosen based on the actual situation, and this embodiment does not impose any restrictions. Using different colors to mark different defect types provides an intuitive visual aid. When the model completes forward inference and outputs the prediction result, the predicted and actual labeled boxes are overlaid on the original SEM image using a specific color mapping. This greatly facilitates quick and accurate manual comparison by R&D personnel or field engineers. This visual distinction effectively reduces the difficulty of manual verification in the validation process and improves the efficiency of evaluating the model's accuracy in classifying and locating various defects.

[0034] Step S32 involves enhancing the preprocessing of the collected SEM images. This enhancement preprocessing maximizes sample diversity and prevents overfitting. Specifically, the enhancement preprocessing procedure may include one or more combinations of randomly triggered geometric transformations, physical model noise injection, and illumination and contrast perturbations on the SEM images. Geometric transformations include one or more combinations of random rotation (±90°) and horizontal / vertical flipping, which can increase the relative frequency of small targets appearing in the image. Physical model noise injection involves injecting Gaussian noise into the training set images with a 50% probability. The Gaussian noise has a mean μ=0 and a variance σ randomly sampled within the range [0.01, 0.03] to simulate imaging conditions with different signal-to-noise ratios, while simultaneously introducing Poisson noise to simulate the shot noise effect of photon / electron counting. Illumination and contrast perturbations involve using an HSV (Hue, Saturation, Value) color space transformation to randomly adjust brightness and contrast, with an adjustment factor range of ±15%, to simulate the contrast differences on the surfaces of different batches of wafers. (Refer to...) Figure 5 As shown, during the enhancement preprocessing, for each original image, one or more combinations of physical model noise injection, illumination and contrast perturbation, or geometric transformation are randomly triggered. The final input dataset to the network consists of both the original images and the enhanced images to maximize sample diversity and prevent overfitting.

[0035] Step S33 is the step of dividing the SEM image after enhancement preprocessing. The specific division ratio can be adjusted according to the actual situation, and this embodiment does not limit it.

[0036] In this invention, the defect detection model used is based on YOLOv8, and its loss function has been optimized. The overall loss function is designed as a hybrid loss function combining classification loss and regression loss. In some possible implementations, when combining classification loss and regression loss, the overall loss function can be obtained by weighting and combining the classification loss and regression loss. Specifically, the overall loss function can be obtained by the following formula:

[0037] in, α and β The balancing coefficient is used to adjust the weights of classification and regression tasks. In this embodiment, the optimal value is determined by performing a grid search on the validation set. α =1.0, β =0.05.

[0038]

[0039] p t is the predicted probability, and is the probability value predicted by the model for this category. γ As a focusing parameter, let γ = 2.0. Its function is to reduce the weight of easily classified samples (such as large areas of clear background) so that the model focuses on difficult-to-classify samples (such as blurry small defects). α t The class balancing parameter is set based on the reciprocal of the frequency of each defect class in the training set. For defect classes with very few samples (such as microbridges), a larger value is set. α t The value is adjusted to increase its contribution to gradient updates.

[0040]

[0041] b pred To predict the coordinates of the center point of the annotation box, b gt The coordinates of the center point of the actual annotation box. ρ This indicates the calculation of the Euclidean distance between two center points. c The diagonal length of the smallest bounding rectangle that simultaneously covers both predicted and ground truth bounding boxes. IoU To predict the intersection-union ratio (IoU) between the bounding boxes and the actual bounding boxes. (Refer to...) Figure 6 As shown in the figure, the black solid line box represents the actual annotation box marked manually, and its center point coordinates are... b gt The green solid-line box represents the predicted bounding box output by the model, and its center point coordinates are... bpred . ρ This represents the Euclidean distance between the center points of the two boxes. c This represents the diagonal length of the smallest bounding rectangle (i.e., the gray dashed box) that can simultaneously contain both predicted and ground truth bounding boxes. This figure visually illustrates... DIoU How to optimize bounding box position regression by minimizing the distance to the center point.

[0042] Compared to GIoU , DIoU Loss Using "center point distance" directly as a penalty in defect detection means that even if the predicted bounding box and the actual bounding box do not overlap at all ( IoU (=0), this loss function can still provide a directional gradient, pulling the predicted box to quickly regress to the actual defect location, which is especially crucial for locating slender or tiny semiconductor defects.

[0043] Finally, the two are combined to form a hybrid loss function to simultaneously optimize classification and localization tasks. At the classification level, the classification loss strengthens the model's focus on minority classes and hard cases; at the localization level, the regression loss improves the geometric consistency of bounding boxes, especially small object boxes. The hybrid overall loss function directly minimizes the center point distance, which ensures higher accuracy in the geometric position of the predicted box than simply maximizing the overlap area. This is particularly important for objects with extreme aspect ratios, significantly reducing predicted box drift and thus improving the localization accuracy of small objects. The balance coefficient in the overall loss function... α and β This is used to adjust the weights of the two types of losses to adapt to the different task requirements regarding classification accuracy and localization accuracy.

[0044] In some possible implementations, while optimizing the loss function, to address the unique challenges of shot noise interference, cross-batch wafer background gray drift, and difficulty in locating nanoscale defects in SEM images, this embodiment also specifically reconstructs the feature extraction network of the YOLOv8 architecture, embedding a frequency domain decoupled edge enhancement and background gray suppression module (FDEE-BSM). Specifically, Figure 7 The schematic diagram illustrates the steps of feature extraction in the defect detection model of the semiconductor manufacturing defect detection method according to an embodiment of the present invention, with reference to... Figure 7 As shown, this process can be implemented by including the following steps: Step S41: Separate the input feature map into low-frequency feature maps and high-frequency feature maps; Step S42: Perform edge-shifting processing on the high-frequency feature map to obtain an enhanced high-frequency feature map; Step S43: Perform calibration processing on the low-frequency feature map to obtain a standard low-frequency feature map; Step S44: The enhanced high-frequency feature map and the standard low-frequency feature map are spliced ​​together to obtain the output feature map.

[0045] Specifically, when performing step S41, the input feature map can be set as follows: X ∈ R C×H×W First, the low-frequency smoothing component of the wafer background is extracted using a local average pooling operation. X low The high-frequency components containing defect edges and shot noise are extracted by subtracting the identity mapping. X high The calculation formula is:

[0046] In semiconductor lithography, metal lines or trench patterns are often horizontally or vertically distributed. When defects such as "line collapse" or "microbridges" occur, they exhibit a highly directional, elongated stripe distribution in the high-frequency domain, while electron shot noise is diffused non-directionally. Conventional symmetrical square convolution kernels (such as 3×3 or 5×5) introduce excessive noise background when extracting such features. Therefore, in step S42, asymmetric stripe convolution (such as parallel or cascaded 1×7 and 7×1 convolution kernels) is used for multi-scale learning. Enhanced high-frequency feature map. X high_enh The calculation is as follows:

[0047] in, Conv This represents the convolution operation. K For asymmetric convolution kernel size, BN σ represents batch normalization, and σ represents the nonlinear activation function. This asymmetric receptive field perfectly matches the prior geometric distribution of slender defects, achieving directional reinforcement of high-frequency defect edges and physical isolation from non-directional shot noise.

[0048] In actual production lines, due to the charging effect of electron beam scanning and process fine-tuning between different batches, the background grayscale of wafers often exhibits significant drift, with this smoothing of brightness primarily concentrated in the low-frequency domain. Specifically, in step S43, this embodiment innovatively constructs a "Background Drift Adaptive Gate" as the first weighting coefficient. Specifically, the low-frequency feature map sequentially passes through a global average pooling layer and a multilayer perceptron to generate dynamic gating weights. These weights are adaptively multiplied back onto the low-frequency feature map, achieving dynamic alignment and calibration of the background contrast of wafers from different batches, completely eliminating the interference of low-frequency background drift. To address the contrast and background grayscale drift phenomena of wafers from different batches, the low-frequency background component... X low Calculate dynamic gating weights W gate ∈ R C×1×1 Perform adaptive calibration:

[0049] in, GAP This indicates Global Average Pooling. W 1 and W 2 represents the learnable weight parameters of the two fully connected layers in a multilayer perceptron (MLP). δ It is the ReLU activation function. This represents element-wise multiplication along the channel dimension. X low_cal This is the calibrated low-frequency feature map.

[0050] Finally, step S44 concatenates the enhanced high-frequency defect edge features and the calibrated low-frequency background features along the channel dimension. Then, spatial attention is used to adaptively weight the reconstructed feature map at the two-dimensional pixel level, greatly improving the network's spatial localization accuracy for extremely small defects. Finally, a residual connection is performed with the original input for output. Specifically, the enhanced defect edges... X high_enh With calibrated background X low_cal We perform concatenation along the channel dimension, and let the concatenated features be... X concat To achieve precise spatial localization of extremely small defects (subpixel-level gaps or microbridges), a two-dimensional spatial attention matrix is ​​introduced. M spatial ∈ R 1×H×W :

[0051] Final module output Y The results were obtained through residual connection calculations:

[0052] Here, ";" indicates a splicing operation along the channel dimension. MaxP channel and AvgP channel These represent max pooling and average pooling along the channel dimension, respectively. Conv 1×1 Used to reduce the dimensionality of the spliced ​​feature channels to that of the original input. X Consistent, This is a broadcast multiplication in spatial dimensions.

[0053] Through the above design, the present invention effectively decouples and adaptively enhances the "large area wafer background" and "fine nanoscale defects" in the frequency domain, significantly improving the model's detection rate of extremely small defects such as bridging and line collapse.

[0054] Returning to steps S22 to S24. In step S22, the defect detection model is trained using the training set. The training period can be set to 100 periods. At the end of each period, forward inference is performed using the validation set data to calculate the intersection-union ratio (IU) of the predicted bounding boxes and the ground truth bounding boxes at the end of each period, and the number of true positives (TP), false positives (FP), and false negatives (FN) are counted. True positives represent the number of correctly detected defects, false positives represent the number of falsely detected defects, and false negatives represent the number of defects missed by the model.

[0055] For example, during training, the image size can be uniformly adjusted to 840×840 pixels to fit the network input layer and retain sufficient texture details. The batch size is set to 16, meaning 16 samples are used for parameter updates during each gradient descent iteration. The training cycle consists of 100 cycles, meaning the entire training set is traversed 100 times. The SGD optimizer is used with an initial learning rate of 0.01, and a cosine annealing strategy is employed to dynamically decay the learning rate during training to prevent the model from getting trapped in local optima.

[0056] Then, step S23 is executed, and the accuracy is determined based on the intersection-union ratio of the predicted and true bounding boxes, the number of true positives, false positives, and false negatives at the end of each cycle. Precision ) and recall rate ( Recall Specifically, the recall rate calculation formula is as follows:

[0057] in, TP (True Positive) represents the number of true positives, indicating the number of defects correctly detected. FN (FalseNegative) represents the number of false negatives, indicating the number of defects missed by the model. High recall means that the system can more effectively cover critical defects with extremely low frequency.

[0058] The accuracy calculation formula is as follows:

[0059] in, TP (True Positive) represents the number of true positives, indicating the number of defects correctly detected. FP (FalsePositive) represents the number of false positives, indicating the number of defects incorrectly detected. High accuracy means the system detects defects with a lower error rate and is less prone to misjudgments.

[0060] Finally, step S24 is executed to determine the training status of the defect detection model based on precision and recall, and to determine whether to stop training early. Specifically, in this step, an early stopping strategy can be adopted: if the performance metrics on the validation set (such as precision and recall) are low, the training can be stopped early. mAP or Recall If the model does not improve within a set value (e.g., 10 epochs), training is terminated early; model weights are only overwritten and saved if the validation metric for the current epoch is better than the historical best, thus ensuring that the final output model has the best generalization performance. This performance metric uses average precision (APS). mAP @0.5) is used as the core indicator, and its calculation formula is as follows:

[0061] Where N is the number of defect categories, AP i For the first i The area under the precision-recall curve (PR curve) for the defect class. In this embodiment, this area is obtained using existing standard calculation techniques in the field of target detection, specifically through interpolation methods in numerical integration.

[0062] Since the model outputs a series of discrete precision and recall data pairs on the validation set, this embodiment uses all-point interpolation, a method found in existing technologies, to calculate the area in order to eliminate local jitter in the curve. Specifically, for the... For each defect, the model prediction results are first sorted from highest to lowest confidence threshold, and then the accuracy at different thresholds is calculated sequentially.P and recall rate R A series of discrete points are obtained. Next, the PR curve is subjected to monotonic smoothing, and the accuracy after smoothing is... Defined as all instances where the current recall rate is greater than or equal to the current recall rate. Take the maximum precision value from the points, that is:

[0063] Finally, the sum of the areas of the rectangles below the smoothed curve is calculated using existing discrete integration methods, thus obtaining the average accuracy for this category. :

[0064] After calculating the area under each category curve using the aforementioned existing technical methods, the comprehensive detection performance of the defect detection model for a specific category of defects can be accurately quantified.

[0065] When implementing the early stopping strategy, the recorded validation loss and mAP If the performance does not improve during 10 consecutive verification cycles, the early stopping mechanism is triggered and the value is saved. mAP The model with the highest weight is used as the final deployed model. (Refer to...) Figure 8 As shown, experimental results indicate that this method is effective overall. mAP It outperformed the baseline model by 95.5%.

[0066] The final trained model can be deployed on edge computing devices or servers, or integrated into production line vision inspection systems to achieve real-time or offline automated defect detection and analysis report generation.

[0067] The semiconductor manufacturing defect detection method of this invention employs a hybrid loss function combining classification and regression losses. The classification loss alleviates class imbalance, while the regression loss improves bounding box regression accuracy by comprehensively considering overlapping areas, center point distance, and aspect ratio similarity. This allows the overall defect detection model to effectively detect defects such as bridging, line collapse, microbridges, and gaps, demonstrating good adaptability and stability. Compared to existing solutions, this method maintains high inference efficiency while significantly improving detection accuracy, generalization ability, and deployment flexibility. This invention enhances the model's ability to extract edge features, improves multi-scale feature fusion methods, and designs a hybrid loss function for imbalanced data and precise localization, ultimately achieving more robust and accurate automated detection of various defects in semiconductor manufacturing, especially subtle and low-contrast defects. Through innovative loss functions, this semiconductor manufacturing defect detection method improves several key indicators, effectively balancing the trade-off between detection accuracy and processing speed. This method provides a practical solution for automated, high-precision defect detection in semiconductor manufacturing and has the potential for direct application in production environments.

[0068] In some embodiments, the present invention provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions, which can be read and executed by an electronic device (including but not limited to a computer, server, or network device, etc.) to perform the semiconductor manufacturing defect detection method of any of the above embodiments of the present invention.

[0069] In some embodiments, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the semiconductor manufacturing defect detection method of any of the above embodiments.

[0070] In some embodiments, the present invention also provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the semiconductor manufacturing defect detection method of any of the above embodiments.

[0071] In some embodiments, the present invention also provides a storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the semiconductor manufacturing defect detection method of any of the above embodiments.

[0072] Figure 9 This is a schematic diagram of the hardware structure of an electronic device for performing a semiconductor manufacturing defect detection method according to another embodiment of this application, as shown below. Figure 9 As shown, the device includes: One or more processors 910 and memory 920, Figure 9 Take the 910 processor as an example.

[0073] The apparatus for performing semiconductor manufacturing defect detection methods may further include an input device 930 and an output device 940.

[0074] The processor 910, memory 920, input device 930, and output device 940 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0075] The memory 920, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the semiconductor manufacturing defect detection method in the embodiments of this application, or a trained defect detection model. The processor 910 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 920, thereby implementing the semiconductor manufacturing defect detection method of the above-described method embodiments.

[0076] The memory 920 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created using semiconductor manufacturing defect detection methods, defect detection models, etc. Furthermore, the memory 920 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 920 may optionally include memory remotely located relative to the processor 910, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0077] Input device 930 can receive input digital or character information and generate signals related to user settings and function control of the image processing device. Output device 940 may include display devices such as a display screen.

[0078] The one or more modules are stored in the memory 920, and when executed by the one or more processors 910, they perform the semiconductor manufacturing defect detection method in any of the above method embodiments.

[0079] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0080] The electronic devices in this application embodiments exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0081] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0082] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0083] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0084] (5) Other electronic devices with data interaction functions.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting defects in semiconductor manufacturing, characterized in that, include: The image to be detected is input into a pre-trained defect detection model, and the defect detection result of the image to be detected is obtained based on the output result of the defect detection model. The image to be detected includes a SEM image. The defect detection model is a target detection model based on YOLOv8, which uses a hybrid loss function as the loss function. The hybrid loss function is constructed based on classification loss and regression loss.

2. The semiconductor manufacturing defect detection method according to claim 1, characterized in that, The defect detection model is trained using the following method: Obtain the dataset, which includes a training set, a validation set, and an independent test set; The defect detection model is trained using the training set within a preset training period. At the end of each training period, forward inference is performed using the validation set to calculate the intersection-union ratio of the predicted bounding boxes and the true bounding boxes at the end of each training period, and to count the number of true positives, false positives and false negatives. The validation set loss, precision, and recall for each training period are determined based on the intersection-union ratio of the predicted bounding boxes and the true bounding boxes, the number of true positives, false positives, and false negatives at the end of each training period. The optimal model is determined as the trained defect detection model based on the validation set loss, precision, and recall for each training cycle.

3. The semiconductor manufacturing defect detection method according to claim 2, characterized in that, The dataset was constructed using the following method: Collect raw SEM images of several labeled defect types and their calibration ranges from different batches and under different noise conditions, covering the target defect types; The original SEM image is subjected to enhancement preprocessing to obtain an enhanced SEM image; The original SEM images and enhanced SEM images are divided into training set, validation set and independent test set according to a preset ratio.

4. The semiconductor manufacturing defect detection method according to claim 3, characterized in that, The target defect type includes at least one of bridging defects, line collapse defects, microbridge defects, and gap defects.

5. The semiconductor manufacturing defect detection method according to claim 2, characterized in that, The enhancement preprocessing includes one or more combinations of geometric transformation, physical model noise injection, and illumination and contrast perturbation, wherein... The geometric transformations include one or more combinations of random rotation (±90°) and horizontal / vertical flipping; The physical model noise injection includes injecting Gaussian noise into the training set images with a 50% probability. The Gaussian noise has a mean μ=0 and a variance σ that is randomly sampled in the interval [0.01, 0.03]. At the same time, Poisson noise is introduced to simulate the shot noise effect of photon / electron counting. The illumination and contrast perturbation includes using HSV (Hue, Saturation, Value) color space transformation to randomly adjust brightness and contrast, with the adjustment factor range set to ±15%.

6. The semiconductor manufacturing defect detection method according to claim 1, characterized in that, The hybrid loss function is constructed based on a weighted combination of classification loss and regression loss, and its formula is as follows: in, L total For a mixed loss function, L focal For classifying losses, L DIoU To regress the loss, α and β This is a balancing coefficient used to adjust the weights of classification loss and regression loss. p t To predict probabilities, γ To focus parameters, α t For class balancing parameters, b pred To predict the coordinates of the center point of the annotation box, b gt The coordinates of the center point of the actual annotation box. ρ This indicates the calculation of the Euclidean distance between two center points. c The diagonal length of the smallest bounding rectangle that simultaneously covers both predicted and ground truth bounding boxes. IoU To predict the intersection-union ratio (IUGR) between the bounding boxes and the actual bounding boxes.

7. The semiconductor manufacturing defect detection method according to claim 6, characterized in that, The category balance parameter α t The settings are based on the reciprocal of the frequency of occurrence of each defect category in the dataset.

8. The semiconductor manufacturing defect detection method according to claim 1, characterized in that, The defect detection model, when performing feature extraction, includes: The input feature map is separated into low-frequency feature maps and high-frequency feature maps; The high-frequency feature map is edge-processed to obtain an enhanced high-frequency feature map; The low-frequency feature map is calibrated to obtain a standard low-frequency feature map; The enhanced high-frequency feature map and the standard low-frequency feature map are spliced ​​together to obtain the output feature map.

9. The semiconductor manufacturing defect detection method according to claim 8, characterized in that, The step of separating the input feature map into low-frequency feature maps and high-frequency feature maps includes: Pooling is performed on the input feature map to obtain low-frequency feature maps; The low-frequency feature map is separated from the input feature map to obtain the high-frequency feature map; The edge-mapping process for the high-frequency feature map includes: Asymmetric strip convolution is applied to the high-frequency feature map to enhance its edges, resulting in an enhanced high-frequency feature map. The standard processing of the low-frequency feature map includes: The low-frequency feature map is processed sequentially by global pooling, multilayer sensing, and activation function to obtain the first weight coefficients. The standard low-frequency feature map is obtained by multiplying each element of the low-frequency feature map by the first weight coefficient. The process of stitching together the enhanced high-frequency feature map and the standard low-frequency feature map includes: The enhanced high-frequency feature map and the standard low-frequency feature map are concatenated, and a two-dimensional spatial weight matrix is ​​generated through a spatial attention network. This matrix is ​​then multiplied with the concatenated feature map and added to the original input feature map by residual addition to obtain the output feature map.

10. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 9.

Images