A deep learning-based EUV mask defect detection method

Abstract

This invention discloses a deep learning-based method for EUV mask defect detection, belonging to the field of extreme ultraviolet lithography mask defect detection technology. The method includes acquiring high-resolution grayscale images of the EUV lithography mask, preprocessing and labeling the grayscale images and dividing the dataset, and using an improved sparrow search algorithm to globally optimize the hyperparameter combination of a pre-defined deep convolutional neural network. Improvements include: adding anti-degradation processing in the initialization of the Tent chaotic mapping; introducing nonlinear adaptive weights in the discoverer position update; and applying Lévy flight perturbation to the globally optimal individual. Both the surrogate evaluation stage and the final training use cosine annealing learning rate decay to maintain consistency. After configuring the network with optimal hyperparameters and training it sufficiently, the model is used for online inference of the mask under test, and outputs the defect category, location, and confidence score after non-maximum suppression. This invention provides a highly automated technical means for EUV mask defect detection.

Description

Technical Field

[0001] This invention relates to the field of extreme ultraviolet lithography mask defect detection technology, specifically to a deep learning-based EUV mask defect detection method. Background Technology

[0002] Extreme ultraviolet (EUV) lithography is one of the core technologies for advancing semiconductor manufacturing processes to smaller nodes. As a key component in the lithography process, the surface quality of the EUV mask has a crucial impact on chip yield and performance. During mask manufacturing, storage, and use, various types of defects can occur, such as bridging, broken lines, bumps, pinholes, and particulate contamination. These defects are often in the micrometer or even nanometer range. Traditional manual visual inspection methods are not only time-consuming and labor-intensive but also prone to missed detections or misjudgments, failing to meet the efficiency and accuracy requirements of modern semiconductor production lines.

[0003] In recent years, object detection methods based on deep convolutional neural networks have shown great application potential in the field of industrial defect detection. These methods typically rely on large amounts of labeled data and appropriate hyperparameter configurations to achieve good detection results. The choice of hyperparameters significantly impacts model performance, but current common hyperparameter tuning methods, such as grid search and random search, suffer from low search efficiency and a tendency to get stuck in local optima. Furthermore, while existing sparrow search algorithms possess some global search capability when applied to hyperparameter optimization, there is still room for improvement in population initialization, the balance between exploration and exploitation during the search process, and escaping local stagnation.

[0004] Therefore, providing a method to improve the automation and accuracy of EUV mask defect detection is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based EUV mask defect detection method. By improving the sparrow search algorithm and introducing anti-degenerative chaotic initialization, nonlinear adaptive weights, and Lévy perturbation, the method effectively enhances the balance between global exploration and local exploitation in hyperparameter search, reducing the risk of getting trapped in local extrema. At the same time, it maintains consistency between surrogate training and final training strategies, improving the reliability of hyperparameter transfer, thereby improving the automation and accuracy of EUV mask defect detection and providing an effective technical means for mask quality control.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep learning-based method for EUV mask defect detection includes: Step 1: Obtain a high-resolution grayscale image of the extreme ultraviolet lithography mask, preprocess and label the grayscale image, obtain all labeled images, construct a dataset, divide the dataset to obtain the training set and test set; Step 2: Construct a deep convolutional neural network. Using the training and test sets as the data foundation, an improved sparrow search algorithm is employed to globally optimize the hyperparameter combination of the deep convolutional neural network to obtain the optimal hyperparameter combination. The improved sparrow search algorithm includes: generating an initial population using Tent chaotic mapping and adding anti-degeneration processing to prevent the sequence from degenerating to a fixed point or a short period during the mapping process; introducing an adaptive weight factor that decreases nonlinearly with the number of iterations in the update of the discoverer's position; and applying a random perturbation operation based on Lévy flight to the current globally optimal individual after each generation of population update. Step 3: Configure a deep convolutional neural network based on the optimal hyperparameter combination, and train the configured network fully using the training set to obtain a trained defect detection model; Step 4: Obtain a high-resolution grayscale image of the extreme ultraviolet lithography mask to be detected, preprocess it, input the preprocessed grayscale image into the trained defect detection model, and output the defect category, location coordinates and confidence level.

[0007] Furthermore, in step 1, the grayscale image is preprocessed and labeled, specifically as follows: The grayscale image is sequentially subjected to median filtering for noise reduction, contrast-limited adaptive histogram equalization for enhancement, and size normalization to obtain normalized image samples with uniform size; defects in the normalized image samples are then labeled with categories and bounding boxes to obtain labeled images.

[0008] Furthermore, in step 2, the hyperparameter combination includes the initial learning rate, momentum factor, and weight decay coefficient of the deep convolutional neural network.

[0009] Further, in step 2, the anti-degradation processing specifically involves: when generating a normalized chaotic sequence using the Tent recursive formula, if the normalized value obtained by recursion is less than the first threshold, it is forcibly set to the first threshold; if the normalized value obtained by recursion is greater than 1 minus the first threshold, it is forcibly set to 1 minus the first threshold.

[0010] Further, in step 2, the non-linearly decreasing adaptive weighting factor is: ; Pick , , This represents the current iteration number. This represents the maximum number of iterations.

[0011] Furthermore, in step 2, the random perturbation operation based on Lévy flight specifically includes: The Mantegna algorithm is used to generate a random step size vector that conforms to a heavy-tailed distribution. This random step size vector is multiplied by a preset perturbation scaling factor and then multiplied element-wise with the current global best individual as a perturbation. This perturbation is then added to the current global best individual to obtain the trial position. The fitness value of the trial position is calculated. If it is better than the current global best individual, it is replaced; otherwise, the original best individual is retained.

[0012] Furthermore, in step 3, a cosine annealing learning rate decay strategy is adopted during the training process.

[0013] Further, in step 4, a high-resolution grayscale image of the extreme ultraviolet lithography mask to be detected is acquired, preprocessed, and then input into the trained defect detection model. The model outputs the defect category, location coordinates, and confidence score, specifically: Obtain the original grayscale image of the extreme ultraviolet lithography mask to be tested, and obtain normalized image samples by the same preprocessing as in step 1; input the normalized image samples into the defect detection model for forward inference, and perform non-maximum suppression postprocessing on the predicted bounding boxes output by the model, and finally output the defect category, location coordinates and confidence level.

[0014] In summary, the beneficial technical effects of the present invention include: Firstly, this invention generates an initial population by combining Tent chaotic mapping with anti-degradation processing, which makes the initial individuals of hyperparameter search more evenly distributed and more ergodic in the search space, reducing the risk of reduced search efficiency due to poor initial point selection.

[0015] Secondly, this invention introduces an adaptive weight factor that decreases non-linearly with the number of iterations in the update of the discoverer's location. This allows the algorithm to maintain a large exploration range in the early stages of the search and focus on fine-tuning the better region in the later stages, which helps to better balance the relationship between global search and local search.

[0016] Third, after each iteration, the present invention applies a random perturbation based on Lévy flight to the globally optimal individual, and uses the heavy-tailed distribution characteristics to help the algorithm escape local extreme value regions during the search process, thereby increasing the possibility of obtaining a better combination of hyperparameters.

[0017] Fourth, the present invention adopts the same learning rate decay strategy as the final training in the hyperparameter proxy evaluation stage, so that the hyperparameter combinations that perform well in the proxy task can be reliably transferred to the complete training scenario, reducing search bias.

[0018] Fifth, this invention provides a flexible solution for image size normalization that balances aspect ratio preservation and local detail retention, and ensures that the data processing flow remains consistent between the training and inference stages, thus providing a foundation for the stable performance of the model in practical applications. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] like Figure 1 As shown, this invention provides a deep learning-based EUV mask defect detection method, comprising: Step 1: Obtain a high-resolution grayscale image of the extreme ultraviolet lithography mask, preprocess and label the grayscale image, obtain all labeled images, construct a dataset, divide the dataset to obtain the training set and test set; Step 2: Construct a deep convolutional neural network. Using the training and test sets as the data foundation, an improved sparrow search algorithm is employed to globally optimize the hyperparameter combination of the deep convolutional neural network to obtain the optimal hyperparameter combination. The improved sparrow search algorithm includes: generating an initial population using Tent chaotic mapping and adding anti-degeneration processing to prevent the sequence from degenerating to a fixed point or a short period during the mapping process; introducing an adaptive weight factor that decreases nonlinearly with the number of iterations in the update of the discoverer's position; and applying a random perturbation operation based on Lévy flight to the current globally optimal individual after each generation of population update. Step 3: Configure a deep convolutional neural network based on the optimal hyperparameter combination, and train the configured network fully using the training set to obtain a trained defect detection model; Step 4: Obtain a high-resolution grayscale image of the extreme ultraviolet lithography mask to be detected, preprocess it, input the preprocessed grayscale image into the trained defect detection model, and output the defect category, location coordinates and confidence level.

[0022] Next, the above steps will be explained in detail with reference to specific parameter settings: In step 1, a high-resolution grayscale image of the extreme ultraviolet lithography mask is acquired, and the grayscale image is preprocessed and labeled. All labeled images are obtained, a dataset is constructed, and the dataset is divided into training and test sets, specifically: The core task of Step 1 is to construct a high-quality, fully annotated EUV mask defect image dataset, providing a standardized data foundation for hyperparameter search in Step 2 and model training in Step 3. Step 1 is broken down into five consecutive sub-steps for detailed explanation: hardware imaging, image preprocessing, normalization, defect annotation, and dataset partitioning. Step 101: High-resolution image acquisition First, a high-resolution image of the EUV mask needs to be acquired. This step is specifically accomplished using an extreme ultraviolet lithography mask defect detection imaging system. This imaging system mainly includes an extreme ultraviolet light source, a high-precision displacement stage for supporting and moving the mask, a set of reflective lenses for focusing and reflecting extreme ultraviolet light, and a high-sensitivity image sensor.

[0023] The specific operation is as follows: Fix the EUV mask to be tested on the loading plate of the displacement stage, and start the vacuum system to evacuate the cavity pressure to... The wavelength emitted by the extreme ultraviolet light source is approximately... An illumination beam of nm is uniformly projected onto a localized area of ​​the mask surface by an illumination optics system. The extreme ultraviolet light reflected from the mask surface is then focused by a numerical aperture of [missing information]. The image is collected by a reflective projection lens and ultimately imaged onto a high-sensitivity charge-coupled device or complementary metal-oxide-semiconductor image sensor. This sensor has... A pixel array with a pixel size of μm. The lateral magnification of this imaging system is fixed at μm. Each pixel of the sensor corresponds to a physical size on the mask surface. This magnification is determined during the system's factory calibration. The method of this invention directly uses the acquired image, and the final physical location of the defect can be obtained by converting pixel coordinates using this calibration coefficient, without affecting the inspection process itself. A high-precision displacement stage moves the mask in a step-scan manner, exposing and stitching together a sub-region each time, until all areas to be inspected on the mask have been traversed. Finally, a complete high-resolution grayscale image of the EUV mask is synthesized, denoted as... .

[0024] Step 102: Image Denoising Processing Due to factors such as extreme ultraviolet photon shot noise and sensor readout noise, the original image Random noise is inevitably present in the sample, which can interfere with the detection of subsequent minor defects. To effectively suppress noise while preserving the edge details of the defects, this step employs median filtering. Process it.

[0025] Select a size of Two-dimensional sliding window of pixels , in Slide pixel by pixel. For coordinates in the image. Pixels at that location, collection window The gray values ​​of all neighboring pixels covered are formed into a set. The values ​​in this set are then sorted, and the median of the sorted values ​​is taken as the new gray value at that coordinate. This operation can be expressed mathematically as follows: ; in, Coordinates after median filtering The pixel grayscale value at that location, This represents the operation of retrieving values ​​from a set. and Uses the relative pixel offset index within the window. Iterates through the entire image. After determining all pixel positions, the denoised image is obtained. .

[0026] Step 103: Image Contrast Enhancement To highlight the small defect areas that originally had low contrast, the denoised image was... Perform contrast-limited adaptive histogram equalization. The core of this operation is to perform histogram equalization independently within each local small region, i.e., block, of the image, while limiting the excessive amplification of local contrast through cropping to prevent noise from being amplified as well.

[0027] The specific operating parameters for this step are as follows: Divided into several Non-overlapping image patches of pixel size. For each image patch, first calculate its grayscale histogram. ,in Use grayscale. Set the cropping limit value. Exceeding the limits in the histogram The portion is cropped out and then evenly redistributed to all gray levels to obtain a restricted histogram. Then on A standard histogram equalization transformation is performed to obtain the grayscale mapping function for the image patch. For each pixel in the image, bilinear interpolation is performed based on the mapping functions of its own block and neighboring blocks to obtain the final enhanced pixel value. After traversing the entire image, a contrast-enhanced image is generated. .

[0028] Step 104: Dimensional Standardization To meet the requirement of fixed-size input for subsequent deep convolutional neural networks, all enhanced images need to be... The resolution is uniformly processed as Pixels. The specific operation is as follows: First, [the text abruptly ends here, likely due to an incomplete sentence Scale proportionally, maintaining the original aspect ratio; the longer side after scaling should not exceed [a certain value]. pixels, then center the scaled image in On a square canvas of pixels, the blank areas are filled with the image's average grayscale, thus achieving a uniform size while avoiding geometric distortion caused by direct stretching. A pixel-normalized image. This step uses bilinear interpolation for scaling. For the target image... median coordinate The pixels at that location are mapped back to the source image. floating-point coordinates : ; in, and They are The original width and height. The grayscale value is determined by the distance in the source image. The grayscale values ​​of the four most recent pixels are in and Linear interpolation is performed in the direction to obtain the result. After this step, the result is... That is, a normalized input image, whose size is strictly defined. Pixel.

[0029] Furthermore, when the original mask image has extremely high resolution and direct proportional scaling would result in the loss of detail due to minor defects, in a preferred embodiment, a sliding window cropping strategy is used instead of the aforementioned scaling strategy: On a square canvas of pixels (obtained by scaling and filling a non-square image proportionally), with a step size Pixels To ensure proper overlap between adjacent windows, the sliding clipping size is... Sub-images of pixels, each containing at least one defect. The subgraph, along with a suitable number of defect-free background subgraphs (randomly sampled), constitutes the normalized sample set for this step. For defects located at the clipping window boundary, their complete bounding boxes are preserved and their coordinates are updated to the subgraph coordinate system. This subgraph construction strategy ensures that the data distribution of the training data is completely consistent with the data distribution when the same sliding window strategy is used in subsequent online detection inference, and the defects maintain their shape and size at the original resolution in the subgraph.

[0030] Step 105: Defect Labeling For each normalized input image obtained This requires manual or semi-automatic annotation tools to accurately label each visible EUV mask defect. The defect categories preset in this invention include, but are not limited to: bridging defects, broken wire defects, protrusion defects, pinhole defects, and particulate contamination, numbered using non-negative integers. Category identification, such as Corresponding bridging, The corresponding broken line, and so on.

[0031] For each defect instance in the image Enclose the bounding box with the smallest bounding rectangle. Record the bounding box parameters of this rectangle in the normalized image coordinate system and express them as a four-dimensional vector. It means that among them These are the pixel coordinates of the top-left corner of the rectangle. These are the pixel coordinates of the bottom right corner of the rectangle.

[0032] Then, the bounding box represented by these pixel coordinates is converted into the normalized center coordinate form required by subsequent networks. The conversion formula is as follows: ; ; in, It is the normalization of the center point of the bounding box. coordinate, These are the normalized width and height of the bounding box. and This involves standardizing the image size. Therefore, each defect instance generates a label record containing its category. and normalized bounding box vector All annotation information is saved in corresponding tag files for each image, with the filename corresponding to the image name.

[0033] Step 106: Dataset Partitioning Finally, all labeled and normalized input image samples and their corresponding label files are considered as a complete sample set. To ensure the objectivity of model evaluation, the sample set is randomly divided into disjoint training sets in an 8:2 ratio. and test set In practice, the index numbers of all samples are first randomly shuffled, and then the first 80% of the samples and their labels are selected as the final index. The last 20% of the samples and their labels were taken as... .

[0034] In step 2, a deep convolutional neural network is constructed. Using the training and test sets as data foundation, an improved sparrow search algorithm is employed to globally optimize the hyperparameter combination of the deep convolutional neural network, obtaining the optimal hyperparameter combination. Specifically: The core task of step 2 is to use the training set constructed in step 1. and test set Based on the data, an improved sparrow search algorithm is used to globally optimize the hyperparameter combination of the preset deep convolutional neural network, ultimately outputting a set of hyperparameters that achieves the optimal target detection performance, which is then used in step 3 to construct the final defect detection model. The following sections will elaborate on the preset deep convolutional neural network structure, the design of the improved sparrow search algorithm, and the specific optimization process, including: 1. Preset deep convolutional neural network structure This step uses the YOLOv5s object detection framework as the preset deep convolutional neural network. YOLOv5s is a single-stage object detection network, which consists of a backbone network, a neck network, and a detection head.

[0035] The backbone network is responsible for extracting multi-level feature maps from the input image. The YOLOv5 backbone network consists of a Focus structure and a series of C3 modules with convolutional layers. The Focus structure is suitable for inputs with a size of... A slicing operation is performed on a single-channel grayscale image, rearranging pixels from the spatial dimension to the channel dimension, with an output size of [size missing]. The feature map is then processed through multiple convolutional layers and the C3 module for feature extraction, with the feature map size progressively downsampled to [size missing]. , , and The C3 module contains several bottleneck residual units, which enhance feature reuse capabilities through cross-layer connections.

[0036] The neck network employs a structure combining a path aggregation network and a feature pyramid network. It receives the last three layers of feature maps output by the backbone network, i.e., features of size [size missing]. , and The feature maps are denoted as P3, P4, and P5, respectively. The neck network first passes deep semantic information to shallow feature maps through a top-down feature pyramid structure, and then passes shallow detail information to deep feature maps through a bottom-up path aggregation structure. After fusion, it outputs three enhanced feature maps N3, N4, and N5, with the same size as P3, P4, and P5, respectively.

[0037] The detection head performs target prediction on feature maps at three different scales. For each grid cell in each feature map, the detection head pre-sets three anchor boxes with different aspect ratios and predicts the bounding box offset, target confidence, and probability of belonging to each category for each anchor box. The total loss function consists of bounding box regression loss. Target confidence loss and classification loss The result is a weighted sum of three parts: ; The loss weight is set to a default value. The bounding box regression loss uses the CIoU loss function, while the target confidence loss and classification loss both use the binary cross-entropy loss.

[0038] Hyperparameter combinations to be optimized The definition is as follows: To train the initial learning rate, The momentum factor for the SGD optimizer. This is the weight decay coefficient. Other hyperparameters, such as anchor frame size, network depth, and width, remain unchanged from the default settings in YOLOv5s.

[0039] 2. Improved Sparrow Search Algorithm Design The improved sparrow search algorithm introduces three improvements based on the standard sparrow search algorithm: Tent chaotic mapping for population initialization, nonlinear adaptive weight correction for discoverer position updates, and global optimal individual perturbation based on Lévy flight. The specific mechanisms and mathematical expressions of each improvement are as follows.

[0040] (1) Initialization of Tent chaotic mapping population Let the population size be The maximum number of iterations is The lower bound of the hyperparameter search space is... The upper boundary is Specifically set as follows: .

[0041] First, generate the normalized coordinates of the first individual sparrow. Each component From the interval The samples were obtained through uniform random sampling within the region. Subsequently, the remaining samples were generated recursively using a Tent chaotic mapping. Normalized coordinates of each individual: ; in, For indexing individuals in the population, This corresponds to three hyperparameter dimensions. This recursive formula allows the individual to... The space is uniformly distributed throughout, avoiding overlaps and gaps that may be caused by random initialization.

[0042] To avoid Tent chaotic mapping in or When it gets stuck at a specific value or enters a short-period cycle (such as...) , This leads to the loss of population diversity. In this embodiment, when the recursive result is... When forced to set ,in ;when When forced to set This anti-degradation process ensures that the chaotic sequence maintains efficient traversal throughout the entire search space.

[0043] By linearly mapping the normalized coordinates to the actual hyperparameter values, we obtain the first value in the initial population. hyperparameter vectors of each individual : ; This generates an initial population with a uniform distribution. This embodiment takes , .

[0044] (2) Nonlinear adaptive weight correction In the standard sparrow search algorithm, the formula for updating the discoverer's position is: ; In the formula, This represents the current iteration number. for Uniform random numbers within, This is a warning value. As a safety threshold, For random numbers that follow a standard normal distribution, It is a three-dimensional row vector with all elements being 1.

[0045] The above update method does not consider the different emphases of exploration and development at different search stages. This invention introduces nonlinear adaptive weights. To correct the discoverer's location update, a weight function is constructed: ; Pick , This weight increases with the number of iterations. It exhibits a quadratic nonlinear decrease, especially in the early stages of the search. The larger size allows individual discoverers to explore a wider area; in the later stages of the search... Rapid reduction allows individual discoverers to focus on the current optimal area for refined development.

[0046] After incorporating nonlinear adaptive weights, the discoverer position update formula is revised as follows: ; (3) Global optimal individual perturbation based on Lévy flight To enhance the algorithm's ability to escape local optima, after updating all individuals in each generation, the globally optimal individual is evaluated. Apply a Lévy flight perturbation. Lévy flight is a random walking pattern that combines short-distance exploration with occasional long-distance jumps. Its step size follows a heavy-tailed distribution, making it well-suited for global optimization. The perturbation operation is defined as follows: ; in, The perturbation step size scaling factor. This indicates element-wise multiplication. It is a with A random vector of the same dimension, each component of which is generated independently, using the Mantegna algorithm: ; ; In the formula, The Lévy index, This is a gamma function. and From the mean of 0 and variance respectively and Sampling is performed within a normal distribution. The new positions after perturbation are calculated. If the fitness value is better than the original best individual, then use... replace Otherwise, retain the original optimal position.

[0047] 3. Improve the optimization process of the sparrow search algorithm. Based on the above design, the specific steps of the entire hyperparameter optimization process are as follows: (1) Initialization Set population size Maximum number of iterations The initial population is generated according to the Tent chaotic mapping method described above. .

[0048] (2) Fitness calculation For each generation Each individual in Its encoded hyperparameters Configure the YOLOv5s network using the provided training set. The network was trained quickly over 20 epochs. During training, the SGD optimizer was used, and the learning rate was scheduled using the same cosine annealing decay strategy as in the final training step 3: that is, based on the current number of individuals... As the initial learning rate, the learning rate is adjusted from [value] during epochs 1 to 20. The cosine annealing gradually decreases until... Momentum adopts Weight decay adopts The batch size is fixed at 16. After training, the batch size is used on the test set. The mean detection accuracy (mAP) at an IoU threshold of 0.5 was calculated and used as the fitness value for that individual. A higher fitness value indicates better model performance. Since both the surrogate training and the final training in step 3 use cosine annealing learning rate scheduling, the relative merits of hyperparameters in the two training scenarios have good transferability: hyperparameter combinations that perform well in the 20-epoch cosine annealing surrogate evaluation tend to perform well in the 150-epoch cosine annealing full training, thus ensuring that the best individuals selected by the surrogate evaluation can be effectively transferred to the final model.

[0049] (3) Population ranking and role classification After calculating the fitness of all individuals in the population, sort them in descending order of fitness value. Then... One individual is defined as the discoverer, and the remaining individuals are defined as the joiners. The current globally optimal individual is also recorded. and its fitness And the worst individual in the whole and its fitness .

[0050] (4) Discoverer location update For each discoverer Its position is updated according to the correction formula incorporating nonlinear adaptive weights: ; in according to Calculation. After the update, the values ​​for each dimension will be restricted to the corresponding search interval.

[0051] (5) Update of joiner location For each participant Move towards the optimal area occupied by the discoverer using the following formula: ; in, The position of the best individual among the discoverers. Randomly select values ​​for each component. or The three-dimensional row vectors, The updated version will also restrict each dimension to within the search boundaries.

[0052] (6) Scout Early Warning Update Randomly select from the population Each individual acts as a scout, providing early warning updates regarding its location: ; In the formula, Follows a standard normal distribution. for Uniform random numbers within, To prevent division by zero of extremely small constants.

[0053] (7) Global optimal individual Lévy perturbation For the current global optimal individual Perform the Lévy flight disturbance described above to generate a probe position. .calculate ,like Then use Replace the original optimal individual.

[0054] (8) Iteration Termination and Output Iteration counter Increment by 1. If Return to the above steps and continue execution; if If the iteration terminates, the final globally optimal individual is output. As the optimal combination of hyperparameters.

[0055] In step 3, a deep convolutional neural network is configured based on the optimal hyperparameter combination, and the configured network is fully trained using the training set to obtain a trained defect detection model, specifically: The core task of step 3 is to find the optimal combination of hyperparameters output in step 2. Based on the configuration, a deep convolutional neural network is instantiated and the training set provided in step 1 is used. After thorough training, a high-precision EUV mask defect detection model is obtained, which is then used for online inference in step 4. The entire process of model construction and training is described in detail below, including the following steps: Step 301: Construction of Deep Convolutional Neural Network The deep convolutional neural network constructed in this step is completely consistent with the preset network structure, still using the YOLOv5s object detection framework, including three main parts: backbone network, neck network, and detection head. Its specific structure is described below to ensure repeatability and deterministic implementation.

[0056] Backbone network: Input layer receiver size is A single-channel grayscale EUV mask normalized image. The first module is a Focus structure, which compresses the spatial size by half and expands the number of channels by four times through interval sampling slicing operations, resulting in an output size of [missing information]. The feature maps are then processed by stacking convolutional blocks and the C3 module sequentially, progressively downsampling and extracting deep semantic features, outputting three feature maps at different scales, denoted as P3 size. P4 size P5 size The standard configuration of the C3 module includes three bottleneck residual units. Each bottleneck residual unit consists of two 1×1 convolutional layers sandwiching a 3×3 convolutional layer, and has cross-layer identity connectivity.

[0057] The neck network receives three feature maps, P3, P4, and P5. Strong semantic information is passed down from top to bottom through a feature pyramid network, and strong localization information is passed up from bottom to top through a path aggregation network. The fusion operation includes upsampling, channel concatenation, and further processing by the C3 module, ultimately outputting three enhanced feature maps N3, N4, and N5, with the same dimensions as P3, P4, and P5, respectively.

[0058] Detection Heads: One detection head is set up at each of the three scales N3, N4, and N5. Each detection head consists of several convolutional layers, with a total number of output channels. ,in This indicates the number of anchor frames preset for each grid cell; 5 indicates... Four bounding box offsets and a target confidence level. The total number of defect categories is determined by the number of defect categories marked in step 1.

[0059] Step 302: Network Hyperparameter Configuration According to the optimal hyperparameter combination output in step 2 The following key training hyperparameters are set for the YOLOv5s network described above: The initial learning rate is set to .

[0060] The momentum factor of the SGD optimizer is set to .

[0061] The weight decay coefficient is set to .

[0062] Other hyperparameters not included in the search scope should retain the official YOLOv5s default values, such as: anchor frame size according to... Nine preset values ​​were recalculated using K-means clustering; the network depth and width multipliers were both kept at 1.0, i.e., the standard YOLOv5s scale was adopted; the activation function was uniformly SiLU; the batch normalized momentum was set to 0.03 by default; and the IoU loss term weights were adjusted accordingly. Confidence loss term weight Classification loss term weights At this point, network initialization is complete, and the network in this configuration is referred to as the model to be trained. .

[0063] Step 303: Training Configuration and Strategy The training process involves constructing the training set in step 1. Execute on the above, and simultaneously the test set of step 1. This serves as a model performance validation set after each epoch.

[0064] Optimizer: A stochastic gradient descent optimizer with momentum is selected, and its update rule is as follows: ; ; in, Indicates the first The set of network weights after one epoch iteration This is the momentum accumulation term. This represents the current actual learning rate. This represents the gradient of the loss function with respect to the weights. Weight decay. L2 regularization is applied to all weight parameters, i.e., a penalty term is added to the loss function. .

[0065] Learning rate scheduling: To achieve rapid convergence in the early stages of training and smooth fine-tuning in the later stages, a cosine annealing decay strategy is used to adjust the learning rate. Let the total number of training epochs be... , No. Learning rate per epoch The calculation method is as follows: ; in, The minimum learning rate is usually set in advance. The initial learning rate is... It gradually decreases with increasing epochs until it approaches... .

[0066] Loss function: The total loss function optimized during training. Consistent with the definition above, it consists of a weighted sum of three parts: ; The detailed definitions of each component are as follows: Bounding box regression loss The CIoU loss is used to calculate the anchor frame for each positive sample: ; In the formula, This represents the total number of positive sample anchor frames. For the first The intersection-union ratio of each predicted bounding box to its corresponding ground truth bounding box. The Euclidean distance between the center points of the two boxes is given. The diagonal length of the smallest bounding rectangle that can simultaneously enclose both the predicted bounding box and the ground truth bounding box. This is a penalty for aspect ratio consistency.

[0067] Target confidence loss and classification loss Both methods employ binary cross-entropy loss, calculating it separately for all positive and negative sample anchor boxes and all positive sample anchor boxes: ; in, This represents the total number of anchor frames involved in the confidence prediction. For the first The true target confidence label for each anchor box. To predict confidence levels; For the first The positive sample anchor frame in the th... Real labels in each category This represents the corresponding predicted probability.

[0068] Step 304: Model Training Process The specific steps for training are detailed below: Data loading: from the training set in batches of 16. Randomly sampled images and their labels were used. Data augmentation was performed online on each sampled image, including mosaic enhancement, random affine transformation, horizontal flipping, and random brightness-contrast jittering (implemented as random scaling of brightness and contrast coefficients on grayscale images, with brightness adjustment ranges from [value missing]). Contrast adjustment range is This is to increase data diversity and enhance model robustness.

[0069] Forward propagation and loss calculation: Batch input of the enhanced images into the current network The predicted tensors at each scale are obtained through layer-by-layer calculations via the backbone network, neck network, and detection head. Based on the matching strategy between the ground truth labels and the preset anchor boxes, positive and negative sample anchor boxes are determined, and the total loss is calculated according to the loss function formula in step 303. .

[0070] Backpropagation and parameter update: Calculating the loss For all learnable parameters in the network gradient The SGD optimizer updates the rules according to step 303, taking into account the current learning rate. ,momentum and weight decay Update network weights.

[0071] Iteration and Validation: Repeat steps 1 to 3 to traverse the entire training set, which constitutes one epoch. After each epoch, on the test set... The mAP@0.5 was calculated as a validation metric. The training process continued. Each epoch is recorded, and the verification mAP@0.5 for each epoch is recorded.

[0072] Model saving and selection: After training, from the 150 epochs of checkpoints, the set of weights that achieves the highest mAP@0.5 on the test set is selected as the final defect detection model, denoted as . The model weight file is persistently stored for later online testing.

[0073] In step 4, a high-resolution grayscale image of the extreme ultraviolet lithography mask to be detected is acquired, preprocessed, and then input into the trained defect detection model. The model outputs the defect category, location coordinates, and confidence score. Specifically: The core task of step 4 is to utilize the defect detection model trained in step 3 in an actual production inspection environment. This process performs real-time defect detection on high-resolution images of newly acquired EUV masks, outputting the precise location, category, and confidence level of each defect to provide data support for mask repair or scrapping decisions. Logically, this step follows the image preprocessing specifications in step 1, the optimal hyperparameter setting in step 2, and the model construction and training results in step 3, forming a complete engineering loop. The following details the four sub-steps: image acquisition, normalized preprocessing, model loading and forward inference, and post-processing and result output. Step 401: Acquisition of the EUV mask image to be tested The image under test is acquired using the same extreme ultraviolet lithography mask defect detection imaging system described above to ensure that the imaging conditions are consistent with the training data. Specifically: Using extreme ultraviolet light source ( nm), reflective projection lens ( High-sensitivity CCD / CMOS sensor (pixel array) pixel size μm).

[0074] The new mask to be tested is mounted on a high-precision displacement stage and evacuated to a high vacuum. Pa).

[0075] The entire mask area image is acquired using a step-scan method, and the complete original grayscale image is obtained by image stitching, denoted as . This image may also contain shot noise, and its resolution is at the original sensor pixel level, consistent with the training data source.

[0076] Step 402: Normalization preprocessing of the image to be tested To ensure that the input to the model inference is the same as the data distribution during training, the image to be tested... The same preprocessing pipeline as described above must be executed, namely median filtering, contrast-limited adaptive histogram equalization, and size normalization. Specific operational procedures and parameters will not be detailed here.

[0077] Step 403: Loading and Forward Inference of the Deep Convolutional Neural Network Model Load the optimal defect detection model saved in step 3. The weight file for this model is based on the optimal hyperparameters. The trained YOLOv5s network. The model input is a single or multiple images (batch) with a size of [missing information]. The grayscale image. The forward inference process is as follows: Normalize the image Convert to a numerical tensor and perform normalization: linearly scale the pixel grayscale values ​​from [0, 255] to the range [0, 1] to obtain the tensor. .

[0078] Will Input Model The forward computation graph passes sequentially through the Focus layer, backbone network, neck network, and three detector heads, yielding three sets of output tensors at different scales, denoted as follows: , and Each output tensor has a dimension of . ,in For batch size, Number of anchor frames per grid This represents the number of grid cells at the corresponding scale. This represents the total number of defect categories marked in step 1.

[0079] Decode each output tensor: the predicted bounding box offsets Using the prior width, height, and grid coordinates of the corresponding anchor frame, the normalized center coordinate representation is restored. And the target confidence is expressed using the Sigmoid function. and the probability of each category Limited to [0, 1]. The final class confidence score for an anchor box. It is given by the following formula: ; Retention score greater than the initial threshold The candidate boxes and their corresponding score vectors.

[0080] Step 404: Post-processing and output of detection results The large number of candidate boxes obtained from forward inference need to be redundant-removed using non-maximum suppression (NMS). The specific process is as follows: Candidate boxes at all scales are processed independently by category.

[0081] For each defect category Collect items that belong to this category and have a score. All candidate boxes are sorted in descending order of score.

[0082] Take the highest-scoring bounding box in sequence and calculate its IoU with the remaining boxes. If the IoU exceeds the NMS threshold... If the remaining boxes are not found, then suppress the remaining boxes. Repeat until there are no remaining boxes in that category.

[0083] After filtering out NMS again, the confidence level was lower than the final threshold. The frame is used to ensure output reliability.

[0084] Through the above process, the final set of test results is obtained. ,in For normalized bounding box coordinates, Number the predicted defect category. This represents the corresponding confidence score.

[0085] Map the normalized bounding box back to the original test image. The actual pixel coordinates are provided for operators to view: ; in and Original image The pixel width and height. The final output detection report should include at least: Mask identifier code and detection timestamp; The category name of each detected defect (by...) Mapped bounding boxes (e.g., bridging, broken lines, etc.) are pixel-level bounding boxes. and confidence level ; Draw bounding boxes on the original image and overlay category and confidence text to generate a visualization.

[0086] If the full-mask detection adopts a block inference strategy, the detection boxes of all sub-images need to be merged into the coordinate system of the full-mask image through coordinate offset, and then the full-image level NMS is performed again to eliminate duplicate boxes at the seams, ensuring that the final output is complete and without redundancy.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A method for detecting defects in EUV masks based on deep learning, characterized in that, include: Step 1: Obtain a high-resolution grayscale image of the extreme ultraviolet lithography mask, preprocess and label the grayscale image, obtain all labeled images, construct a dataset, divide the dataset to obtain the training set and test set; Step 2: Construct a deep convolutional neural network. Using the training and test sets as the data foundation, an improved sparrow search algorithm is employed to globally optimize the hyperparameter combination of the deep convolutional neural network to obtain the optimal hyperparameter combination. The improved sparrow search algorithm includes: generating an initial population using Tent chaotic mapping and adding anti-degeneration processing to prevent the sequence from degenerating to a fixed point or a short period during the mapping process; introducing an adaptive weight factor that decreases nonlinearly with the number of iterations in the update of the discoverer's position; and applying a random perturbation operation based on Lévy flight to the current globally optimal individual after each generation of population update. Step 3: Configure a deep convolutional neural network based on the optimal hyperparameter combination, and train the configured network fully using the training set to obtain a trained defect detection model; Step 4: Obtain a high-resolution grayscale image of the extreme ultraviolet lithography mask to be detected, preprocess it, input the preprocessed grayscale image into the trained defect detection model, and output the defect category, location coordinates and confidence level.

2. The EUV mask defect detection method based on deep learning according to claim 1, characterized in that, In step 1, the grayscale image is preprocessed and labeled, specifically as follows: The grayscale image is sequentially subjected to median filtering for noise reduction, contrast-limited adaptive histogram equalization for enhancement, and size normalization to obtain normalized image samples with uniform size; defects in the normalized image samples are then labeled with categories and bounding boxes to obtain labeled images.

3. The EUV mask defect detection method based on deep learning according to claim 2, characterized in that, In step 2, the hyperparameter combination includes the initial learning rate, momentum factor, and weight decay coefficient of the deep convolutional neural network.

4. The EUV mask defect detection method based on deep learning according to claim 3, characterized in that, In step 2, the anti-degradation process specifically involves: when generating a normalized chaotic sequence using the Tent recursive formula, if the normalized value obtained by the recursion is less than the first threshold, it is forcibly set to the first threshold. When the normalized value obtained by recursion is greater than 1 minus the first threshold, it is forcibly set to 1 minus the first threshold.

5. The EUV mask defect detection method based on deep learning according to claim 4, characterized in that, In step 2, the non-linearly decreasing adaptive weighting factor is: ； Pick , , This represents the current iteration number. This represents the maximum number of iterations.

6. The EUV mask defect detection method based on deep learning according to claim 5, characterized in that, In step 2, the random perturbation operation based on Lévy flight specifically includes: The Mantegna algorithm is used to generate a random step size vector that conforms to a heavy-tailed distribution. This random step size vector is multiplied by a preset perturbation scaling factor and then multiplied element-wise with the current global best individual as a perturbation. This perturbation is then added to the current global best individual to obtain the trial position. The fitness value of the trial position is calculated. If it is better than the current global best individual, it is replaced; otherwise, the original best individual is retained.

7. The EUV mask defect detection method based on deep learning according to claim 6, characterized in that, In step 3, a cosine annealing learning rate decay strategy is used during training.

8. The EUV mask defect detection method based on deep learning according to claim 7, characterized in that, In step 4, a high-resolution grayscale image of the extreme ultraviolet lithography mask to be detected is acquired, preprocessed, and then input into the trained defect detection model. The model outputs the defect category, location coordinates, and confidence score. Specifically: Obtain the original grayscale image of the extreme ultraviolet lithography mask to be tested, and obtain normalized image samples by the same preprocessing as in step 1; input the normalized image samples into the defect detection model for forward inference, and perform non-maximum suppression postprocessing on the predicted bounding boxes output by the model, and finally output the defect category, location coordinates and confidence level.

Images