Layout understanding model device and application integrating mask optimization and hotspot area detection

Abstract

The layout understanding model device and application integrating mask optimization and hot spot area detection belong to the semiconductor field. The technical key points are that the model includes a LUM encoder, whose input includes layout blocks and types, and maps the layout blocks to a latent space for latent feature representation; and a LUM decoder, whose input includes the latent feature representation and type output by the LUM encoder, and decodes and outputs the target image or optimizes the mask and hot spot area detection classification of the layout graphics according to the type, with the effect of achieving satisfactory performance on various layout understanding tasks.

Description

Technical Field

[0001] The present invention belongs to the field of semiconductors and relates to a layout understanding model device integrating mask optimization and hotspot area detection and its application. Background Art

[0002] With the rapid development of semiconductors, the size of transistors continues to shrink. The reduction in circuit size poses a huge challenge to optical proximity correction (OPC) and hotspot detection (HSD). The latest advances in OPC and HSD usually use deep neural networks to achieve satisfactory performance within an efficient running time. Based on these works, the present invention observes that both HSD and OPC models based on deep learning need to understand the layout structure information. In addition, these two tasks are closely related to the lithography process in the chip manufacturing process.

[0003] In chip manufacturing, due to the proximity effect in the lithography process, mask optimization technology is needed to obtain the desired mask graphics and reduce printing errors. Inverse lithography technology (ILT) is a mathematically rigorous method that optimizes the shape on the mask to achieve the desired wafer pattern. However, traditional ILT algorithms usually use iterative methods such as gradient descent to optimize the difference between the printed image on the wafer and the target pattern. The whole process requires many iterations to converge and is time-consuming. Therefore, researchers have proposed a variety of algorithms to improve the efficiency of ILT. Some methods attempt to optimize traditional ILT algorithms. For example, the GLS-ILT work relies on the CUDA toolkit to implement a GPU-accelerated Fourier transform algorithm, which accelerates a critical but time-consuming step in the lithography simulation model.

[0004] In recent years, advances in convolutional neural networks (CNNs) have spawned powerful techniques for accelerating ILT. Related works attempt to learn ILT solvers using stacked convolutional layers. By fully utilizing the massive computational resources of GPUs, they can output optimized mask graphics in a short running time. For example, GAN-OPC uses a CNN-based generative model GAN ​​to quickly approximate the initial mask solution of a test target, and then further optimizes the initial mask to improve the solution quality.

[0005] Despite considerable progress in mask optimization, layout defects still exist, which can cause open or short circuit failures. Figure 1As shown, conventional defect detection methods usually sample a series of points on the optimized mask and then measure the edge position error (EPE). Locations where the EPE is unacceptable are considered hotspot areas. But the main problem is that these methods cannot achieve end-to-end defect detection because they always require mask optimization results, resulting in additional time overhead. In addition, the calculation of EPE is not simple, and the number of sampling points varies significantly due to the different complexity of the layout, making parallel calculation challenging. Therefore, the efficiency of identifying potential defects after lithography simulation is not high. In addition, since this detection process involves sampling, the accuracy is also affected by the sampling frequency. A higher sampling frequency will improve the detection accuracy, but the efficiency is reduced. The present invention hopes to improve the defect detection process. By performing hotspot detection before lithography simulation, potential hotspot areas can be identified early, allowing the design team to take proactive measures to address these hotspots before lithography simulation, such as applying design rule modifications or layout optimization. Early detection of hotspots enables designers to mitigate potential problems and improve the overall manufacturability of the chip.

[0006] In recent years, various practical and efficient hotspot detection methods have been developed to overcome these limitations. The main idea is to detect hotspot areas directly from a given target layout, rather than analyzing mask graphics during the time-consuming mask optimization process. In this way, the overall running time can be significantly reduced. Recent hotspot detection is generally divided into two categories: 1. Pattern matching-based techniques. These methods first collect a set of layout graphics containing hotspots and use them to scan new design layouts. Any matching graphics identified will be considered as hotspots. 2. The hotspot detection task is formulated as a binary classification problem, which can be effectively solved by a deep neural network (DNN) model. Related work is inspired by the great success of learning models in the field of image classification.

[0007] For pattern matching methods, although they can overcome the time-consuming problem, their generalization ability is often unsatisfactory. Specifically, when a given hotspot pattern cannot match any pattern in the pre-collected dataset, these methods may not detect them. Therefore, deep learning models are more favored in the state-of-the-art because they can automatically learn to extract key features from layout graphics. However, there are still some problems with learning-based methods:

[0008] (1) Current deep learning-based hotspot detection models are usually trained and tested on small-scale annotated datasets, such as the ICCAD 2012 dataset. These small datasets lead to large data requirements in EDA academic research and easily lead to overfitting of learning models. Therefore, in real industrial scenarios, these learning models trained with these datasets usually perform poorly.

[0009] (2) As mentioned above, hotspot detection is closely related to mask optimization, as the quality of mask optimization will determine whether any hotspots exist. However, these methods only regard hotspot detection as an image classification problem and fail to effectively utilize the prior knowledge of mask optimization and lithography process. In order to improve the generalization ability, the present invention aims to design a large pre-trained model that can utilize more training data. Considering that both hotspot detection and mask optimization require understanding of layout geometry information and knowledge of lithography process, the present invention believes that hotspot detection and mask optimization should be integrated into a unified deep learning model, such as Figure 2 shown. Summary of the invention

[0010] In order to solve the above problems, in the first aspect, a layout understanding model device according to some embodiments of the present application is used for mask optimization and hot spot area detection of layout graphics, including:

[0011] The LUM encoder, whose input includes a layout tile and a type, maps the layout tile to a latent space for latent feature representation; and

[0012] The LUM decoder, whose input includes the latent feature representation and type output by the LUM encoder, decodes and outputs the target image or optimized mask and hot spot area detection classification of the layout graphics according to the type;

[0013] Wherein, the type includes an output target image or an optimized mask, the type is embedded in a LUM encoder and a LUM decoder, and the type embedding matrix is ​​set to be trainable;

[0014] Among them, the LUM decoder includes

[0015] The OPC decoder is used to output the target image or optimize the mask;

[0016] The HSD decoder is used to detect and classify hotspot areas in the output layout graphics.

[0017] According to the layout understanding model device of some embodiments of the present application, the LUM encoder includes

[0018] Tile merging layer, used for merging tiles; and

[0019] Swin Transformer block.

[0020] According to the layout understanding model device of some embodiments of the present application, the Swin Transformer block includes

[0021] The window multi-head self-attention layer W-MSA is used to partition the window so that the multi-head self-attention mechanism is applied in each window to calculate the self-attention of the window;

[0022] The offset window multi-head self-attention layer SW-MSA is used to shift the window partitions so that the model can bridge windows between different layers;

[0023] The multi-layer perceptron layer MLP is used to further extract features.

[0024] According to the layout understanding model device of some embodiments of the present application, in the Swin Transformer block, it is characterized in that the window multi-head self-attention layer W-MSA is used for input, and its output is added to the input as the first input and input into the first multi-layer perceptron layer MLP;

[0025] The output of the first multi-layer perceptron layer MLP is added to the first input as the second input and input to the offset window multi-head self-attention layer SW-MSA;

[0026] The output of the offset window multi-head self-attention layer SW-MSA is added to the second input as the third input and input to the second multi-layer perceptron layer MLP;

[0027] The output of the second multi-layer perceptron layer MLP is added to the third input as the output of the Swin Transformer block.

[0028] According to some embodiments of the layout understanding model device of the present application, the OPC decoder receives the output features from the four stages of the LUM encoder; the OPC decoder adopts a top-down path, which uses convolutional layers, residual connections and pyramid pooling modules to process the input feature map to generate a multi-scale feature map containing features at different abstract levels; the OPC decoder integrates feature information at different levels through lateral connections, which combine feature maps from different levels to facilitate the creation of a single comprehensive feature map, and generates a final output by reading the convolutional layer to represent the generated layout graphics.

[0029] According to the layout understanding model device of some embodiments of the present application, the HSD decoder is composed of three consecutive convolutional layers, receives the output of the previous encoder and predicts whether there is a hotspot in each tile, and the output of the HSD decoder is an N×N feature map, in which each pixel value represents the probability of a hotspot area.

[0030] In a second aspect, an embodiment of the present application also provides a pre-training method for any of the above-mentioned layout understanding model devices, including reconstructing a target.

[0031] In a third aspect, the embodiments of the present application also provide a method for fine-tuning any one of the layout understanding model devices, including S10. training the OPC decoder;

[0032] S20. Train the HSD decoder.

[0033] In a fourth aspect, embodiments of the present application also provide application of any of the layout understanding model devices for mask optimization and hot spot area detection and classification.

[0034] In a fifth aspect, embodiments of the present application also provide the application of any one of the layout understanding model devices in a layout understanding task.

[0035] Beneficial effects:

[0036] In the first aspect, the present invention proposes a powerful pre-trained layout understanding model (LUM), in which a customized training method called cross-domain masked layout modeling is designed. It helps LUM to better capture the geometric information of layout graphics, which helps the ability of mask optimization and hotspot detection. The present invention first generates a large amount of layout data and randomly masks part of the layout blocks. Then, a training scheme called cross-domain masked layout modeling is proposed to guide model training, and LUM is responsible for recovering the target image and ILT results from the given masked image. In this way, the present invention can not only fully pre-train the model with a large amount of masked layout data, but also embed the awareness of lithography process and mask optimization into LUM. Experimental results show that after pre-training, the present invention can easily fine-tune the pre-trained LUM with less data, while achieving satisfactory performance on various layout understanding tasks (including OPC and HSD).

[0037] On the second aspect, the layout understanding model (LUM) proposed in the present invention is a model based on deep learning, which has achieved remarkable performance in both mask optimization and hotspot detection tasks. The model structure of the present invention is inspired by the mask modeling method, which masks part of the input signal and requires the model to restore the content. This model can use more training data to improve generalization ability. The present invention proposes different decoders for HSD and OPC tasks while maintaining the same encoder components. During the training phase, the present invention designs multiple reconstruction tasks so that the model can effectively learn the geometric information of the layout graphics and the process of mask optimization. Experimental results show that the model of the present invention exhibits significant performance on the OPC task, and the learned mask optimization knowledge helps to improve the HSD performance. The present invention hopes to provide a method for layout pre-training models through this work. In the future, the present invention will continue to explore how to design task-specific decoder frameworks to further improve the performance of the model on various tasks.

[0038] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1.Traditional mask optimization and hotspot detection process, hotspot detection relies on the results of mask optimization.

[0040] Figure 2 .The unified framework proposed by the present invention.

[0041] Figure 3 .Modeling mechanisms for cross-domain masking maps.

[0042] Figure 4 (a) PVB quantifies the difference between the maximum and minimum process angles. (b) L2 measures the difference between the nominal resist image and the desired target image. (c) EPE estimates the distortion of the geometrically nominal resist image.

[0043] Figure 5 .Overview of the proposed framework.

[0044] Figure 6 .The structure of the Transformer block.

[0045] Figure 7 Comparison of shifted windows used to compute self-attention in the encoder architecture.

[0046] Figure 8 .Illustration of a stage in the Swin Transformer architecture.

[0047] Fig. 9 .LUM decoder.

[0048] Fig.10 .Fine-tune the algorithm.

[0049] Fig.11 .Graphic example of a composite layout that follows legal design rules.

[0050] Fig.12 .Pre-training loss function curve.

[0051] Fig.13 .(a) Loss function curve in the OPC fine-tuning stage (b) Accuracy change curve in the hotspot detection stage.

[0052] Fig.14 Examples of OPC results. (a) shows the mask output by the pre-trained LUM without further training. (b) is a well-optimized mask from the “Ours-Exact” setting. (c)(d)(e) show three examples from the “Ours-Fast” setting.

[0053] Fig.15 HSD and OPC results of a layout. (a) shows the hotspot detection result and (b) the corresponding mask optimization result.

[0054] Fig.16 . Experimental results of ablation study. DETAILED DESCRIPTION

[0055] The following describes the embodiments of the present application in detail with reference to the accompanying drawings, examples of which are shown in the accompanying drawings. The present application provides a method and an electronic device. The method and the device are based on the same technical concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0056] In one embodiment, the present invention relates to a layout understanding model device, comprising

[0057] The LUM encoder, whose input includes the layout tiles and types, maps the layout tiles to the latent space for latent feature representation.

[0058] The LUM decoder, whose input includes the latent feature representation and type output by the LUM encoder, decodes and outputs the target image or optimized mask according to the type.

[0059] The type includes outputting a target image or optimizing a mask. The type is embedded in a LUM encoder and a LUM decoder, and the type embedding matrix is ​​set to be trainable.

[0060] In one embodiment, the LUM encoder comprises

[0061] Tile merging layer, used for merging tiles.

[0062] Swin Transformer block, including window multi-head self-attention layer W-MSA, offset window multi-head self-attention layer SW-MSA and multi-layer perceptron layer MLP. Among them:

[0063] The LUM encoder groups layout tiles into multiple windows.

[0064] The window multi-head self-attention layer W-MSA performs window partitioning so that the multi-head self-attention mechanism is applied in each window to calculate the self-attention of the window.

[0065] The offset window multi-head self-attention layer SW-MSA performs shifted window partitioning, allowing the model to bridge windows between different layers.

[0066] The multi-layer perceptron layer MLP further extracts features.

[0067] In one embodiment, in the Swin Transformer block, a window multi-head self-attention layer W-MSA is used as input, and its output is added to the input as the first input and input to the first multi-layer perceptron layer MLP.

[0068] The output of the first multi-layer perceptron layer MLP is added to the first input as the second input and input to the offset window multi-head self-attention layer SW-MSA.

[0069] The output of the offset window multi-head self-attention layer SW-MSA is added to the second input as the third input and input to the second multi-layer perceptron layer MLP.

[0070] The output of the second multi-layer perceptron layer MLP is added to the third input as the output of the Swin Transformer block.

[0071] In one approach, the LUM decoder receives output features from four stages of the LUM encoder.

[0072] LUM Decoder The decoder adopts a top-down path that processes the input feature map using convolutional layers, residual connections, and pyramid pooling modules to generate multi-scale feature maps containing features at different levels of abstraction.

[0073] LUM Decoder The decoder integrates feature information from different levels through lateral connections. The connections combine feature maps from different levels to facilitate the creation of a single comprehensive feature map. The final feature map is generated by reading the convolutional layer to generate the final output, representing the generated layout graphic.

[0074] In one embodiment, the present invention relates to a pre-training method of the layout understanding model, comprising:

[0075] The layout understanding model obtains the target image or optimized mask corresponding to a given layout tile.

[0076] The masking strategy is used to mask parts of a given tile to reconstruct the target.

[0077] The layout understanding model outputs the target image or optimized mask corresponding to the layout tile of the reconstructed target.

[0078] In one approach, the type embedding is represented by the following formula:

[0079]

[0080] In the formula, X P Indicates type embedding, X E Represents the characteristics of the input layout, E T is the embedding matrix when the input is the target layout, E M It represents the embedding matrix when the input is a mask.

[0081] In one embodiment, the masking strategy has a masking ratio set to 50%.

[0082] In one embodiment, the reconstruction target is expressed by the following formula:

[0083]

[0084] Where, L cross (X) represents the total loss function, Represents the reconstruction result after inputting the masked layout, represents the mask optimization result after masking the layout, X T represents the target layout, represents the lithography simulation estimate after inputting the mask, X M represents the mask image, Represents the mask reconstruction result after the input mask is covered.

[0085] In one embodiment, the present invention relates to a method for fine-tuning a model obtained after pre-training of the layout understanding model, wherein the method integrates mask optimization and hotspot detection of the obtained model, wherein the parameters of the LUM encoder of the pre-trained layout understanding model are frozen, and the OPC decoder and HSD decoder of the LUM decoder are trained. The OPC decoder outputs a target image or an optimized mask. The HSD decoder outputs a hotspot area detection classification of the layout graphics.

[0086] It is understood that in the first embodiment, the LUM decoder is an OPC decoder, wherein the OPC decoder receives output features from four stages of the LUM encoder.

[0087] OPC Decoder The decoder adopts a top-down path that processes the input feature map using convolutional layers, residual connections, and pyramid pooling modules to generate multi-scale feature maps containing features at different abstraction levels.

[0088] OPC Decoder The decoder integrates feature information from different levels through lateral connections. The connections combine feature maps from different levels to facilitate the creation of a single comprehensive feature map. The final feature map is generated by reading the convolutional layer to generate the final output, representing the generated layout graphic.

[0089] The HSD decoder consists of three consecutive convolutional layers, which receives the output of the previous encoder and predicts whether there is a hotspot in each tile. The output of the HSD decoder is an N×N feature map, in which each pixel value represents the probability of a hotspot area.

[0090] In one solution, the OPC decoder is trained, and after the OPC decoder training is completed, the HSD decoder is trained, wherein the training of the OPC decoder includes:

[0091] Fixed parameters of LUM encoder.

[0092] Initialize the parameters of the OPC decoder and train it through multiple training cycles (Epochs).

[0093] In each training cycle, a training set is sampled from the data set, and the OPC decoder is trained with the training set, with the aim of minimizing the loss function of the OPC decoder.

[0094] Among them, the loss function is defined as:

[0095] L OPC (X T , X M )=L M (X T , X M )+L L2 (X T )+L PVB (X T ).

[0096] Where, L OPC (X T ,X M ) represents the loss function for mask optimization.

[0097] Minimize the similarity L between the predicted mask and the optimized mask M (X T ,X M ), defined as:

[0098]

[0099] In the formula, f TM (X T ) represents the mask optimization result after inputting the layout, X M Represents the mask image.

[0100] Minimize the distance L between the printed image predicted by mask optimization and the target image L2 (X T ), defined as:

[0101]

[0102] In the formula, f Z (f TM (X T )) represents the result of mask optimization after input layout and then lithography simulation, X T Indicates the target layout.

[0103] Minimize the distance L between the printed images of the maximum and minimum process angles PVB (X T ), defined as:

[0104]

[0105] In the formula, It indicates the result of mask optimization after inputting layout and then undergoing abnormal lithography simulation. It represents the result of mask optimization after inputting layout and then undergoing abnormal lithography simulation.

[0106] The training of the HSD decoder includes

[0107] Fixed parameters of LUM encoder.

[0108] Initialize the parameters of the HSD decoder and train it through multiple training cycles (Epochs).

[0109] In each training cycle, a training set is sampled from the data set, and the HSD decoder is trained with the training set, with the aim of minimizing the loss function of the HSD decoder.

[0110] Among them, the loss function is defined as:

[0111]

[0112] Where, L HSD (X T ) represents the loss function of hotspot detection, N represents the number of hotspot areas, x represents the horizontal serial number of the hotspot area, y represents the vertical serial number of the hotspot area, and p xy represents the true probability of the hotspot area, Represents the predicted probability of the hot spot area.

[0113] In one embodiment, a method for mask optimization and hotspot area detection and classification, applying the model, includes obtaining a layout block by the model;

[0114] The model outputs a target image or an optimized mask, and outputs a hotspot area detection classification of the layout graphics.

[0115] I. Overview

[0116] This paper proposes a powerful pre-trained layout understanding model (LUM), in which a customized training scheme called cross-domain masked layout modeling is designed. The proposed framework is inspired by large pre-trained models in deep learning, such as MAE and GPT, which have achieved amazing performance in many downstream tasks. In LUM, the paper first generates a large amount of layout data and randomly masks parts of the layout tiles. Then, a training scheme called cross-domain masked layout modeling is proposed to guide model training, such as Figure 3As shown. It can be seen that LUM is responsible for recovering the target image and ILT results from a given mask image. In this way, the present invention can not only fully pre-train the model with a large amount of masked layout data, but also embed the awareness of the lithography process and mask optimization into LUM. After pre-training, the present invention can easily fine-tune the pre-trained LUM with less data, while achieving satisfactory performance on various layout understanding tasks (including OPC and HSD).

[0117] II. Preliminary Knowledge

[0118] In this section, the author will introduce some preliminary knowledge related to this work.

[0119] A. Optical Proximity Correction

[0120] In the photolithography process, the input mask M is first converted into an aerial image I by an optical projection system. The distribution of light intensity on the wafer plane forms the final printed image Z through the development and etching processes. OPC is a widely used mask optimization technology used to find the optimized mask M. opt =f -1 (Z t ), where Z t is the design target, and f(·) represents the lithography process under normal process conditions.

[0121] The present invention proposes a mathematical model to simulate the photolithography process. The model consists of two parts: an optical projection model and a photoresist model. For the optical projection process, the Hopkins diffraction model of the partially coherent imaging system is used to approximate the projection behavior. Mathematically, the spatial image I can be approximated by convolving the mask M with a set of optical kernels, as shown below:

[0122]

[0123] where h k is the kth optical kernel in the optical kernel set H, w k is the corresponding coherent system weight. represents the convolution operation. In order to save computing resources, for the partially coherent system N h The order approximation is expressed as:

[0124]

[0125] In the work of the present invention, the number of cores N h Set to 24. After optical simulation, the aerial image I is input to the image with light intensity threshold I th In the photoresist model, the light intensity threshold I th Represents the exposure level. The final binary print image Z is calculated by the following step function:

[0126]

[0127] In order to evaluate the quality of the optimized mask, the present invention defines the following three indicators:

[0128] Process variation band (PVB). In actual lithography systems, process variations can lead to deviations in the final printed image, resulting in printing failure. Under different lithography conditions, such as focus / defocus depth and incident light intensity, the printed image has different profile results to evaluate the robustness of the printing, and the formula is as follows:

[0129]

[0130] Squared L2 Error. Given a target image and an image representing an image printed by normal lithography process conditions, the squared L2 loss is calculated as follows:

[0131]

[0132] Edge Position Error (EPE). Edge Position Error is used to evaluate the difference between the target design's outline and the image. When calculating EPE, a series of points are sampled on the target design's outline. If the distance D(x,y) between the target design and the printed image is greater than the EPE constraint, the point (x,y) is marked as an EPE violation.

[0133]

[0134] Figure 4 (a) PVB quantifies the difference between the maximum and minimum process angles. (b) L2 measures the difference between the nominal resist image and the desired target image. (c) EPE estimates the distortion of the geometrically nominal resist image.

[0135] B. Hotspot Detection

[0136] The photolithography process always involves many changes, and some patterns are sensitive to these changes, which may lead to potential open or short circuit failures. These failures will undoubtedly reduce the manufacturing yield. The layout pattern area that is sensitive to changes in the photolithography process is defined as a hot spot area. In order to improve the manufacturing yield, efficient and accurate hot spot detection technology must be developed to help locate the defective position of the layout. A high-performance hot spot detector should be able to correctly detect as many hot spots as possible and avoid misjudging non-hot spot patterns as hot spot patterns. In order to evaluate the performance, the present invention defines the following indicators:

[0137] Accuracy. The ratio of the number of correctly detected hotspots to the number of true hotspots.

[0138] False positive rate. The number of non-hotspots predicted as hotspots by the classifier.

[0139] C. Mask Modeling

[0140] Mask modeling is a learning task that masks part of the input content and attempts to recover the masked content. Masked language modeling, including BERT and GPT, is the first successful application of mask modeling in natural language processing. Typically, this is a fill-in-the-blank self-supervised learning task, where the model learns representations by predicting the masked words based on the surrounding context words. More recently, masked image modeling learns representations in a similar way, by predicting the missing parts at the pixel or block level. Once the model is trained and evaluated, it can be deployed in production systems for a variety of applications such as object detection, instance segmentation, semantic segmentation, or medical image analysis. Mask modeling plays a vital role in computer vision tasks that require precise localization and pixel-level understanding of objects or regions.

[0141] However, the above techniques have only been proven to be useful for natural language and image modeling. Mask modeling has not been fully explored in the field of EDA. It is well known that many layout-related tasks require powerful layout understanding capabilities of deep learning models. In this work, the present invention aims to explore the application of mask modeling in more robust layout feature extraction. The present invention masks a part of the layout, and the layout understanding model is responsible for restoring the masked layout image. In addition, considering the uniqueness of EDA layouts and tasks, the present invention makes many customizations to the masked layout modeling mechanism, which is significantly different from the masked modeling mechanism in the fields of computer vision and natural language processing, which will be described in detail in Section III.E.

[0142] Combined with the evaluation indicators defined in Section 2.A and Section 2.B, the present invention expresses the objectives of LUM as follows:

[0143] Problem 1 (Layout Understanding Model). Given a set of layout graphics, the goal of the layout understanding model is to learn to capture geometric information in order to achieve satisfactory performance on OPC and HSD. For OPC, the printed image should be close to the target image under different process conditions to minimize EPE, L2 loss, and PVB. For HSD, LUM should locate and classify all hotspots and non-hotspots to maximize detection accuracy and minimize false positives.

[0144] III. MLAE Algorithm

[0145] In this section, the architecture of the proposed layout understanding model (LUM) will be explained in detail. The key components that make LUM have strong generalization ability are introduced, which makes LUM different from previous layout feature learning models. The prior knowledge of mask optimization is also integrated into LUM, which significantly improves the performance of HSD and OPC.

[0146] A. Overview

[0147] Figure 5 The figure shows an overview of the proposed framework, including pre-training, fine-tuning, and co-evolution stages. The solid and dashed lines represent feedforward and back-propagation in training, respectively. The proposed framework consists of two stages:

[0148] Pre-training stage. In this stage, the present invention adopts the proposed masked layout modeling method to pre-train the model. The masking strategy (Section 3, Section B) appropriately masks specific parts of the input tile, the LUM encoder (Section 3, Section C) maps the input tile to the latent space, and the LUM decoder (Section 3, Section D) aims to restore the desired layout graphics. In the pre-training stage, the input and output can be the target image or the optimized mask. This training scheme enables the model to acquire knowledge in two fields. The reconstruction target (Section 3, Section E) guides the pre-training process through back-propagation.

[0149] Fine-tuning phase. Since the framework of the present invention targets OPC and HSD, the present invention requires an OPC decoder and an HSD decoder to complete these two tasks based on the encoding of the LUM encoder. The present invention fine-tunes the LUM decoder used in the pre-training phase to build an OPC decoder. In addition, the present invention designs a new HSD decoder for classification. Section III-F shows how to fine-tune the model on these tasks.

[0150] B. Concealment Strategy

[0151] Before training the LUM framework of the present invention to recover the layout pattern, the first step is to appropriately mask a portion of the layout tile. The latest mask modeling methods, such as MAE and SimMIM, use uniform random masking methods at the block level. Inspired by these works, the layout pattern of the present invention is first divided into non-overlapping blocks of size 4×4, and some blocks are randomly masked.

[0152] The masking ratio is crucial for model training. The right ratio can ensure that the masked layout graphics effectively eliminate redundancy, causing the task to be unable to be solved by extrapolating from visible adjacent blocks. In previous visual work, a commonly used masking ratio is 75%. However, the present invention finds that this high ratio is not suitable for the task of the present invention because the semantic information of the layout graphics is sparser than that of conventional images. In order to determine the appropriate masking ratio, the present invention conducted a series of experiments and finally set the masking ratio to 50%.

[0153] C. Encoder

[0154] The LUM encoder is responsible for modeling the latent feature representation of the masked block, which is then used to predict the original signal of the masked area. The learned encoder should be adaptable to different tasks. In recent years, Transformer has become a powerful encoder in vision and language fields. Figure 6Schematic diagram of the Transformer block structure. The Transformer encoder consists of multiple layers, the most important of which is multi-head self-attention, which enables the model to globally capture information at different locations. To process image input, VisionTransformer (ViT) divides the image into fixed-size blocks, each of which is regarded as a token in sequential data. Given an input layout of size H×W×C and a block size P, the representation is first converted into blocks The Transformer encoder first packs these blocks into a matrix, represented as Next, the fully connected layer maps the block to the input embedding To distinguish the location of the block, the position code is then added to the input embedding.

[0155] X P =X E +E P

[0156] Given an embedding matrix X P , the multi-head self-attention layer projects it into three different subspaces, which can be expressed as:

[0157] {Q, KV} = {X P W Q , X P W K , X P W V},

[0158] Where W Q ,W K ,W V Are three projection matrices, X P Projected onto Q, K, V. The output of the multi-head self-attention layer can be expressed as:

[0159] MultiHead(Q,K,V)=Concat(H1,...,H h )W O ,

[0160] Among them, H i ,i∈{1,2,…,h} is the output of a single scaled dot-product attention head, such as Figure 6 As shown, h is the number of heads. i The calculation of is as follows:

[0161]

[0162] Figure 7is a schematic diagram of the shift window method for calculating self-attention in the encoder architecture proposed in the present invention. In the W-MSA layer, the present invention adopts a conventional window partitioning scheme, such as Figure 7 As shown on the left, self-attention is calculated within each local window. In the SW-MSA layer, the window is shifted, as shown in Figure 7 Shown right.

[0163] For each attention head, the original input Q, K, V is passed through the projection matrix Further projection to different subspaces allows different heads to process different inputs to learn richer information. Then, the attention head calculates the similarity between the projected query and the key through the scaled dot product and the softmax function to obtain the weight of the projected value. The multi-head self-attention layer connects all the outputs Hi from different heads and then passes another projection matrix W O Concat(H1,…,H h ) to a low dimension. However, the original Transformer is extremely computationally expensive. Inspired by the Swin Transformer, the encoder module of the present invention introduces three additional mechanisms to further improve efficiency while maintaining the advantages of the Transformer. Figure 8 A diagram showing a stage in the Swin Transformer architecture, which consists of a patch merging layer and N i The first is the local attention mechanism. The present invention groups blocks into W×W windows and applies the multi-head self-attention layer only within each window. This mechanism is called the window multi-head self-attention (W-MSA) layer. Local attention improves computational efficiency but lacks global perception. The second mechanism solves this problem by introducing an offset window, which is implemented by moving the starting position of the window partition. Specifically, the local attention mechanism uses the conventional window partitioning strategy starting from the upper left corner, while the offset window mechanism shifts the window Figure 7 The cross-connection capability of the offset window mechanism is demonstrated. This mechanism is called the offset window multi-head self-attention (SW-MSA) layer. In addition, the block merging mechanism reduces the feature scale by connecting adjacent blocks, thereby enabling the aggregation of multi-scale information. The encoder of the present invention includes multiple stages, Figure 8 A single module of one stage is shown, which consists of a block merging layer and Ni Swin Transformer blocks. Each Swin Transformer block consists of W-MSA, MLP, SW-MSA, and MLP layers, where MLP stands for multi-layer perceptron.

[0164] D.Decoder

[0165] The LUM decoder is designed based on the Feature Pyramid Network (FPN), a widely used and effective technique in computer vision. As a versatile feature extractor, FPN fully exploits the inherent pyramidal hierarchical structure of features. This allows the extraction of multi-level feature representations.

[0166] Fig. 9 The LUM decoder is shown in FIG. The input of the LUM decoder is the output features from the four stages of the LUM encoder. Through the top-down path, multi-scale feature maps can be generated, each of which contains features at different levels. In the framework of the present invention, the LUM decoder receives the features output by the LUM encoder from the four stages, such as Fig. 9 As shown in Figure 2. These features serve as the basis for the subsequent decoding process. In order to generate feature maps with different levels of detail, the LUM decoder adopts a top-down path. This path uses convolutional layers, residual connections, and pyramid pooling modules to process the input feature maps to generate multi-scale feature maps containing features at different levels of abstraction.

[0167] In addition, lateral connections are introduced to integrate feature information from different levels. These connections combine feature maps from different levels, leading to the creation of a single comprehensive feature map. This final feature map is then passed through the convolutional layer to generate the final output, representing the generated layout graph.

[0168] E. Target Reconstruction

[0169] The model of the present invention can receive or output a target image or an optimized mask. For example, receiving a target image and generating an optimized mask is equivalent to performing OPC. Receiving a masked target image and generating a restored target image is a layout reconstruction process. Combining different inputs and reconstruction targets can enable the LUM of the present invention to acquire knowledge in different fields.

[0170] In order to distinguish the target image and optimize the mask, the present invention designs additional type embedding for the LUM encoder. Specifically, given the input embedding, the present invention applies the type embedding in the following way:

[0171]

[0172] In the formula, X P Indicates type embedding, X E Represents the characteristics of the input layout, E T is the embedding matrix when the input is the target layout, E M It represents the embedding matrix when the input is a mask.

[0173] Similarly, the present invention introduces type embedding for the LUM decoder. The embedding matrix is ​​added to the input of the MLM decoder. The present invention sets the type embedding matrix to be trainable. This mechanism enables the encoder and decoder to distinguish between the two domains. In the pre-training stage, given a layout tile, the present invention first obtains its corresponding target image and optimized mask. Next, the present invention masks these images, and the reconstruction target can be expressed as:

[0174]

[0175] In the above formula, the two subscripts of f represent the input and output domains. Specifically, in the formula, L cross (X) represents the total loss function, Represents the reconstruction result after inputting the masked layout, represents the mask optimization result after masking the layout, X T represents the target layout, represents the lithography simulation estimate after inputting the mask, X M represents the mask image, Represents the mask reconstruction result after the input mask is covered.

[0176] F. Fine-tuning

[0177] In the fine-tuning stage, the present invention freezes the parameters of the pre-trained LUM encoder, adapts the pre-trained model to the HSD and OPC tasks, and trains the OPC decoder and HSD decoder. The fine-tuning process is shown in the algorithm:

[0178]

[0179] During fine-tuning, the parameters of the LUM encoder are first fixed, which means that the encoder does not change during training.

[0180] Next, the algorithm initializes the parameters of the OPC decoder and trains it through multiple training cycles (epochs). In each training cycle, the algorithm samples a training set from the dataset and uses this training set to train the OPC decoder in order to minimize the OPC loss function. After the OPC decoder is trained, the algorithm performs similar steps for the HSD decoder: initialize the parameters of the HSD decoder, sample a training set from the dataset, and try to minimize the HSD loss function through training.

[0181] Throughout the process, the two decoders are trained independently, each optimizing its own parameters and loss functions, while the shared encoder parameters remain unchanged. This training strategy aims to improve the performance of the decoder on a specific task without affecting the stability of the encoder.

[0182] For OPC, the present invention simply fine-tunes the pre-trained mask layout modeling decoder to predict the optimized mask. Given the target image and the true optimized mask, the loss function is defined as:

[0183] L OPC (X T , X M )=L M (X T , X M )+L L2 (X T )+L PVB (X T ).

[0184] Where, L OPC (X T ,X M ) represents the loss function for mask optimization.

[0185] The loss function consists of the following parts:

[0186] By minimizing the similarity L between the predicted mask and the optimized mask M (X T ,X M ), defined as:

[0187]

[0188] In the formula, f TM (X T ) represents the mask optimization result after inputting the layout, X M Represents the mask image.

[0189] Minimize the distance L between the printed image predicted by mask optimization and the target image L2 (X T ), defined as:

[0190]

[0191] In the formula, f Z (f TM (X T )) represents the result of mask optimization after input layout and then lithography simulation, X T Indicates the target layout.

[0192] By minimizing the distance L between the printed images of the maximum and minimum process angles PVB (X T ), improve the robustness of OPC results, and the PVB loss is defined as:

[0193]

[0194] In the formula, It indicates the result of mask optimization after inputting layout and then undergoing abnormal lithography simulation (overexposure). It represents the result of mask optimization after inputting layout and then going through abnormal lithography simulation (defocusing).

[0195] Since the goals of HSD are very different from OPC tasks, a custom HSD decoder was designed specifically for classification. Fig. 9 In , the HSD decoder consists of three consecutive convolutional layers, which receive the output of the previous encoder and predict whether there is a hotspot in each block. Therefore, the output of the HSD decoder is an N×N feature map, where each pixel value represents the probability of a hotspot area. In order to guide the learning of the HSD decoder, the present invention adopts a cross-entropy-based loss function, which is defined as:

[0196]

[0197] Where, L HSD (X T ) represents the loss function of hotspot detection, N represents the number of hotspot areas, x represents the horizontal serial number of the hotspot area, y represents the vertical serial number of the hotspot area, and p xy and Respectively represent the true probability and predicted probability of the hot spot area. Once the present invention constructs the training data and completes the model training, it will no longer be necessary to perform lithography simulation when using the model of the present invention to find hot spots in the test data, thereby greatly improving the detection efficiency. Therefore, the present invention only needs to make a one-time effort to achieve a one-time completion process.

[0198] IV. Experiment

[0199] The present invention uses the widely used deep learning library Pytorch to implement the entire framework LUM. The model is tested on a Linux system equipped with a 2.3GHz Intel Xeon CPU and a single NVIDIA GeForce RTX 3090 GPU. In order to verify the efficiency and robustness of the method of the present invention, the present invention synthesized a large layout dataset SynLayoutDataset to test the performance of LUM.

[0200] A. SynLayout Dataset

[0201] In previous work, two commonly used layout datasets, ICCAD-2012 benchmark and ICCAD-2013, were used for hotspot detection and mask optimization. However, these benchmarks are only used for academic research and contain a very limited number of layouts. Specifically, ICCAD-2013 contains only ten 2μm×2μm metal layer fragments, which is too simple for AI solutions. In order to create a larger dataset, GAN-OPC released about 4k synthesized metal layer tiles. However, its scale is still not enough to meet the training needs of deep learning models currently applied in layout-related tasks. Learning models previously trained by these small datasets usually perform poorly when applied to real industrial scenarios due to weak generalization capabilities.

[0202] In order to further improve the generalization ability of the model, the present invention should generate and utilize more layout graphic data to effectively train the LUM of the present invention. Therefore, the present invention creates a new layout dataset SynLayout, following the layout generation method proposed in the literature. The SynLayout dataset is based on the 32nm technology node, which is the same as the ICCAD 2013 benchmark. The present invention adjusts the line width size to ensure that the shape of the synthesized layout graphics is similar to that in the given benchmark. In order to generate experimental units, all shapes are randomly placed together according to a series of design rules as shown in Table 1. The present invention synthesized 16,000 tiles, and used 70% of the data for training and 30% of the data for testing.

[0203] In addition, the present invention needs to obtain the corresponding optimized masks and hot spots of all generated layout graphics as supervisory signals. An optimization-based ILT method is used to obtain an accurate optimized mask for the generated layout of the present invention. L2 loss, PVB loss and curvature loss are used in the ILT method to ensure the mask while minimizing the maskability. In order to find the hot spots that cause open circuits and short circuits, the present invention samples a series of points, calculates their EPE values, and identifies the locations of EPE violations as hot spots. Fig.11 An example of a synthetic layout pattern that follows legal design rules is shown. The detailed information of the dataset of the present invention is listed in Table 2, where the columns "Train#HS" and "Test#HS" indicate the total number of hotspots in the training set and the test set.

[0204] This data generation process enables LUM to learn from diverse layout patterns and acquire the knowledge required to perform accurate downstream layout tasks, including OPC and HSD. Although generating this training data may require additional effort, it plays a vital role in training a robust and effective model. Fig.12As shown in Figure 1, with the help of the SynLayout dataset, during the training phase, LUM optimizes its parameters through an iterative process, gradually improving its ability to accurately understand the layout. Such a phase is also necessary for fine-tuning the model and making it generalize well to unseen data. Fig.13 (a) and Fig.13 The performance variation curve is depicted in (b).

[0205] Table 1. Design rules followed by the SynLayout dataset

[0206]

[0207] Table 2. SynLayout dataset

[0208]

[0209] B. Comparison of results

[0210] Table 3 shows the comparison results of the framework proposed in this invention with several other state-of-the-art hotspot detectors. The comparison results show that the model LUM of this invention has satisfactory performance. Specifically, the average accuracy of the framework of this invention is 94.21%, compared with 91.94% for ICCAD'21, 89.76% for DAC'19, and 85.66% for TCAD'19. In addition, the framework of this invention also has advantages in suppressing false alarms, reducing the false alarm rates reported by TCAD'19, DAC'19, and ICCAD'21 by 55.2%, 30.0%, and 11.8%, respectively. As for the running time, LUM takes an average of 0.22 seconds to detect hotspots on a single layout, which is slightly slower than ICCAD'21 (0.12 seconds), but faster than the other two methods. This is because the model of this invention is designed for multiple tasks, including hotspot detection and mask optimization, while the architecture of LUM is more complex than that of ICCAD'21. In contrast, the model of ICCAD'21 is only used for hotspot detection and has a simple structure. However, the present invention recognizes that the running times are still comparable and that the better performance is worthwhile for the limited additional time invested.

[0211] Table 3. Performance comparison with other SOTA hotspot detection methods on the SynLayout dataset

[0212]

[0213] What did C.LUM learn?

[0214] The motivation for designing LUM in the present invention is to incorporate the prior knowledge of mask optimization into the model, thereby improving the accuracy of hotspot detection. In order to prove that LUM also learns the process of OPC, the present invention connects the encoder and the OPC decoder and lets LUM receive the target image and directly output the optimized mask without any fine-tuning process. Fig.14 (a) shows an example of pre-training results. Fig.14 (a) with Fig.14 Comparing with (b), we can see that the masks generated without OPC-specific training are very close to the well-optimized masks. This observation suggests that the pre-trained LUM can be equipped with OPC-based knowledge, which contributes to the excellent HSD results.

[0215] Table 4. Mask printability comparison with state-of-the-art methods on ICCAD 2013 benchmark

[0216]

[0217] Table 5. Running time comparison with state-of-the-art methods on the ICCAD 2013 benchmark

[0218]

[0219] To provide quantitative analysis, the present invention further performs OPC-specific training. The methods are tested on the ICCAD 2013 benchmark. Tables 4 and 5 compare the mask printability and runtime performance of various OPC methods. For fair comparison, the present invention adopts the same optimization region as the comparison method. The optimization region is a 1280×1280 region around the center of the layout. “MOSAIC” is a pixel-based ILT method. “DevelSet” is a level set-based ILT method combined with a DNN model to reduce optimization time. “CTM-SRAF” proposes a robust constraint-aware SRAF generation method based on continuous transfer mask (CTM). Then, it jointly optimizes the generated SRAF with the main target graphics with the ILT method to evaluate the effectiveness of the SRAF. Multi-ILT proposes a lithography simulation scheme that works at multiple resolutions of a given mask, achieving satisfactory ILT acceleration. “Ours-Fast” demonstrates the performance of the mask generated by the OPC decoder of the present invention. Fig.14 Examples of OPC results are shown. (a) shows the mask output by the pre-trained LUM without further training. (b) is a well-optimized mask from the “Ours-Exact” setting. (c)(d)(e) show three examples from the “Ours-Fast” setting. Fig.14 (c) Fig.14 (d) and Fig.14(e) shows three examples of masks generated under the “Ours-Fast” setting. “Ours-Exact” is obtained by fine-tuning the mask using the optimization-based ILT method of the present invention. Fig.14 (b) shows Fig.14 (c) is a fine-tuned version of (c). It can be seen that “Ours-Fast” achieves satisfactory results in L2 and PVB performance while significantly reducing the runtime. With acceptable runtime overhead, “Ours-Exact” not only improves L2 but also achieves significant EPE results. The excellent performance shows that the LUM-based OPC of the present invention can achieve better manufacturability. In summary, “Ours-Fast” achieves on-the-fly mask optimization and obtains good results, while “Ours-Exact” achieves excellent performance within an acceptable runtime. The method of the present invention has indeed not yet surpassed Multi-ILT in every indicator, but overall, the present invention believes that they are comparable.

[0220] Fig.15 The HSD and OPC results of a layout are illustrated. (a) shows the hotspot detection result and (b) is the corresponding mask optimization result. The main goal of the present invention's work is to develop a pre-trained layout understanding model that integrates various layout tasks into one framework. Therefore, achieving results comparable to the state-of-the-art model Multi-ILT in mask optimization and surpassing other state-of-the-art work in hotspot detection can be considered as the first step in this direction. In the future, the present invention plans to study the creation of a dedicated decoder framework to further improve the performance of the present invention's model on various tasks.

[0221] Finally, the present invention is also Fig.15 (a) and Fig.15 (b) visualizes the results obtained by applying the OPC decoder and the HSD decoder to the same layout, aiming to show that the detected hot spots correspond to the potential open or short-prone areas in the optimized mask. It can be observed that as the light intensity increases, the probability of short circuits occurring in the highlighted areas of the optimized mask increases significantly. This further proves the accuracy of the LUM framework of the present invention in terms of OPC and HSD.

[0222] D. Ablation studies

[0223] The training of LUM aims to give it the ability to understand the geometric information of layout graphics, so that it can be applied to various downstream layout tasks. To further verify the benefits of unified hotspot detection and mask optimization, the present invention conducts additional ablation studies to evaluate the impact by removing certain supervision loss signals. Fig.16As shown in (a), in the case of OPC, even if the same model architecture and training data are used, the performance of the model on OPC will decrease if only the target input and mask output are used for training. It can be seen that the unified model is 28.7% lower on average in L2 error than the model trained with only OPC. For HSD, the performance of the model on HSD will also decrease if only the target input and hotspot prediction output are used for training. Fig.16 As shown in (b), the hotspot detection accuracy of the unified model is improved by 6.6% compared with the HSD-only training model. Therefore, these ablation studies verify that the proposed LUM model has the ability to understand layout geometry information during the pre-training stage, which is further beneficial for layout tasks, including OPC and HSD.

[0224] V. Conclusion

[0225] In this work, the present invention proposes a layout understanding model (LUM), which is a deep learning-based model that achieves significant performance in both mask optimization and hotspot detection tasks. The model structure of the present invention is inspired by the mask modeling method, which masks part of the input signal and requires the model to restore the content. This model can utilize more training data to improve generalization ability. The present invention proposes different decoders for HSD and OPC tasks while maintaining the same encoder components. During the training phase, the present invention designs multiple reconstruction tasks so that the model can effectively learn the geometric information of the layout graphics and the process of mask optimization. Experimental results show that the model of the present invention exhibits significant performance on the OPC task, and the learned mask optimization knowledge helps to improve the HSD performance. The present invention hopes to provide a method for layout pre-training models through this work. In the future, the present invention will continue to explore how to design task-specific decoder frameworks to further improve the performance of the model on various tasks.

[0226] From the above, the present invention proposes to integrate OPC and HSD into a unified deep learning model, which will help improve the performance of both tasks. In order to bridge the relationship between OPC and HSD, the present invention first pre-trains a layout understanding model (LUM) built based on the modeling technology of the mask layout, which effectively captures the layout geometry information, and then can be fine-tuned on HSD and OPC with limited data. In order to fully pre-train the layout understanding model (LUM), the present invention uses layout generation technology to create a large dataset, solving the problem of large data requirements. Experimental results show that the fine-tuned LUM model has achieved significant performance on both OPC and HSD tasks.

[0227] [7]G.Chen,Z.Yu,H.Liu,Y.Ma,and B.Yu,“DevelSet:Deep neural level setfor instant mask optimization,”in IEEE / ACM International Conference onComputer-Aided Design(ICCAD),2021

[0228]

[14] S.Sun,F.Yang,B.Yu,L.Shang,and X.Zeng,“Efficient ilt via multi-level lithography simulation,”in ACM / IEEE Design Automation Conference(DAC),2023.

[0229]

[25] R.Chen,W.Zhong,H.Yang,H.Geng,X.Zeng,and B.Yu,“Faster region-basedhotspot detection,”in ACM / IEEE Design Automation Conference(DAC),2019,pp.146:1–146:6.

[0230]

[42] B.Zhu,R.Chen,X.Zhang,F.Yang,X.Zeng,B.Yu,and M.D.Wong,“Hotspotdetection via multi-task learning and transformer encoder,”in IEEE / ACMInternational Conference on Computer-Aided Design(ICCAD).IEEE,2021,pp.1–8.

[0231]

[43] H.Yang, J.Su, Y.Zou, Y.Ma, B.Yu, and EFYYoung, "Layout hotspot detection with feature tensor generation and deep biased learning," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), vol.38, no.6, pp.1175–1187, 2019.

[0232]

[44] J.-R.Gao,

[0233] Based on the above embodiments, the embodiments of the present application further provide a computer program, which, when executed on a computer, enables the computer to execute the methods provided in the above embodiments.

[0234] Based on the above embodiments, the embodiments of the present application further provide a computer storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.

[0235] The storage medium may be any available medium that can be accessed by a computer. For example, but not limited to, a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0236] Based on the above embodiments, an embodiment of the present application further provides a chip, which is used to read a computer program stored in a memory to implement the method provided in the above embodiments.

[0237] Based on the above embodiments, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the method provided in the above embodiments is implemented.

[0238] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0239] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0240] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0241] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0242] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A layout understanding model device for mask optimization and hot spot detection of layout graphics, characterized in that: include The LUM encoder, whose input includes the layout tile and type, maps the layout tile to the latent space for latent feature representation; as well as The LUM decoder, whose input includes the latent feature representation and type output by the LUM encoder, decodes and outputs the target image or optimized mask and hot spot area detection classification of the layout graphics according to the type; Wherein, the type includes an output target image or an optimized mask, the type is embedded in a LUM encoder and a LUM decoder, and the type embedding matrix is ​​set to be trainable; Among them, the LUM decoder includes The OPC decoder is used to output the target image or optimize the mask; The HSD decoder is used to detect and classify hotspot areas in the output layout graphics.

2. The layout understanding model device according to claim 1, characterized in that: The LUM encoder comprises a tile merging layer for merging tiles; and Swin Transformer block.

3. The layout understanding model device according to claim 2, characterized in that: The Swin Transformer block includes The window multi-head self-attention layer W-MSA is used to partition the window so that the multi-head self-attention mechanism is applied in each window to calculate the self-attention of the window; The offset window multi-head self-attention layer SW-MSA is used to shift the window partitions so that the model can bridge windows between different layers; The multi-layer perceptron layer MLP is used to further extract features.

4. The layout understanding model device according to claim 3, characterized in that: In the Swin Transformer block, it is characterized in that a window multi-head self-attention layer W-MSA is used for input, and its output is added to the input as the first input and input to the first multi-layer perceptron layer MLP; The output of the first multi-layer perceptron layer MLP is added to the first input as the second input and input to the offset window multi-head self-attention layer SW-MSA; The output of the offset window multi-head self-attention layer SW-MSA is added to the second input as the third input and input to the second multi-layer perceptron layer MLP; The output of the second multi-layer perceptron layer MLP is added to the third input as the output of the Swin Transformer block.

5. The layout understanding model device according to claim 1, characterized in that: in, The OPC decoder receives output features from four stages of the LUM encoder; The OPC decoder uses a top-down path that uses convolutional layers, residual connections, and pyramid pooling modules to process input feature maps to generate multi-scale feature maps containing features at different levels of abstraction; the OPC decoder integrates feature information at different levels through lateral connections that combine feature maps from different levels to facilitate the creation of a single comprehensive feature map, and generates the final output by reading the convolutional layer to represent the generated layout graphics.

6. The layout understanding model device according to claim 1, characterized in that: in, The HSD decoder consists of three consecutive convolutional layers, which receives the output of the previous encoder and predicts whether there is a hotspot in each tile. The output of the HSD decoder is an N×N feature map, in which each pixel value represents the probability of the hotspot area.

7. A pre-training method for a layout understanding model device according to any one of claims 1 to 6, characterized in that: Includes rebuild targets.

8. A method for fine-tuning the layout understanding model device according to any one of claims 1 to 6, comprising: S10. Training the OPC decoder; S20. Train the HSD decoder.

9. An application of the layout understanding model device described in any one of claims 1 to 6 for mask optimization and hot spot area detection and classification.

10. Application of the layout understanding model device described in any one of claims 1 to 6 in a layout understanding task.

Images