Mask defect image generation method and device, equipment and medium

Abstract

The invention discloses a mask defect image generation method and device, equipment and a medium, and relates to the field of machine learning, and the mask defect image generation method comprises the steps: carrying out the feature coding and decoding restoration of a mask defect design drawing through employing a pre-trained mask generation network, and obtaining a mask defect image; obtaining a first mask defect image corresponding to the mask defect design drawing; the mask generation network is obtained based on difference training between a real mask defect image corresponding to a sample mask defect design drawing and a sample mask defect image, and the sample mask defect image is obtained by performing feature coding and decoding reduction on the sample mask defect design drawing through the mask generation network; the method can solve the problem of insufficient mask defect samples in the prior art.

Description

Technical Field

[0001] This invention relates to the field of machine learning, and in particular to a method, apparatus, device, and medium for generating mask defect images. Background Technology

[0002] EUV (Extreme Ultraviolet Lithography) is the mainstream technology for 7nm and below nodes. The EUV mask is the main technical challenge of EUV lithography. Because extreme ultraviolet light is strongly absorbed by most materials, reflective optical components, including the mask, must be used in the optical path. Therefore, unlike traditional lithography masks, EUV masks employ a multi-layer film structure. The EUV mask fabrication process is very complex, and each process step inevitably introduces defects. Defects on the EUV mask directly affect the yield of the lithography process and must be strictly controlled. Therefore, EUV mask defect detection technology has become the core key technology for ensuring defect-free EUV mask fabrication.

[0003] The mask defect images obtained by EUV mask defect detection equipment require high-precision image processing to achieve mask defect classification and recognition. Image processing algorithms for mask defect detection can be divided into traditional image processing algorithms and deep learning methods. The key steps in traditional algorithms are image matching and feature extraction. Commonly used image matching algorithms fall into two main categories: grayscale-based and feature-point-based. Grayscale-based matching algorithms are fewer, while feature-point-based matching methods are numerous, with commonly used algorithms including FAST, BRIEF, SURF, SIFT, and ORB. Image feature extraction commonly uses image segmentation, aiming to extract the contour features of defects (particle information) from the image. Segmentation methods mainly include thresholding, edge analysis, region analysis, clustering, graph theory, and specific theoretical segmentation methods. These traditional detection algorithms are primarily applicable to wafer or DUV defect detection, possessing high detection accuracy and efficiency. However, in EUV detection, the requirements for detection accuracy increase further, and the detection results are increasingly affected by the mask image resolution. The efficiency and sensitivity of traditional algorithms are constrained by the front-end detection equipment, resulting in insufficient accuracy for EUV mask detection.

[0004] Deep learning can achieve higher accuracy and lower cost in mask defect detection. However, the main difficulty in deep learning detection of EUV masks lies in obtaining training samples. The amount of sample data required for training deep learning models is enormous. Obtaining a large number of samples by taking EUV mask images using a scanning electron microscope is extremely costly, and insufficient sample size will lead to insufficient accuracy and robustness of the trained model. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, and medium for generating mask defect images, so as to solve the problem of insufficient mask defect samples in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for generating a mask defect image, comprising:

[0008] Using a pre-trained mask generation network, feature encoding and decoding are performed on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map;

[0009] The mask generation network is trained based on the difference between the real mask defect image and the sample mask defect image corresponding to the sample mask defect design map. The sample mask defect image is obtained by feature encoding and decoding the sample mask defect design map through the mask generation network.

[0010] In one optional embodiment of this application, the mask generation network includes: an encoder and a decoder;

[0011] The step of using a pre-trained mask generation network to perform feature encoding and decoding on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map includes:

[0012] The encoder is used to perform feature encoding on the mask defect design diagram to obtain the mask features of the mask defect design diagram;

[0013] The mask features are decoded and restored using the decoder to obtain a first mask defect image corresponding to the mask defect design drawing.

[0014] In one optional embodiment of this application, it further includes:

[0015] A discriminant network is used to construct a discriminant loss between the sample mask defect image and the real mask defect image;

[0016] The mask generation network is trained based on the discrimination loss.

[0017] In one optional embodiment of this application, the step of constructing a discriminative loss between the sample mask defect image and the real mask defect image using a discriminative network includes:

[0018] The sample mask defect image and the real mask defect image are segmented using a discriminant network to obtain each first segment image of the sample mask defect image and each second segment image of the real mask defect image.

[0019] Furthermore, the discriminant network is used to determine the realism of each first block image and each second block image, thereby obtaining the first realism of each first block image and the second realism of each second block image.

[0020] Based on the first realism and the second realism, the discrimination loss between the sample mask defect image and the real mask defect image is determined.

[0021] In one optional embodiment of this application, determining the discrimination loss between the sample mask defect image and the real mask defect image based on the first realism and the second realism includes:

[0022] Determine the first cross-entropy loss between the first realism and the third realism of each of the first image blocks;

[0023] A second cross-entropy loss is determined between the second and fourth realism values ​​of each second image block; wherein the third realism value indicates that the image block is not real, and the fourth realism value indicates that the image block is real;

[0024] The discrimination loss is determined based on the average of the first cross-entropy loss and the second cross-entropy loss.

[0025] In one optional embodiment of this application, it further includes:

[0026] Determine the first location of the mask defect in the first mask defect image;

[0027] Using a mask defect recognition model, defect recognition is performed on the first mask defect image to determine the second location of the mask defect in the first mask defect image;

[0028] The mask defect recognition model is trained based on the positional difference between the first position and the second position.

[0029] In one optional embodiment of this application, the step of using a mask defect recognition model to perform defect recognition on the first mask defect image and determine the second location of the mask defect in the first mask defect image includes:

[0030] The first mask defect image is input into the mask defect recognition model, and the first feature map of the first mask defect image is obtained by extracting features from the first mask defect image through the convolutional layer of the mask defect recognition model.

[0031] Furthermore, by combining a mask defect recognition model with a selective search algorithm, candidate defect locations in the first mask defect image are marked to obtain a first mask defect image including candidate defect boxes, wherein the candidate defect boxes include candidate defect points of the first mask defect image;

[0032] The first feature map and the first mask defect image including the candidate defect box are pooled through the pooling layer of the mask defect recognition model to obtain a second feature map including the candidate defect box.

[0033] The mask defect recognition model is used to perform defect regression classification on the candidate defect boxes in the second feature map, determine the defect boxes in the candidate defect boxes, and determine the second position of the mask defect in the first mask defect image based on the position of the defect boxes.

[0034] Compared with existing technologies, the mask defect image generation method provided by this invention utilizes a mask generation network trained based on the differences between the real mask defect image corresponding to the sample mask defect design map and the sample mask defect image to perform feature encoding and decoding reconstruction of the mask defect design map, thereby obtaining a first mask defect image. This method can generate a large number of mask defect samples based on the mask defect design map, which is beneficial for constructing the training set of the mask defect recognition model and improving the accuracy of the mask defect recognition model.

[0035] The present invention also provides an apparatus for generating a mask defect image, comprising:

[0036] The mask generation unit is used to use a pre-trained mask generation network to perform feature encoding and decoding on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map;

[0037] The mask generation network is trained based on the difference between the real mask defect image and the sample mask defect image corresponding to the sample mask defect design map. The sample mask defect image is obtained by feature encoding and decoding the sample mask defect design map through the mask generation network.

[0038] Compared with the prior art, the beneficial effects of the mask defect image generation device provided by the present invention are the same as those of the mask defect image generation method described in the above technical solutions, and will not be repeated here.

[0039] The present invention also provides an electronic device, comprising:

[0040] processor;

[0041] Memory used to store the processor's executable instructions;

[0042] The processor is configured to execute the above-described method for generating mask defect images by running instructions in the memory.

[0043] Compared with the prior art, the beneficial effects of the electronic device provided by the present invention are the same as those of the mask defect image generation device described in the above technical solution, and will not be repeated here.

[0044] The present invention also provides a computer storage medium storing instructions that, when executed, implement the above-described method for generating mask defect images.

[0045] Compared with the prior art, the beneficial effects of the computer storage medium provided by the present invention are the same as those of the mask defect image generation device described in the above technical solution, and will not be repeated here. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0047] Figure 1 A flowchart illustrating a method for generating a mask defect image according to an embodiment of this application;

[0048] Figure 2 This is a diagram of the mask generation network structure provided in the embodiments of this application;

[0049] Figure 3 A schematic diagram of a mask defect design provided in an embodiment of this application;

[0050] Figure 4 A flowchart illustrating the training process of the mask generation network provided in this application embodiment;

[0051] Figure 5 A schematic diagram of a first mask defect image provided in an embodiment of this application;

[0052] Figure 6 A flowchart illustrating the image processing of the mask defect recognition model provided in this application embodiment;

[0053] Figure 7 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0054] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0055] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0056] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0057] EUV (Extreme Ultraviolet Lithography) is the mainstream technology for 7nm and below nodes. The EUV mask is the main technical challenge of EUV lithography. Because extreme ultraviolet light is strongly absorbed by most materials, reflective optical components, including the mask, must be used in the optical path. Therefore, unlike traditional lithography masks, EUV masks employ a multi-layer film structure. The EUV mask fabrication process is very complex, and each process step inevitably introduces defects. Defects on the EUV mask directly affect the yield of the lithography process and must be strictly controlled. Therefore, EUV mask defect detection technology has become the core key technology for ensuring defect-free EUV mask fabrication.

[0058] The mask defect images obtained by EUV mask defect detection equipment require high-precision image processing to achieve mask defect classification and recognition. Image processing algorithms for mask defect detection can be divided into traditional image processing algorithms and deep learning methods. The key steps in traditional algorithms are image matching and feature extraction. Commonly used image matching algorithms fall into two main categories: grayscale-based and feature-point-based. Grayscale-based matching algorithms are fewer, while feature-point-based matching methods are numerous, with commonly used algorithms including FAST, BRIEF, SURF, SIFT, and ORB. Image feature extraction commonly uses image segmentation, aiming to extract the contour features of defects (particle information) from the image. Segmentation methods mainly include thresholding, edge analysis, region analysis, clustering, graph theory, and specific theoretical segmentation methods. These traditional detection algorithms are primarily applicable to wafer or DUV defect detection, possessing high detection accuracy and efficiency. However, in EUV detection, the requirements for detection accuracy increase further, and the detection results are increasingly affected by the mask image resolution. The efficiency and sensitivity of traditional algorithms are constrained by the front-end detection equipment, resulting in insufficient accuracy for EUV mask detection.

[0059] Deep learning can achieve higher accuracy and lower cost in mask defect detection. However, the main difficulty in deep learning detection of EUV masks lies in obtaining training samples. The amount of sample data required for training deep learning models is enormous. Obtaining a large number of samples by taking EUV mask images using a scanning electron microscope is extremely costly, and insufficient sample size will lead to insufficient accuracy and robustness of the trained model.

[0060] In view of the above-mentioned technical status, this application provides a method, apparatus, device and medium for generating mask defect images, which will be described in detail in the following embodiments.

[0061] This application first provides a method for generating mask defect images, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a method for generating a mask defect image, as provided in an embodiment of this application.

[0062] like Figure 1 The method for generating the mask defect image includes:

[0063] S101, using a pre-trained mask generation network, feature encoding and decoding are performed on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map; the mask generation network is trained based on the difference between the real mask defect image corresponding to the sample mask defect design map and the sample mask defect image, and the sample mask defect image is obtained by feature encoding and decoding the sample mask defect design map through the mask generation network.

[0064] The mask generation network can be understood as a U-Net architecture, which is often used for image-to-image tasks such as semantic segmentation and image inpainting. In this embodiment, the mask generation network includes an encoder and a decoder.

[0065] Specifically, S101 above includes:

[0066] The encoder is used to perform feature encoding on the mask defect design diagram to obtain the mask features of the mask defect design diagram;

[0067] The mask features are decoded and restored using the decoder to obtain a first mask defect image corresponding to the mask defect design drawing.

[0068] For further details, please refer to... Figure 2 , Figure 2 This is a diagram of the mask generation network structure provided in an embodiment of this application.

[0069] like Figure 2 As shown, the mask generation network includes an encoder 201 and a decoder 202.

[0070] The input to encoder 201 is a mask defect design map. In one optional embodiment of this application, the mask defect design map can be an RGB image with a size of 256×256×3. The first layer of encoder 201 uses 64 4×4 convolutional kernels to convolve the mask defect design map. Each convolutional kernel slides and calculates the dot product on the image. The convolutional result is a convolutional feature map with a size of 128×128×64 (width and height halved, number of channels doubled). The ReLU activation function is then applied to this convolutional feature map to introduce non-linearity.

[0071] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a mask defect design provided in an embodiment of this application.

[0072] like Figure 3 As shown, Figure 3 The design includes a mask defect design drawing 301, which includes the defect location 302.

[0073] Each layer after encoder 201 contains two consecutive convolutional operations, with the depth of the convolutional feature map doubling and the width and height halved after each convolution. For example, the second layer has 128 4×4 convolutional kernels, and the output convolutional feature map size becomes 64×64×128. This process continues until the last layer of encoder 201, which outputs 512 1×1 convolutional feature maps.

[0074] The input to decoder 202 is the convolutional feature map output from the last layer of encoder 201, i.e., 512 1×1 convolutional feature maps. The first layer of decoder 202 uses 512 4×4 transposed convolutional kernels (deconvolutional kernels) for upsampling, which doubles the size of the convolutional feature map, resulting in a 2×2×512 deconvolutional feature map.

[0075] Each layer after decoder 202 is used for deconvolution operations. After each deconvolution, the size of the deconvolution feature map doubles and the number of channels is halved. For example, the second deconvolution will result in a 4×4×256 deconvolution feature map. This process is repeated three times, eventually resulting in an 8×8×1024 deconvolution feature map.

[0076] Since deconvolution can only enlarge an image but not restore it, in order to reduce data loss, after each deconvolution operation, a convolutional feature map of the same size as the deconvolution feature map obtained by deconvolution can be copied from the convolutional layer. This convolutional feature map is then concatenated with the deconvolution feature map to obtain a concatenated feature map. In the next deconvolution process, the concatenated feature map is deconvolved to obtain the corresponding deconvolution feature map.

[0077] In the last layer of the decoder, three 4×4 convolutional kernels are used to convert the deconvolutional feature maps back into an RGB image of size 256×256×3.

[0078] Furthermore, to facilitate understanding of the principle of the mask generation network for generating mask defect images provided in the embodiments of this application, the following description is based on the training process of the mask generation network.

[0079] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating the training process of the mask generation network provided in an embodiment of this application.

[0080] like Figure 4 As shown, during the training of the mask generation network, the first step is to input the sample mask defect design map into the mask generation network to obtain the sample mask defect image output by the mask generation network.

[0081] Among them, the sample mask defect design drawing can be understood as a clear EUV mask defect drawing by the technician.

[0082] Furthermore, after obtaining the sample mask defect image, the sample mask defect image and the real mask defect image of the EUV mask defect map are input into the discrimination network to construct the discrimination loss between the sample mask defect image and the real mask defect image through the discrimination network. Finally, the mask generation network is trained based on the discrimination loss.

[0083] Specifically, the discriminant loss for constructing the sample mask defect image and the real mask defect image includes the following S1 to S3:

[0084] S1, the sample mask defect image and the real mask defect image are segmented using a discriminant network to obtain each first segment image of the sample mask defect image and each second segment image of the real mask defect image;

[0085] S2, the discriminant network is used to determine the realism of each first block image and each second block image, so as to obtain the first realism of each first block image and the second realism of each second block image;

[0086] S3, based on the first realism and the second realism, determine the discrimination loss between the sample mask defect image and the real mask defect image.

[0087] Specifically, S3 includes: determining a first cross-entropy loss between a first realism and a third realism of each first segmented image; determining a second cross-entropy loss between a second realism and a fourth realism of each second segmented image; wherein the third realism indicates that the segmented image is not real, and the fourth realism indicates that the segmented image is real; and determining the discrimination loss based on the average of the first cross-entropy loss and the second cross-entropy loss.

[0088] Specifically, the discriminant network can adopt the PatchGAN structure. In one optional embodiment of this application, the discriminant network performs convolution and regularization on the sample mask defect image and the real mask defect image several times, thereby dividing the two images into 30×30×1 block images, namely, the first block image and the second block image.

[0089] Subsequently, the discriminant network classifies the first block image of the sample mask defect image and the second block image of the real mask defect image, respectively, and determines the first realism of the first block image as a real image and the second realism of the second block image as a real image. The first realism and the second realism can be understood as the probability that the block image is a real image.

[0090] For each of the first block images, calculate the first cross-entropy between the first realism and the third realism (i.e., the probability that the first block image is a real image is 0); and calculate the second cross-entropy between the second realism and the fourth realism (i.e., the probability that the second block image is a real image is 1).

[0091] Finally, the discrimination loss is determined by adding the first cross-entropy and the second cross-entropy and averaging them.

[0092] In this embodiment of the application, the first mask defect image generated in the above manner can largely restore the real mask defect image. Furthermore, the first mask defect image can also participate in the training of the mask defect recognition model to improve the accuracy of the mask defect recognition model.

[0093] Specifically, the mask defect recognition model can be understood as a deep learning model used to identify defects in a mask, thereby achieving mask defect detection. The mask defect recognition model can be trained using the following steps S4 to S6:

[0094] S4, determine the first position of the mask defect in the first mask defect image.

[0095] The first position of the mask defect in the first mask defect image can be marked by relevant personnel in the first mask defect image. Specifically, the first mask defect image can be marked according to the defect position in the mask defect design drawing.

[0096] Please refer to Figure 5 , Figure 5 A schematic diagram of the first mask defect image provided in the embodiments of this application, as shown below. Figure 5 As shown, Figure 5 It includes a first mask defect image 501 and a first position 502 of the mask defect in the first mask defect image.

[0097] S5, using the mask defect recognition model, perform defect recognition on the first mask defect image to determine the second position of the mask defect in the first mask defect image.

[0098] For details, please refer to Figure 6 , Figure 6 This is a flowchart illustrating the image processing of the mask defect recognition model provided in this application embodiment.

[0099] like Figure 6 As shown, after inputting the first mask defect image into the mask defect recognition model, the first feature map of the first mask defect image can be obtained by extracting features from the convolutional layer of the mask defect recognition model.

[0100] Furthermore, by combining a mask defect recognition model with a selective search algorithm, candidate defect locations in the first mask defect image are marked to obtain a first mask defect image including candidate defect boxes, wherein the candidate defect boxes include candidate defect points of the first mask defect image;

[0101] Then, the first feature map and the first mask defect image including the candidate defect box are pooled through the pooling layer of the mask defect recognition model to obtain a second feature map including the candidate defect box.

[0102] The mask defect recognition model is used to perform defect regression classification on the candidate defect boxes in the second feature map, determine the defect boxes in the candidate defect boxes, and determine the second position of the mask defect in the first mask defect image based on the position of the defect boxes.

[0103] Finally, a first mask defect image containing a defect box is output, the defect box corresponding to the second position of the mask defect.

[0104] S6, The mask defect recognition model is trained based on the positional difference between the first position and the second position.

[0105] In practical applications, the mask generation network can be used to generate multiple first mask defect images in batches, where different first mask defect images correspond to different mask defect design drawings; and these first mask defect images are divided into training sets and test sets to complete the training of the mask defect recognition model.

[0106] In the experiment, the trained mask defect recognition model was able to identify defects of different types and sizes on the mask. In the test set, the defect recognition accuracy rate reached over 95%, demonstrating good recognition precision and accuracy.

[0107] In summary, the mask defect image generation method provided in this application utilizes a mask generation network trained based on the differences between the real mask defect image corresponding to the sample mask defect design map and the sample mask defect image to perform feature encoding and decoding reconstruction on the mask defect design map, thereby obtaining a first mask defect image. This method can generate a large number of mask defect samples based on the mask defect design map, which is beneficial for constructing the training set of the mask defect recognition model and improving the accuracy of the mask defect recognition model.

[0108] This application embodiment also provides an apparatus for generating a mask defect image, comprising:

[0109] The mask generation unit is used to use a pre-trained mask generation network to perform feature encoding and decoding on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map;

[0110] The mask generation network is trained based on the difference between the real mask defect image and the sample mask defect image corresponding to the sample mask defect design map. The sample mask defect image is obtained by feature encoding and decoding the sample mask defect design map through the mask generation network.

[0111] In one optional embodiment of this application, the mask generation network includes: an encoder and a decoder;

[0112] The step of using a pre-trained mask generation network to perform feature encoding and decoding on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map includes:

[0113] The encoder is used to perform feature encoding on the mask defect design diagram to obtain the mask features of the mask defect design diagram;

[0114] The mask features are decoded and restored using the decoder to obtain a first mask defect image corresponding to the mask defect design drawing.

[0115] In one optional embodiment of this application, the device is further configured to:

[0116] A discriminant network is used to construct a discriminant loss between the sample mask defect image and the real mask defect image;

[0117] The mask generation network is trained based on the discrimination loss.

[0118] In one optional embodiment of this application, the step of constructing a discriminative loss between the sample mask defect image and the real mask defect image using a discriminative network includes:

[0119] The sample mask defect image and the real mask defect image are segmented using a discriminant network to obtain each first segment image of the sample mask defect image and each second segment image of the real mask defect image.

[0120] Furthermore, the discriminant network is used to determine the realism of each first block image and each second block image, thereby obtaining the first realism of each first block image and the second realism of each second block image.

[0121] Based on the first realism and the second realism, the discrimination loss between the sample mask defect image and the real mask defect image is determined.

[0122] In one optional embodiment of this application, determining the discrimination loss between the sample mask defect image and the real mask defect image based on the first realism and the second realism includes:

[0123] Determine the first cross-entropy loss between the first realism and the third realism of each of the first image blocks;

[0124] A second cross-entropy loss is determined between the second and fourth realism values ​​of each second image block; wherein the third realism value indicates that the image block is not real, and the fourth realism value indicates that the image block is real;

[0125] The discrimination loss is determined based on the average of the first cross-entropy loss and the second cross-entropy loss.

[0126] In one optional embodiment of this application, the device is further configured to:

[0127] Determine the first location of the mask defect in the first mask defect image;

[0128] Using a mask defect recognition model, defect recognition is performed on the first mask defect image to determine the second location of the mask defect in the first mask defect image;

[0129] The mask defect recognition model is trained based on the positional difference between the first position and the second position.

[0130] In one optional embodiment of this application, the step of using a mask defect recognition model to perform defect recognition on the first mask defect image and determine the second location of the mask defect in the first mask defect image includes:

[0131] The first mask defect image is input into the mask defect recognition model, and the first feature map of the first mask defect image is obtained by extracting features from the first mask defect image through the convolutional layer of the mask defect recognition model.

[0132] Furthermore, by combining a mask defect recognition model with a selective search algorithm, candidate defect locations in the first mask defect image are marked to obtain a first mask defect image including candidate defect boxes, wherein the candidate defect boxes include candidate defect points of the first mask defect image;

[0133] The first feature map and the first mask defect image including the candidate defect box are pooled through the pooling layer of the mask defect recognition model to obtain a second feature map including the candidate defect box.

[0134] The mask defect recognition model is used to perform defect regression classification on the candidate defect boxes in the second feature map, determine the defect boxes in the candidate defect boxes, and determine the second position of the mask defect in the first mask defect image based on the position of the defect boxes.

[0135] The apparatus embodiments provided in this example belong to the same concept as the method embodiments of this application, and can execute the mask defect image generation method provided in any of the above embodiments of this application. It possesses the corresponding functional modules and beneficial effects of the mask defect image generation method. Technical details not described in detail in this example can be found in the specific processing content of the mask defect image generation method provided in the above embodiments of this application, and will not be repeated here.

[0136] It should be understood that the units in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0137] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0138] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0139] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0140] This application also provides an electronic device, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of an electronic device structure provided in an embodiment of this application.

[0141] like Figure 7 As shown, the electronic device includes:

[0142] Processor 210;

[0143] Memory 200 for storing executable instructions of the processor 210;

[0144] The processor 210 is configured to execute the mask defect image generation method disclosed in any of the above embodiments by running instructions in the memory 200.

[0145] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0146] A bus can include a pathway for transmitting information between various components of a computer system.

[0147] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0148] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0149] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0150] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, touch screen, etc.

[0151] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0152] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0153] The processor 210 executes the program stored in the memory 200 and calls other devices, and can be used to implement each step of any of the mask defect image generation methods provided in the above embodiments of this application.

[0154] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the mask defect image generation method of various embodiments of this application.

[0155] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0156] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the steps of the mask defect image generation method of various embodiments of this application.

[0157] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0158] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0159] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0160] The modules and sub-modules in the apparatus and terminal in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0161] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0162] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0163] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0164] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0166] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0167] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a mask defect image, characterized in that, include: Using a pre-trained mask generation network, feature encoding and decoding are performed on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map; The mask generation network is trained based on the difference between the real mask defect image and the sample mask defect image corresponding to the sample mask defect design map. The sample mask defect image is obtained by feature encoding and decoding the sample mask defect design map through the mask generation network.

2. The method according to claim 1, characterized in that, The mask generation network includes: an encoder and a decoder; The step of using a pre-trained mask generation network to perform feature encoding and decoding on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map includes: The encoder is used to perform feature encoding on the mask defect design diagram to obtain the mask features of the mask defect design diagram; The mask features are decoded and restored using the decoder to obtain a first mask defect image corresponding to the mask defect design drawing.

3. The method according to claim 1, characterized in that, Also includes: A discriminant network is used to construct a discriminant loss between the sample mask defect image and the real mask defect image; The mask generation network is trained based on the discrimination loss.

4. The method according to claim 3, characterized in that, The method of constructing a discriminative loss between the sample mask defect image and the real mask defect image using a discriminative network includes: The sample mask defect image and the real mask defect image are segmented using a discriminant network to obtain each first segment image of the sample mask defect image and each second segment image of the real mask defect image. Furthermore, the discriminant network is used to determine the realism of each first block image and each second block image, thereby obtaining the first realism of each first block image and the second realism of each second block image. Based on the first realism and the second realism, the discrimination loss between the sample mask defect image and the real mask defect image is determined.

5. The method according to claim 4, characterized in that, The step of determining the discrimination loss between the sample mask defect image and the real mask defect image based on the first realism and the second realism includes: Determine the first cross-entropy loss between the first realism and the third realism of each of the first image blocks; A second cross-entropy loss is determined between the second and fourth realism values ​​of each second image block; wherein the third realism value indicates that the image block is not real, and the fourth realism value indicates that the image block is real; The discrimination loss is determined based on the average of the first cross-entropy loss and the second cross-entropy loss.

6. The method according to claim 1, characterized in that, Also includes: Determine the first location of the mask defect in the first mask defect image; Using a mask defect recognition model, defect recognition is performed on the first mask defect image to determine the second location of the mask defect in the first mask defect image; The mask defect recognition model is trained based on the positional difference between the first position and the second position.

7. The method according to claim 6, characterized in that, The step of using a mask defect recognition model to identify defects in the first mask defect image and determine the second location of the mask defect in the first mask defect image includes: The first mask defect image is input into the mask defect recognition model, and the first feature map of the first mask defect image is obtained by extracting features from the first mask defect image through the convolutional layer of the mask defect recognition model. Furthermore, by combining a mask defect recognition model with a selective search algorithm, candidate defect locations in the first mask defect image are marked to obtain a first mask defect image including candidate defect boxes, wherein the candidate defect boxes include candidate defect points of the first mask defect image; The first feature map and the first mask defect image including the candidate defect box are pooled through the pooling layer of the mask defect recognition model to obtain a second feature map including the candidate defect box. Using the mask defect recognition model, defect regression classification is performed on the candidate defect boxes in the second feature map to determine the defect boxes in the candidate defect boxes, and the second position of the mask defect in the first mask defect image is determined based on the position of the defect boxes.

8. An apparatus for generating a mask defect image, characterized in that, include: The mask generation unit is used to use a pre-trained mask generation network to perform feature encoding and decoding on the mask defect design map to obtain a first mask defect image corresponding to the mask defect design map; The mask generation network is trained based on the difference between the real mask defect image and the sample mask defect image corresponding to the sample mask defect design map. The sample mask defect image is obtained by feature encoding and decoding the sample mask defect design map through the mask generation network.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the method for generating a mask defect image according to any one of claims 1 to 7 by running instructions in the memory.

10. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed, implement the method for generating a mask defect image as described in any one of claims 1 to 7.

Images