Mask defect detection method, device and equipment and storage medium

Abstract

The embodiment of the invention discloses a mask defect detection method and device, equipment and a storage medium. The method comprises the following steps: acquiring an original image corresponding to a mask to be detected; obtaining a mask of a pre-designed to-be-generated image area, and performing image restoration on the original reflection image and the original transmission image based on the mask of the to-be-generated image and a pre-trained neural network model to obtain a restored target reflection image and a target transmission image; and determining the defect condition of the mask to be detected according to the target reflection image, the target transmission image, the original reflection image and the original transmission image. According to the technical scheme, image restoration is carried out on the original image through the obtained mask of the to-be-generated image in combination with the neural network model, so that the defect condition of the to-be-detected mask plate is determined according to the restored image and the original image, accurate defect detection of the mask plate image can be realized, and the defect detection accuracy of the to-be-detected mask plate is improved. And the efficiency and the accuracy of mask defect detection are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for detecting defects in photomasks. Background Technology

[0002] In the field of mask defect detection, there are generally two methods. One method involves using strictly repeating patterns on the mask. By setting the positions of these repeating patterns and comparing them, the specific location of the defect can be determined. This method requires only the detection of repeating patterns and is prone to false positives. The other method involves simulating the mask design drawings to generate a simulated optical image. This simulated image is then compared with the actual optical image to obtain relevant defect information. This method requires design drawings and has high requirements for simulation imaging conditions and computing power, resulting in low efficiency. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, device and storage medium for detecting defects in mask images, which can achieve accurate defect detection of mask images and improve the efficiency and accuracy of mask defect detection.

[0004] According to one aspect of the present invention, an embodiment of the present invention provides a method for detecting defects in a mask, the method comprising:

[0005] Acquire the original image corresponding to the mask to be detected; wherein, the original image includes: the original reflection image and the original transmission image;

[0006] Obtain a pre-designed mask for the region of the image to be generated, and perform image inpainting on the original reflection image and the original transmission image based on the mask and a pre-trained neural network model to obtain the repaired target reflection image and target transmission image.

[0007] The defects of the mask to be detected are determined based on the target reflection image, the target transmission image, the original reflection image, and the original transmission image.

[0008] According to another aspect of the present invention, embodiments of the present invention also provide a mask defect detection device, the device comprising:

[0009] An image acquisition module is used to acquire the original image corresponding to the mask to be detected; wherein, the original image includes: an original reflection image and an original transmission image; the imaging parameters corresponding to the original reflection image and the original transmission image are different;

[0010] The image inpainting module is used to obtain a pre-designed mask of the image region to be generated, and to perform image inpainting on the original reflection image and the original transmission image based on the mask of the image to be generated and a pre-trained neural network model, so as to obtain the repaired target reflection image and target transmission image.

[0011] The defect detection module is used to determine the defect status of the mask to be detected based on the target reflection image, the target transmission image, the original reflection image, and the original transmission image.

[0012] According to another aspect of the present invention, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the mask defect detection method according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute and implement the mask defect detection method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, an embodiment of the present invention also provides a computer program product, characterized in that the computer program product includes a computer program, which, when executed by a processor, implements the mask defect detection method described in any embodiment of the present invention.

[0018] The above-described technical solution of this invention uses the acquired mask of the image to be generated and a neural network model to perform image inpainting on the original image, thereby obtaining the repaired target reflection image and target transmission image. Based on the target reflection image, target transmission image, original reflection image, and original transmission image, the defect status of the mask to be detected is determined. This method can combine the transmission and reflection acquisition device and the neural network model to achieve accurate defect detection of the mask image, thereby improving the efficiency and accuracy of mask defect detection.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a mask defect detection method provided in an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a mask for two symmetrical binary images provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram illustrating the result of integrating all masks according to an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram illustrating a randomly contaminated image according to an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram illustrating an example of an image that was originally contaminated, as provided in an embodiment of the present invention.

[0026] Figure 6 A flowchart illustrating another mask defect detection method provided in an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of the structure of an image acquisition device according to an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of a neural network model architecture provided in an embodiment of the present invention;

[0029] Figure 9 This is a structural block diagram of a mask defect detection device provided in an embodiment of the present invention;

[0030] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] In one embodiment, Figure 1 This is a flowchart illustrating a mask defect detection method according to an embodiment of the present invention. This embodiment is applicable to situations where defects are detected in a mask. The method can be executed by a mask defect detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0034] S110. Acquire the original image corresponding to the mask to be detected; wherein, the original image includes: the original reflection image and the original transmission image.

[0035] The mask to be inspected is the mask for which defect detection is required. The original reflected image can be understood as the image obtained by reflecting off the mask; the original transmitted image can be understood as the image obtained by transmitting off the mask.

[0036] In this embodiment, the original reflected image and the original transmitted image corresponding to the mask pattern can be acquired by a corresponding acquisition device. Specifically, a light source, a reflection camera, a transmission camera, etc. can be added to the acquisition device to perform imaging, thereby obtaining the reflected image and the transmitted image. The imaging parameters corresponding to the original reflected image and the original transmitted image are different. Of course, in addition to the acquisition device mentioned above for acquiring the original transmitted image and the original reflected image, the original image can also be obtained by other means. This embodiment does not limit this.

[0037] S120. Obtain a pre-designed mask for the region of the image to be generated. Based on the mask of the image to be generated and a pre-trained neural network model, perform image inpainting on the original reflection image and the original transmission image to obtain the repaired target reflection image and target transmission image.

[0038] In this embodiment, the mask of the image region to be generated is the mask corresponding to the original image. The mask of the image region to be generated can be understood as an indicator map that defines "where to be processed". Typically, the mask is a binary map with the same size as the original image, and the binary map only has two regions: black and white.

[0039] The neural network model in this embodiment can be the U-Net model, or other neural network model architectures. This embodiment does not impose any specific restrictions.

[0040] In one embodiment, the design of the mask for the image region to be generated includes: designing at least two complementary binary images of the mask to obtain at least two first mask images and second mask images with complementary features after design; wherein, the complementary first mask image and second mask image are merged to form the original image; wherein, the first mask image and the second mask image are both single-channel binary images, the white area in each binary image represents the effective area, usually with a value of 1 or 255, the effective area is the area to be filled by the neural network model, and the black area represents the invalid area, usually with a value of 0, the invalid area is the original value retention area, that is, the pre-supported area, the area is complete, and can provide clues and constraints for the neural network model so that the neural network model can know how to generate the effective area, that is, the area to be generated by the neural network model.

[0041] It should be noted that when designing the mask, it can be two complementary binary images, or three, four, or more binary images with complementary forms. That is, it can be customized according to the user's needs, and this embodiment does not impose specific limitations. For example, to better understand the mask design and the process of combining the designed mask with a neural network model for original image restoration, Figure 2 This is a schematic diagram of a mask for two symmetrical binary images provided in an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the result of integrating all masks according to an embodiment of the present invention. Figure 2 As shown, left 'a' can be understood as the first designed mask image, and right 'b' can be understood as the second designed mask image. The two mask images are complementary and both consist of white and black areas. This can be understood as... Figure 2 The white blocks represent the areas the model will generate, and the black areas represent the areas where the original values ​​are preserved. Since there are two complementary mask images, the model needs to process them twice. The generated parts, i.e., the white blocks, are then complemented to obtain the uncontaminated optical image of the target, which is the restored image. The outermost ring completely preserves the original image and is not included in the detection. Except for the outermost ring, the interior forms complementary patterns, which, when combined, can create an image like... Figure 3 The sum of all the masks shown.

[0042] In this embodiment, based on the designed mask of the image region to be generated, a neural network model is used to perform corresponding image inpainting on the original reflected image and the original transmitted image obtained by reflecting and transmitting the original image, thereby obtaining the repaired target reflected image and target transmitted image. Specifically, the original reflected image, the imaging parameters corresponding to the reflected image, the original transmitted image, the imaging parameters corresponding to the transmitted image, and the first mask image can be input into the neural network model to perform the first image restoration on the original reflected image and the original transmitted image, and output the predicted first restoration result. Based on this, the original reflected image, the imaging parameters corresponding to the reflected image, the original transmitted image, the imaging parameters corresponding to the transmitted image, and the second mask image are input into the neural network model again to perform the second image restoration on the original reflected image and the original transmitted image, and output the predicted second restoration result. Thus, the white areas of the first restoration result and the white areas of the second restoration result are merged to obtain the target restored image.

[0043] In one embodiment, the pre-training process of the neural network model includes: acquiring reflection and transmission images corresponding to the mask to be detected under different imaging parameters using a reflection camera and a transmission camera; filtering each reflection and transmission image to obtain uncontaminated reflection and transmission images, and treating the uncontaminated reflection and transmission images as ideal reflection and ideal transmission images, respectively; performing random contamination labeling on the selected ideal reflection and ideal transmission images to form contaminated reflection and transmission images, and determining the mask images corresponding to the contaminated reflection and transmission images; forming a sample training set by combining the ideal reflection images, the ideal transmission images, and the mask images corresponding to the contaminated reflection and transmission images; and training the neural network model using the sample training set until the loss function of the neural network model reaches its minimum, thereby obtaining a trained neural network model.

[0044] In this embodiment, the reflection camera and the transmission camera are used to acquire images of the mask under different optical paths. Different images correspond to different imaging parameters, which may include, but are not limited to, coma (aberration), telecentricity, modulation transfer function (MTF), etc. Multiple original reflection and transmission images with different imaging parameters can be obtained by modifying the imaging parameters of the cameras multiple times. This allows for subsequent screening of each reflection and transmission image to obtain uncontaminated reflection and transmission images. These uncontaminated images are then considered ideal reflection and transmission images, respectively. A portion of the reflection and transmission images are randomly labeled with contamination markers to form contaminated reflection and transmission images. The corresponding mask images for the contaminated reflection and transmission images are then determined. A sample training set is formed from the ideal reflection and transmission images, as well as the mask images corresponding to the contaminated reflection and transmission images. This sample training set is used to train the neural network model. In this embodiment, by introducing different imaging parameters, a positive impact is brought to the model, increasing its adjustability and solving the problem of data imbalance in the traditional image processing application of defect detection.

[0045] Specifically, the loss function corresponding to this neural network model is expressed as follows: In the formula, a, b, c, and d are all pre-defined constants, which are assigned different parameter values ​​at different training stages. The target reflection image output by the neural network model; The target transmission image output by the neural network model; Represents a mask image. This represents the result of inverting the mask image (i.e., the result after swapping the black and white areas in the mask image). , These are ideal reflection and ideal transmission images. This can be understood as... and This refers to the first mask image and the second mask image with complementary features in the above embodiments.

[0046] In this embodiment, to facilitate a better understanding of the labeling process for random contamination, Figure 4 This is a schematic diagram of a randomly contaminated image provided in an embodiment of the present invention. Of course, in addition to random contamination, the originally contaminated image can also be annotated accordingly. Figure 5 This is a schematic diagram illustrating an example of an image that was originally contaminated, as provided in an embodiment of the present invention. For example... Figure 4 As shown, Figure 4 In the image, 'a' represents the original, uncontaminated optical image. Figure 4In this model, 'b' represents an artificially added contamination image. The mask image 'c', input into the model, can completely cover the defects in image 'b'. For example... Figure 5 As shown, Figure d is the original, uncontaminated optical image, with the original defect in the upper left corner. Figure e is an artificially added contamination image. The mask image f, which is input into the model, can completely cover the defect in Figure e.

[0047] S130. Determine the defects of the mask to be detected based on the target reflection image, target transmission image, original reflection image, and original transmission image.

[0048] In this embodiment, after obtaining the restored defect-free images, namely the target reflection image and the target transmission image, through the neural network model, the target reflection image and the target transmission image can be compared with the original reflection image and the original transmission image, respectively, to determine the defect status of the mask to be detected based on the comparison results. Specifically, the grayscale value of each pixel in the image can be compared to obtain the corresponding comparison results, and the defect status of the mask to be detected can be determined based on any one of the comparison results.

[0049] The above-described technical solution of this invention uses the obtained mask of the image to be generated and a neural network model to perform image inpainting on the original image to obtain the repaired target reflection image and target transmission image; based on the target reflection image, target transmission image, original reflection image and original transmission image, the defect status of the mask to be detected is determined, which can achieve accurate defect detection of the mask image and improve the efficiency and accuracy of mask defect detection.

[0050] In one embodiment, Figure 6 This is a flowchart of another mask defect detection method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment acquires the original image corresponding to the mask to be detected; performs image inpainting on the original reflection image and the original transmission image based on the mask of the image to be generated and a pre-trained neural network model to obtain the repaired target reflection image and target transmission image; and further refines the determination of the defect situation of the mask to be detected based on the target reflection image, the target transmission image, the original reflection image and the original transmission image.

[0051] like Figure 6 As shown, the mask defect detection method in this embodiment may specifically include the following steps:

[0052] S610. Acquire the original reflected image corresponding to the mask to be detected using the reflection camera in the preset image acquisition device, and acquire the original transmitted image corresponding to the mask to be detected using the transmission camera in the preset image acquisition device.

[0053] In this embodiment, the preset image acquisition device includes a reflection camera and a transmission camera. The reflection camera can be used to acquire the original reflection image corresponding to the mask to be detected, and the transmission camera can be used to acquire the original transmission image corresponding to the mask to be detected. Specifically, in addition to the reflection camera and the transmission camera, the preset image acquisition device may also include at least: a light source, a reflection optical path, a mask to be detected, and a transmission optical path; the reflection camera and the transmission camera each correspond to corresponding imaging parameters; the imaging parameters corresponding to the original reflection image and the original transmission image are different.

[0054] For example, to facilitate a better understanding of the preset image acquisition device, Figure 7 This is a schematic diagram of the structure of an image acquisition device according to an embodiment of the present invention, as shown below. Figure 7 As shown, the image acquisition device includes: a light source 710, a reflection optical path 720, a reflection camera 730, a mask to be detected 740, a transmission optical path 750, and a transmission camera 760. In this embodiment, the light emitted from the light source 710 passes through the reflection optical path 720 and illuminates the mask 740. The reflected light then passes through the reflection optical path 720 and enters the reflection camera 730 to form a reflection path image. The transmitted light passes through the transmission optical path 750 and enters the transmission camera 760 to form a transmission path image. It should be noted that the images obtained from the cameras in the reflection and transmission paths are relatively large, and due to the imperfections of the optical system, there are some effects such as coma and telecentrism, resulting in slightly different imaging performance at each field of view.

[0055] S620: Obtain the pre-designed mask for the image region to be generated.

[0056] S630. Input the original reflection image, the first imaging parameter, the original transmission image, the second imaging parameter, and the first mask image into the neural network model respectively to perform the first image restoration on the original reflection image and the original transmission image, and output the predicted first restoration result.

[0057] Wherein, the first imaging parameter is the imaging parameter of the original reflected image; the second imaging parameter is the image parameter of the original transmitted image.

[0058] In this embodiment, the mask for the image region to be generated includes at least two first mask images and second mask images with complementary features. Each mask image is a single-channel binary image. After acquiring the optical images (original reflection image and original transmission image), an image restoration process is performed using a neural network model to obtain a restored defect-free image. Specifically, the original reflection image, first imaging parameters, original transmission image, second imaging parameters, and the first mask image are input into the neural network model for image restoration to obtain the predicted first restoration result.

[0059] S640. The original reflection image, the first imaging parameters, the original transmission image, the second imaging parameters, and the second mask image are input into the neural network model again to perform a second image restoration on the original reflection image and the original transmission image, and the predicted second restoration result is output.

[0060] In this embodiment, after obtaining the first output of the model, i.e. the first restoration result, a second repair process is required. This involves inputting the original reflection image, the first imaging parameters, the original transmission image, the second imaging parameters, and the second mask image back into the neural network model to perform a second image restoration on the original reflection image and the original transmission image, and outputting the predicted second restoration result.

[0061] S650. Merge the white areas of the first restoration result and the white areas of the second restoration result to obtain the target restoration image; wherein, the target restoration image includes: the restored target reflection image and the target transmission image.

[0062] In this embodiment, the white areas generated in the first restoration result and the white areas generated in the second restoration result are merged to obtain the target restoration image; the target restoration image includes: the restored target reflection image and the target transmission image.

[0063] In this embodiment, to facilitate a better understanding of the architecture of the neural network model, Figure 8 This is a schematic diagram of a neural network model architecture provided in an embodiment of the present invention, as shown below. Figure 8 As shown, 801-804 are the input layers of the model. 801 corresponds to the imaging parameters of the entire acquisition system, such as coma, telecentricity, and MTF, which can be obtained through other measurement methods. 802 corresponds to the reflected image, 803 corresponds to the transmitted image, and 804 corresponds to the mask for the region to be generated, which is a black and white image. An example pattern is shown above. Figure 2 As shown. 805-807 are the second layer of neurons immediately following the input layer. Neuron 805 corresponds to the system imaging parameters and is only connected to 801. Neuron 806 is used for learning pattern features and connects to all neurons in the upper layer. Neuron 807 is used for learning the influence of the image mask and is also connected to all neurons in the upper layer. The design of the entire deep learning model follows the design of 805-807, such as 808-810. The neural network model in this embodiment can be a UET or other complex network. For this field, each layer of neurons can be decomposed to achieve the task described in this paper. 811-812 are the output layers, corresponding to the output reflection map and transmission map.

[0064] S660. Compare the grayscale value of each pixel in the target reflection image with that in the original reflection image to obtain the corresponding first comparison result.

[0065] In this embodiment, the first comparison result refers to the image comparison result between the target reflection image and the original reflection image. The first comparison result is obtained by extracting the grayscale value of each pixel in both the target reflection image and the original reflection image, and then comparing the corresponding grayscale values.

[0066] S670. Compare the grayscale values ​​of each pixel in the target transmission image with those in the original transmission image to obtain the corresponding second comparison result.

[0067] In this embodiment, the second comparison result refers to the image comparison result between the target transmission image and the original transmission image. The second comparison result is obtained by extracting the grayscale value of each pixel in the target transmission image and the original transmission image, and then comparing the corresponding grayscale values.

[0068] S680. Determine the defect status of the mask to be inspected based on the first comparison result and / or the second comparison result.

[0069] In this embodiment, the defect status of the mask to be detected can be determined through the first comparison result and / or the second comparison result. This can be understood as follows: if any at least one of the first and second comparison results is abnormal, the mask to be detected is considered defective. Specifically, if the difference in grayscale values ​​between the target reflection image and the original reflection image exceeds a preset threshold, the first comparison result is considered abnormal, indicating that the mask to be detected is defective; if the difference in grayscale values ​​between the target reflection image and the original reflection image does not exceed the preset threshold, the first comparison result is considered normal, indicating that the mask to be detected is defect-free. If the difference in grayscale values ​​between the target transmission image and the original transmission image exceeds a preset threshold, the second comparison result is considered abnormal, indicating that the mask to be detected is defective; if the difference in grayscale values ​​between the target transmission image and the original transmission image does not exceed the preset threshold, the second comparison result is considered normal, indicating that the mask to be detected is defect-free.

[0070] In this embodiment, the above-described technical solution uses a pre-set image acquisition device with a reflection camera and a transmission camera to acquire the original reflection image and the original transmission image corresponding to the mask to be detected, respectively. Based on this, and combined with the mask of the area to be generated, the original reflection image, first imaging parameters, original transmission image, second imaging parameters, and the first mask image are input into a neural network model to perform a first image restoration on the original reflection image and the original transmission image, outputting a predicted first restoration result. The original reflection image, first imaging parameters, original transmission image, second imaging parameters, and the second mask image are then input into the neural network model again to perform a second image restoration on the original reflection image and the original transmission image. Image restoration outputs a predicted second restoration result. The white areas of the first and second restoration results are merged to obtain the target restored image. By comparing the gray values ​​of each pixel in the target reflection image with those in the original reflection image, and the gray values ​​of each pixel in the target transmission image with those in the original transmission image, the defects of the mask to be detected can be obtained. The neural network model can learn some elements on the mask to perform image inference of the contaminated area and achieve the image repair effect. It can further combine the transmission and reflection acquisition device and the neural network model to achieve accurate defect detection of the mask image, improving the efficiency and accuracy of mask defect detection.

[0071] In one embodiment, Figure 9 This is a structural block diagram of a mask defect detection device according to an embodiment of the present invention. This device is suitable for detecting defects in masks and can be implemented in hardware or software. It can be configured in an electronic device to implement a mask defect detection method according to an embodiment of the present invention. Figure 9 As shown, the device includes: an image acquisition module 910, an image restoration module 920, and a defect detection module 930.

[0072] The image acquisition module 910 is used to acquire the original image corresponding to the mask to be detected; wherein the original image includes: an original reflection image and an original transmission image; the imaging parameters corresponding to the original reflection image and the original transmission image are different;

[0073] Image inpainting module 920 is used to obtain a pre-designed mask of the image region to be generated, and to perform image inpainting on the original reflection image and the original transmission image based on the mask of the image to be generated and a pre-trained neural network model to obtain the repaired target reflection image and target transmission image.

[0074] The defect detection module 930 is used to determine the defect status of the mask to be detected based on the target reflection image, the target transmission image, the original reflection image, and the original transmission image.

[0075] In this embodiment of the invention, the image restoration module uses the acquired mask of the image to be generated and a neural network model to perform image restoration on the original image, obtaining the restored target reflection image and target transmission image; the defect detection module determines the defect status of the mask to be detected based on the target reflection image, target transmission image, original reflection image, and original transmission image. It can combine the transmission and reflection acquisition device and the neural network model to achieve accurate defect detection of the mask image, improving the efficiency and accuracy of mask defect detection.

[0076] In one embodiment, the image acquisition module 910 includes:

[0077] A reflection image acquisition unit is used to acquire the original reflection image corresponding to the mask to be detected through a reflection camera in a preset image acquisition device;

[0078] A transmission image acquisition unit is used to acquire the original transmission image corresponding to the mask to be detected through the transmission camera in the preset image acquisition device; wherein, the preset image acquisition device includes at least: a light source, a reflection optical path, a reflection camera, a mask to be detected, a transmission optical path, and a transmission camera; the reflection camera and the transmission camera are respectively corresponding to corresponding imaging parameters; the imaging parameters corresponding to the original reflection image and the original transmission image are different.

[0079] In one embodiment, the mask for the image region to be generated is the mask corresponding to the original image; wherein, the design of the mask for the image region to be generated includes:

[0080] The mask is designed using at least two complementary binary images to obtain at least two first mask images and second mask images with complementary features; wherein the complementarity represents that the first mask image and the second mask image are merged to form the original image;

[0081] In this process, both the first mask image and the second mask image are single-channel binary images. The white areas in each binary image represent the areas to be filled by the neural network model, and the black areas represent the areas where the original values ​​are preserved.

[0082] In one embodiment, the imaging parameters include a first imaging parameter corresponding to the original reflected image and a second imaging parameter corresponding to the original transmitted image;

[0083] Correspondingly, the image restoration module 920 includes:

[0084] The first restoration unit is used to input the original reflection image, the first imaging parameter, the original transmission image, the second imaging parameter, and the first mask image into the neural network model respectively, so as to perform the first image restoration on the original reflection image and the original transmission image, and output the predicted first restoration result;

[0085] The second restoration unit is used to input the original reflection image, the first imaging parameter, the original transmission image, the second imaging parameter, and the second mask image back into the neural network model to perform a second image restoration on the original reflection image and the original transmission image, and output the predicted second restoration result.

[0086] The merging unit is used to merge the white areas of the first restoration result and the white areas of the second restoration result to obtain the target restoration image; wherein, the target restoration image includes: the restored target reflection image and the target transmission image.

[0087] In one embodiment, the pre-training process of the neural network model includes:

[0088] The reflective and transmissive images of the mask to be detected are acquired using a reflective camera and a transmissive camera, respectively, under different imaging parameters.

[0089] Each of the reflected images and each of the transmitted images is screened to obtain uncontaminated reflected images and transmitted images, and the uncontaminated reflected images and transmitted images are regarded as ideal reflected images and ideal transmitted images, respectively;

[0090] Random contamination labeling is performed on selected ideal reflection and ideal transmission images to form contamination-labeled reflection and transmission images, and the mask images corresponding to the contamination-labeled reflection and transmission images are determined.

[0091] The ideal reflection image and the ideal transmission image, as well as the mask images corresponding to the reflection image and the transmission image with contamination markings, are used to form a sample training set;

[0092] The neural network model is trained using the sample training set until the loss function of the neural network model reaches its minimum, thus obtaining a trained neural network model.

[0093] In one embodiment, the loss function is expressed as: In the formula, a, b, c, and d are all pre-defined constants, which are assigned different parameter values ​​at different training stages. The target reflection image output by the neural network model; The target transmission image output by the neural network model; Represents a mask image. This represents the result of inverting the mask image. , These are represented as ideal reflection image and ideal transmission image, respectively.

[0094] In one embodiment, the defect detection module 930 includes:

[0095] The first comparison unit is used to compare the gray value of each pixel in the target reflection image with that in the original reflection image to obtain the corresponding first comparison result;

[0096] The second comparison unit is used to compare the grayscale value of each pixel in the target transmission image with that in the original transmission image to obtain the corresponding second comparison result.

[0097] The defect determination unit is used to determine the defect status of the mask to be detected based on the first comparison result and / or the second comparison result.

[0098] The mask defect detection device provided in this embodiment of the invention can execute the mask defect detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0099] In one embodiment, Figure 10 This is a schematic diagram of an electronic device provided for an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0100] like Figure 10As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0101] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0102] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as mask defect detection methods.

[0103] In some embodiments, the mask defect detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the mask defect detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the mask defect detection method by any other suitable means (e.g., by means of firmware).

[0104] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable mask defect detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0106] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0108] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0109] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0110] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0111] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting defects in a photomask, characterized in that, The method includes: Acquire the original image corresponding to the mask to be detected; wherein, the original image includes: the original reflection image and the original transmission image; Obtain a pre-designed mask for the region of the image to be generated, and perform image inpainting on the original reflection image and the original transmission image based on the mask and a pre-trained neural network model to obtain the repaired target reflection image and target transmission image. The defects of the mask to be detected are determined based on the target reflection image, the target transmission image, the original reflection image, and the original transmission image.

2. The method according to claim 1, characterized in that, The acquisition of the original image corresponding to the mask to be detected includes: The original reflected image corresponding to the mask to be detected is acquired by the reflection camera in the preset image acquisition device, and the original transmitted image corresponding to the mask to be detected is acquired by the transmission camera in the preset image acquisition device. The preset image acquisition device includes at least: a light source, a reflective optical path, a reflective camera, a mask to be detected, a transmittance optical path, and a transmittance camera; the imaging parameters corresponding to the original reflective image and the original transmittance image are different.

3. The method according to claim 1, characterized in that, The mask for the region of image to be generated is the mask corresponding to the original image; The design of the mask for the image region to be generated includes: The mask is designed using at least two complementary binary images to obtain at least two first mask images and second mask images with complementary features after design. In this process, both the first mask image and the second mask image are single-channel binary images. The white areas in each binary image represent the areas to be filled by the neural network model, and the black areas represent the areas where the original values ​​are preserved.

4. The method according to claim 3, characterized in that, The imaging parameters include a first imaging parameter corresponding to the original reflected image and a second imaging parameter corresponding to the original transmitted image; Accordingly, the image inpainting of the original reflection image and the original transmission image based on the mask of the image to be generated and the pre-trained neural network model, to obtain the repaired target reflection image and target transmission image, includes: The original reflection image, the first imaging parameter, the original transmission image, the second imaging parameter, and the first mask image are respectively input into the neural network model to perform the first image restoration on the original reflection image and the original transmission image, and output the predicted first restoration result. The original reflection image, the first imaging parameter, the original transmission image, the second imaging parameter, and the second mask image are respectively input into the neural network model again to perform a second image restoration on the original reflection image and the original transmission image, and output the predicted second restoration result; The white areas in the first restoration result and the white areas in the second restoration result are merged to obtain the target restoration image; wherein, the target restoration image includes: the restored target reflection image and the target transmission image.

5. The method according to claim 1, characterized in that, The pre-training process of the neural network model includes: The reflective and transmissive images of the mask under test are acquired using a reflective camera and a transmissive camera, respectively, under different imaging parameters. Each of the reflected images and each of the transmitted images is screened to obtain uncontaminated reflected images and transmitted images, and the uncontaminated reflected images and transmitted images are regarded as ideal reflected images and ideal transmitted images, respectively; Random contamination labeling is performed on selected ideal reflection and ideal transmission images to form contamination-labeled reflection and transmission images, and the mask images corresponding to the contamination-labeled reflection and transmission images are determined. The ideal reflection image and the ideal transmission image, as well as the mask images corresponding to the reflection image and the transmission image with contamination markings, are used to form a sample training set; The neural network model is trained using the sample training set until the loss function of the neural network model reaches its minimum, thus obtaining a trained neural network model.

6. The method according to claim 5, characterized in that, The loss function is expressed as: In the formula, a, b, c, and d are all pre-defined constants, with different parameter values ​​assigned at different training stages. The target reflection image output by the neural network model; The target transmission image output by the neural network model; Represents a mask image. This represents the result of inverting the mask image. , These are represented as ideal reflection image and ideal transmission image, respectively.

7. The method according to claim 1, characterized in that, Determining the defect status of the mask to be detected based on the target reflection image, the target transmission image, the original reflection image, and the original transmission image includes: The grayscale value of each pixel in the target reflection image is compared with that in the original reflection image to obtain the corresponding first comparison result; The grayscale value of each pixel in the target transmission image is compared with that in the original transmission image to obtain the corresponding second comparison result; The defects of the mask to be detected are determined based on the first comparison result and / or the second comparison result.

8. A mask defect detection device, characterized in that, The device includes: An image acquisition module is used to acquire the original image corresponding to the mask to be detected; wherein, the original image includes: an original reflection image and an original transmission image; the imaging parameters corresponding to the original reflection image and the original transmission image are different; The image inpainting module is used to obtain a pre-designed mask of the image region to be generated, and to perform image inpainting on the original reflection image and the original transmission image based on the mask of the image to be generated and a pre-trained neural network model, so as to obtain the repaired target reflection image and target transmission image. The defect detection module is used to determine the defect status of the mask to be detected based on the target reflection image, the target transmission image, the original reflection image, and the original transmission image.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the mask defect detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the mask defect detection method according to any one of claims 1-7.

Images