Registration method, device and equipment for mask image, medium and product

Abstract

The invention discloses a mask image registration method and device, equipment, a medium and a product, and belongs to the technical field of image processing, the method comprises the steps that a first frequency domain feature image is acquired, and the first frequency domain feature image is an image obtained by converting a mask gray level image to be registered based on a preset spatial domain and frequency domain conversion algorithm; processing the first frequency domain feature image based on a preset sub-pixel image registration algorithm, and determining a sub-pixel registration displacement of the first frequency domain feature image; performing phase modulation on the first frequency domain feature image based on the sub-pixel registration displacement to generate a second frequency domain feature image; and converting the second frequency domain feature image based on a preset frequency domain-space domain conversion algorithm to obtain a registered mask gray level image, and by registering the image in the frequency domain space, the configuration efficiency of the mask image can be improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a mask image registration method, device, equipment, medium and product. Background Art

[0002] In the integrated circuit production process, the mask is a very important process carrier in the integrated circuit manufacturing process, so the detection of the mask is particularly important.

[0003] At present, the detection of the mask is mainly carried out by collecting the mask image through the acquisition device, and then the mask image is pre-registered with the reference image to obtain the registered image, and the mask defect detection is performed based on the registered image. The existing technology mainly registers the mask image through the square difference matching algorithm or the normalized matching algorithm to generate the registered mask image, but the calculation amount of the above matching algorithm is large, which makes the registration process inefficient. Summary of the invention

[0004] The embodiments of the present application provide a mask image registration method, device, equipment, medium and product, which can improve the registration efficiency of the mask grayscale image.

[0005] In a first aspect, an embodiment of the present application provides a method for registering a mask image, comprising:

[0006] Acquire a first frequency domain characteristic image, where the first frequency domain characteristic image is an image obtained by converting the mask grayscale image to be registered based on a preset spatial domain to frequency domain conversion algorithm;

[0007] Processing the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine a sub-pixel registration displacement of the first frequency domain feature image;

[0008] Performing phase modulation on the first frequency domain feature image based on the sub-pixel registration displacement to generate a second frequency domain feature image;

[0009] The second frequency domain feature image is converted based on a preset frequency domain and space domain conversion algorithm to obtain a registered mask grayscale image.

[0010] In a second aspect, an embodiment of the present application provides a mask image registration device, comprising:

[0011] An acquisition module, used for acquiring a first frequency domain characteristic image, where the first frequency domain characteristic image is an image obtained by converting a grayscale image of a mask to be registered based on a preset spatial domain to frequency domain conversion algorithm;

[0012] A determination module, configured to process the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine a sub-pixel registration displacement of the first frequency domain feature image;

[0013] A generating module, configured to perform phase modulation on the first frequency domain feature image based on the sub-pixel registration displacement to generate a second frequency domain feature image;

[0014] The generating module is further used to convert the second frequency domain feature image based on a preset frequency domain and spatial domain conversion algorithm to obtain a registered mask grayscale image.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising:

[0016] a processor and a memory storing computer program instructions;

[0017] When the processor executes the computer program instructions, it is used to perform the mask image registration method of the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the mask image registration method of the first aspect described above is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when processed by a processor, implements the mask image registration method of the first aspect.

[0020] The mask image registration method, device, equipment, medium and product provided in the embodiments of the present application obtain a first frequency domain feature image, and then process the first frequency domain feature image with a preset sub-pixel image registration algorithm to determine the sub-pixel registration displacement of the first frequency domain feature image, and then phase modulate the first frequency domain feature image to generate a second frequency domain feature image, and convert based on the second frequency domain feature image to obtain a registered mask grayscale image. The above method can convert the mask grayscale image into a frequency domain feature image, determine the sub-pixel registration displacement based on the frequency domain feature image and perform phase modulation, which can overcome the problem of low registration rate for spatial domain images in the prior art and improve the registration efficiency of the mask image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A schematic flow chart of a mask image registration method provided for some embodiments of the present application.

[0023] Figure 2 A schematic flow chart of another mask image registration method provided for some embodiments of the present application.

[0024] Figure 3 A schematic flow chart of yet another mask image registration method provided for some embodiments of the present application.

[0025] Figure 4 A schematic flow chart of yet another mask image registration method provided for some embodiments of the present application.

[0026] Figure 5 A schematic diagram of a mask image registration device provided in some embodiments of the present application.

[0027] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0030] Before describing the technical solutions provided by the embodiments of the present application, in order to facilitate the understanding of the embodiments of the present application, the present application first specifically describes the problems existing in the related art:

[0031] With the development of integrated circuits, masks play an important role in the manufacturing process of integrated circuits, so defect detection of masks is particularly important.

[0032] At present, the detection of the mask is mainly carried out by collecting the mask image through the acquisition device, and the defect detection is performed based on the mask image. However, before the defect detection is performed based on the mask image, the mask image and the reference image need to be configured and registered to obtain the registered image, and the defect detection is performed based on the registered image. However, the applicant has found that the registration algorithm of the prior art, such as the square difference matching algorithm or the normalized matching algorithm, has a large amount of calculation for the registration of the mask image, which makes the registration process inefficient.

[0033] Based on this, the embodiments of the present application provide a mask image registration method, device, equipment, medium and product, which can solve the above problems.

[0034] Below, a mask image registration method provided in an embodiment of the present application is described in detail.

[0035] In some embodiments, Figure 1 As shown, the embodiment of the present application provides a method for registering a mask image, which may include the following steps S110-S140:

[0036] S110: Acquire a first frequency domain characteristic image, where the first frequency domain characteristic image is an image obtained by converting the mask grayscale image to be registered based on a preset spatial domain to frequency domain conversion algorithm.

[0037] Here, the mask is a product required in the integrated circuit production process, and the mask is usually composed of glass, quartz substrate, chrome layer and photoresist layer. Its graphic structure can be obtained through plate making process, and the commonly used processing equipment is direct writing lithography equipment, such as laser direct writing lithography machine, electron beam lithography machine, etc.

[0038] A first frequency domain feature image can be obtained. It can be imagined that the above-mentioned first frequency domain feature image is an image obtained by converting the grayscale image of the mask to be registered based on a preset spatial domain-frequency domain conversion algorithm. The above-mentioned spatial domain-frequency domain conversion algorithm may include any one of the following: Fourier transform algorithm, discrete cosine transform algorithm, and wavelet algorithm.

[0039] In some examples, to further improve the calculation rate, the Fourier transform algorithm may include a fast Fourier transform algorithm (Fast Fourier Transform, FFT).

[0040] For example, when the spatial domain image is as shown in the following expression (1), it can be converted into a frequency domain feature image as shown in expression (2) through the fast Fourier transform algorithm.

[0041]

[0042] Among them, M is the length of the image, N is the width of the image, x and y are the pixel coordinates of the image space, and u and v are the frequency domain coordinates of the frequency space.

[0043] Here, the spatial domain feature image is a spatial domain image based on the pixel composition of the image, and the frequency domain feature image is a feature image that describes the characteristics of the image with frequency as the independent variable. The spatial variation of the grayscale value of an image is decomposed into a linear superposition of simple vibration functions with different amplitudes, spatial frequencies and phases.

[0044] S120: Processing the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine a sub-pixel registration displacement of the first frequency domain feature image.

[0045] The above sub-pixel registration displacement is the pixel displacement that each sub-pixel in the grayscale image of the mask to be registered needs to be translated.

[0046] Here, the first frequency domain feature image can be processed by a sub-pixel registration algorithm to determine the sub-pixel registration displacement of the first frequency domain feature image. In some examples, the first frequency domain feature image and the reference frequency domain image can be registered by a sub-pixel image registration algorithm to determine the above sub-pixel registration displacement. For example, feature points are extracted from the first frequency domain feature image and the reference frequency domain image using a feature point extraction algorithm such as the Harris algorithm, and the sub-pixel registration displacement is determined based on matching of the extracted feature points.

[0047] In some examples, the image gradient of the first frequency domain feature image may be obtained by using a gradient algorithm included in the sub-pixel image registration algorithm, and the sub-pixel registration displacement may be determined based on the image gradient.

[0048] S130: performing phase modulation on the first frequency domain feature image based on the sub-pixel registration displacement to generate a second frequency domain feature image.

[0049] The first frequency domain characteristic image may be phase modulated based on the above sub-pixel registration displacement to generate a second frequency domain characteristic image.

[0050] Here, the above-mentioned sub-pixel registration displacement can determine the pixel displacement that each sub-pixel in the grayscale image of the mask to be registered needs to be translated, for example, a 1.5 pixel displacement is required. According to the preset phase difference in the frequency domain between the two images with rigid translation in the spatial domain, the first frequency domain feature image can be phase modulated based on the above-mentioned sub-pixel registration displacement and the preset phase difference to generate a second frequency domain feature image.

[0051] S140: Convert the second frequency domain feature image based on a preset frequency domain and space domain conversion algorithm to obtain a registered mask grayscale image.

[0052] After obtaining the second frequency domain feature image, the second frequency domain feature image can be converted based on a preset frequency domain spatial domain conversion algorithm to obtain a registered mask grayscale image. For example, the frequency domain spatial domain conversion algorithm can be an inverse Fourier transform algorithm.

[0053] In some examples, the inverse Fourier transform algorithm includes a fast inverse Fourier transform algorithm (DiscreteFourier Transform, DFT).

[0054] In some examples, when determining the frequency domain feature image of the above expression (2), an inverse Fourier transform algorithm may be performed thereon to obtain the spatial domain feature image of the above expression (1).

[0055] The embodiment of the present application obtains a first frequency domain feature image, processes the first frequency domain feature image based on a preset sub-pixel image registration algorithm, determines the sub-pixel registration displacement of the first frequency domain feature image, performs phase modulation on the first frequency domain feature image, generates a second frequency domain feature image, and performs conversion based on the second frequency domain feature image to obtain a registered mask grayscale image, converts the mask grayscale image into a frequency domain feature image, determines the sub-pixel registration displacement based on the frequency domain feature image and performs phase modulation, thereby overcoming the problem of low registration rate for spatial domain images in the prior art and improving the registration efficiency of mask images.

[0056] It can be imagined that after obtaining the registered grayscale image of the mask, detection can be performed based on the registered grayscale image of the mask to further determine possible defects of the mask.

[0057] In some embodiments, Figure 2 As shown, the registered mask grayscale image includes a plurality of first sub-pixels with different grayscale values, and each first sub-pixel corresponds to a first sub-pixel coordinate;

[0058] Step S140: After converting the second frequency domain feature image based on a preset frequency domain and space domain conversion algorithm to obtain a registered mask grayscale image, the method may further include the following steps S210-S230:

[0059] S210: Calculate the difference between the grayscale value of each first sub-pixel in the registered mask grayscale image and the grayscale value of the corresponding sub-pixel in the reference grayscale image.

[0060] It can be imagined that the registered mask grayscale image has the same size as the reference grayscale image, and the registered mask grayscale image includes a plurality of first sub-pixels with different grayscale values, each of which corresponds one-to-one to a sub-pixel in the reference grayscale image.

[0061] The difference between the grayscale value of each first sub-pixel and the grayscale value of the corresponding sub-pixel in the reference grayscale image may be calculated to obtain a plurality of difference values.

[0062] S220: When the target difference is greater than or equal to a preset threshold, determining a target first sub-pixel corresponding to the target difference in the registered mask grayscale image.

[0063] Among the multiple differences, a target difference greater than or equal to a preset threshold can be determined, and a target first sub-pixel corresponding to the target difference can be determined in the registered mask grayscale image. The coordinates of the target first sub-pixel can be mapped to a position where a defect may exist in the mask.

[0064] S230: Determine a target position corresponding to the coordinates of the target first sub-pixel in the mask.

[0065] A target position corresponding to the coordinates of the target first sub-pixel may be determined in the mask, and then prompt information carrying the target position may be sent to the user device to prompt the user to detect the target position of the mask.

[0066] The embodiment of the present application obtains the registered mask grayscale image, calculates the difference between the grayscale value of each first sub-pixel in the registered mask grayscale image and the grayscale value of the corresponding sub-pixel in the reference grayscale image, and determines a target difference whose difference is greater than or equal to a preset threshold value, and determines the target position in the mask by the coordinates of the target first sub-pixel corresponding to the target difference, thereby realizing defect detection of the mask, improving the integrated circuit manufacturing process, and facilitating improving product yield.

[0067] In some embodiments, step S120: processing the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine the sub-pixel registration displacement of the first frequency domain feature image may include:

[0068] The image gradient of the first frequency domain feature image is determined based on a preset Taylor algorithm; and the sub-pixel registration displacement is determined based on the image gradient.

[0069] Here, the image gradient of the first frequency domain feature image can be determined based on the Taylor algorithm. For example, by performing a first-order expansion of the above expression (2) based on the Taylor algorithm, the partial derivatives of the sub-pixel registration displacements Δx and Δy can be obtained. The above partial derivatives can be used as the image gradient, and the sub-pixel registration displacements Δx and Δy of the first frequency domain feature image can be determined based on the above image gradient.

[0070] The embodiment of the present application determines the image gradient of the first frequency domain feature image based on the Taylor algorithm; determines the sub-pixel registration displacement based on the image gradient, which can improve the efficiency of determining the sub-pixel registration displacement, thereby improving the registration efficiency of the mask grayscale image.

[0071] It can be imagined that, usually, the size of the original mask grayscale image captured by the acquisition device may be different from that of the reference grayscale image. Therefore, the original mask grayscale image needs to be processed to obtain the mask grayscale image to be registered so that it has the same size as the reference grayscale image.

[0072] In some embodiments, Figure 3 As shown, before step S110: acquiring the first frequency domain feature image, the registration method may further include:

[0073] S310: Acquire a third frequency domain characteristic image and a reference frequency domain image, wherein the third frequency domain characteristic image is an image determined based on the original mask grayscale image, and the reference frequency domain image is an image obtained by converting the reference grayscale image based on a preset spatial domain to frequency domain conversion algorithm.

[0074] Here, a third frequency domain characteristic image and a reference frequency domain image may be obtained. The third frequency domain characteristic image is an image determined based on the original mask grayscale image. The step of determining the third frequency domain characteristic image may include:

[0075] The target pixel with the largest grayscale value is determined in the above-mentioned original mask grayscale image, and a grayscale image with the same size as the reference grayscale image is determined in the original mask grayscale image using the target pixel as the initial most matching pixel point. The above-mentioned grayscale image is subjected to a spatial domain to frequency domain conversion algorithm to obtain the above-mentioned third frequency domain feature image.

[0076] The above spatial domain to frequency domain conversion algorithm is the same as the spatial domain to frequency domain conversion algorithm in step S110.

[0077] S320: Based on the preset first convolution kernel number convolution algorithm, feature matching is performed on the third frequency domain feature image and the reference frequency domain image, and the target frequency domain coordinates with the largest matching value are determined in the third frequency domain feature image, the third frequency domain feature image includes multiple frequency domain feature coordinates, the original mask grayscale image includes multiple second sub-pixels, one second sub-pixel corresponds to one second sub-pixel coordinate, and one frequency domain feature coordinate corresponds to one second sub-pixel coordinate.

[0078] Based on a convolution algorithm with a preset first number of convolution kernels, features of the third frequency domain feature image and the baseline frequency domain image can be extracted and feature matching can be performed respectively, and the target frequency domain coordinates with the largest matching value can be determined in the third frequency domain feature image, which is equivalent to determining the frequency domain coordinates with the highest matching degree with the baseline frequency domain image in the third frequency domain feature image.

[0079] S330: Determine the target second sub-pixel coordinates corresponding to the target frequency domain coordinates in the original mask grayscale image.

[0080] The target second sub-pixel coordinates corresponding to the target frequency domain coordinates can be determined in the original mask grayscale image, that is, the target second sub-pixel coordinates of the sub-pixel with the highest matching degree with the reference grayscale image can be determined in the original mask grayscale image.

[0081] S340: cropping the original mask grayscale image based on the target second sub-pixel corresponding to the target second sub-pixel coordinates to obtain the mask grayscale image to be registered, wherein the mask grayscale image to be registered includes the target second sub-pixel corresponding to the target second sub-pixel coordinates.

[0082] The original mask grayscale image can be cropped with the target second sub-pixel coordinates as the center point to obtain the mask grayscale image to be registered. Here, the mask grayscale image to be registered has the same size as the reference grayscale image.

[0083] The embodiment of the present application obtains a third frequency domain feature image by converting the original mask grayscale image, performs feature matching on the third frequency domain feature image and the reference frequency domain image, determines the target frequency domain coordinates with the highest matching degree in the third frequency domain feature image, and then determines the corresponding target second sub-pixel in the original mask grayscale image, and obtains the mask grayscale image to be registered by cropping the original mask grayscale image, which can realize pixel registration of the original mask grayscale image with an integer pixel size, thereby facilitating further registration of the mask grayscale image to be registered.

[0084] It can be imagined that in order to further improve the accuracy of registration, feature matching can be further performed based on the above-mentioned determination of the target second sub-pixel coordinates to determine the pixel coordinates with the highest matching degree in the original mask grayscale image.

[0085] In some embodiments, Figure 4 As shown, the original mask grayscale image includes a sub-image, the sub-image includes a plurality of sub-pixels, the plurality of sub-pixels of the sub-image include a target second sub-pixel, the third frequency domain feature image includes a sub-frequency domain feature image, the sub-frequency domain feature image is a frequency domain feature image corresponding to the sub-image, and one sub-image corresponds to one sub-frequency domain feature image;

[0086] Step S330: Based on the preset first convolution kernel number convolution algorithm, feature matching is performed on the third frequency domain feature image and the reference frequency domain image, and after the target frequency domain coordinates with the largest matching value are determined in the third frequency domain feature image, the registration method may further include:

[0087] S410: Based on a convolution algorithm with a preset second number of convolution kernels, feature matching is performed on the sub-frequency domain feature image and the reference frequency domain image, and the target sub-frequency domain coordinates with the largest matching value are determined in the sub-frequency domain feature image, wherein the second number of convolution kernels is greater than the first number of convolution kernels.

[0088] Here, after determining the target second sub-pixel, the sub-image can be determined in the original mask grayscale image with the target second sub-pixel as the center. It can be imagined that a convolution algorithm based on the second number of convolution kernels can be used to perform feature matching on the sub-frequency domain feature image and the reference frequency domain image, and determine the target sub-frequency domain coordinates with the largest matching value in the sub-frequency domain feature image corresponding to the sub-image.

[0089] Step S330: determining the target second sub-pixel coordinates corresponding to the target frequency domain coordinates in the original mask grayscale image may include:

[0090] S420: Determine, in the sub-image, target sub-pixel coordinates corresponding to the target sub-frequency domain coordinates.

[0091] Step S340: cropping the original mask grayscale image based on the target second sub-pixel corresponding to the target second sub-pixel coordinates to obtain the mask grayscale image to be registered may include:

[0092] S430: Crop the sub-image based on the target sub-pixel corresponding to the target sub-pixel coordinates to obtain the mask grayscale image to be registered. Afterwards, the target sub-pixel coordinates corresponding to the target frequency domain coordinates can be determined in the sub-image, and the sub-image is cropped with the target sub-pixel coordinates to obtain the mask grayscale image to be registered.

[0093] Here, in the case where the sub-image is smaller than the reference grayscale image, the original mask grayscale image may be cropped with the target sub-pixel coordinates as the center to obtain the mask grayscale image to be registered.

[0094] The embodiment of the present application increases the number of convolution kernels after obtaining the target frequency domain coordinates, and performs feature point matching again on the sub-frequency domain feature image and the reference frequency domain image, which is equivalent to improving the matching accuracy. Feature matching is first performed based on a convolution algorithm with a first number of convolution kernels, and then feature matching is performed based on a convolution algorithm with a second number of convolution kernels. This can improve both the matching efficiency and the matching accuracy.

[0095] In some embodiments, the spatial-frequency-domain conversion algorithm includes a Fourier transform algorithm, and the frequency-space-domain conversion algorithm includes an inverse Fourier transform algorithm.

[0096] Through the above algorithm, it is possible to realize the back-and-forth conversion of the image between the spatial domain and the frequency domain, improve the conversion efficiency, and align the image based on the frequency domain to increase the calculation rate, thereby improving the efficiency of image configuration.

[0097] Based on the same inventive concept, Figure 5 As shown, the embodiment of the present application also provides a mask image registration device, which may include:

[0098] An acquisition module 501 is used to acquire a first frequency domain characteristic image, where the first frequency domain characteristic image is an image obtained by converting a grayscale image of a mask to be registered based on a preset spatial domain to frequency domain conversion algorithm;

[0099] A determination module 502 is used to process the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine the sub-pixel registration displacement of the first frequency domain feature image;

[0100] A generating module 503, configured to perform phase modulation on the first frequency domain feature image based on sub-pixel registration displacement to generate a second frequency domain feature image;

[0101] The generating module 503 is further used to convert the second frequency domain feature image based on a preset frequency domain and space domain conversion algorithm to obtain a registered mask grayscale image.

[0102] In some embodiments, the registered mask grayscale image includes a plurality of first sub-pixels of different grayscale values, and each first sub-pixel corresponds to a first sub-pixel coordinate;

[0103] Generates modules, also used to:

[0104] Calculating the difference between the grayscale value of each first sub-pixel in the registered mask grayscale image and the grayscale value of the corresponding sub-pixel in the reference grayscale image;

[0105] When the target difference is greater than or equal to a preset threshold, determining a target first sub-pixel corresponding to the target difference in the registered mask grayscale image;

[0106] A target position corresponding to the coordinates of the target first sub-pixel is determined in the mask.

[0107] In some embodiments, the determining module is further configured to:

[0108] Determine the image gradient of the first frequency domain feature image based on a preset Taylor algorithm;

[0109] Determine sub-pixel registration displacement based on image gradients.

[0110] In some embodiments, the acquisition module is further configured to:

[0111] Acquire a third frequency domain characteristic image and a reference frequency domain image, wherein the third frequency domain characteristic image is an image determined based on the original mask grayscale image, and the reference frequency domain image is an image obtained by converting the reference grayscale image based on a preset spatial domain frequency domain conversion algorithm;

[0112] Based on a preset convolution algorithm with a first convolution kernel quantity, feature matching is performed on the third frequency domain feature image and the reference frequency domain image, and a target frequency domain coordinate with a maximum matching value is determined in the third frequency domain feature image, where the third frequency domain feature image includes a plurality of frequency domain feature coordinates, and the original mask grayscale image includes a plurality of second sub-pixels;

[0113] Determining the target second sub-pixel coordinates corresponding to the target frequency domain coordinates in the original mask grayscale image;

[0114] The original mask grayscale image is cropped based on the target second sub-pixel corresponding to the target second sub-pixel coordinates to obtain the mask grayscale image to be registered, wherein the mask grayscale image to be registered includes the target second sub-pixel corresponding to the target second sub-pixel coordinates.

[0115] In some embodiments, the original mask grayscale image includes a sub-image, the sub-image includes a plurality of sub-pixels, the plurality of sub-pixels of the sub-image include a target second sub-pixel, and the sub-frequency domain feature image is a frequency domain feature image corresponding to the sub-image;

[0116] Get module, also used for:

[0117] Based on a convolution algorithm with a preset second number of convolution kernels, feature matching is performed on the sub-frequency domain feature image and the reference frequency domain image, and the target sub-frequency domain coordinates with the maximum matching value are determined in the sub-frequency domain feature image, wherein the second number of convolution kernels is greater than the first number of convolution kernels;

[0118] In the sub-image, the target sub-pixel coordinates corresponding to the target sub-frequency domain coordinates are determined; the sub-image is cropped based on the target sub-pixels corresponding to the target sub-pixel coordinates to obtain the mask grayscale image to be registered. In some embodiments, the spatial domain to frequency domain conversion algorithm includes a Fourier transform algorithm, and the frequency domain to spatial domain conversion algorithm includes an inverse Fourier transform algorithm.

[0119] The device of the above embodiment is used to implement the corresponding mask image registration method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0120] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an application embodiment.

[0121] The electronic device 600 may include a processor 601 and a memory 602 storing computer program instructions.

[0122] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0123] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0124] In certain embodiments, memory 602 includes a read-only memory (ROM). The ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of the above, where appropriate.

[0125] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of the present application.

[0126] The processor 601 reads and executes the computer program instructions stored in the memory 602 to implement any one of the mask image registration methods in the above embodiments.

[0127] In one example, the electronic device may further include a communication interface 603 and a bus 604. Figure 6 The processor 601, the memory 602, and the communication interface 603 are connected via a bus 604 and communicate with each other.

[0128] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0129] Bus 604 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 604 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0130] The electronic device of the above embodiment is used to implement the corresponding mask image registration method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0131] In addition, in combination with the mask image registration method in the above embodiment, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any mask image registration method in the above embodiment is implemented.

[0132] In addition, in combination with the mask image registration method in the above embodiment, the present application embodiment can provide a computer program product to implement. When the computer program product instructions are executed by a processor of an electronic device, any mask image registration method in the above embodiment is implemented.

[0133] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0134] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0135] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or devices based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0136] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (device) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0137] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.

Claims

1. A mask image registration method is used to achieve mask image registration, characterized in that: include: Acquire a first frequency domain characteristic image, where the first frequency domain characteristic image is an image obtained by converting the mask grayscale image to be registered based on a preset spatial domain to frequency domain conversion algorithm; Processing the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine a sub-pixel registration displacement of the first frequency domain feature image; Performing phase modulation on the first frequency domain feature image based on the sub-pixel registration displacement to generate a second frequency domain feature image; The second frequency domain feature image is converted based on a preset frequency domain and space domain conversion algorithm to obtain a registered mask grayscale image.

2. The mask image registration method according to claim 1, characterized in that: Before acquiring the first frequency domain feature image, the registration method further includes: Acquire a third frequency domain characteristic image and a reference frequency domain image, wherein the third frequency domain characteristic image is an image determined based on the original mask grayscale image, and the reference frequency domain image is an image obtained by converting the reference grayscale image based on a preset spatial domain frequency domain conversion algorithm; Based on a preset first convolution kernel quantity convolution algorithm, feature matching is performed on the third frequency domain feature image and the reference frequency domain image, and a target frequency domain coordinate with a maximum matching value is determined in the third frequency domain feature image, wherein the third frequency domain feature image includes a plurality of frequency domain feature coordinates, and the original mask grayscale image includes a plurality of second sub-pixels; Determining a target second sub-pixel coordinate corresponding to the target frequency domain coordinate in the original mask grayscale image; The original mask grayscale image is cropped based on the target second sub-pixel corresponding to the target second sub-pixel coordinates to obtain the mask grayscale image to be registered, wherein the mask grayscale image to be registered includes the target second sub-pixel corresponding to the target second sub-pixel coordinates.

3. The mask image registration method according to claim 2, characterized in that: The original mask grayscale image includes a sub-image, the sub-image includes a plurality of sub-pixels, the plurality of sub-pixels of the sub-image include the target second sub-pixel, and one sub-image corresponds to one sub-frequency domain feature image; After performing feature matching on the third frequency domain feature image and the reference frequency domain image based on the convolution algorithm of the preset first convolution kernel number, and determining the target frequency domain coordinates with the maximum matching value in the third frequency domain feature image, the registration method further includes: Based on a convolution algorithm with a preset second number of convolution kernels, feature matching is performed on the sub-frequency domain feature image and the reference frequency domain image, and the target sub-frequency domain coordinates with the maximum matching value are determined in the sub-frequency domain feature image, wherein the second number of convolution kernels is greater than the first number of convolution kernels; The step of determining the target second sub-pixel coordinates corresponding to the target frequency domain coordinates in the original mask grayscale image includes: Determining, in the sub-image, target sub-pixel coordinates corresponding to the target sub-frequency domain coordinates; The method of cropping the original mask grayscale image based on the target second sub-pixel corresponding to the target second sub-pixel coordinates to obtain the mask grayscale image to be registered includes: The sub-image is cropped based on the target sub-pixels corresponding to the target sub-pixel coordinates to obtain the mask grayscale image to be registered.

4. The mask image registration method according to claim 1, characterized in that: The registered mask grayscale image includes a plurality of first sub-pixels of different grayscale values, each first sub-pixel corresponding to a first sub-pixel coordinate; After converting the second frequency domain feature image based on the preset frequency domain and space domain conversion algorithm to obtain the registered mask grayscale image, the method further includes: Calculating the difference between the grayscale value of each first sub-pixel in the registered mask grayscale image and the grayscale value of the corresponding sub-pixel in the reference grayscale image; When the target difference is greater than or equal to a preset threshold, determining a target first sub-pixel corresponding to the target difference in the registered mask grayscale image; A target position corresponding to the coordinates of the target first sub-pixel is determined in the mask.

5. The mask image registration method according to claim 1, characterized in that: The step of processing the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine the sub-pixel registration displacement of the first frequency domain feature image includes: Determining the image gradient of the first frequency domain feature image based on a preset Taylor algorithm; The sub-pixel registration shift is determined based on the image gradient.

6. The mask image registration method according to claim 1, characterized in that: The spatial domain to frequency domain conversion algorithm includes a Fourier transform algorithm, and the frequency domain to spatial domain conversion algorithm includes an inverse Fourier transform algorithm.

7. A mask image registration device is used to realize the mask image registration, characterized in that: include: An acquisition module, used for acquiring a first frequency domain characteristic image, where the first frequency domain characteristic image is an image obtained by converting a grayscale image of a mask to be registered based on a preset spatial domain to frequency domain conversion algorithm; A determination module, configured to process the first frequency domain feature image based on a preset sub-pixel image registration algorithm to determine a sub-pixel registration displacement of the first frequency domain feature image; A generating module, configured to perform phase modulation on the first frequency domain feature image based on the sub-pixel registration displacement to generate a second frequency domain feature image; The generating module is further used to convert the second frequency domain feature image based on a preset frequency domain and spatial domain conversion algorithm to obtain a registered mask grayscale image.

8. An electronic device, characterized in that: The device comprises: a processor, and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the mask image registration method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the mask image registration method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is processed by a processor, the method for registering a mask image according to any one of claims 1 to 6 is implemented.

Images