Image exposure correction method and system

Abstract

The invention provides an image exposure correction method and system, and the method comprises the steps: obtaining a to-be-corrected input image, carrying out the multi-scale decomposition of the input image and a reverse image, and obtaining a corresponding low-frequency component and a corresponding high-frequency component respectively; performing exposure correction processing on the low-frequency component by using a neural network, and performing detail enhancement processing on the high-frequency component; reconstructing the processed low-frequency component and the processed high-frequency component to obtain at least one intermediate correction image; the input image and at least one intermediate correction image are subjected to adaptive fusion, an exposure correction image is output, the complete morphology and clear edge of the mark are effectively recovered, the high-quality and high-reliability input image is provided for a subsequent high-precision positioning algorithm, and the performance basis of an alignment system is fundamentally improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of exposure, and particularly relates to an image exposure correction method and system. BACKGROUND

[0002] In the photolithography, a core process of semiconductor integrated circuit manufacturing, the performance of an exposure machine (Stepper or Scanner) directly determines the line width and yield of a chip. The exposure machine precisely exposes and replicates a pattern on a mask (Mask / Reticle) to a silicon wafer (Wafer) coated with photoresist through a projection objective. In this process, a highly precise alignment system (Alignment System) is required to ensure accurate overlay of the current layer and the previous layer pattern.

[0003] Modern alignment systems usually use image sensors based on machine vision to achieve precise positioning by collecting images of specific alignment marks (Alignment Mark) on the mask and the silicon wafer and calculating the positional deviation thereof.

[0004] However, in the actual imaging process, due to the thickness and composition of different thin film materials (such as silicon nitride and polysilicon) on the silicon wafer surface and the photoresist, as well as the structure of the marks (such as trenches and bars), there are significant differences in their reflectivity. High-reflectivity areas (such as metal marks) are prone to overexposure (saturation) in the image, resulting in blurred edges and loss of topographic details of the marks; while low-reflectivity areas may be underexposed, with reduced signal-to-noise ratio and difficulty in extracting mark features. SUMMARY

[0005] To solve the above technical problems, the application provides an image exposure correction method and system to solve the technical problems in the prior art.

[0006] In one aspect, the application provides the following technical solution, an image exposure correction method, the method comprising:

[0007] Obtaining an input image to be corrected, and performing multi-scale decomposition on the input image and its inverse image to obtain corresponding low-frequency components and high-frequency components, respectively;

[0008] Performing exposure correction processing on the low-frequency components and detail enhancement processing on the high-frequency components using a neural network;

[0009] Reconstructing the processed low-frequency components and high-frequency components to obtain at least one intermediate correction image;

[0010] Adaptively fusing the input image and at least one intermediate correction image to output an exposure correction image.

[0011] Compared with the prior art, the application has the beneficial effects that through multi-scale decomposition and frequency processing, the method can simultaneously solve the two big problems of the most difficult overexposure leading to saturation of the mark edge and underexposure leading to too low signal-to-noise ratio of the mark in the exposure machine image. The complete appearance and clear edge of the mark are effectively restored, and a high-quality and high-credibility input image is provided for the subsequent high-precision positioning algorithm, thereby fundamentally improving the performance basis of the alignment system.

[0012] Further, before the adaptive fusion step, the method further comprises:

[0013] obtaining an original image containing alignment marks;

[0014] identifying a region of interest in which the alignment marks in the original image are located;

[0015] generating a region importance weight map with the same size as the input image based on the identified region of interest, wherein a pixel located in the region of interest is assigned a first weight value, a pixel located outside the region of interest is assigned a second weight value, and the first weight value is greater than the second weight value;

[0016] inputting the input image, the at least one intermediate correction image and the region importance weight map into a fusion network together to output a corresponding weight vector;

[0017] performing weighted fusion on the input image and the at least one intermediate correction image using the weight vector to output a final exposure correction image.

[0018] Further, the step of identifying the region of interest comprises:

[0019] based on the mask of the original image and the preset or historical alignment information of the silicon wafer, calculating the predicted region in the current original image using an alignment information calculation formula, wherein the original image contains the first alignment mark located on the mask and the second alignment mark located on the silicon wafer;

[0020] defining the predicted region as the region of interest;

[0021] The alignment information calculation formula is as follows:

[0022]

[0023] wherein (X1, Y1) and (X2, Y2) are the coordinates of two feature points on the first alignment mark obtained from a pre-stored reference image, and wherein, X1 and Y2 are the intermediate correction images, w1, w2, w3 are the weight vectors related to the image content generated by the fusion network.

[0024] Further, the step of identifying the region of interest comprises:

[0025] performing preliminary edge detection or feature extraction on the original image to directly locate the rough outline of the alignment mark;

[0026] extending the rough outline outward by a predetermined number of pixels to form a bounding box, and defining the region within the bounding box as the region of interest.

[0027] Further, the step of adaptive fusion is realized by the following formula:

[0028] Y = X1 x w1 + X2 x w2 + X x w3

[0029] wherein, Y is the final exposure correction image, X is the input image, X1 and Y2 are the intermediate correction images, w1, w2, w3 are the weight vectors related to the image content generated by the fusion network.

[0030] Further, the method further comprises:

[0031] judging whether the color balance of the exposure correction image meets the requirements;

[0032] if the color balance of the exposure correction image does not meet the requirements, inputting the exposure correction image into a prediction neural network for inference to predict the required red channel gain value and blue channel gain value of the current image;

[0033] correcting the exposure correction image based on the parameters of the red channel gain value and the blue channel gain value using a correction formula to obtain the exposure correction image corrected again.

[0034] Further, the prediction neural network is an improved network based on the ResNet34 architecture, and the improvements include:

[0035] replacing the activation function from ReLU to Leaky ReLU;

[0036] replacing the global average pooling operation at the end of the network with a local pooling operation;

[0037] the final output layer of the network is a fully connected layer, and the output dimension is 2, corresponding to the red channel gain value and the blue channel gain value respectively;

[0038] The training of the prediction neural network adopts a combined loss function;

[0039] The combined loss function comprises:

[0040] L=L1+L2

[0041] Wherein, L1 is a Log-Cosh loss function, L2 is a CosineEmbedding loss function, and L is a combined loss function.

[0042] The correction formula comprises

[0043] Y o =[Y R / R gain ,Y G ,Y B / B gain

[0044] Wherein Y R , Y G , Y B are red, green and blue channel components of an exposure corrected image respectively, and Y o represents the exposure corrected image again.

[0045] In a second aspect, the present application provides the following technical solutions, an image exposure correction system, the system comprises:

[0046] A decomposition module is configured to obtain an input image to be corrected, and perform multi-scale decomposition on the input image and an inverse image thereof to obtain corresponding low-frequency components and high-frequency components respectively;

[0047] A processing module is configured to perform exposure correction processing on the low-frequency components and detail enhancement processing on the high-frequency components by using a neural network;

[0048] A reconstruction module is configured to reconstruct the processed low-frequency components and the high-frequency components to obtain at least one intermediate corrected image;

[0049] A fusion module is configured to perform adaptive fusion on the input image and at least one intermediate corrected image to output an exposure corrected image.

[0050] In a third aspect, the present application provides the following technical solutions, a computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the image exposure correction method as described above when executing the computer program.

[0051] ​In a fourth aspect, the present application provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the image exposure correction method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0053] Figure 1 A flow chart of the image exposure correction method provided by the first embodiment of the present application;

[0054] Figure 2 A structural block diagram of the image exposure correction system provided by the second embodiment of the present application;

[0055] Figure 3 A hardware structure schematic diagram of the computer provided by the third embodiment of the present application.

[0056] The embodiments of the present application will be further described below with reference to the drawings. DETAILED DESCRIPTION

[0057] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, in which the same or similar notations refer to the same or similar elements or elements having the same or similar functions. The embodiments described below by reference to the drawings are exemplary and are intended to explain the embodiments of the present application, and cannot be understood as a limitation of the present application.

[0058] In the description of the embodiments of the present application, it should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0059] In addition, the terms "first", "second", etc. are used only for descriptive purposes and should not be construed as indicating or implying relative importance or an indicated number of technical features. Therefore, the features defined as "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly and specifically limited.

[0060] Embodiment one

[0061] In the first embodiment of the present application, referring to Figure 1 , an image exposure correction method includes the following steps S01 to S04:

[0062] S01, obtaining an input image to be corrected, and performing multi-scale decomposition on the input image and its inverse image to obtain corresponding low-frequency components and high-frequency components, respectively;

[0063] In this embodiment, taking a semiconductor wafer alignment image as an example, the specific processing process is as follows:

[0064] Obtain the semiconductor wafer alignment image to be corrected (i.e. the input image X), which is usually an underexposed or overexposed RGB image with a size of HxWx3 (including RGB three channels).

[0065] To construct a double-path processing basis, generate the inverse image X inv of the input image X, and the calculation formula is:

[0066] X inv =1-X

[0067] Through this operation, for an underexposed image X, its inverse image X inv presents an overexposed state; for an overexposed image X, its inverse image X inv presents an underexposed state, thereby converting the exposure correction problem of a single image into the cooperative processing of a pair of complementary exposure problems.

[0068] Subsequently, the input image X and its inverse image X inv are respectively subjected to Gaussian pyramid decomposition. Taking the input image X as an example, L-layer decomposition (L=3 in this embodiment) is performed to obtain a corresponding low-frequency component set and a high-frequency component set Among them, the bottom layer is the lowest frequency component (presenting a highly blurred image state), is the high-frequency detail under the original image resolution. For the inverse image X invThe same decomposition process as the input image X is performed to obtain the corresponding low-frequency component set and high-frequency component set

[0069] S02, exposure correction processing is performed on the low-frequency components by using a neural network, and detail enhancement processing is performed on the high-frequency components;

[0070] In this embodiment, the low-frequency components and high-frequency components obtained by decomposition are processed respectively, and the specific process is as follows:

[0071] The low-frequency components are stacked with the low-frequency components of the inverse image and then input to the exposure correction network. The network adopts a lightweight U-Net structure, and the encoder and the decoder each contain 4 convolutional layers. After processing by the network, the output is the corrected low-frequency component with the same size as

[0072] The high-frequency components are stacked with the high-frequency components of the inverse image and then input to the detail enhancement network. The network is composed of three consecutive convolutional layers, and the final output after processing is the detail-enhanced high-frequency component

[0073] S03, the low-frequency components and the high-frequency components after processing are reconstructed to obtain at least one intermediate correction image;

[0074] In this embodiment, the Laplacian pyramid reconstruction algorithm is used for image reconstruction, and the specific process is as follows:

[0075] The processed low-frequency components the unprocessed high-frequency components and the enhanced top layer high-frequency components are upsampled and fused to reconstruct an intermediate correction image X1.

[0076] By adjusting the network parameters, this embodiment can generate two intermediate correction images X1 and X2 with different focuses. For example, one of the intermediate correction images focuses on the optimization of global contrast, and the other focuses on the recovery of local details to meet the correction needs in different scenarios. The task of the fusion network is to intelligently select the best part for each pixel from X, X1, and X2. The main method of generating X2 is as follows:

[0077] In the training stage, two sub-networks with different optimization objectives can be trained to produce different outputs.

[0078] Different loss function weights:

[0079] The network used to generate X1: When training its exposure correction and detail enhancement modules, its loss function may focus more on the smooth transition and naturalness of the overall brightness (e.g., L1 Loss with higher weights).

[0080] The network used to generate X2 may have a training loss function that focuses more on local contrast and detail sharpening (e.g., MS-SSIM Loss or GAN Loss with higher weights).

[0081] Results: During inference, the same input image, when passed through these two trained networks, will produce X1 (more natural) and X2 (sharper) with different emphases.

[0082] Fine-tuning of network structure:

[0083] The same network architecture can be used, but different model weight files can be used. One set of weights (Weight A) is optimized to produce conservative correction results (X1), and another set of weights (Weight B) is optimized to produce aggressive augmentation results (X2).

[0084] S04, adaptively fuse the input image with at least one of the intermediate correction images to output an exposure correction image.

[0085] Specifically, prior to the adaptive fusion step, the method further includes:

[0086] S41, Obtain the original image containing alignment marks;

[0087] S42, Identify the region of interest where the alignment markers are located in the original image;

[0088] S43, Based on the identified region of interest, generate a region importance weight map with the same size as the input image, wherein pixels located within the region of interest are assigned a first weight value, pixels located outside the region of interest are assigned a second weight value, and the first weight value is greater than the second weight value.

[0089] S44, input the input image, the at least one intermediate corrected image, and the region importance weight map into a fusion network, and output the corresponding weight vector;

[0090] S45, the input image and the at least one intermediate correction image are weighted and fused using the weight vector to output the final exposure correction image.

[0091] Optionally, the step of identifying the region of interest includes:

[0092] Based on the mask of the original image and the preset or historical alignment information of the silicon wafer, the first alignment mark and the second alignment mark of the original image are calculated by using an alignment information calculation formula to obtain a predicted region appearing in the current original image, wherein the original image contains the first alignment mark on the mask and the second alignment mark on the silicon wafer;

[0093] The predicted region is defined as the region of interest;

[0094] The alignment information calculation formula is as follows:

[0095]

[0096] Wherein (X1, Y1) and (X2, Y2) are the coordinates of two feature points on the first alignment mark obtained from a pre-stored reference image, And (X3, Y3) and (X4, Y4) are the coordinates of two feature points on the second alignment mark obtained from the pre-stored reference image, d is the distance between the two feature points on the second alignment mark, ΔX is the offset of the mask and the silicon wafer in the X direction, ΔY is the offset of the mask and the silicon wafer in the Y direction, and ΔΘ is the relative rotation angle of the mask and the silicon wafer.

[0097] Optionally, the step of identifying the region of interest comprises:

[0098] Performing preliminary edge detection or feature extraction on the original image to directly locate the rough outline of the alignment mark;

[0099] Extending the rough outline outward by a preset number of pixels to form a bounding box, and defining the region within the bounding box as the region of interest.

[0100] In this embodiment, first, an original image I containing mask alignment marks and silicon wafer alignment marks is obtained.

[0101] The pre-stored reference image feature point coordinates are read from the system database, including mask mark points (X1, Y1) and (X2, Y2) and silicon wafer mark points And

[0102] According to ΔX, ΔY, and ΔΘ calculated by the alignment information calculation formula, the mark region in the reference image is subjected to affine transformation to predict its position appearing in the current image I, and the region is defined as the region of interest.

[0103] Optionally, the original image I is grayed and Canny edge detection is performed, and the rough outline of all alignment marks is located by Hough transformation or contour finding function. The minimum circumscribed rectangle of all contours is expanded outward by 20 pixels to form a large bounding box, and the region is the region of interest.

[0104] A full zero matrix M with the same size as the input image x is created (the size is HxW). Set all pixel values in the ROI region to 0.8 (the first weight value), and set the pixel values outside the region to 0.2 (the second weight value). Then, Gaussian blur filtering is performed on M to make the weight transition smooth and avoid obvious boundaries after fusion.

[0105] Specifically, the adaptive fusion step is realized by the following formula:

[0106] Y = X1xw1 + X2xw2 + Xxw3

[0107] Where Y is the final exposure correction image, X is the input image, X1 and X2 are the intermediate correction images, and w1, w2, w3 are the weight vectors related to the image content generated by the fusion network.

[0108] In this embodiment, the input image X, the two intermediate correction images X1 and X2, and the region importance weight map are stacked in the channel dimension to form a HxWx7 tensor (where 3 channels come from X, 3 channels come from X1, 3 channels come from X2, and 1 channel comes from M, for a total of 3+3+3+1 = 7 channels). The tensor is input into a fusion network.

[0109] Fusion network structure: The network consists of four convolutional layers, and the last layer uses a Sigmoid activation function. The network outputs three weight maps w1, w2, w3 with the same size as the input image, each with a size of (each weight map has a size of HxWx1), corresponding to the weights of X1, X2 and X respectively, and satisfying the weight normalization at each pixel position: w1+w2+w3=1.

[0110] According to the formula of adaptive fusion, the final exposure correction image Y is generated by pixel-by-pixel weighted fusion.

[0111] Optionally, the method further comprises:

[0112] determining whether the color balance of the exposure correction image meets the requirements;

[0113] If the color balance of the exposure correction image does not meet the requirements, the exposure correction image is input into a prediction neural network for inference to predict the required red channel gain value and blue channel gain value for the current image.

[0114] correcting the exposure correction image based on parameters of the red channel gain value and the blue channel gain value using a correction formula, to obtain the exposure correction image corrected again.

[0115] Further, the prediction neural network is an improved network based on a ResNet34 architecture, and the improvements include:

[0116] replacing the activation function from ReLU to Leaky ReLU;

[0117] replacing the global average pooling operation at the end of the network with a local pooling operation;

[0118] the final output layer of the network is a fully connected layer, and the output dimension is 2, corresponding to the red channel gain value and the blue channel gain value respectively;

[0119] The training of the prediction neural network uses a combined loss function;

[0120] The combined loss function includes:

[0121] L = L1 + L2

[0122] wherein L1 is a Log-Cosh loss function, L2 is a CosineEmbedding loss function, and L is a combined loss function;

[0123] The correction formula includes

[0124] Y o = [Y R / R gain , Y G , Y B / B gain ]

[0125] wherein Y R , Y G , Y B are the red, green, and blue channel components of the exposure correction image respectively, and Y o represents the exposure correction image corrected again.

[0126] In this embodiment, the mean values of the exposure correction image Y on the RGB three channels are calculated, and the color balance state is evaluated by judging the deviation of mean(R) / mean(G) and mean(B) / mean(G) from the ideal value (usually 1). If the deviation exceeds a set threshold (such as 15%), it is determined that the image color is unbalanced and color correction is needed.

[0127] After scaling the exposure correction image Y to 224x224 size, it is input to the white balance prediction network for gain value prediction.

[0128] The network is improved based on ResNet34 architecture, and the specific improvements include:

[0129] All ReLU activation functions are replaced with Leaky ReLU (negative slope is 0.01);

[0130] The last two layers (global average pooling layer and fully connected layer) of the original network are removed;

[0131] Replace with a local average pooling layer with a kernel size of 4x4, followed by a fully connected layer with an output dimension of 2

[0132] The network finally outputs two scalar values, red channel gain Rgain and blue channel gain Bgain.

[0133] In summary, an image exposure correction method has the following effects:

[0134] Through multi-scale decomposition and frequency processing strategy, the key problems of mark edge saturation caused by overexposure and low signal-to-noise ratio caused by underexposure in exposure machine image are solved. After multi-scale decomposition of the input image and its inverse image, the low-frequency component is corrected for exposure, and the high-frequency component is enhanced for details. While effectively improving the overall exposure level, the image detail information is maximally preserved and enhanced. This feature ensures the clear presentation of small features such as alignment marks in semiconductor images, providing high-quality and high-reliability input images for subsequent high-precision positioning algorithms, fundamentally improving the performance of the alignment system and significantly improving the accuracy of subsequent feature recognition.

[0135] Through the design of regional importance weight map, the calculation resources of the fusion network and the attention of the alignment algorithm are preferentially concentrated in the mark area (ROI). The exposure correction effect and detail clarity of the mark area are optimized, while the background area remains natural transition. This "precise power" mode maximizes the positioning error introduced by image quality problems, directly reduces the overlay error, and has key significance for improving the yield of advanced process chips.

[0136] Adaptive fusion with dynamic weight vectors generated based on the fusion network can dynamically adjust the fusion ratio of the input image and the intermediate corrected image according to the image content characteristics. Compared with the fixed weight fusion strategy, it has stronger scene adaptability. This makes the system can intelligently cope with image changes caused by different reflectivity materials and different process layers, and can output stable and reliable correction results in various working conditions, reduce equipment alarms or production interruptions caused by image quality fluctuations, and meet the diversified image correction needs in semiconductor manufacturing.

[0137] In the ROI positioning aspect, the prediction method based on historical positioning information has extremely fast calculation speed, almost no increase in processing time, and perfect match with the rhythm requirement of the high-speed and continuous production of the exposure machine; and the ROI positioning method based on images as a redundant backup can still work reliably in the initialization or abnormal condition, and the combination of the two ensures the continuity and reliability of the production process.

[0138] For the color deviation problem of different films on the surface of a silicon wafer under specific illumination, the automatic white balance post-processing function judges and corrects the color balance of the corrected image through a prediction neural network, effectively corrects the color distortion related to the process material, and avoids the color deviation problem that may occur in the exposure correction process. This ensures that the image reflects the true geometric shape rather than color interference, further improves the accuracy and consistency of the alignment measurement, and significantly improves the reliability of image analysis for semiconductor defect detection and other links that rely on color information.

[0139] Embodiment Two

[0140] As shown in Figure 2 The second embodiment of the present application provides an image exposure correction system, which comprises:

[0141] A decomposition module 10 is configured to obtain an input image to be corrected, and perform multi-scale decomposition on the input image and its inverse image to obtain corresponding low-frequency components and high-frequency components, respectively;

[0142] A processing module 20 is configured to perform exposure correction processing on the low-frequency components and detail enhancement processing on the high-frequency components by using a neural network;

[0143] A reconstruction module 30 is configured to reconstruct the processed low-frequency components and high-frequency components to obtain at least one intermediate correction image;

[0144] A fusion module 40 is configured to perform adaptive fusion on the input image and at least one intermediate correction image to output an exposure correction image.

[0145] The image exposure correction system provided by the embodiment of the present application has the same implementation principle and technical effects as the foregoing method embodiment, and for brevity of description, the part not mentioned in the system embodiment can be referred to the corresponding content in the foregoing method embodiment.

[0146] Embodiment Three

[0147] As shown in Figure 3As shown in the third embodiment of the present application, the embodiment of the present application provides the technical scheme of a computer, comprising a memory 202, a processor 201, and a computer program stored in the memory 202 and capable of running on the processor 201, and the processor 201 implements the image exposure correction method as described above when executing the computer program.

[0148] Specifically, the processor 201 can include a central processing unit, or a specific integrated circuit, or can be configured as one or more integrated circuits implementing the embodiments of the present application.

[0149] The memory 202 can include a mass storage for data or instructions. For example, but not limited to, the memory 202 can include a hard disk drive, a floppy disk drive, a solid-state drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus drive, or a combination of two or more of the above. The memory 202 can include a removable or non-removable medium, as appropriate. The memory 202 can be internal or external to the data processing device, as appropriate. In a particular embodiment, the memory 202 is a non-volatile memory. In a particular embodiment, the memory 202 includes a read-only memory and a random access memory. The ROM can be a mask-programmed ROM, a programmable ROM, an erasable PROM, an electrically erasable PROM, an electrically reprogrammable ROM, or a flash memory, or a combination of two or more of the above, as appropriate. The RAM can be a static random access memory or a dynamic random access memory, as appropriate, wherein the DRAM can be a fast page mode dynamic random access memory, an extended data output dynamic random access memory, a synchronous dynamic random access memory, etc.

[0150] The memory 202 can be used to store or cache various data files required for processing and / or communication, and possible computer program instructions executed by the processor 201.

[0151] The processor 201 reads and executes the computer program instructions stored in the memory 202 to implement the image exposure correction method described above.

[0152] In some embodiments, the computer can further include a communication interface 203 and a bus 200. As Figure 3 shown, the processor 201, the memory 202, and the communication interface 203 are connected by the bus 200 and complete communication with each other.

[0153] The communication interface 203 is configured to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 203 can also realize the data communication between other components, such as external devices, image / data acquisition devices, databases, external storage, image / data processing workstations and the like.

[0154] Bus 200 includes hardware, software, or both, to couple components of the computer to each other and to couple components to other systems. Bus 200 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, a peripheral bus, a local bus, a global bus. By way of example and not limitation, bus 200 can include a graphics accelerator interface or other graphics bus, enhanced industry standard architecture bus, frontside bus, ultra path interconnect, industry standard architecture bus, wireless bandwidth interconnect, low pin count bus, memory bus, micro channel architecture bus, peripheral component interconnect bus, PCI Express bus, serial advanced technology attachment bus, video electronics standards association local bus, or other suitable bus or combination of two or more of these. Where appropriate, bus 200 can include one or more buses. Although the present embodiments describe and show a particular bus, the present application contemplates any suitable bus or interconnect.

[0155] Embodiment Four

[0156] In the fourth embodiment of the present application, in combination with the image exposure correction method described above, the embodiment of the present application provides the following technical solution, a storage medium, the storage medium has a computer program stored thereon, the computer program is executed by the processor to realize the image exposure correction method described above.

[0157] Those skilled in the art can understand that the logic and / or steps described in the flowchart and / or otherwise described herein, for example, can be considered as a sequence of data tables of executable instructions for realizing the logic function, which can be specifically embodied in any computer readable medium for use by or in conjunction with an instruction execution system, device or equipment. For the present specification, the "computer readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in conjunction with an instruction execution system, device or equipment.

[0158] More specific examples of the computer-readable medium include the following: an electrical connection having one or more wires, a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CD-ROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then employable by a computer. Examples of computer-readable media that are further within the spirit of the present application are a computer program product, a computer readable storage medium, and a computer.

[0159] It should be understood that aspects of the application can be implemented in hardware, software, firmware, or combinations thereof. In the embodiments described above, various steps or methods can be implemented, for example, in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following techniques, which are well known in the art, can be used to implement the application: a hybrid of the techniques mentioned above; a combination of one or more of the techniques mentioned above; or one or more other techniques that are well known in the art; or some combination thereof.

[0160] The technical features of the above-described embodiments can be combined in any manner, and in order to make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present disclosure.

[0161] The above-described embodiments are merely representative of several embodiments of the present application, and the description is relatively specific and detailed, but it should not be understood as limiting the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image exposure correction method characterized by, The method comprises: obtaining an input image to be corrected, and performing multi-scale decomposition on the input image and its inverse image to obtain corresponding low-frequency components and high-frequency components respectively; performing exposure correction processing on the low-frequency components and detail enhancement processing on the high-frequency components by using a neural network; reconstructing the processed low-frequency components and high-frequency components to obtain at least one intermediate corrected image; performing adaptive fusion on the input image and the at least one intermediate corrected image to output an exposure corrected image.

2. The image exposure correction method according to claim 1, characterized by, Before the step of adaptive fusion, the method further comprises: obtaining an original image containing alignment marks; identifying a region of interest in the original image where the alignment marks are located; generating a region importance weight map with the same size as the input image based on the identified region of interest, wherein pixels located in the region of interest are assigned a first weight value, and pixels located outside the region of interest are assigned a second weight value, and the first weight value is greater than the second weight value; inputting the input image, the at least one intermediate corrected image and the region importance weight map into a fusion network to output a corresponding weight vector; performing weighted fusion on the input image and the at least one intermediate corrected image by using the weight vector to output a final exposure corrected image.

3. The image exposure correction method according to claim 2, characterized by, The step of identifying the region of interest comprises: calculating a predicted region in the current original image where the first alignment mark and the second alignment mark of the original image appear based on a mask of the original image and preset or historical alignment information of a silicon wafer by using an alignment information calculation formula, wherein the original image contains the first alignment mark located on the mask and the second alignment mark located on the silicon wafer; defining the predicted region as the region of interest; the alignment information calculation formula is as follows: wherein (X1, Y1) and (X2, Y2) are coordinates of two feature points on the first alignment mark obtained from a pre-stored reference image, and (X1, Y1) and (X2, Y2) are coordinates of two feature points on the second alignment mark obtained from a pre-stored reference image, d is a distance between the two feature points on the second alignment mark, ΔX is an offset of the mask and the silicon wafer in the X direction, ΔY is an offset of the mask and the silicon wafer in the Y direction, and ΔΘ is a relative rotation angle of the mask and the silicon wafer.

4. The image exposure correction method according to claim 2, characterized by, The step of identifying the region of interest comprises: directly locating a rough outline of the alignment mark by performing preliminary edge detection or feature extraction on the original image; extending the rough outline outward by a preset number of pixels to form a bounding box, and defining the region in the bounding box as the region of interest.

5. The image exposure correction method according to claim 2, characterized by, The step of adaptive fusion is realized by the following formula: Y = X1 × w1 + X2 × w2 + X × w3 wherein Y is the final exposure corrected image, X is the input image, X1 and X2 are the intermediate corrected images, and w1, w2 and w3 are weight vectors related to image content generated by the fusion network.

6. The image exposure correction method according to claim 1, characterized by, The method further comprises: determining whether the color balance of the exposure corrected image meets the requirements; if the color balance of the exposure corrected image does not meet the requirements, inputting the exposure corrected image into a prediction neural network for inference to predict the required red channel gain value and blue channel gain value of the current image; correcting the exposure corrected image based on the parameters of the red channel gain value and the blue channel gain value by using a correction formula to obtain a re-corrected exposure corrected image.

7. The image exposure correction method according to claim 6, characterized by, The prediction neural network is an improved network based on the ResNet34 architecture, and the improvement includes: replace the activation function from ReLU to Leaky ReLU; replace the global average pooling operation at the end of the network with a local pooling operation; the final output layer of the network is a fully connected layer, and the output dimension is 2, corresponding to the red channel gain value and the blue channel gain value respectively; the training of the prediction neural network adopts a combined loss function; the combined loss function includes: L=L1+L2 wherein, L1 is a Log-Cosh loss function, L2 is a CosineEmbedding loss function, and L is a combined loss function; the correction formula includes Y o = [Y R / R gain , Y G , Y B / B gain ] where Y R , Y G , Y B are the red, green, blue channel components of the exposure corrected image, respectively, and Y o represents the exposure corrected image again.

8. An image exposure correction system characterized by comprising: the system includes: a decomposition module configured to obtain an input image to be corrected, and perform multi-scale decomposition on the input image and its inverse image to obtain corresponding low-frequency components and high-frequency components respectively; a processing module configured to perform exposure correction processing on the low-frequency components and detail enhancement processing on the high-frequency components by using a neural network; a reconstruction module configured to reconstruct the processed low-frequency components and high-frequency components to obtain at least one intermediate correction image; a fusion module configured to perform adaptive fusion on the input image and at least one intermediate correction image to output an exposure correction image.

9. A computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the image exposure correction method of any one of claims 1-7.

10. A storage medium, characterized by The storage medium has a computer program stored thereon, and the computer program is executed by the processor to implement the image exposure correction method of any one of claims 1-7.

Images