Image quality improvement system and image quality improvement method

Abstract

This invention provides a high-precision and highly reliable image quality improvement system and method that can learn with appropriate teaching information even for samples whose images are prone to change with each shooting, in the context of image quality improvement of low-quality images through machine learning. A system for improving the image quality of low-quality images includes: an image quality improvement unit for improving the image quality of low-quality images; a deformation prediction unit for predicting the amount of deformation between a first low-quality image included in an input low-quality image column and a second low-quality image different from the first low-quality image; and a deformation correction unit for correcting any one of a first predicted image obtained by applying the image quality improvement unit to the first low-quality image, the second low-quality image, and a second predicted image obtained by applying the image quality improvement unit to the second low-quality image based on the deformation amount predicted by the deformation prediction unit, and for learning such that the evaluation of the loss function between the first predicted image corrected by the deformation correction unit and the second low-quality image or the second predicted image decreases, or the evaluation of the loss function between the first predicted image and the second low-quality image or the second predicted image corrected by the deformation correction unit decreases.

Description

TECHNICAL FIELD

[0001] The present application relates to a structure of an inspection / measurement apparatus that performs inspection or measurement using an electron microscope and control thereof, and particularly relates to a technology effective for inspection or measurement of a semiconductor wafer or a liquid crystal panel in which a shot damage caused by an electron beam is likely to occur. BACKGROUND

[0002] In a production line of a semiconductor, a liquid crystal panel, or the like, if a defect is generated at an early stage of a process, the operation of a subsequent process is completely useless, and thus an inspection / measurement process is provided at each main part of the process, and manufacturing is performed while confirming / maintaining a certain yield. In these inspection / measurement processes, for example, a length measurement SEM (CD-SEM: Critical Dimension-SEM) and a defect review SEM (Defect Review-SEM) using a scanning electron microscope (SEM) are used.

[0003] In inspection / measurement using an electron microscope, a high-quality image is generated by accumulating a plurality of shot results to improve accuracy and is used. However, since the increase in the number of shots leads to a decrease in throughput, it is required to generate a high-quality image with as few shots as possible.

[0004] As background art of the present technical field, for example, there is a technology like Patent Literature 1. In Patent Literature 1, “an image noise reduction method that generates a training image containing noise, generates a teacher image containing less noise than the training image, and configures a feedforward neural network for the training image whose input and output correspond to an image of the teacher image” is disclosed.

[0005] Prior Art Documents

[0006] Patent Literature

[0007] Patent Literature 1: Japanese Patent Application Publication No. 2019-8599 SUMMARY

[0008] Problems to be Solved by the Invention

[0009] In the above-described Patent Literature 1, a technology of inputting a low-accumulation image, giving a high-accumulation image as teaching information, and predicting a high-accumulation image from a low-accumulation image is described. In Patent Literature 1, a high-accumulation image with less noise is predicted from a low-accumulation image with fewer shots.

[0010] However, in the inspection or measurement of a fine circuit pattern such as a semiconductor, there is a case where a shot damage caused by electron beam irradiation occurs in the circuit pattern at each time of shooting, and the circuit shape is deformed. In such a case, it is difficult to generate appropriate teaching information in a high-quality image generated by averaging low-quality images. In addition, in addition to the deformation of the circuit shape, in a case where a field shift or a brightness change due to charging occurs at each time of shooting in a plurality of shootings, it is difficult to generate appropriate teaching information in a simple average image.

[0011] Therefore, an object of the present application is to provide a high-precision and high-reliability quality improvement system and quality improvement method in which, in a quality improvement system and quality improvement method in which quality improvement of a low-quality image is performed by machine learning, appropriate teaching information can be learned even for a sample in which an image easily changes at each time of shooting.

[0012] Means for solving the problem

[0013] To solve the above problem, the present application is a quality improvement system that performs quality improvement of a low-quality image, characterized by including: a quality improvement section that performs quality improvement of a low-quality image; a deformation prediction section that predicts a deformation amount that occurs between a first low-quality image included in a series of input low-quality images and a second low-quality image different from the first low-quality image; and a deformation correction section that corrects any one of a first predicted image obtained by applying processing of the quality improvement section to the first low-quality image, the second low-quality image, and a second predicted image obtained by applying processing of the quality improvement section to the second low-quality image, based on the deformation amount predicted by the deformation prediction section, performs learning so that an evaluation of a loss function of the first predicted image after correction by the deformation correction section and the second low-quality image or the second predicted image becomes smaller, or so that an evaluation of a loss function of the first predicted image and the second low-quality image or the second predicted image after correction by the deformation correction section becomes smaller.

[0014] Further, the present application is a quality improvement method characterized by comprising the steps of: (a) a step of acquiring a plurality of inspection images; (b) a step of, after the step (a), applying a quality improvement model to the acquired inspection images to acquire predicted images for each inspection image; (c) a step of, after the step (b), predicting a deformation amount between the acquired predicted images; (d) a step of, after the step (c), generating a corrected predicted image in which an arbitrary predicted image is deformed into a predicted image for a different inspection image based on the predicted deformation amount; (e) a step of, after the step (d), evaluating an error of quality improvement using the generated corrected predicted image and an inspection image to be corrected; and (f) a step of, after the step (e), updating a parameter of the quality improvement model so as to reduce the evaluated error of quality improvement.

[0015] Effects of Invention

[0016] According to the present application, in a quality improvement system and a quality improvement method in which quality improvement of a low-quality image is performed by machine learning, a high-precision and high-reliability quality improvement system and a quality improvement method in which learning with appropriate teaching information is possible even for a sample in which an image easily changes at each time of shooting are realized.

[0017] Thus, rapid and high-precision inspection and measurement of electronic devices can be performed.

[0018] The above-described objects, structures, and effects other than the above become clear through the following embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Conceptually represents quality improvement of the present application.

[0020] Figure 2 Is a block diagram representing the structure of quality improvement model learning of Embodiment 1 of the present application.

[0021] Figure 3 Is a block diagram representing the structure of a quality improvement system of Embodiment 1 of the present application.

[0022] Figure 4 Is a flowchart representing a quality improvement method (learning stage) of Embodiment 1 of the present application.

[0023] Figure 5 Is a flowchart representing a quality improvement method (inference stage) of Embodiment 1 of the present application.

[0024] Figure 6 Is a block diagram representing the structure of a deformation prediction unit of Figure 2

[0025] Figure 7A ​Conceptual representation of the relationship between the number of shots and precision in the present application.

[0026] Figure 7B Conceptual representation of the relationship between the number of shots and precision in the related art.

[0027] Figure 8 Learning GUI of Embodiment 1 of the present application.

[0028] Figure 9 Inference GUI of Embodiment 1 of the present application.

[0029] Figure 10 is a block diagram showing the structure of the quality improvement model learning of Embodiment 2 of the present application.

[0030] Figure 11 is a block diagram showing the structure of the deformation prediction section of Figure 11

[0031] Figure 12 Conceptual representation of the quality improvement of the related art. DETAILED DESCRIPTION

[0032] Hereinafter, an embodiment of the present application will be described using the drawings. In addition, detailed description will be omitted for portions that are repeated.

[0033] Embodiment 1

[0034] In order to easily understand the quality improvement of the present application, first, the quality improvement of the related art will be described using Figure 12 Figure 12 Conceptual representation of the quality improvement of the above Patent Document 1.

[0035] As shown in Figure 12 , in the related art, among a plurality of (n) low-quality images obtained by shooting the same part of the same sample (for example, a semiconductor wafer), a low-quality image 1 is extracted as a correction target, and a quality improvement process using machine learning is performed by a quality improvement section, and a predicted image is output. On the other hand, a low-quality image 2-n different from the low-quality image 1 is subjected to accumulation processing (for example, averaging processing) to generate a high-quality image, and the high-quality image is used as a teacher image to teach the predicted image of the low-quality image 1.

[0036] As described above, in this method, in the inspection or measurement of a fine circuit pattern such as a semiconductor integrated circuit, a shot damage caused by electron beam irradiation occurs in the circuit pattern at each time of shooting, and the circuit shape can be deformed, and it is difficult to generate appropriate teaching information (high-quality image).

[0037] Next, with reference to Figures 1 to 9 ​​The image quality improvement system and image quality improvement method of Embodiment 1 of the present invention will be described. Figure 1 This invention conceptually represents the image quality improvement.

[0038] like Figure 1 As shown, in this invention, among multiple (n) low-quality images obtained by photographing the same part of the same sample (e.g., a semiconductor wafer), low-quality image 1 is extracted as the correction target. The image quality improvement unit performs image quality improvement processing using machine learning and outputs a predicted image. On the other hand, the deformation that occurs between low-quality image 1 and each of the other low-quality images 2-n (different from low-quality image 1) is predicted. The deformation-corrected image generated by the deformation correction unit is then used as a teacher image to demonstrate the predicted image of low-quality image 1.

[0039] Regarding low-quality images 2-n, it is sufficient to compare the deformation with low-quality image 1, or it can be set to only one low-quality image 2. That is, by obtaining at least two captured images, low-quality image 1 and low-quality image 2, the deformation can be predicted to generate the teacher image (deformation-corrected image).

[0040] use Figure 2 For use in Figure 1 The specific structure of the functions described in the text will be explained. Figure 2 This is a block diagram illustrating the learning structure of the image quality improvement model in this embodiment.

[0041] like Figure 2 As shown, the image improvement system 1 of this embodiment is configured to include an image quality improvement unit 2, a deformation prediction unit 4, a deformation correction unit 5, a corrected image 6, an image quality improvement error evaluation unit 7, and an image quality improvement parameter update unit 8.

[0042] Image quality improvement unit 2 applies image quality improvement processing to the low-quality image i9 (first low-quality image) contained in the input low-quality image column to generate a prediction image i3 (first prediction image). In the image quality improvement model of image quality improvement unit 2, for example, a CNN (Convolutional Neural Network) of the Encoder-Decoder type such as U-Net or a CNN with other structures is used.

[0043] Deformation prediction unit 4 utilizes a deformation database pre-saved (described later). Figure 6 The deformation prediction unit 4 uses deformation data such as the deformation amount DB19 to predict the deformation amount of the prediction image i3 (the first prediction image). The deformation prediction unit 4 predicts the deformation amount (D) of each pixel in the prediction image.

[0044] The deformation correction unit 5 corrects the predicted image i3 (the first predicted image) based on the deformation amount predicted by the deformation prediction unit 4. For example, assuming that the corrected image Y' is the image after the deformation correction unit 5 deforms the predicted image Y by a deformation amount (D), the [i, j] pixels of Y' become the information of the [i+D[i, j, 0], j+D[i, j, 1]] pixels of Y. Here, the deformation amount D is a dual-channel image with the same height and width as the predicted image Y, and each channel has the deformation amount in the height direction and width direction in each image coordinate. When D[i, j] is not an integer, the corrected image Y' is generated by methods such as bilinear sampling. The corrected image 6 is an image obtained by deforming the predicted image i3 based on the deformation amount predicted by the deformation prediction unit 4 so that the low-quality image j10 is consistent with the shape of the circuit pattern.

[0045] Furthermore, the deformation correction unit 5 not only predicts circuit deformation caused by shooting damage, but also predicts field of view shifts and brightness changes in each shot. These can be corrected through positional correction based on inter-image matching or correction based on changes in brightness value distribution.

[0046] The image quality improvement error evaluation unit 7 evaluates the error between the corrected image 6 (first predicted image) after deformation correction by the deformation correction unit 5 and the low-quality image j10 (second low-quality image). The error function or loss function used in the image quality improvement error evaluation unit 7 is, for example, a likelihood function based on absolute error, variance, or Gaussian, Poisson, gamma distribution, etc.

[0047] Based on the evaluation results of the image quality improvement error evaluation unit 7, the image quality improvement parameter update unit 8 updates and optimizes the parameters of the image quality improvement model in the image quality improvement unit 2 to reduce the evaluation of the loss function between the corrected image 6 (first predicted image) corrected by the deformation correction unit 5 and the low-quality image j10 (second low-quality image). This update is performed, for example, by the probabilistic gradient descent method. Furthermore, the error function or loss function used for error evaluation in the image quality improvement error evaluation unit 7 and the image quality improvement parameter update unit 8 can also be calculated using combinations other than the first predicted image and the second low-quality image. For example, the loss function can be calculated using the first predicted image and the second predicted image obtained by applying image quality improvement processing to the second low-quality image.

[0048] Furthermore, since the distortion correction from the first low-quality image to the second low-quality image is reversible, the distortion correction performed by the distortion correction unit 5 can also be applied to the second low-quality image or the second predicted image. That is, when correcting shooting damage, if damage occurs in the first low-quality image that thins the circuitry, correction can be performed to thicken the circuitry in the second low-quality image or the second predicted image. Similarly, reversible position correction and brightness value correction can be performed for changes in field of view shift and brightness value distribution.

[0049] Furthermore, the error function or loss function used for error evaluation in the image quality improvement error evaluation unit 7 and the image quality improvement parameter update unit 8 can be calculated by combining the first predicted image and the first low-quality image. In this case, deformation correction by the deformation correction unit 5 can be omitted.

[0050] Furthermore, the error function or loss function used for error evaluation in the image quality improvement error evaluation unit 7 and the image quality improvement parameter update unit 8 can also be obtained by combining or weighting the error function or loss function.

[0051] Figure 3 Indicates will Figure 2 Example of a specific system structure when the image quality improvement system 1 described herein is mounted on the inspection device 16.

[0052] The inspection device 16, based on the imaging scheme 15, acquires multiple (n) low-quality images 17 obtained by photographing the same part of the sample 14 (e.g., a semiconductor wafer).

[0053] The image quality improvement system 1 is configured to include an image database (DB) 13, a computer 11, and a learning result database (DB) 12.

[0054] The image database (DB)13 stores two or more columns of non-cumulative images and shooting conditions.

[0055] The image quality improvement model, which has been learned and processed by the computer 11, is stored in the learning results database (DB) 12. Additionally, Figure 3 The example shown is of a learning results database (DB) 12 being included in the image quality improvement system 1, but it can also be configured to be externally located via a centralized monitoring system or the like.

[0056] Computer 11 learns and processes an image quality improvement model based on the captured low-quality image 17 and information read from the image database (DB) 13, and outputs an image quality improved image 18.

[0057] use Figure 4 and Figure 5The representative processing (image quality improvement method) of the image quality improvement system 1 described above will be explained. Figure 4 This is a flowchart illustrating the learning phase of the image quality improvement method in this embodiment.

[0058] First, in step S101, the inspection device 16 acquires two or more inspection images (low-quality images 17) from one or more samples 14 at one or more shooting points based on the shooting scheme 15, and stores them in the image database (DB) 13.

[0059] Next, in step S102, computer 11 begins learning processing of the image quality improvement model.

[0060] Next, in step S103, the computer 11 obtains two or more inspection images of the same wafer and the same shooting point from the image database (DB) 13.

[0061] Next, in step S104, the computer 11 applies an image quality improvement model to the obtained inspection images to obtain predicted images for each inspection image.

[0062] Subsequently, in step S105, the deformation prediction unit 4 predicts the amount of deformation between the predicted images.

[0063] Next, in step S106, the deformation correction unit 5 deforms any predicted image into a predicted image for different inspection images based on the predicted deformation amount, and generates a corrected image.

[0064] Next, in step S107, the image quality improvement error evaluation unit 7 uses the corrected image and the inspection image that is the object of correction to evaluate the error in image quality improvement. Here, the corrected image is... Figure 2 The corrected image 6 in the image is the inspection image that becomes the object of correction. Figure 2 Low-resolution image 10.

[0065] Next, in step S108, the image quality improvement parameter update unit 8 updates the parameters of the image quality improvement model to reduce the image quality improvement error.

[0066] Next, in step S109, it is determined whether the learning termination condition has been met. If the learning termination condition has been met (yes), the process proceeds to step S110, where the computer 11 saves the image quality improvement model to the learning result database (DB) 12, ending the learning process. On the other hand, if the learning termination condition has not been met (no), the process returns to step S103, and step S103 and subsequent processes are executed again.

[0067] Figure 5 This is a flowchart illustrating the inference phase of the image quality improvement method in this embodiment.

[0068] First, in step S201, the inspection device 16 acquires more than one inspection image from the sample 14 at more than one shooting point based on the shooting scheme 15.

[0069] Next, in step S202, the computer 11 reads the image quality improvement model from the learning results database (DB) 12.

[0070] Finally, in step S203, the computer 11 applies the image quality improvement model to the inspected image and outputs the image quality improved image.

[0071] Figure 6 This is a block diagram illustrating the structure of the deformation prediction unit in this embodiment. For example... Figure 6 As shown, the deformation prediction unit 21 predicts the deformation of the prediction image i20 based on the deformation data stored in the deformation database (DB) 19 in advance, and outputs the deformation amount i22.

[0072] Generally, deformations in circuit patterns occur at the ends of the circuit. Furthermore, the deformation occurs in the direction that makes the circuit pattern thinner. Therefore, the deformation prediction unit 21 extracts the edges of the circuit pattern from the prediction image i20 and calculates the amount of deformation that causes the pattern to thin. Figure 6 The deformation prediction unit 21 includes an edge direction detection unit and a deformation amount generation unit. The edge direction detection unit detects the edges of the pattern and uses the direction towards the center of the pattern for each edge as the edge direction. Then, the deformation amount generation unit generates a deformation amount based on the edge direction detected by the edge direction detection unit. Regarding the deformation amount generated here, the deformation amount generated for each shooting condition is referred to the deformation amount stored in the deformation amount DB19 to calculate the deformation amount corresponding to the input image, and the deformation amount is assigned to the region of each edge to form a dual-channel image with the same height and width as the predicted image i20. In addition, the deformation amount data stored in the deformation amount DB19 may depend not only on the shooting conditions but also on the edge shape.

[0073] use Figure 7A as well as Figure 7B The effects of the present invention will be explained. Figure 7A This conceptually illustrates the relationship between the number of shots and accuracy in this invention. Figure 7B This conceptually represents the relationship between the number of shots and accuracy in existing technologies.

[0074] like Figure 7BAs shown, in the prior art, such as Patent Document 1 mentioned above, when the sample is difficult to deform (shrink), the image quality of the high-quality image (the teacher image) improves with each increase in the number of shots, and the accuracy of the inspection / measuring device also improves. On the other hand, when the sample is easily deformable (shrinks), the sample deforms (shrinks) with each increase in the number of shots, making it difficult to generate a suitable teacher image (high-quality image). Inspection / measuring is then performed using an image with improved image quality that includes information about the sample's deformation, resulting in a decrease in the accuracy of the inspection / measuring device. In contrast, as... Figure 7A As shown, in this invention, even for samples whose shape is easily deformed (shrinks) during shooting, an appropriate teacher image (deformation correction image) can be generated. Appropriate learning can be performed through this teacher image (deformation correction image), thus ensuring the accuracy of a stable inspection / measuring device regardless of the number of shots.

[0075] use Figure 8 as well as Figure 9 This section will illustrate a specific example of the input / output device GUI (Graphical User Interface) used in the control of the image quality improvement system 1. Figure 8 Showing the learning GUI, Figure 9 The inference is shown in the GUI.

[0076] like Figure 8 As shown, the learning GUI includes (1) a learning data selection section, (2) an evaluation data selection section, (3) a learning condition setting section, (4) a learning mode selection section, (5) a learning result confirmation section, and (6) a learning instruction section.

[0077] (5) For example, an image verification section and a result verification section are set up in the learning result verification section. By arranging and displaying the low-quality image before image quality improvement and the image quality improved image after image quality improvement in the image verification section, the operator can proceed with the operation while verifying the image quality improvement effect of the image quality improvement system 1.

[0078] (3) In the learning condition setting section, the structure of the CNN used in the image quality improvement section 2, the loss function used, the coefficients used in the weighted average, the number of learning iterations, the learning rate, and other learning schedule information are set. In addition, when performing deformation correction, the database used for deformation correction can also be specified here.

[0079] (4) In the learning mode selection section, select whether to perform deformation correction. For example, deformation correction may not be performed for specimens that are difficult to deform.

[0080] like Figure 9As shown, the inference GUI includes (1) an inference data selection unit, (2) a learning model selection unit, (3) an inference execution unit, (4) an inference result confirmation unit, (5) a post-processing result confirmation unit, and (6) a deformation confirmation unit.

[0081] (4) In the inference result confirmation section, for example, the low-quality image before image quality improvement and the image quality improved after image quality improvement are displayed in an array.

[0082] (5) In the post-processing result confirmation section, for example, the results of applying post-processing to the low-quality image before image quality improvement and the improved image after image quality improvement are displayed. Here, post-processing refers to edge extraction, length measurement of specific parts, or defect inspection of the captured image, etc. Figure 9 Edge detection is illustrated in the example. The user can use this post-processing result confirmation unit to confirm whether the desired result is obtained when post-processing is applied to the image with image quality improvement, thereby deciding whether the obtained image quality improvement unit can be used.

[0083] In addition, the deformation confirmation section (6) displays multiple low-quality images and improved images, and displays deformation images calculated based on these images.

[0084] In addition, this embodiment shows an example of applying deformation correction to a predicted image after image quality improvement, but it can also be applied directly to low-quality images. Alternatively, teacher images can be generated and used for learning by averaging the results of applying the correction to multiple predicted images or low-quality images.

[0085] Example 2

[0086] Reference Figure 10 as well as Figure 11 The image quality improvement system and image quality improvement method of Embodiment 2 of the present invention will be described. Figure 10 This is a block diagram illustrating the learning structure of the image quality improvement model in this embodiment, equivalent to Embodiment 1. Figure 2 A variation of Example 1. In this embodiment, it is similar to Example 1. Figure 2 The difference is that the deformation prediction unit does not use the deformation amount DB19, but is constructed using a CNN in the same way as the image quality improvement unit 26. The other basic structures are the same as in Embodiment 1.

[0087] like Figure 10 As shown, the image quality improvement system 23 of this embodiment is configured to include an image quality improvement unit 26, a deformation prediction unit 29, a deformation correction unit 30, a corrected image comparison unit 31, an image quality improvement error evaluation unit 32, and an image quality improvement parameter update unit 33.

[0088] The low-quality image i24 (first low-quality image) and the low-quality image j25 (second low-quality image) which are different from the low-quality image i24 (first low-quality image) in the input low-quality image column are both processed in the image quality improvement unit 26 to generate prediction image i27 and prediction image j28 respectively.

[0089] Both predicted images i27 and j28 are input into the deformation prediction unit 29, which predicts the deformation between the predicted images. The deformation prediction unit 29, like the image quality improvement unit 26, is a CNN. The learning process of the deformation prediction unit 29 is... Figure 11 The structure shown is followed.

[0090] The deformation correction unit 30 corrects the predicted image i27 based on the deformation amount between each predicted image predicted by the deformation prediction unit 29.

[0091] The image comparison unit 31 compares the corrected prediction image i, which has been corrected by the deformation correction unit 30, with the corrected prediction image j.

[0092] The image quality improvement error evaluation unit 32 evaluates the error between the corrected prediction image i and the corrected prediction image j after being corrected by the deformation correction unit 30, based on the comparison results of the corrected image comparison unit 31.

[0093] Based on the evaluation results of the image quality improvement error evaluation unit 32, the image quality improvement parameter update unit 33 updates the parameters of the image quality improvement model in the image quality improvement unit 26 to optimize them, so as to reduce the evaluation of the error function between the corrected predicted image i and the corrected predicted image j after being corrected by the deformation correction unit 30.

[0094] Figure 11 This is a block diagram illustrating the structure of the deformation prediction unit during learning in this embodiment. For example... Figure 11 As shown, the deformation prediction unit 37 takes the predicted image i35 and the predicted image j36 as inputs and predicts the deformation amount i38. This deformation amount i38 is the deformation amount used in the deformation correction unit 30. Figure 11This is a structure used to learn the deformation prediction unit 37, which calculates an appropriate deformation amount i38. The deformation prediction unit 37 uses the predicted image i35 and the predicted image j36 as input to predict the deformation amount i38. Then, in the deformation correction unit 39, the predicted image i35 is deformed to match the circuit pattern shape of the predicted image j36 based on the predicted image i35 and the deformation amount i38, resulting in a corrected image 40. Then, the deformation prediction error evaluation unit 41 evaluates the error between the corrected image 40 and the predicted image j36. The error evaluated here is, for example, based on the absolute error, variance, or the likelihood function or Kullback-Leibler information of a Gaussian distribution, Poisson distribution, gamma distribution, etc. The deformation prediction unit parameter update unit 42 updates the parameters of the deformation prediction unit 37 to reduce the error predicted by the deformation prediction error evaluation unit 41. This update is performed, for example, by the probability gradient descent method.

[0095] Furthermore, the predictions performed here can not only predict deformations related to the shape of the circuit pattern, but also, as in Embodiment 1, predict the field of view offset and the correction amount for the brightness value distribution. In this case, the parameters of the deformation prediction unit 37 are updated based on the predicted deformation of the circuit pattern shape, the field of view offset, and the correction amount for the brightness value distribution, so as to reduce the error function or loss function between the corrected image 40 after the deformation prediction unit 37 corrects the predicted image i35 and the predicted image j36.

[0096] In addition, similar to Example 1, the prediction of the amount of deformation performed here can be a deformation that makes the predicted image i35 consistent with the predicted image j36, or conversely, a deformation that makes the predicted image j36 consistent with the predicted image i35.

[0097] Such learning and Figure 4 The learning process for the image quality improvement unit shown is performed in the same way. In addition, the learning of the deformation prediction unit 37 can be performed simultaneously with the image quality improvement unit, or it can be performed separately.

[0098] When using the deformation prediction unit 37 described in Embodiment 2, it is also possible to... Figure 8 In section (3), learning condition setting items related to the learning of deformation prediction unit 37 can be added. That is, items related to the network structure, loss function, and learning schedule of deformation prediction unit 37 can also be added.

[0099] Furthermore, the present invention is not limited to the above embodiments, but includes various modifications. For example, the above embodiments are detailed examples to aid in understanding the present invention and are not limited to having all the described structures. Additionally, a portion of the structure of one embodiment can be replaced with the structure of another embodiment, and it is also possible to add structures of other embodiments to the structure of one embodiment. Furthermore, for a portion of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0100] Explanation of reference numerals in the attached figures

[0101] 1, 23… Image quality improvement system

[0102] 2, 26… Image Quality Improvement Department

[0103] 3, 20, 27, 35… Predict image i

[0104] 4, 21, 29, 34, 37… Deformation Prediction Department

[0105] 5, 30, 39… Deformation and Correction Section

[0106] 6, 31, 40… Correct the image (corrected image i > j)

[0107] 7, 32… Image Quality Improvement Error Evaluation Department

[0108] 8, 33… Image Quality Improvement Parameter Update Department

[0109] 9, 24… Low-resolution images i

[0110] 10, 25... Low-resolution images j

[0111] 11…computer

[0112] 12…Learning Outcomes Database (DB)

[0113] 13…Image Database (DB)

[0114] 14…samples

[0115] 15… Shooting Plan

[0116] 16… Inspection device

[0117] 17…Low-resolution images

[0118] 18…Image quality improvement

[0119] 19… Deformation Database (DB)

[0120] 20…Predicted image i

[0121] 22, 38… Deformation amount i

[0122] 28, 36… Predicted image j

[0123] 41…Deformation Prediction Error Evaluation Department

[0124] 42…Deformation prediction parameter update section.

Claims

1. An image quality improvement system for improving the image quality of low-quality images, characterized in that, have: The image quality improvement department improves the image quality of low-resolution images. The deformation prediction unit predicts the amount of deformation that occurs between a first low-quality image included in the input low-quality image column and a second low-quality image that is different from the first low-quality image. as well as The deformation correction unit corrects any one of the following based on the deformation amount predicted by the deformation prediction unit: a first predicted image obtained by applying the image quality improvement unit to the first low-quality image, a second low-quality image, and a second predicted image obtained by applying the image quality improvement unit to the second low-quality image. The learning process reduces the evaluation of the loss function between the first predicted image after deformation correction and the second low-quality image or the second predicted image, or reduces the evaluation of the loss function between the first predicted image and the second low-quality image or the second predicted image after deformation correction.

2. The image quality improvement system according to claim 1, characterized in that, The deformation prediction unit uses a pre-designed deformation database to predict the amount of deformation generated in the first low-quality image. or The first low-resolution image or the first predicted image and the second low-resolution image or the second predicted image are used as inputs to predict the amount of deformation, thereby reducing the evaluation of the loss function of the two inputs after deformation correction.

3. The image quality improvement system according to claim 1, characterized in that, The low-quality image series is an image series obtained by taking more than two pictures of the same part of the same sample.

4. The image quality improvement system according to claim 1, characterized in that, The deformation prediction unit predicts the deformation between each prediction image based on the deformation data pre-stored in the deformation database.

5. The image quality improvement system according to claim 1, characterized in that, The image quality improvement unit uses machine learning with CNN to obtain predicted images for each low-quality image, where CNN is a convolutional neural network.

6. The image quality improvement system according to claim 1, characterized in that, The image quality improvement system includes an image quality improvement error evaluation unit, which evaluates the image quality improvement error using a corrected predicted image corrected by the deformation correction unit and a low-quality image that is the object of correction. The image quality improvement error evaluation unit uses a likelihood function based on any one of absolute error, variance, Gaussian distribution, Poisson distribution, or gamma distribution to evaluate the image quality improvement error of the image quality improvement unit.

7. The image quality improvement system according to claim 6, characterized in that, The image quality improvement system includes an image quality improvement parameter update unit, which updates the parameters of the image quality improvement model in the image quality improvement unit based on the evaluation results of the image quality improvement error evaluation unit. The parameters of the image quality improvement model are updated to reduce the error in the image quality improvement of the image quality improvement unit.

8. The image quality improvement system according to claim 1, characterized in that, The image quality improvement system has the following features: An image database that stores low-resolution image columns and shooting conditions; and The computer processes the image quality improvement model through learning. The computer uses the deformation prediction unit to predict the deformation between columns of low-quality images read from the image database, and uses the deformation correction unit to correct the first predicted image based on the predicted deformation amount.

9. A method for improving image quality, characterized in that, Include: Step a: Obtain multiple inspection images; After step a, step b involves applying an image quality improvement model to the obtained inspection images to obtain predicted images for each inspection image. Following step b, step c is performed to predict the amount of deformation between the obtained predicted images. After step c, step d is performed to generate a corrected prediction image that deforms any prediction image into a prediction image for different inspection images based on the predicted deformation amount. Following step d, step e evaluates the error in image quality improvement using the generated corrected predicted image and the inspection image that becomes the object of correction; Following step e, step f updates the parameters of the image quality improvement model to reduce the error in the evaluated image quality improvement.

10. The image quality improvement method according to claim 9, characterized in that, In step a, two or more inspection images of the same part of the same sample are obtained.

11. The image quality improvement method according to claim 9, characterized in that, In step c, the amount of deformation between the predicted images is predicted based on pre-saved deformation data.

12. The image quality improvement method according to claim 9, characterized in that, In step b, a predicted image for each inspected image is obtained by using machine learning with a CNN, where CNN is a convolutional neural network.

13. The image quality improvement method according to claim 9, characterized in that, In step e, the error in image quality improvement is evaluated using a likelihood function based on absolute error, variance, or any one of Gaussian, Poisson, or gamma distributions.

Images