Image inspection device and image processing method

Abstract

Provided is an image inspection device which can prevent a degradation in the accuracy of an estimation value of a probability distribution caused by position displacement between design data and a captured image in image processing in which a model that estimates a pixel value probability distribution of a captured image is trained by using design data and the captured image of a sample. The image inspection device inspects a captured image by using design data and the capture image of a sample, the device being characterized by comprising: a training processing unit which trains a probability distribution estimation model that estimates, from the design data, a pixel value probability distribution of the captured image; and an inspection processing unit which inspects a captured image for inspection by using the probability distribution estimation model created in the training processing unit, design data for inspection, and the captured image for inspection, wherein the training processing unit includes: a probability distribution estimation unit which estimates, from sample design data for training, a pixel value probability distribution of a sample captured image for training; a position displacement amount estimation unit which estimates a position displacement amount between a probability distribution during training estimated by the probability distribution estimation unit, and the captured image for training; a position displacement reflection unit which reflects, to the probability distribution during training, the estimation position displacement amount estimated by the position displacement amount estimation unit; and a model update unit which evaluates the probability distribution estimation model of the probability distribution estimation unit by using the captured image for training and the probability distribution during training to which the position displacement calculated by the position displacement reflection unit has been reflected, and updates parameters of the probability distribution estimation model according to the evaluation values.

Description

[Technical field]

[0001] The present invention relates to an image processing technique for processing image data, and more particularly to a technique that is effective when applied to inspections using image data. [Background technology]

[0002] 2. Description of the Related Art For evaluation such as defect inspection of semiconductor circuits, design data of a sample to be inspected is compared with image data obtained by imaging the inspection target.

[0003] As semiconductor circuit patterns become finer, it is becoming more difficult to form circuit patterns on wafers as designed, and defects in which the width and shape of wiring differ from the design values ​​are more likely to occur. Such defects are called systematic defects, and because they occur in common on all dies, they are difficult to detect using a method that detects defects by comparing the die being inspected with adjacent dies (die-to-die inspection). Semiconductor devices manufactured from dies containing undetected defects may be defective in other inspections performed in final tests, resulting in reduced yields.

[0004] In contrast, there is a method (die-to-database inspection) that detects defects by comparing the design data image, which is an image of design data such as CAD data, with the die to be inspected, instead of adjacent dies. Since die-to-database inspection compares the design data with the die to be inspected, it is theoretically possible to detect systematic defects.

[0005] Background art in this technical field includes, for example, a technique such as that described in Patent Document 1. Patent Document 1 discloses a method for estimating a probability distribution of pixel values ​​of an image captured from design data by machine learning, and expressing a shape deviation that is allowable as a manufacturing margin as a variation in the probability distribution, in order to allow false reports due to a shape deviation between design data and an image captured to be inspected that does not affect the electrical characteristics of a semiconductor device. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2020 / 250373 Summary of the Invention [Problem to be solved by the invention]

[0007] When learning the probability distribution of pixel values ​​of a captured image as in Patent Document 1, it is important to align the design data used for the learning data with the pattern of the captured image in advance, as a mismatch in the patterns reduces the learning accuracy of the probability distribution and causes a deterioration in inspection performance.

[0008] However, images captured by inspection devices may have image distortion caused by the imaging process, and nonlinear and local misalignment may occur between the design data and the captured image, making it difficult to align them in advance. For example, when capturing images using a scanning electron microscope (SEM), image distortion may occur due to the charging of the sample by the electron beam.

[0009] In Patent Document 1, in order to learn the probability distribution of pixel values ​​of a captured image, it is assumed that the learning design data and the captured image have been sufficiently aligned in advance. If learning is performed using a captured image that has a nonlinear and local misalignment, including image distortion caused by imaging, and for which it is difficult to align in advance, the misalignment is modeled as a manufacturing margin, and the variance of the probability distribution increases. As a result, there is a problem that the inspection sensitivity decreases in an inspection in which the captured image is compared with the probability distribution.

[0010] Therefore, an object of the present invention is to provide an image inspection device and an image processing method that can prevent a decrease in accuracy of the estimated value of probability distribution due to a positional shift between design data and an captured image in image processing that uses design data and an captured image of a sample to learn a model that estimates the probability distribution of pixel values ​​of the captured image. [Means for solving the problem]

[0011] In order to solve the above-mentioned problems, the present invention provides an image inspection device that uses design data of a sample and an image of the sample to inspect the image, the image comprising: a learning processing unit that learns a probability distribution estimation model that estimates a probability distribution of pixel values ​​of the image from design data; and an inspection processing unit that inspects the inspection image using the probability distribution estimation model created by the learning processing unit, inspection design data, and the inspection image, wherein the learning processing unit has: a probability distribution estimation unit that estimates a probability distribution of pixel values ​​of the learning image of the sample from the learning design data of the sample; a positional deviation estimation unit that estimates a positional deviation between the learning-time probability distribution estimated by the probability distribution estimation unit and the learning-time image; a positional deviation reflection unit that reflects the estimated positional deviation estimated by the positional deviation estimation unit in the learning-time probability distribution; and a model evaluation unit that evaluates the probability distribution estimation model of the probability distribution estimation unit using the learning-time probability distribution after positional deviation reflection calculated by the positional deviation reflection unit and the learning image, and updates parameters of the probability distribution estimation model in accordance with the evaluation value.

[0012] The present invention also provides an image processing method for learning a model that estimates a probability distribution of pixel values ​​of an captured image using design data of a sample and an captured image of the sample, comprising the steps of: (a) estimating a learning probability distribution of pixel values ​​of a training captured image of the sample from training design data of the sample; (b) estimating an amount of positional deviation between the learning probability distribution estimated in step (a) and the training captured image; (c) reflecting the amount of positional deviation estimated in step (b) in the learning probability distribution; and (d) evaluating the probability distribution estimation model estimated in step (a) using the learning probability distribution after reflecting the positional deviation calculated in step (c) and the training captured image, and updating parameters of the probability distribution estimation model in accordance with the evaluation value. Effect of the Invention

[0013] According to the present invention, in image processing in which a model for estimating a probability distribution of pixel values ​​of an captured image is learned using design data and the captured image of a sample, it is possible to realize an image inspection device and an image processing method that can prevent a decrease in accuracy of the estimated value of the probability distribution due to a positional shift between the design data and the captured image.

[0014] This makes it possible to learn a model that estimates a probability distribution suitable for image inspection, which prevents an increase in the variation in the probability distribution due to pattern misalignment between the design data and the captured image and can take into account only deformation due to manufacturing tolerances.

[0015] As a result, the inspection accuracy can be improved in image inspection in which a captured image is compared with a probability distribution.

[0016] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0017] [Figure 1A] FIG. 2 is a diagram illustrating an example of design data. [Figure 1B] FIG. 4 is a diagram showing an example of a captured image. [Figure 1C] FIG. 13 is a diagram showing an example of a positional deviation between design data and a captured image; [Figure 2A] FIG. 13 is a diagram showing an example of a probability distribution in which the variation increases due to a positional shift between design data included in learning data and a captured image. [Figure 2B] FIG. 2 is a diagram showing an example of a probability distribution estimated by a model trained by a learning processing unit according to an embodiment of the present invention. [Diagram 3] 1 is a functional block diagram showing an example of the overall configuration of an inspection device according to an embodiment of the present invention; [Figure 4] FIG. 2 is a functional block diagram illustrating a configuration of a learning processing unit according to the first embodiment. [Diagram 5] 4 is a flowchart showing a processing operation of a learning processing unit according to the first embodiment. [Figure 6]5 is a flowchart showing a processing operation of an inspection processing unit of the inspection device according to the first embodiment. [Figure 7] FIG. 11 is a functional block diagram illustrating a configuration of a learning processing unit according to a second embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of a position shift estimation setting amount according to the second embodiment. [Figure 9] 10 is a flowchart showing a processing operation of a learning processing unit according to the second embodiment. [Figure 10] 13 is a diagram showing an example of a GUI screen of a learning progress display unit and a positional deviation estimated set amount update unit according to the second embodiment; FIG. [Figure 11] FIG. 11 is a functional block diagram illustrating a configuration of a learning processing unit according to a third embodiment. [Figure 12] 13 is a flowchart showing a processing operation of a learning processing unit according to the third embodiment. [Figure 13] 13 is a flowchart showing a processing operation of an inspection processing unit of the inspection device according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and detailed description of overlapping parts will be omitted.

[0019] The inspection device exemplified in this specification relates to an image inspection device and an image processing method using the same that can prevent an increase in variance in an estimated probability distribution by estimating a positional shift between design data and a captured image and learning the model that estimates a probability distribution of pixel values ​​of a captured image, thereby reflecting the shift in position between the design data and the captured image and learning the model.

[0020] In this specification, the sample and its captured image are exemplified by a semiconductor circuit captured by a scanning electron microscope (SEM), but the present invention is not limited to this. Needless to say, the present invention can also be applied to images captured by other imaging devices. EXAMPLES

[0021] First Embodiment An image inspection device and an image processing method using the same according to a first embodiment of the present invention will be described with reference to FIGS. 1A to 6. FIG.

[0022] First, an example of design data and a captured image in this embodiment will be described with reference to FIGS. 1A to 1C.

[0023] FIG. 1A is a diagram showing an example of design data for a semiconductor circuit. As shown in FIG. 1A, the design data is an image of design data such as layout data for a semiconductor circuit and CAD data in which manufacturing conditions are registered. The design data 101 in FIG. 1A shows an example of a binary image in which the wiring portion and the space portion of a circuit pattern are colored differently, but the wiring portion of a semiconductor circuit may have two or more layers. For example, if the wiring is in one layer, it can be used as a binary image of the wiring portion and the space portion, and if the wiring is in two layers, it can be used as a ternary image of the lower layer wiring portion, the upper layer wiring portion, and the space portion.

[0024] In addition, the design data may be an image having two or more dimensional values, such as a color image, to represent the manufacturing conditions of the semiconductor circuit, or an image expressed in continuous values, and is not limited to an image of one-dimensional discrete values.

[0025] FIG. 1B is a diagram showing an example of a captured image corresponding to design data 101. FIG. 1C is an example in which the design data 101 is displayed by a dotted line and superimposed on the captured image 102, and shows a positional deviation of a circuit pattern between the design data 101 and the captured image 102. As shown in FIG. 1C, there is a positional deviation between the design data 101 and the captured image 102, and the magnitude of the positional deviation increases toward the left side of the image. Such a positional deviation occurs due to factors such as poor control of an electron beam emitted from an electron source in imaging by a scanning electron microscope, and image distortion caused by a sample being charged by scanning with an electron beam, which causes a change in the amount or electron trajectory of secondary electrons or backscattered electrons emitted from the sample.

[0026] Here, as an example of misalignment, misalignment in the left-right direction is shown, but is not limited thereto. For example, any misalignment may be used, such as a non-linear translational misalignment or rotational misalignment, or a wavy misalignment in which wiring is wavy. In addition, in the case of misalignment caused by the configuration or setting amount of a semiconductor circuit manufacturing device or an imaging device, it is possible to obtain the magnitude and direction of the misalignment by simulation or the like through a prior analysis. In such a case, the captured image may be one in which the misalignment caused by the device has been corrected in advance.

[0027] FIG. 3 is a functional block diagram showing an example of the overall configuration of an inspection device according to an embodiment of the present invention.

[0028] 3, the inspection device is composed of a learning processing unit 303 and an inspection processing unit 307. Here, the learning processing unit 303 and the inspection processing unit 307 are realized by, for example, a processor such as a CPU (not shown), a ROM for storing various programs, a RAM for temporarily storing data in the calculation process, and a storage device such as an external storage device, and the processor such as a CPU reads and executes various programs stored in the ROM, and stores the calculation results, which are the execution results, in the RAM, the external storage device, or cloud storage via a network connection or the like.

[0029] A learning processing unit 303 uses learning design data 301 and learning captured images 302 to learn a model for estimating a probability distribution of pixel values ​​of a captured image from the design data.

[0030] The inspection processing unit 307 inspects the inspection captured image 306 using the model data 304 created by the learning processing unit 303 , the inspection design data 305 , and the inspection captured image 306 , and outputs the inspection result 308 .

[0031] Here, the learning process in the learning processing unit 303 may be performed simultaneously with the inspection process in the inspection processing unit 307, or may be performed separately. In addition, the inspection processing unit 307 may be configured to be executed by a computer different from the learning processing unit 303, as long as the computer executing the inspection processing unit 307 can acquire the model data 304 via a network connection or the like.

[0032] FIG. 4 is a functional block diagram showing a specific example of the configuration of a learning processing unit 401 of this embodiment, which corresponds to the learning processing unit 303 of FIG.

[0033] The learning processing unit 401 of this embodiment is composed of a probability distribution estimation unit 402, a position shift amount estimation unit 404, a position shift reflection unit 406, and a model evaluation unit 408, and outputs model data 410 upon completion of a predetermined learning process, and stores the model data 410 in a RAM, an external storage device, or in cloud storage via a network connection, etc.

[0034] The probability distribution estimation unit 402 estimates a probability distribution of pixel values ​​of the corresponding learning captured image 302 from the learning design data 301 using a model using machine learning, and outputs a learning probability distribution 403. The estimated probability distribution is represented by parameters of the probability distribution corresponding to each pixel of the design data or the captured image.

[0035] Examples of the probability distribution to be estimated include the mean and standard deviation in the case of a normal distribution, and the arrival rate in the case of a Poisson distribution. For the probability distribution estimation model to estimate the probability distribution of pixel values ​​of a captured image, for example, an Encoder-Decoder type CNN (Convolution Neural Network) such as U-Net or a CNN having another structure is used, but is not limited to CNN.

[0036] The positional deviation estimation unit 404 estimates the amount of positional deviation between the learning probability distribution 403 estimated by the probability distribution estimation unit 402 and the learning captured image 302, and outputs the estimated positional deviation 405. The estimated positional deviation 405 is expressed as a two-dimensional vector amount (dx, dy) corresponding to each pixel of the design data or the captured image, and is set so that when each pixel of the estimated probability distribution is moved by the corresponding vector amount, the pixel value of the captured image will better follow the distribution. Here, a vector amount is given as the format of the estimated positional deviation 405, but if it is assumed that the amount of distortion is a formulatable amount such as a rotational deviation or a translational deviation, the estimated positional deviation 405 may be a parameter such as a rotation angle or a translational deviation, or may be a combination of multiple formats.

[0037] A position shift reflection unit 406 reflects the position shift represented by the estimated position shift amount 405 in the learning-time probability distribution 403 , and outputs a position shift-reflected learning-time probability distribution 407 .

[0038] The model evaluation unit 408 evaluates the probability distribution estimation model of the probability distribution estimation unit 402 using the learning captured image 302 and the learning-time probability distribution after reflecting positional displacement 407, calculates the update amount of the parameters of the probability distribution estimation model according to the evaluation value, and updates the parameters of the probability distribution estimation model according to the update amount (model parameter update amount 409). At this time, the parameter update amount is calculated so that the pixel values ​​of the learning captured image 302 better follow the learning-time probability distribution after reflecting positional displacement 407.

[0039] 5 is a flowchart showing the processing operation of the learning processing unit 401 according to this embodiment. As shown in FIG. 5, when the learning processing starts, the learning captured image 302 and the learning design data 301 are input to the learning processing unit 401 in step S501.

[0040] In step S502, the probability distribution estimation unit 402 constituting the learning processing unit 401 estimates the probability distribution of the pixel values ​​of the corresponding learning captured image 302 from the input learning design data 301 using a probability distribution estimation model, and outputs the learning-time probability distribution 403.

[0041] In step S503, the positional deviation estimation unit 404 constituting the learning processing unit 401 estimates the positional deviation between the learning probability distribution 403 and the learning captured image 302 based on the learning probability distribution 403 and the input learning captured image 302.

[0042] As a method of estimation, for example, there is a method of setting an arbitrary or random initial value of the amount of positional deviation, and updating the amount of positional deviation according to an evaluation value in step S504 described later.

[0043] More specifically, there is a method of solving an optimization problem by dynamic programming or the like to minimize an evaluation value calculated by an evaluation function d(R,f(I,D)) for a probability distribution R, a captured image I, a positional shift reflection processing function f, an estimated positional shift amount D, and a distance function d that evaluates the difference between the probability distribution R and the captured image I. The distance function d evaluates the difference between the probability distribution and the captured image, and an example of the distance function is the negative logarithmic likelihood. If the probability distribution is a normal distribution, there is an absolute error or square error between the average of the pixel values ​​of the captured image.

[0044] In step S504, the misalignment estimation unit 404 evaluates the misalignment estimated in step S503, determines whether the evaluation value satisfies an evaluation criterion, and if the evaluation criterion is satisfied (YES), outputs the estimated misalignment 405. If the evaluation criterion is not satisfied (NO), the process returns to step S503 and executes the process of step S503 again.

[0045] The evaluation value may be a function value that evaluates the difference between the probability distribution calculated by the distance function d described above and the captured image. The evaluation criteria may be such that the smaller the evaluation value, the smaller the value of the captured image that follows the probability distribution, the greater the evaluation value, the greater the value of the captured image that follows the probability distribution, and the greater the value of the captured image that follows the probability distribution, and the greater the number of times that the process from step S503 to step S504 is performed.

[0046] In step S505, the position shift reflection unit 406 constituting the learning processing unit 401 reflects the position shift represented by the estimated position shift amount 405 in the learning-time probability distribution 403, and outputs the learning-time probability distribution 407 after reflecting the position shift.

[0047] As a method of reflecting the amount of positional deviation, for example, if the amount of positional deviation is in a two-dimensional vector format, the value of each pixel in the probability distribution can be shifted to another pixel according to the vector amount. In addition, if the amount of positional deviation is a parameter that can be formulated, such as a translational deviation or a rotational deviation, an affine transformation using the parameter can be used.

[0048] In step S506, the model evaluation unit 408 constituting the learning processing unit 401 evaluates an error function or a loss function of the probability distribution estimation model of the probability distribution estimation unit 402 by using the input learning captured image 302 and the learning-time probability distribution after reflecting positional displacement 407. Examples of the error function or loss function of the probability distribution estimation model include a negative log-likelihood of the learning-time probability distribution after reflecting positional displacement 407 with respect to the learning captured image 302, an absolute error or squared error of pixel values ​​between an image sampled from the learning-time probability distribution after reflecting positional displacement 407 and the learning captured image 302, etc.

[0049] In step S507, the model evaluation unit 408 calculates an update amount for the parameters of the probability distribution estimation model based on the evaluation result in step S506 so as to reduce the error function or loss function of the probability distribution estimation model of the probability distribution estimation unit 402, and updates the parameters according to the update amount. This update is performed by, for example, the stochastic gradient descent method.

[0050] In step S508, the learning processing unit 401 judges whether or not the learning end condition has been reached, and if it is judged that the learning end condition has been reached (YES), the process proceeds to step S509, where the learning processing unit 401 saves the model data 410 including the parameters of the probability distribution estimation model of the probability distribution estimation unit 402, and ends the learning process. On the other hand, if it is judged that the learning end condition has not been reached (NO), the process returns to step S501, and the processes from step S501 onwards are executed again.

[0051] The conditions for terminating the learning include whether the process from step S501 to step S507 has been repeated a predetermined number of times or more, whether the value of the error function of the probability distribution estimation model calculated in step S506 does not become smaller even after the process from step S501 to step S507 has been repeated a predetermined number of times, and therefore it is determined that the learning of the probability distribution estimation model in the probability distribution estimation unit 402 has converged, etc.

[0052] FIG. 6 is a flowchart showing the processing operation of the inspection processing unit 307A of this embodiment, which corresponds to the inspection processing unit 307 of FIG.

[0053] As shown in FIG. 6, when the inspection process is started, in step S601, the inspection captured image 306, the inspection design data 305, and the model data 410 including parameters of the probability distribution estimation model learned by the learning processing unit 401 are input to the inspection processing unit 307A.

[0054] In step S602, the inspection processing unit 307A uses the input inspection design data 305 and a probability distribution estimation model included in the model data 410 to estimate a probability distribution of pixel values ​​of the corresponding inspection captured image 306.

[0055] In step S603, the inspection processing unit 307A estimates the positional deviation between the probability distribution estimated in step S602 and the inspection captured image 306 in the same manner as in step S503 of the learning processing unit 401 shown in FIG.

[0056] Here, an example is shown in which the amount of misalignment is estimated in the inspection process, but if the same estimated amount of misalignment 405 is obtained in most cases in combinations of multiple design data and captured images included in the learning design data 301 and the learning captured images 302 in the learning processing unit 401, it can be determined that the misalignment between the probability distribution estimated from the design data and the captured images is a misalignment caused by the device. In such a case, the average value of the estimated amounts of misalignment obtained in the learning process may be stored as a representative amount of misalignment and output in step S603.

[0057] In step S604, the inspection processing unit 307A evaluates the amount of positional deviation estimated in step S603, determines whether the evaluation value satisfies an evaluation criterion, and if the evaluation criterion is satisfied (YES), outputs the estimated amount of positional deviation of the inspection captured image 306. If the evaluation criterion is not satisfied (NO), the process returns to step S603 and executes the process of step S603 again.

[0058] The evaluation criteria include the value of a function that evaluates the difference between a probability distribution similar to step S504 of the learning processing unit 401 shown in FIG. 5 and a captured image, and performing the processing from step S603 to step S604 a specified number of times or more.

[0059] In step S605, the inspection processing unit 307A reflects the position shift using the position shift amount estimated in step S603 on the probability distribution estimated in step S602, and outputs the probability distribution after reflecting the position shift.

[0060] In step S606, the inspection processing unit 307A compares the position shift reflected probability distribution obtained in step S605 with the inspection captured image 306, and performs defect inspection.

[0061] As a comparison method, when the probability distribution corresponding to the pixel value x of the inspection image follows a normal distribution, the degree of anomaly expressed as |x-μ| / σ is calculated using the mean μ and standard deviation σ, and pixels with an anomaly degree exceeding a specified threshold are deemed to be defective.

[0062] In step S607, the inspection processing unit 307A outputs the inspection result of step S606, stores it in a RAM, an external storage device, cloud storage, or the like, or displays it on a GUI (Graphical User Interface), and the inspection process ends.

[0063] The effect of this embodiment will be described with reference to Figures 2A and 2B. Figure 2A is an example of an estimated probability distribution (average image, standard deviation image) of a probability distribution estimation model trained using the method described in Patent Document 1 for training data having a positional shift between design data and captured images as shown in Figures 1A and 1B. Also, Figure 2B is an example of an estimated probability distribution of a probability distribution estimation model trained by this embodiment. Each image shown in Figures 2A and 2B shows the value of the probability distribution estimated by the brightness value.

[0064] The method described in Patent Document 1 assumes that there is no positional deviation between the design data and the captured image other than due to manufacturing tolerances. Therefore, when the learning data includes a captured image having a nonlinear and local positional deviation that is difficult to match in advance, the positional deviation of pixel values ​​is learned as a variation in pixel values ​​due to manufacturing tolerances.

[0065] 2A, the edges of the circuit pattern represented by the average image of the estimated probability distribution become unclear, and the standard deviation image takes large values ​​at the edges of the circuit pattern. If the defect inspection in step S606 of the inspection processing unit 307A described above is performed using such a probability distribution, the degree of abnormality near the edges of the circuit pattern will be evaluated as low, resulting in undetected defects.

[0066] In contrast, in this embodiment, in learning a model that estimates the probability distribution of pixel values ​​of a captured image, the positional shift between the estimated probability distribution and the captured image is sequentially estimated, and learning is performed to optimize the probability distribution after reflecting the positional shift reflected in the estimated probability distribution, thereby preventing an increase in the variation in the probability distribution caused by positional shifts between the design data and the captured image other than the manufacturing tolerance.

[0067] As a result, it is possible to obtain an average image in which the edges of the circuit pattern are clear, as shown in FIG. 2B, and a standard deviation image in which only the manufacturing tolerance is the variation.

[0068] Another example of the effect of this embodiment is that it is possible to reduce the decrease in learning accuracy of the probability distribution due to misalignment between the learning design data and the learning captured images, thereby relaxing the required accuracy for prior alignment of the design data and the captured images, and thereby reducing the cost of generating learning data. EXAMPLES

[0069] Second Embodiment An image inspection device and an image processing method using the same according to a second embodiment of the present invention will be described with reference to FIGS.

[0070] FIG. 7 is a functional block diagram showing the configuration of a learning processing unit 701 according to the second embodiment of the present invention.

[0071] The learning processing unit 701 of this embodiment corresponds to the learning processing unit 401 in the above-mentioned embodiment 1 (FIG. 4), but differs in that the misalignment amount estimation unit 702 has an input of a misalignment estimation setting amount 704 that stabilizes the learning of a probability distribution suitable for inspection by restricting the misalignment amount according to the number of learning steps, such as the maximum misalignment amount to be estimated.

[0072] Another difference is that the learning processing unit 701 has a learning progress display unit 705 which records the visualization results of the learning probability distribution 403, the learning probability distribution after reflecting the position shift 407, and the estimated position shift amount 703 for each learning step and displays them on the GUI, and a position shift estimation setting amount update unit 706 which updates the position shift estimation setting amount 704 that allows the user to stabilize the learning of the probability distribution based on the display results of the learning progress display unit 705.

[0073] 7, a learning processing unit 701 of this embodiment is composed of a probability distribution estimation unit 402, a positional deviation estimation unit 702, a positional deviation reflection unit 406, a model evaluation unit 408, a learning progress display unit 705, and a positional deviation estimation set amount update unit 706, and outputs model data 410 when a predetermined learning process is completed, and stores it in a RAM, an external storage device, or cloud storage via a network connection, etc. Below, differences from the first embodiment will be described.

[0074] The positional deviation estimation unit 702 estimates the amount of positional deviation between the learning-time probability distribution 403 estimated by the probability distribution estimation unit 402 and the learning captured image 302, and outputs an estimated positional deviation 703. At this time, the positional deviation estimation unit 702 estimates an amount of positional deviation that satisfies a constraint condition corresponding to the learning step set by the positional deviation estimation setting amount 704.

[0075] A learning progress display unit 705 records the visualization results of the learning probability distribution 403, the learning probability distribution after reflecting the position shift 407, and the estimated position shift amount 703 for each learning step, and displays them on the GUI.

[0076] The positional deviation estimation set amount updating unit 706 updates the positional deviation estimation set amount 704 for stabilizing the learning of the probability distribution based on the display result of the learning progress display unit 705, and performs the learning process again with the updated positional deviation estimation set amount 704.

[0077] FIG. 8 is a diagram showing an example of the positional deviation estimation setting amount 704 in this embodiment.

[0078] The misalignment estimation setting amount 704 prevents the estimated misalignment amount 703 from becoming a large vector amount exceeding the image size in the early stage of learning when the learning probability distribution 403 cannot fully express the circuit pattern by setting the presence or absence of misalignment reflection processing corresponding to the learning step and an upper limit on the magnitude of the estimated misalignment amount. Also, in a circuit with a repetitive pattern such as that shown in Fig. 1A, a misalignment amount that is shifted by one period is not predicted.

[0079] As a result, in the present invention in which learning is performed to optimize the probability distribution after reflecting positional deviations, it is possible to stabilize the learning of the probability distribution estimation model.

[0080] 8, it is set that the processing of the positional deviation reflection unit 406 is not performed when the number of learning steps is from 0 to 400. At this time, the norm of the vector of the estimated positional deviation amount is 0 for all pixels. It is also set that the norm of the estimated positional deviation amount is limited to 2 pixels when the number of learning steps is from 400 to 1000, that the norm of the estimated positional deviation amount is limited to 5 pixels when the number of learning steps is from 1000 to 10000, and that the limit on the norm of the estimated positional deviation amount is removed when the number of learning steps is 10000 or more.

[0081] In particular, since the accuracy of misalignment amount estimation decreases in the early stage of learning and the learning of the probability distribution estimation model becomes unstable, it is recommended to relax the constraints as the number of learning steps increases. The misalignment estimation setting amounts shown in Fig. 8 are just an example, and the contents of the constraints and the parameter values ​​are determined according to the circuit size and circuit pattern shape of the sample to be learned and the misalignment amount estimation method, so they are not limited to this.

[0082] 9 is a flowchart showing the processing operation of the learning processing unit 701 of this embodiment. Differences from the first embodiment shown in FIG. 5 will be described.

[0083] In step S 903 , the positional deviation estimation unit 702 receives an input of the positional deviation estimated setting amount 704 .

[0084] In step S904, the positional deviation estimation unit 702 constituting the learning processing unit 701 estimates the positional deviation between the learning probability distribution 403 and the learning captured image 302 based on the learning probability distribution 403 and the input learning captured image 302.

[0085] At this time, a positional deviation amount that satisfies the constraint condition set in the positional deviation estimated set amount 704 is estimated. More specifically, as described above in step S503 of Fig. 5, for the probability distribution R, the captured image I, the positional deviation reflection processing function f, the estimated positional deviation amount D, and the distance function d that evaluates the difference between the probability distribution R and the captured image I, the problem of minimizing the evaluation value calculated by the evaluation function d(R,f(I,D)) may be solved by a constrained optimization problem that adds the constraint condition set in the positional deviation estimated set amount 704, using dynamic programming or the like.

[0086] In step S905, the misalignment estimation unit 702 evaluates the misalignment estimated in step S904, determines whether the evaluation value satisfies an evaluation criterion, and if the evaluation criterion is satisfied (YES), outputs the estimated misalignment 703. If the evaluation criterion is not satisfied (NO), the process returns to step S904 and executes the process of step S904 again.

[0087] The evaluation value may be a function value for evaluating the difference between the probability distribution calculated by the distance function d described above and the captured image. The evaluation criteria may be that the smaller the evaluation value, the smaller the value is that the captured image follows the probability distribution and is equal to or smaller than a specified value, that the larger the evaluation value, the larger the value is that the captured image follows the probability distribution and is equal to or larger than a specified value, that the process from step S904 to step S905 is performed a specified number of times or more, and that the constraint conditions set in the positional deviation estimation setting amount 704 are satisfied a predetermined number of times or more.

[0088] In step S909, the learning processing unit 701 stores the learning progress, including the learning probability distribution 403 estimated in step S902, the estimated position shift amount 703 estimated in step S904, and the learning probability distribution 407 after reflecting the position shift calculated in step S906, in RAM, an external memory device, or storage, linked to the number of learning steps, and outputs it to the GUI of the learning progress display unit 705 to present it to the user.

[0089] At this time, the learning probability distribution 403, the estimated positional shift amount 703, and the learning probability distribution after reflecting the positional shift 407 are converted into a format that can be recognized by the user and displayed on the GUI.

[0090] For example, the probability distribution can be displayed as an image in which the parameter values ​​corresponding to each pixel are represented as brightness values. Also, by superimposing the corresponding learning design data 301 or learning captured image 302, the probability distribution can be displayed in a format that allows confirmation of whether the probability distribution is suitable for inspection.

[0091] If the displacement amount is in vector format, it can be displayed as an image in which an arrow showing the norm and direction of the vector amount is drawn at each pixel or at a specified pixel interval. It can also be converted to a color image in the HSV color space with the norm of the vector amount as brightness and the direction as hue and displayed. If the displacement amount can be formulated and its parameters are in numerical format, it can be displayed as a graph with the number of learning steps on the horizontal axis and the numerical value on the vertical axis, or the numerical value can be displayed directly as a string.

[0092] FIG. 10 shows an example of a GUI screen of the learning progress display unit 705 and the position deviation estimated set amount update unit 706. In FIG.

[0093] 10 includes an inference result selection unit 1001, an inference image display unit 1002, a display step number selection unit 1003, a coordinate / magnification setting unit 1004, a positional deviation estimation setting amount input unit 1005, and a positional deviation estimation setting amount determination unit 1006. User input operations in each unit constituting the GUI 1000 are performed using a mouse, a keyboard, a touch panel, or the like.

[0094] The inference result selection unit 1001 selects one or more of the learning processes including the learning probability distribution 403, the estimated position shift amount 703, and the learning probability distribution after reflecting the position shift 407 stored in step S909 of the learning processing unit 701, and displays them on the inference image display unit 1002.

[0095] The inference image display unit 1002 displays, as images or graphs, the results of the learning process selected by the inference result selection unit 1001 at the number of learning steps specified by a display step number selection unit 1003, which will be described later. Here, an example is given in which one image is displayed, but multiple images or graphs may be displayed side by side.

[0096] The display step number selection unit 1003 can change the number of learning steps of the learning process displayed in the inference image display unit 1002, and switch images and graphs.

[0097] A coordinate / magnification setting section 1004 can change the display magnification and position of the image or graph to be displayed.

[0098] A positional deviation estimated set amount input section 1005 displays each item of the positional deviation estimated set amount 704 input to the positional deviation amount estimating section 702, and the user specifies the numerical parameters and contents by keyboard input or pull-down.

[0099] The positional deviation estimated set amount determining unit 1006 performs a process of inputting the contents inputted by the positional deviation estimated set amount input unit 1005 to the positional deviation estimated set amount updating unit 706 in Fig. 7, and updates the contents of the positional deviation estimated set amount 704. After the update process, the learning process of the learning processing unit 701 in Fig. 7 is performed again.

[0100] According to this embodiment, the positional deviation estimation unit 702 estimates a positional deviation amount that satisfies the constraint condition set in the positional deviation estimation setting amount 704, thereby stabilizing learning of the probability distribution estimation model in the early stage of learning.

[0101] As an example of a case where this embodiment is applied, in a circuit with a repeating pattern as shown in Fig. 1A, an excessive misalignment amount shifted by one period may be estimated. Also, in a line-and-space circuit with linear patterns, a misalignment amount that does not exist in the direction in which the line pattern extends may be estimated. In such cases, it may not be possible to normally learn the probability distribution of pixel values ​​of the captured image at a location corresponding to the design data, but by applying this embodiment, it is possible to prevent excessive or non-existent misalignment amounts from being estimated and to stably learn the probability distribution of pixel values ​​of the captured image. EXAMPLES

[0102] Third Embodiment An image inspection device and an image processing method using the same according to a third embodiment of the present invention will be described with reference to FIGS.

[0103] FIG. 11 is a functional block diagram showing the configuration of a learning processing unit 1101 according to the third embodiment of the present invention.

[0104] The learning processing unit 1101 of this embodiment corresponds to the learning processing unit 701 in the above-mentioned embodiment 2 (Figure 7), but differs in that the position shift estimation unit 1102 estimates the position shift between the learning probability distribution 403 and the learning captured image 302 using a position shift estimation model generated by machine learning.

[0105] Another difference is that the model evaluation unit 1104 evaluates the above-mentioned position shift amount estimation model in addition to the probability distribution estimation model, calculates update amounts for the parameters of the position shift amount estimation model according to the evaluation value, and updates the parameters of the position shift amount estimation model according to the update amounts.

[0106] 11, a learning processing unit 1101 of this embodiment is composed of a probability distribution estimation unit 402, a positional deviation estimation unit 1102, a positional deviation reflection unit 406, a model evaluation unit 1104, a learning progress display unit 705, and a positional deviation estimation set amount update unit 706, and outputs model data 1108 when a predetermined learning process is completed, and stores it in a RAM, an external storage device, or cloud storage via a network connection, etc. Below, differences from the first and second embodiments will be described.

[0107] The misalignment estimation unit 1102 estimates the amount of misalignment between the learning probability distribution 403 estimated by the probability distribution estimation unit 402 and the learning captured images 302 using a misalignment estimation model created by machine learning, and outputs an estimated misalignment amount 1103. At this time, the misalignment amount is estimated so as to satisfy the constraint condition corresponding to the learning step set by the misalignment estimation set amount 1107, which corresponds to the misalignment estimation set amount 704 in FIG. 7.

[0108] For the position shift estimation model that estimates the position shift amount, for example, an Encoder-Decoder type CNN such as U-Net or a CNN having another structure is used, but is not limited to CNN.

[0109] The model evaluation unit 1104 evaluates the probability distribution estimation model of the probability distribution estimation unit 402 using the learning captured image 302 and the learning-time probability distribution after reflecting the positional displacement 407, calculates the update amount of the parameters of the probability distribution estimation model according to the evaluation value, and updates the parameters of the probability distribution estimation model according to the update amount (probability distribution estimation model parameter update amount 1105). Also, the model evaluation unit 1104 evaluates the positional displacement estimation model of the positional displacement estimation unit 1102, calculates the update amount of the parameters of the positional displacement estimation model according to the evaluation value, and updates the parameters of the positional displacement estimation model according to the update amount (positional displacement estimation model parameter update amount 1106). At this time, the parameter update amount is calculated so that the pixel value of the learning captured image 302 better follows the learning-time probability distribution after reflecting the positional displacement 407.

[0110] 12 is a flowchart showing the processing operation of the learning processing unit 1101 of this embodiment. The points different from the second embodiment shown in FIG. 9 will be described.

[0111] In step S1203, the learning processing unit 1101 accepts an input of a misalignment estimation setting amount 1107. The misalignment estimation setting amount 1107 corresponds to the misalignment estimation setting amount 704 in Fig. 7, and in addition to the example of constraint conditions shown in Fig. 8, additional constraints can be set to an error function and a loss function for evaluating a misalignment estimation model of the misalignment estimation unit 1102 in step S1208 described later.

[0112] In step S1204, the positional deviation estimation unit 1102 constituting the learning processing unit 1101 estimates the positional deviation between the learning probability distribution 403 and the learning captured image 302 using a positional deviation estimation model, and estimates an estimated positional deviation 1103.

[0113] At this time, the misalignment amount is estimated so as to satisfy the constraint condition set in the misalignment estimation setting amount 1107. For example, when an upper limit value of the norm of the estimated misalignment amount is specified, an activation function that makes the norm of the misalignment amount equal to or less than the specified upper limit value is applied to the estimation of the misalignment amount estimation model.

[0114] In step S1208, the model evaluation unit 1104 constituting the learning processing unit 1101 evaluates the error function or loss function of the position shift estimation model of the position shift estimation unit 1102 using the input learning captured images 302 and the learning probability distribution after reflecting the position shift 407.

[0115] Examples of the error function or loss function of the misalignment estimation model include the negative logarithmic likelihood of the learning probability distribution 407 after reflecting the misalignment with respect to the learning captured image 302, and the absolute error or square error of pixel values ​​between an image sampled from the learning probability distribution 407 after reflecting the misalignment and the learning captured image 302. Examples of the error function or loss function include an additional error function and loss function set in the misalignment estimation setting amount 1107, such as a function that, when the norm of the estimated misalignment exceeds a specified value, reduces the norm of the estimated misalignment using a function that evaluates the difference with that value. In this case, the model evaluation unit 1104 also uses the estimated misalignment amount 1103 to evaluate the misalignment estimation model.

[0116] In step S1209, the model evaluation unit 1104 calculates an update amount for the parameters of the positional deviation estimation model based on the evaluation result in step S1208 so as to reduce the error function or loss function of the probability distribution estimation model of the positional deviation estimation unit 1102, and updates the parameters according to the update amount. This update is performed by, for example, the stochastic gradient descent method.

[0117] In step S1211, the learning processing unit 1101 judges whether or not the learning end condition has been reached, and if it is judged that the learning end condition has been reached (YES), the process proceeds to step S1212, where the learning processing unit 1101 stores model data 1108 including the parameters of the probability distribution estimation model of the probability distribution estimation unit 402 and the parameters of the position shift amount estimation model of the position shift amount estimation unit 1102, and ends the learning process. On the other hand, if it is judged that the learning end condition has not been reached (NO), the process returns to step S1201, and the processes from step S1201 onwards are executed again.

[0118] The conditions for terminating the learning include whether the process from step S1201 to step S1210 has been repeated a predetermined number of times or more, whether the value of the error function of the probability distribution estimation model calculated in step S1206 and the value of the error function of the position shift estimation model calculated in step S1208 do not become smaller even after the process from step S1201 to step S1210 has been repeated a predetermined number of times, and therefore it is determined that the learning of the probability distribution estimation model of the probability distribution estimation unit 402 and the position shift estimation model of the position shift estimation unit 1102 has converged, etc.

[0119] 13 is a flowchart showing the processing operation of the inspection processing unit 307B in this embodiment. Differences from the first embodiment (FIG. 6) will be described.

[0120] As shown in FIG. 13, in step S1301, the inspection captured image 306, the inspection design data 305, the parameters of the probability distribution estimation model learned by the learning processing unit 1101, and the model data 1108 including the parameters of the positional deviation estimation model are input to the inspection processing unit 307B.

[0121] In step S1303, the inspection processing unit 307B estimates the positional deviation between the probability distribution estimated in step S1302 and the inspection captured image 306 using the positional deviation estimation model included in the input model data 1108.

[0122] In this way, when machine learning is used for the misalignment estimation model, changing the configuration of the misalignment estimation model results in a trade-off with the estimation accuracy, but has the advantage of being able to reduce the memory usage and calculation time of the computer that performs the learning process and the inspection process. Methods for reducing the calculation time by changing the configuration of the misalignment estimation model include, for example, reducing the number of channels in the convolution layer used in CNN or reducing the number of layers.

[0123] As described above, according to this embodiment, in addition to the effects of the first and second embodiments, it is possible to reduce the memory usage and the calculation time of a computer that performs an inspection by changing the configuration of the position deviation estimation model.

[0124] The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0125] 101...design data, 102...captured image, 301...learning design data, 302...learning captured image, 303, 401, 701, 1101...learning processing unit, 304, 410, 1108...model data, 305...inspection design data, 306...inspection captured image, 307, 307A, 307B...inspection processing unit, 308...inspection result, 402...probability distribution estimation unit, 403...learning probability distribution, 404, 702, 1102...positional deviation amount estimation unit, 405, 703, 1103...estimated positional deviation amount, 406...positional deviation reflection unit, 407...learning time after positional deviation reflection Probability distribution, 408, 1104...model evaluation unit, 409...model parameter update amount, 704, 1107...positional deviation estimation setting amount, 705...learning progress display unit, 706...positional deviation estimation setting amount update unit, 1000...GUI, 1001...inference result selection unit, 1002...inference image display unit, 1003...display step number selection unit, 1004...coordinate / magnification setting unit, 1005...positional deviation estimation setting amount input unit, 1006...positional deviation estimation setting amount determination unit, 1105...probability distribution estimation model parameter update amount, 1106...positional deviation amount estimation model parameter update amount.

Claims

1. 1. An image inspection apparatus for inspecting a captured image of a sample using design data of the sample, the image comprising: a learning processing unit that learns a probability distribution estimation model that estimates a probability distribution of pixel values ​​of a captured image from design data; an inspection processing unit that inspects an inspection captured image using a probability distribution estimation model created by the learning processing unit, inspection design data, and the inspection captured image, The learning processing unit includes a probability distribution estimation unit that estimates a probability distribution of pixel values ​​of a learning captured image of a sample from learning design data of the sample; a positional deviation amount estimation unit that estimates a positional deviation amount between the learning probability distribution estimated by the probability distribution estimation unit and the learning captured image; a position shift reflection unit that reflects the estimated position shift amount estimated by the position shift amount estimation unit in the learning probability distribution; and a model evaluation unit that evaluates a probability distribution estimation model of the probability distribution estimation unit using the learning probability distribution after reflecting the position shift calculated by the position shift reflection unit and the learning captured images, and updates parameters of the probability distribution estimation model in accordance with the evaluation value.

2. 2. The image inspection device according to claim 1, an input of a positional deviation estimation setting amount that stabilizes learning of a probability distribution of pixel values ​​of a captured image by constraining the positional deviation amount according to a number of learning steps including a maximum value of the positional deviation amount estimated by the positional deviation amount estimation unit.

3. 3. The image inspection apparatus according to claim 2, a learning progress display unit that records the learning probability distribution, the learning probability distribution after reflecting the position shift, and the visualization results of the estimated position shift amount for each learning step and displays them on a GUI; and a positional deviation estimated set amount update unit that enables a user to update the positional deviation estimated set amount based on a display result of the learning progress display unit.

4. 2. The image inspection device according to claim 1, The image inspection device, wherein the positional deviation estimation unit estimates the amount of positional deviation between the learning probability distribution and the learning captured image using a positional deviation estimation model created by machine learning.

5. 5. The image inspection apparatus according to claim 4, the model evaluation unit evaluates the probability distribution estimation model and the positional deviation estimation model, an image inspection apparatus that updates parameters of the probability distribution estimation model and parameters of the positional deviation estimation model in accordance with an evaluation value;

6. 1. An image processing method for learning a model that estimates a probability distribution of pixel values ​​of a captured image by using design data of a sample and a captured image of the sample, comprising: (a) estimating a learning probability distribution of pixel values ​​of a learning captured image of the sample from learning design data of the sample; (b) estimating a positional deviation amount between the learning probability distribution estimated in the step (a) and the learning captured image; (c) reflecting the positional deviation amount estimated in the step (b) in the learning probability distribution; (d) evaluating the probability distribution estimation model estimated in the step (a) using the learning probability distribution after reflecting the position shift calculated in the step (c) and the learning captured images, and updating parameters of the probability distribution estimation model in accordance with the evaluation value.

7. 7. The image processing method according to claim 6, The image processing method is characterized in that in the step (b), a positional deviation estimation setting amount is input that stabilizes learning of the probability distribution of pixel values ​​of the captured image by constraining the positional deviation amount according to the number of learning steps including a maximum value of the positional deviation amount to be estimated.

8. 8. The image processing method according to claim 7, The learning probability distribution, the learning probability distribution after reflecting the position shift, and the visualization results of the position shift amount are recorded for each learning step and displayed on a GUI; The image processing method according to the present invention, wherein a user can update the positional deviation estimation setting amount based on the display result of the GUI.

9. 7. The image processing method according to claim 6, An image processing method, comprising: estimating, in the step (b), the amount of positional deviation between the learning probability distribution and the learning captured image using a positional deviation estimation model created by machine learning.

10. 10. The image processing method according to claim 9, In the step (d), the probability distribution estimation model and the position deviation estimation model are evaluated, an image processing method comprising updating parameters of the probability distribution estimation model and parameters of the positional deviation estimation model in accordance with an evaluation value;

Images