Information processing device, inspection device, information processing method, inspection method, and learning method

Abstract

The present invention provides an information processing device, an inspection device, an information processing method, an inspection method, and a learning method that can improve the handling of areas prone to noise. [Solution] The information processing device 200 according to this embodiment includes a learning unit 210 for training a model, an image acquisition unit 220 for acquiring an image captured CI, a reference image generation unit 230 for generating a reference image RI, and an evaluation unit 240 for evaluating the object 300 based on a comparison between the reference image RI and the image captured CI. The learning unit 210 includes a residual calculation unit 212 for calculating a residual LS by comparing an estimated image PI output from the model during the training process with a training image TI including the image captured CI, and a determination unit 213 for determining that the model training is complete when the residual LS satisfies predetermined conditions. The residual calculation unit 212 calculates the residual LS based on the difference image DI.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an inspection apparatus, an information processing method, an inspection method, and a learning method.

Background Art

[0002] Patent Document 1 describes an inspection method for inspecting a photomask by comparing a captured image of a photomask manufactured based on design data with a reference image generated from the design data. The inspection method of Patent Document 1 generates a reference image from design data using a machine learning model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A machine learning model such as that of Patent Document 1 generally determines that the training of the model is completed based on the fact that the difference (residual or loss) between the output image by the model in the training process and the captured image is below a predetermined threshold. [[ID=3X6]]

[0005] However, according to the inventors' findings, depending on the nature of the object to be imaged, noise is likely to be superimposed on a partial region of the captured image. Here, since the residual may include a residual based on insufficient training of the model and a residual based on the noise of the captured image, a region where noise is likely to be superimposed is more likely to generate a residual than a region where noise is less likely to be superimposed. If the residual is calculated without considering this and the completion of the model training is managed, there are disadvantages such as insufficient generation accuracy of the output image by the trained model.

[0006] Note: There seems to be an error in the original text where line number 19 in the source text has an incorrect tag , and I've marked it as <X19> in the translation for clarity. Similarly, line 33 in the source text has an incorrect tag

[0004] , and I've marked it as <X33>. Also, line 36 in the source text has an incorrect tag , and I've marked it as <3X6>. These should be corrected in the original text for proper translation and understanding.This disclosure was made in consideration of these problems and provides an information processing device, inspection device, information processing method, inspection method, and learning method that can improve the handling of areas prone to noise. [Means for solving the problem]

[0007] An information processing device according to one aspect of this embodiment includes: a learning unit for training a model; an image acquisition unit for acquiring an image of an object; a reference image generation unit for generating a reference image based on the design data of the object and the model; and an evaluation unit for evaluating the object based on a comparison between the reference image and the image. The learning unit includes: a residual calculation unit for calculating residuals by comparing an estimated image output from the model during the training process with a training image including the image; and a determination unit for determining that the training of the model is complete when the residual satisfies predetermined conditions. The residual calculation unit calculates each pixel of the estimated image. The residual is calculated based on the difference image obtained by subtracting the information from the information of each pixel of the training image. A corrected evaluation value is calculated by assigning different weights to the evaluation value of the first pixel of the difference image corresponding to a pixel whose information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a first brightness, and to the evaluation value of the second pixel of the difference image corresponding to a pixel whose information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a second brightness lower than the first brightness. The residual of the difference image is then calculated by arithmetic processing of the corrected evaluation value.

[0008] In the information processing device described above, the residual calculation unit may assign a larger weight to the evaluation value of the second pixel than the weight assigned to the evaluation value of the first pixel.

[0009] In the above-described information processing device, the residual calculation unit may calculate the residual of the difference image by performing arithmetic operations on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels.

[0010] In the above-described information processing device, the residual calculation unit may obtain statistical values ​​for multiple pixels of the multiple captured images, based on a comparison of multiple captured images of substantially identical regions of the object, indicating the degree of variation in pixel information among the multiple captured images, and calculate the weighting of the evaluation value of the difference image based on the obtained statistical values.

[0011] An inspection apparatus according to one aspect of this embodiment comprises an imaging device for imaging the object and the information processing device described above.

[0012] An information processing method according to one aspect of this embodiment includes the steps of training a model, acquiring captured images of an object, generating a reference image based on the design data of the object and the model, and evaluating the object based on a comparison of the reference image and the captured image, wherein the step of training the model includes the steps of calculating residuals by comparing an estimated image output from the model during the training process with a training image including the captured image, and determining that the training of the model is complete when the residual satisfies predetermined conditions, and in the step of calculating the residuals, the estimated The residual is calculated based on a difference image obtained by subtracting the information of each pixel in the fixed image from the information of each pixel in the training image. A corrected evaluation value is calculated by assigning different weights to the evaluation value of the first pixel in the difference image corresponding to a pixel whose information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a first brightness, and to the evaluation value of the second pixel in the difference image corresponding to a pixel whose information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a second brightness lower than the first brightness. The residual of the difference image is then calculated by arithmetic processing of the corrected evaluation value.

[0013] In the above-described information processing method, in the step of calculating the residual, the weighting of the second pixel to the evaluation value may be greater than the weighting of the first pixel to the evaluation value.

[0014] In the above-described information processing method, the residual of the difference image may be calculated in the step of calculating the residual by performing an arithmetic operation on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels.

[0015] In the above information processing method, in the step of calculating the residual, statistical values ​​indicating the degree of variation in pixel information among the multiple captured images are obtained for multiple pixels of the captured images based on a comparison of multiple captured images of substantially identical regions of the object, and the weighting of the evaluation value of the difference image is calculated based on the obtained statistical values.

[0016] An inspection method according to one aspect of this embodiment comprises the steps of imaging an object and performing information processing using the information processing method described above.

[0017] A learning method according to one aspect of this embodiment includes the steps of: calculating residuals by comparing an estimated image output from a model in the training process with a training image including an image of an object; and determining that the training of the model is complete when the residual satisfies predetermined conditions. In the step of calculating residuals, the residuals are calculated based on a difference image obtained by differentiating the information of each pixel in the estimated image with the information of each pixel in the training image; calculating corrected evaluation values ​​by assigning different weights to the evaluation value of a first pixel in the difference image corresponding to a pixel whose information based on at least one of the pixel information of the estimated image and the pixel information of the training image indicates a first brightness; and calculating corrected evaluation values ​​by arithmetic processing of the corrected evaluation values.

[0018] In the learning method described above, in the step of calculating the residual, the weighting of the second pixel to the evaluation value may be greater than the weighting of the first pixel to the evaluation value.

[0019] In the learning method described above, the residual of the difference image may be calculated in the step of calculating the residual by performing an arithmetic operation on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels.

[0020] In the learning method described above, in the step of calculating the residual, statistical values ​​indicating the degree of variation in pixel information among multiple captured images may be obtained for multiple pixels of the captured images based on a comparison of multiple captured images of substantially identical regions of the object, and the weighting of the evaluation value of the difference image may be calculated based on the obtained statistical values.

[0021] A learning method according to one aspect of this embodiment comprises the steps of: calculating residuals by comparing an estimated image output from a model in the training process with a training image including an image of an object; determining that the training of the model is complete when the residual satisfies predetermined conditions; and classifying multiple pixels of an image into either a first group of pixels or a second group of pixels based on a comparison of multiple images of substantially identical regions of the object, wherein the pixels of the second group are pixels in which the variation of pixel information among multiple images of substantially identical regions of the object is smaller than that of the pixels of the first group; and in the step of calculating residuals, the residuals are calculated based on a difference image obtained by differentiating the information of each pixel in the estimated image with the information of each pixel in the training image; and different weights are assigned to the evaluation value of the first pixel corresponding to the pixels of the first group in the difference image and the evaluation value of the second pixel corresponding to the pixels of the second group in the difference image. The corrected evaluation value is calculated by performing the above, and the residual of the difference image is calculated by arithmetic processing of the corrected evaluation value.

[0022] A learning method according to one aspect of this embodiment includes the steps of: calculating residuals by comparing an estimated image output from a model in the training process with a training image including an image of an object; determining that the training of the model is complete when the residual satisfies predetermined conditions; and classifying multiple pixels of an image into either a first group of pixels or a second group of pixels based on a comparison of multiple images of substantially identical regions of the object, wherein the pixels of the second group are more than the pixels of the first group when comparing multiple images of substantially identical regions of the object. In the step of calculating residuals, a pixel has small variation in information, and based on the difference image obtained by differentiating the information of each pixel in the estimated image with the information of each pixel in the training image, a first residual for the first pixel corresponding to the first group of pixels in the difference image and a second residual for the second pixel corresponding to the second group of pixels in the difference image are calculated, and in the step of determining that the training of the model is complete when the residuals satisfy predetermined conditions, a predetermined second condition applied to the second residual is set to be stricter than the predetermined first condition applied to the first residual, and it is determined that the training of the model is complete when the second residual satisfies the predetermined second condition and the first residual satisfies the predetermined first condition.

[0023] A learning method according to one aspect of this embodiment comprises the steps of: classifying multiple pixels of an image into either a first group of pixels or a second group of pixels based on a comparison of multiple image images of substantially identical regions of an object; and training a model that outputs a reference image to be compared with the image based on the design data, using the image of the object and an image based on the design data of the object as training data, wherein the pixels of the second group are pixels in which the variation of pixel information among multiple image images of substantially identical regions of the object is smaller than that of the pixels of the first group, and in the step of training the model, the training data includes the image images containing the first group of pixels and the image images containing the second group of pixels, and the number of image images containing the first group of pixels is greater than the number of image images containing the second group of pixels.

Advantages of the Invention

[0024] According to the present disclosure, there are provided an information processing apparatus, an inspection apparatus, an information processing method, an inspection method, and a learning method capable of improving the handling of areas where noise is likely to be superimposed.

Brief Description of the Drawings

[0025] [Figure 1] It is a schematic diagram illustrating an inspection apparatus according to Embodiment 1. [Figure 2] It is a diagram illustrating the training of a rendering model in the information processing apparatus according to Embodiment 1. [Figure 3] It is a diagram illustrating the inspection using the trained rendering model in the information processing apparatus according to Embodiment 1. [Figure 4] It is a configuration diagram illustrating an imaging apparatus according to Embodiment 1. [Figure 5] It is a configuration diagram illustrating another imaging apparatus according to Embodiment 1. [Figure 6] It is a block diagram illustrating the configuration of the information processing apparatus according to Embodiment 1. [Figure 7] It is a diagram illustrating the residual calculation performed by the residual calculation unit in the information processing apparatus according to Embodiment 1. [Figure 8] It is a plan view illustrating an estimated image in the information processing apparatus according to Embodiment 1. [Figure 9] It is a plan view illustrating a teacher image in the information processing apparatus according to Embodiment 1. [Figure 10] It is a plan view illustrating a difference image in the information processing apparatus according to Embodiment 1. [Figure 11] It is a graph illustrating a function used by the residual calculation unit for weighting in the information processing apparatus according to Embodiment 1. The horizontal axis represents luminance, and the vertical axis represents the standard deviation as weighting. [Figure 12]This graph illustrates the distribution of evaluation values ​​(difference values) before weighting in the information processing device according to Embodiment 1. The horizontal axis represents sample pixel information (luminance), and the vertical axis represents evaluation values ​​(difference values). [Figure 13] This graph illustrates the distribution of weighted evaluation values ​​(difference values) in the inspection apparatus according to Embodiment 1, with the horizontal axis representing sample pixel information (luminance) and the vertical axis representing the corrected evaluation value. [Figure 14] This graph illustrates the distribution of evaluation values ​​(difference values) of pixels in a difference image obtained by differentiating two captured images in the information processing device according to Embodiment 1. The horizontal axis represents the brightness of pixels in the captured image, and the vertical axis represents the evaluation value (difference value). [Figure 15] This flowchart illustrates an information processing method using the information processing device according to Embodiment 1. [Figure 16] This flowchart illustrates a learning method using a learning unit in the information processing device according to Embodiment 1. [Figure 17] This flowchart illustrates the residual calculation method performed by the residual calculation unit in the inspection apparatus according to Embodiment 1. [Figure 18] This is a flowchart illustrating an example of the inspection method according to Embodiment 1. [Figure 19] This is a block diagram illustrating the configuration of an information processing device according to a modified example of Embodiment 1. [Figure 20] This graph illustrates the distribution of evaluation values ​​(difference values) of pixels in a difference image obtained by differentiating two captured images in the information processing device according to Embodiment 2. The horizontal axis represents the brightness of pixels in the captured image, and the vertical axis represents the evaluation value (difference value). [Figure 21] This flowchart illustrates a learning method using the learning unit in the information processing device according to Embodiment 3. [Figure 22] This flowchart illustrates a learning method using a learning unit in the information processing device according to Embodiment 5. [Modes for carrying out the invention]

[0026] Embodiments of the present disclosure will be described below with reference to the drawings. The following description illustrates preferred embodiments of the present disclosure and does not limit the scope of the present disclosure to the following embodiments. In the following description, the same reference numerals indicate substantially the same thing. Some reference numerals may be omitted to avoid cluttering the drawings.

[0027] <Embodiment 1> Embodiment 1 will now be described. First, the <inspection device> will be described, followed by the <imaging device> and <information processing device> within the inspection device. Then, the <information processing method> and <learning method> will be described, followed by the <inspection method>.

[0028] <Inspection equipment> An inspection apparatus according to Embodiment 1 will now be described. Figure 1 is a schematic diagram illustrating an inspection apparatus according to Embodiment 1. As shown in Figure 1, the inspection apparatus 1 according to this embodiment includes an imaging device 100 and an information processing device 200. In Figure 1, the imaging device 100 and the information processing device 200 are shown separately. However, the inspection apparatus 1 may be an integrated unit with the imaging device 100 and the information processing device 200, or the imaging device 100 and the information processing device 200 may function as separate units.

[0029] The inspection device 1 of this embodiment inspects the object 300. For example, the inspection device 1 inspects for defects present in the object 300. The object 300 may be an EUV (Extreme Ultra Violet) photomask used in lithography using EUV light. The EUV photomask is simply referred to as the EUV mask 310. The object 300 may also be a photomask used in lithography using light other than EUV light. Note that the object 300 is not limited to a photomask; it may also be a semiconductor device, as long as a pattern is formed on it.

[0030] In the following, the object 300 may be described as an EUV mask 310 as an example. In that case, the inspection device 1 is an EUV mask inspection device that inspects the EUV mask 310. The inspection device 1 inspects the EUV mask 310 by capturing an image CI of the EUV mask 310 including the pattern and comparing the captured image CI with a reference image RI. The following describes the outline of the inspection performed by the inspection device 1.

[0031] (1) The inspection device 1 first converts the design data D10 of the EUV mask 310 into a reference image RI. The design data D10 may include design CAD (Computer Aided Design) data. The design data D10 may also include vector data. The process of converting the design data D10 into a reference image RI is called rendering.

[0032] (2) Next, the inspection device 1 acquires an image CI by imaging the EUV mask 310 with the imaging device 100. The inspection device 1 then compares and verifies the reference image RI and the acquired image CI. The inspection device 1 detects defects in the EUV mask 310 from the difference obtained by the comparison and verification.

[0033] In this embodiment, the inspection device 1 generates and trains a rendering model M10 (converter) that performs the conversion process before carrying out (1). The rendering model M10 may sometimes be simply referred to as the model. The rendering model M10 is generated and trained using machine learning techniques. The information processing device 200 generates a reference image RI using the trained rendering model M10. One of the features of this embodiment is the method for determining the completion of training of the rendering model M10. That is, if the rendering model M10 can output a predetermined image as the reference image RI, it is determined that the training is complete.

[0034] Figure 2 illustrates the training of the rendering model M10 in the information processing device 200 according to Embodiment 1. Figure 3 illustrates the inspection using the rendering model M10 after training has been completed in the information processing device 200 according to Embodiment 1.

[0035] As shown in Figure 2, the information processing device 200 trains the rendering model M10 during training. For example, the rendering model M10 includes a network NW such as a multilayer neural network and coefficients KS. The rendering model M10 is input with the design data D10 of the object 300. The rendering model M10, having received the design data D10 of the object 300 as input, outputs an image. Hereinafter, the image output from the rendering model M10 during training will be called the estimated image PI.

[0036] This embodiment introduces residuals as a method for determining the completion of training for the rendering model M10. The residuals are calculated based on the difference between the captured image CI, which serves as the training image TI, and the estimated image PI, which is output by the model M10 during the training process. Specifically, the residuals are calculated based on the difference (e.g., brightness difference) between the information of each pixel in the captured image CI (e.g., brightness) and the information of each pixel in the estimated image PI (e.g., brightness). In this embodiment, training for the rendering model M10 is considered complete if the residuals satisfy predetermined conditions. If the residuals do not satisfy the predetermined conditions, the coefficient KS is updated and training for the rendering model M10 continues.

[0037] In captured image CIs, there are areas that are prone to noise due to factors such as pattern shape. Residuals are more likely to occur in these noise-prone areas than in areas that are less prone to noise. Generally, areas with high brightness are more prone to noise than areas with low brightness. Areas with high brightness are, for example, the multilayer areas in an EUV mask. Areas with low brightness are, for example, the absorber areas in an EUV mask.

[0038] During the training process, the absorber region has less noise. Therefore, the absorber region contributes to reducing residuals. On the other hand, the multi-layer region has more noise. Therefore, the multi-layer region contributes to increasing residuals. The residuals between the estimated image PI and the training image TI during the training process include residuals due to insufficient training and residuals due to noise. Therefore, when calculating residuals using the loss function LOS based on the difference between the estimated image PI and the training image TI, even if the residuals meet the specified conditions, the training may be insufficient. In particular, it is difficult to adequately train the absorber region where the pitch between patterns where proximity effects occur is narrow.

[0039] Therefore, the difference between the estimated image PI and the training image TI is weighted using parameters such as brightness. The weighting may be calculated based on at least one of the brightness of each pixel in the estimated image PI and the brightness of each pixel in the training image TI. The coefficients KS of the rendering model M10 are updated until the residuals satisfy predetermined conditions. Then, the training of the rendering model M10 and the calculation of residuals continue. When the residuals satisfy the predetermined conditions, the training of the rendering model M10 is completed.

[0040] As shown in Figure 3, the information processing device 200 outputs a reference image RI by inputting the design data D10 of the object 300 into the trained rendering model M10 during inspection. The configurations of the <imaging device> and <information processing device> in the inspection device 1 of this embodiment will be described below.

[0041] <Imaging device> First, the imaging device 100 will be described with reference to the figures. Figure 4 is a configuration diagram illustrating the imaging device 100 according to Embodiment 1. Figure 5 is a configuration diagram illustrating another imaging device 100a according to Embodiment 1. As shown in Figure 4, the imaging device 100 may image the EUV mask 310 using transmitted illumination. Also, as shown in Figure 5, the imaging device 100a may image the EUV mask 310 using reflected illumination. As shown in Figure 4, the imaging device 100 includes an illumination light source 110, an illumination optical system 120, a lens 130, a stage 140, a lens 150, a detection optical system 160, and a detector 170.

[0042] In this explanation, we will use an EUV mask 310 with a pattern 311 as the object 300. However, the object 300 is not limited to an EUV mask 310; it may also be a mask used for lithography other than EUV light, or a semiconductor device, as long as it has a pattern 311. When the object 300 is an EUV mask 310, the imaging device 100 functions as an imaging device that images the EUV mask 310 with the pattern 311.

[0043] The illumination light source 110 generates illumination light L10 to illuminate the EUV mask 310. The illumination light L10 from the illumination light source 110 is incident on the illumination optical system 120. The illumination optical system 120 is equipped with optical components such as relay lenses and mirrors, and guides the illumination light L10 to the lens 130. The illumination optical system 120 may also be equipped with an optical scanner and an autofocus (AF) function. The illumination light L10 is focused by the lens 130 and incident on the EUV mask 310. The lens 130 focuses the illumination light L10 on the pattern surface of the EUV mask 310 where the pattern 311 is formed. As a result, the EUV mask 310 is illuminated.

[0044] The transmitted light L20 that has passed through the EUV mask 310 passes through the stage 140, which is transparent to the transmitted light L20, and enters the lens 150. The lens 150 is an objective lens and focuses the transmitted light L20 from the EUV mask 310. The transmitted light L20 enters the detection optical system 160 via the lens 150. The detection optical system 160 is equipped with optical components such as an imaging lens and a mirror and guides the transmitted light L20 to the detector 170. The detection optical system 160 forms an image of the EUV mask 310 on the light-receiving surface of the detector 170.

[0045] The detector 170 is a line sensor or a two-dimensional array sensor such as a CCD (Charged Coupled Device) or CMOS camera (Complementary Metal Oxide Semiconductor) containing multiple pixels. A TDI (Time Delay Integration) sensor can also be used as the detector 170. Therefore, the detector 170 images the EUV mask 310 on which the pattern 311 is provided. The reflectance and transmittance to the illumination light L10 differ depending on whether the pattern 311 is present or not. For example, in the case of the EUV mask 310, the transmittance is lower where the pattern 311 is present and higher where it is not. Therefore, the amount of light received changes depending on whether the pattern 311 is present or not. Note that the difference in transmittance depending on the presence or absence of the pattern is just one example, and the opposite may also be true.

[0046] The EUV mask 310 is placed on the stage 140. The stage 140 is an XY stage, and moves the EUV mask 310 in the X-axis and Y-axis directions. The movement coordinates of the stage 140 are input to the information processing device 200. While the stage 140 is moving the EUV mask 310, the detector 170 images the EUV mask 310. In this way, an image CI of the entire EUV mask 310 or a desired region can be obtained. Depending on the presence or absence of the pattern 311, the transmittance to the illumination light L10 is different. Therefore, depending on the presence or absence of the pattern 311, the brightness value, i.e., the intensity of the detection signal, is different.

[0047] The detector 170 outputs a detection signal to the information processing device 200 according to the amount of light received. This inputs the captured image CI to the information processing device 200. Each pixel of the captured image CI has a grayscale value set according to the amount of light received. The information processing device 200 performs image processing on the detection signal. For example, the information processing device 200 is a computer equipped with a processor, memory, etc., as will be described later.

[0048] As shown in Figure 5, the EUV mask 310 may also be imaged using an imaging device 100a that employs reflective illumination. The imaging device 100a includes an illumination light source 110a, an illumination optical system 120a, a mirror 130a, a stage 140, a detection optical system 160a, and a detector 170. When illuminating and imaging the EUV mask 310 with light of a wavelength in the EUV region as illumination light L30, it is preferable to configure the imaging device 100a as a reflective optical system.

[0049] The illumination light source 110a generates illumination light L30 to illuminate the EUV mask 310. The illumination light L30 from the illumination light source 110a is incident on the illumination optical system 120a. The illumination optical system 120a is equipped with optical components such as an elliptical reflector and guides the illumination light L30 to the mirror 130a. The illumination optical system 120a may also be equipped with an optical scanner or AF function. The illumination light L30 is reflected by the mirror 130a and incident on the EUV mask 310. The mirror 130a focuses the illumination light L30 onto the pattern surface of the EUV mask 310 where the pattern 311 is formed. As a result, the EUV mask 310 is illuminated.

[0050] The reflected light L40 reflected by the EUV mask 310 enters the detection optical system 160a. The detection optical system 160a includes optical components such as a mirror and guides the reflected light L40 to the detector 170. The detection optical system 160a forms an image of the EUV mask 310 on the light-receiving surface of the detector 170.

[0051] <Information Processing Device> Figure 6 is a block diagram illustrating the configuration of the information processing device 200 according to Embodiment 1. As shown in Figure 6, the information processing device 200 comprises a learning unit 210, an image acquisition unit 220, a reference image generation unit 230, an evaluation unit 240, a learning storage unit 250, and a control unit 260. The learning unit 210 has an estimated image generation unit 211, a residual calculation unit 212, a determination unit 213, and a training unit 214. The control unit 260 has a processor PRC, a memory MMR, a storage device STR, and a user interface UI. The information processing device 200 includes information processing equipment such as a PC (Personal Computer), a server, and a tablet.

[0052] First, the functions of the control unit 260 will be explained. The storage device STR stores programs for the processes to be executed by each component of the information processing device 200. The processor PRC loads the programs from the storage device STR into the memory MMR and executes them. In this way, the processor PRC realizes the functions of each component of the information processing device 200, such as the learning unit 210, the image acquisition unit 220, the reference image generation unit 230, and the evaluation unit 240. The user interface UI may include input devices such as a keyboard, mouse, and imaging equipment, as well as output devices such as a display, printer, and speaker.

[0053] Each component of the information processing device 200 may be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits and processors (PRCs), or combinations thereof. These may be implemented by a single chip or by multiple chips connected via a bus. Some or all of each component may be implemented by a combination of the aforementioned circuits and processors (PRCs) and a program. As the processor (PRC), a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-programmable Gate Array), a quantum processor (quantum computer control chip), etc., can be used.

[0054] Furthermore, if some or all of the components of the information processing device 200 are implemented by multiple information processing devices or circuits, these devices and circuits may be centrally located or distributed. For example, the information processing devices and circuits may be implemented in a form where they are connected via a communication network, such as a client-server system or a cloud computing system. Also, the functions of the information processing device 200 may be provided in SaaS (Software as a Service) format.

[0055] The learning unit 210 trains the rendering model M10. The learning unit 210 trains the rendering model M10 by operating the estimated image generation unit 211, the residual calculation unit 212, the judgment unit 213, and the training unit 214.

[0056] The estimated image generation unit 211 generates an estimated image PI by inputting the design data D10 of the object 300 into the rendering model M10 in the training process. As mentioned above, the rendering model M10 includes a network NW and coefficients KS.

[0057] The residual calculation unit 212 calculates the residual by comparing the estimated image PI output from the rendering model M10 during the training process with the training image TI, which includes the captured image CI of the object 300. For example, the residual calculation unit 212 calculates the residual by comparing the estimated image PI output from the rendering model M10 during the training process, which has been input with the design data D10 of the object 300, with the training image TI, which includes the captured image CI of the object 300. Specifically, the rendering model M10 is input with a design image obtained by rasterizing the design data D10 of the object 300 as the design data D10 of the object 300. Note that the design data D10 and the design image may not be distinguished and are simply referred to as design data D10.

[0058] Figure 7 is a diagram illustrating the residual calculation performed by the residual calculation unit 212 in the information processing device 200 according to Embodiment 1. Figure 8 is a plan view illustrating the estimated image PI in the information processing device 200 according to Embodiment 1. Figure 9 is a plan view illustrating the training image TI in the information processing device 200 according to Embodiment 1. Figure 10 is a plan view illustrating the difference image DI in the information processing device 200 according to Embodiment 1. As shown in Figures 7 to 10, the residual calculation unit 212 calculates the residual based on the difference image DI, which is obtained by subtracting the information of each pixel in the estimated image PI from the information of each pixel in the training image TI.

[0059] The estimated image PI is output from the training rendering model M10, which has been input with the design data D10 of the object 300. The estimated image PI may also be a portion of the larger image output from the rendering model M10. The estimated image PI may include, for example, multiple pixels arranged in a matrix. In the estimated image PI, the direction in which the pixels are arranged is defined as the α-axis direction, and the direction intersecting the α-axis direction is defined as the β-axis direction. The estimated image PI may include multiple pixels arranged in L rows along the α-axis direction and M columns along the β-axis direction. Each pixel in the estimated image PI has pixel information. The information for each pixel in the estimated image PI includes, for example, the brightness of the pixel.

[0060] Note that "each pixel" may mean all the pixels that make up the image, or it may mean some of the pixels (however, multiple pixels) that make up the image. In this embodiment, unless otherwise specified, the expression "each pixel" is used to mean either.

[0061] The training image TI may include an captured image CI. The training image TI may also be a portion of the captured image CI in which the object 300 is captured. The training image TI may include, for example, a plurality of pixels arranged in a matrix. In the training image TI, the direction in which the pixels are arranged is defined as the α-axis direction, and the direction intersecting the α-axis direction is defined as the β-axis direction. The training image TI may include a plurality of pixels arranged in L rows along the α-axis direction and M columns along the β-axis direction. Each pixel in the training image TI has pixel information. The information of each pixel in the training image TI includes, for example, the brightness of the pixel.

[0062] The residual calculation unit 212 generates a difference image DI between the estimated image PI and the training image TI. The difference image DI may include, for example, a plurality of pixels arranged in a matrix. In the difference image DI, one direction in which the pixels are arranged is defined as the α-axis direction, and the direction intersecting the α-axis direction is defined as the β-axis direction. The difference image DI may include a plurality of pixels arranged in L rows along the α-axis direction and M columns along the β-axis direction. Each pixel in the difference image DI has pixel information. The information of each pixel in the difference image DI includes, for example, an evaluation value. The difference image DI is called a difference image DI because it may be data containing information that can be used for image generation. However, it is sufficient if the information is used for evaluating residuals, and therefore, it is not essential that the difference image DI be displayed on a display unit or the like so that it can be recognized by the user as an image. Accordingly, the difference image DI may also be called difference data.

[0063] The evaluation value is the difference between the information of each pixel in the estimated image PI and the information of each pixel in the training image TI corresponding to each pixel in the estimated image PI. A pixel in the training image TI corresponding to a pixel in the estimated image PI means that the position of the pixel in the estimated image PI on the α-axis and β-axis is the same as the position of the pixel in the training image TI on the α-axis and β-axis. The evaluation value includes, for example, the brightness difference obtained by the difference between the brightness of each pixel in the estimated image PI and the brightness of each pixel in the training image TI. Therefore, each pixel in the difference image DI has an evaluation value that includes the brightness difference as pixel information. In addition, each pixel in the difference image DI may further include the information of the corresponding pixel in the estimated image PI and the information of the corresponding pixel in the training image TI as pixel information.

[0064] For example, if the brightness of a pixel in the corresponding estimated image PI is 15, and the brightness of a pixel in the corresponding training image TI is 10, and the brightness difference between the two is 5, then the pixel G1 in the corresponding difference image DI may include G1(15, 10, 5) as information for pixel G1.

[0065] Furthermore, if the brightness of a pixel in the corresponding estimated image PI is 9, and the brightness of a pixel in the corresponding training image TI is 4, and the brightness difference between the two is 5, then the pixel G2 in the corresponding difference image DI may include G2(9, 4, 5) as information for pixel G2.

[0066] The residual calculation unit 212 weights the evaluation value of each pixel in the difference image DI. The residual calculation unit 212 designates the pixel in the difference image DI corresponding to the pixel whose pixel information (which may be called sample pixel information) based on at least one of the pixel information (e.g., brightness) of the estimated image PI and the pixel information (e.g., brightness) of the training image TI shows a first brightness as the first pixel. The residual calculation unit 212 designates the pixel in the difference image DI corresponding to the pixel whose pixel information (which may be called sample pixel information) based on at least one of the pixel information (e.g., brightness) of the estimated image PI and the pixel information (e.g., brightness) of the training image TI shows a second brightness as the second pixel. Any pixel other than the first pixel may be designated as the second pixel. For example, the second brightness is lower than the first brightness. That is, first brightness > second brightness. The residual calculation unit 212 assigns different weights to the evaluation value of the first pixel in the difference image DI and the evaluation value of the second pixel in the difference image DI. The first and second brightness may include a predetermined range of brightness.

[0067] The sample pixel information is pixel information (e.g., brightness) based on at least one of the following: the information of a pixel in the estimated image PI (e.g., brightness) and the information of a pixel in the training image TI corresponding to that pixel in the estimated image PI (e.g., brightness). For example, the sample pixel information is the greater of the brightness of a pixel in the estimated image PI and the brightness of a pixel in the training image TI corresponding to that pixel in the estimated image PI. The sample pixel information may be determined under predetermined conditions based on at least one of the information of a pixel in the estimated image PI and the information of a pixel in the training image TI corresponding to that pixel in the estimated image PI. The brightness of the pixel that makes the difference value of defects etc. easier to see between the case where the brightness of a pixel in the estimated image PI is used and the case where the brightness of a pixel in the training image TI is used may be used as the sample pixel information. It may also be determined according to the progress of training of the rendering model M10. For example, at the beginning of training, the brightness of a pixel in the training image TI may be used as the sample pixel information, and towards the end of training, the brightness of a pixel in the estimated image PI may be used as the sample pixel information. Here, the progress of training of the rendering model M10 may be evaluated based on the magnitude of the residual LS value described later. The sample pixel information may be the average or weighted average of the pixel information (e.g., brightness) of the estimated image PI and the pixel information (e.g., brightness) of the training image TI corresponding to the pixels of the estimated image PI. The weights in the weighted average may be changed according to the training progress of the rendering model M10.

[0068] For example, let's assume that the sample pixel information is the larger of the brightness of the pixels in the estimated image PI and the brightness of the pixels in the training image TI. Also, let's assume that the first brightness is between 10 and 20, and the second brightness is between 0 and 10. Then, for pixel G1 (15, 10, 5) in the difference image DI described above, the brightness of the corresponding pixel in the estimated image PI is 15, and the brightness of the corresponding pixel in the training image TI is 10. Therefore, the sample pixel information is 15, and pixel G1 in the difference image DI is the first pixel. For pixel G2 (9, 4, 5) in the difference image DI described above, the brightness of the corresponding pixel in the estimated image PI is 9, and the brightness of the corresponding pixel in the training image TI is 4. Therefore, the sample pixel information is 9, and pixel G2 in the difference image DI is the second pixel. Accordingly, the residual calculation unit 212 assigns different weights to the evaluation value (5) of pixel G1 in the difference image DI and the evaluation value (5) of pixel G2 in the difference image DI.

[0069] Figure 11 is a graph illustrating a function used for weighting by the residual calculation unit 212 in the information processing device 200 according to Embodiment 1, where the horizontal axis represents luminance and the vertical axis represents the correction coefficient as weighting. As shown in Figure 11, the residual calculation unit 212 may use the correction coefficient F(x) shown in the following equation (1) as an example of weighting for the evaluation value. Here, a and b are constants that depend on the specifications of the inspection device 1, etc. x represents luminance as sample pixel information.

[0070] F(x) = 1 - a × exp(-b × x) (1)

[0071] F(x) is a function that takes larger values ​​as the brightness increases. F(x) asymptotically approaches 1 for high brightness. By adjusting the values ​​of a and b, F(x) > 0 for x greater than or equal to 0.

[0072] The residual calculation unit 212 calculates the corrected evaluation value L by weighting the evaluation value of each pixel in the difference image DI using the following equation (2). The weighted evaluation value is called the corrected evaluation value L. Here, diff represents the evaluation value of each pixel in the difference image DI, such as the brightness difference. K is a constant that depends on the specifications of the inspection device 1.

[0073] L(diff, x) = K × diff / F(x) =K × diff / {{1 - a × exp(-b × x)}} (2)

[0074] The residual calculation unit 212 may, instead of using equation (2), calculate the corrected evaluation value L based on the standard deviation σ(x) of diff as shown in equation (3) below. Here, diffavex is the average value of diff in the set of pixels whose brightness as sample pixel information is x.

[0075] L(diff, x)=K×(diff−diffavex) / σ(x) (3)

[0076] According to equation (3) above, the residual calculation unit 212 calculates a corrected evaluation value L by standardizing evaluation values ​​such as luminance difference with respect to luminance x.

[0077] Figure 12 is a graph illustrating the distribution of evaluation values ​​(diff) (difference values, e.g., brightness difference) for the difference image DI before weighting in the information processing device 200 according to Embodiment 1, where the horizontal axis represents sample pixel information (brightness) and the vertical axis represents the evaluation value (diff) (difference values, e.g., brightness difference). Figure 13 is a graph illustrating the distribution of corrected evaluation values ​​L for the difference image DI after weighting in the information processing device 200 according to Embodiment 1, where the horizontal axis represents sample pixel information (brightness) and the vertical axis represents the corrected evaluation value L. As shown in Figure 12, the residual calculation unit 212 may, for example, obtain the average value diffavex and the standard deviation σ(x) of the difference value diff for each brightness range portion where the brightness width of the pixels is within the range of ±1.

[0078] As shown in Figures 12 and 13, the residual calculation unit 212 divides the evaluation value of the first pixel corresponding to a pixel whose sample pixel information indicates a first brightness (> second brightness) by a large F(x) (Equation (2)) or standardizes it based on the standard deviation σ (Equation (3)). This allows the evaluation value of the first pixel in the difference image DI to be underestimated. On the other hand, the residual calculation unit 212 divides the evaluation value of the second pixel corresponding to a pixel whose sample pixel information indicates a second brightness (< first brightness) by a small F(x) (Equation (2)) or standardizes it based on the standard deviation σ (Equation (3)). This allows the evaluation value of the second pixel in the difference image DI to be overestimated. In this way, the residual calculation unit 212 calculates a corrected evaluation value L by assigning different weights to the evaluation value of the first pixel and the evaluation value of the second pixel by dividing the evaluation value (diff) for the difference image DI by a function of x (for example, F(x) above) that has a value greater than 0 and increases monotonically with respect to brightness x, thereby assigning different weights to the evaluation value of the first pixel and the evaluation value of the second pixel. Alternatively, the residual calculation unit 212 calculates a corrected evaluation value L by standardizing the evaluation value (diff) for the difference image DI by luminance x, thereby assigning different weights to the evaluation value of the first pixel and the evaluation value of the second pixel. The variability of the evaluation value (diff) for the difference image DI differs depending on the luminance x. Therefore, standardizing the evaluation value of the first pixel and the evaluation value of the second pixel by luminance x can be said to correct the evaluation value (diff) for the difference image DI by assigning different weights according to the variability of the evaluation values.

[0079] The residual calculation unit 212 may divide the sample pixel information (luminance) into three or more luminance levels, such as first luminance > second luminance > third luminance, etc. The residual calculation unit 212 may also divide the pixels of the difference image DI corresponding to each luminance of the sample pixel information (luminance) into three or more pixels, such as first pixel, second pixel, third pixel, etc. Furthermore, the residual calculation unit 212 may assign predetermined weights to the evaluation values ​​of the first pixel, second pixel, third pixel, etc. This allows for overestimation and underestimation of the evaluation values ​​of the first pixel, second pixel, third pixel, etc. of the difference image DI by predetermined proportions.

[0080] As shown in Figure 13, the residual calculation unit 212 calculates a corrected evaluation value by assigning different weights to the evaluation value of the first pixel of the difference image DI and the evaluation value of the second pixel of the difference image DI. For example, the residual calculation unit 212 assigns a greater weight to the evaluation value of the second pixel than to the evaluation value of the first pixel. This makes it possible to equalize the variation in the corrected evaluation value at the first pixel and the variation in the corrected evaluation value at the second pixel.

[0081] The residual calculation unit 212 calculates the residual of the difference image DI by arithmetically processing the corrected evaluation value. The residual calculation unit 212 may calculate and obtain the residual of the difference image DI based on the corrected evaluation value. The residual calculation unit 212 may use the corrected evaluation value as the residual of the difference image DI. The residual calculation unit 212 may calculate the residual from multiple reference values ​​VA to VD. For example, the reference value VA is the maximum value in the corrected evaluation value of multiple pixels in the difference image DI. The reference value VB is the average value in the corrected evaluation value of multiple pixels in the difference image DI. The reference value VC is the maximum value in the corrected evaluation value of multiple pixels corresponding to the pattern edge portion of the difference image DI. The reference value VD is the average value in the corrected evaluation value of multiple pixels corresponding to the pattern edge portion of the difference image DI. Note that the reference values ​​are not limited to the four reference values ​​VA to VD.

[0082] The residual calculation unit 212 calculates a single residual LS using equation (4), with coefficients J1 to J4 for the reference values ​​VA to VD, respectively.

[0083] LS=(J1×VA)+(J2×VB)+(J3×VC)+(J4×VD) (4)

[0084] Thus, the residual calculation unit 212 may calculate the residual LS of the difference image DI by arithmetically processing at least one of the average value of the correction evaluation values ​​of multiple pixels, including the first and second pixels, in the difference image DI, and the maximum value of the correction evaluation values ​​of multiple pixels.

[0085] Figure 14 is a graph illustrating the distribution of evaluation values ​​(difference values) of pixels in a difference image obtained by differentiating captured images CI in the information processing device 200 according to Embodiment 1. The horizontal axis represents the brightness of pixels in the captured image CI, and the vertical axis represents the evaluation value (difference value). As shown in Figure 14, the residual calculation unit 212 may obtain statistical values ​​for multiple pixels of the captured image CI, indicating the degree of variation in pixel information between multiple captured image CIs, based on a comparison of multiple captured image CIs for substantially identical regions of the object 300. The residual calculation unit 212 then calculates the weighting of the evaluation values ​​of the difference image based on the obtained statistical values. Specifically, as an example, the residual calculation unit 212 may obtain the evaluation values ​​(difference values) of multiple pixels in a difference image obtained by differentiating the information of each pixel in multiple captured image CIs in advance. Substantially identical regions include, for example, regions with the same design information but different coordinates on the object 300, or regions on the object 300 (e.g., a photomask) that belong to different dies but have the same relative coordinates within the die. Statistical values ​​include, for example, standard deviation and mean. The residual calculation unit 212 may then calculate weights for the evaluation values ​​of pixels in the difference image DI based on the acquired statistical values. In other words, the residual calculation unit 212 may obtain weights for the evaluation values ​​of pixels in the difference image DI by calculating F(x) and standard deviation σ(x) for the difference image based on statistical values ​​of the evaluation values ​​of multiple pixels in the difference image. The residual calculation unit 212 may also calculate corrected evaluation values ​​by applying the acquired F(x) and standard deviation σ(x) to the evaluation values ​​(diff) of pixels in the difference image DI between the training image TI and the estimated image PI.

[0086] Furthermore, the residual calculation unit 212 may obtain weights for the evaluation values ​​of pixels in the difference image DI by calculating statistical values ​​for each luminance of the luminance difference of multiple pixels in the difference image obtained by pre-calculating the luminance of each pixel in a plurality of captured image CIs. Also, "for each luminance" may mean for each range of luminance. For example, the residual calculation unit 212 may obtain statistical values ​​for each luminance for ranges H1, H2, H3, and H4, etc., as shown in Figure 14, by the residual calculation unit 212 calculating them. Then, the residual calculation unit 212 may calculate weights for each luminance difference in the difference image D1 based on the obtained statistical values ​​for each luminance.

[0087] As will be described later, the residual calculation unit 212 may obtain statistical values ​​from a processing unit that performs various statistical processing to calculate statistical values.

[0088] The determination unit 213 determines that training of the rendering model M10 is complete when the residual LS satisfies predetermined conditions.

[0089] The training unit 214 trains the rendering model M10 using the estimated image PI and the training image TI.

[0090] The image acquisition unit 220 acquires an image CI from the imaging device 100. The image acquisition unit 220 acquires the image CI based on the detection signal from the detector 170 of the imaging device 100. The image acquisition unit 220 acquires a two-dimensional image of the EUV mask 310 by associating the coordinates of the stage 140 with the intensity of the detection signal. The image CI is an image acquired by imaging the object 300. The image acquisition unit 220 may also acquire an image CI that has been previously stored in a storage medium such as a storage device STR from the storage device STR.

[0091] The reference image generation unit 230 generates a reference image RI based on the design data D10 of the object 300, such as the EUV mask 310. The reference image generation unit 230 may also generate the reference image RI based on the design data D10 of the object 300 and a trained rendering model M10. Specifically, the reference image generation unit 230 generates the reference image RI from the design data D10 using the rendering model M10 trained in the learning unit 210. In other words, the reference image generation unit 230 generates the reference image RI by applying the rendering model M10, which is a converter that performs conversion processing and is trained in the learning unit 210, to the design data D10.

[0092] The evaluation unit 240 evaluates the object 300, such as the EUV mask 310, based on a comparison between the reference image RI and the captured image CI.

[0093] The learning memory unit 250 may store training data used for learning in the learning unit 210. The learning memory unit 250 may also store coefficients of the rendering model M10 that is learned in the learning unit 210.

[0094] <Information Processing Methods> Next, the information processing method performed by the information processing device 200 in the inspection apparatus according to this embodiment will be described. Figure 15 is a flowchart illustrating an information processing method using the information processing device 200 according to Embodiment 1. As shown in Figure 15, the information processing method of this embodiment includes a step S10 to train a model, a step S20 to acquire an image CI of the object 300, a step S30 to generate a reference image RI based on the design data D10 and rendering model M10 of the object 300, and a step S40 to evaluate the object 300 based on a comparison of the reference image RI and the image CI.

[0095] In step S10, the learning unit 210 trains the rendering model M10. Specifically, the learning unit 210 uses the estimated image PI and the captured image CI of the object 300 as training images TI to train the rendering model M10. The learning unit 210 continues to train the rendering model M10 until the determination unit 213 determines that the training of the rendering model M10 is complete.

[0096] In step S20, the image acquisition unit 220 acquires, for example, an image CI of the object 300 captured by the imaging device 100. The image acquisition unit 220 may also acquire an image CI stored in a storage medium such as a storage device STR.

[0097] In step S30, the reference image generation unit 230 generates a reference image RI based on the design data D10 of the object 300 and the trained rendering model M10. Step S30 may be performed before step S20.

[0098] In step S40, the evaluation unit 240 compares the reference image RI and the captured image CI, and evaluates defects and other issues contained in the object 300 based on the difference between the two.

[0099] <Learning Method> Next, a learning method using the learning unit 210 of this embodiment will be described. Figure 16 is a flowchart illustrating a learning method using the learning unit 210 in the information processing device 200 according to Embodiment 1. As shown in Figure 16, the learning method of this embodiment includes a step S11 of outputting an estimated image PI, a step S12 of calculating the residual LS of the difference image DI between the estimated image PI and the training image TI, a step S13 of determining whether the residual LS satisfies predetermined conditions, and a step S14 of training a rendering model M10 using the estimated image PI and the training image TI.

[0100] In step S11, the estimated image generation unit 211 generates an estimated image PI by inputting the design data D10 of the object 300 into the rendering model M10 in the training process.

[0101] In step S12, the residual calculation unit 212 calculates the residual by comparing the estimated image PI output from the training process rendering model M10, which has the design data D10 of the object 300 as input, with the training image TI, which includes the captured image CI of the object 300. Specifically, the residual calculation unit 212 calculates the residual based on the difference image DI, which is the difference between the information of each pixel in the estimated image PI and the information of each pixel in the training image TI.

[0102] In step S13, the determination unit 213 determines whether the residuals satisfy predetermined conditions. If the residuals satisfy the predetermined conditions (Yes) in step S13, the determination unit 213 determines that the training of the rendering model M10 is complete. In this case, the process ends. On the other hand, if the residuals do not satisfy the predetermined conditions (No) in step S13, the determination unit 213 determines that the training of the rendering model M10 is not complete. In this case, the process proceeds to step S14.

[0103] In step S14, the training unit 214 trains the rendering model M10 using the estimated image PI and the training image TI. Then, it returns to step S11 and performs steps S11 to S13.

[0104] Next, the residual calculation method performed by the residual calculation unit 212 will be described. Figure 17 is a flowchart illustrating the residual calculation method performed by the residual calculation unit 212 in the inspection apparatus 1 according to Embodiment 1. As shown in Figure 17, the residual calculation method performed by the residual calculation unit 212 includes a step S21 of calculating the evaluation value of each pixel of the difference image DI, a step S22 of calculating a corrected evaluation value by applying weighting, and a step S23 of calculating the residual LS of the difference image DI using the corrected evaluation value.

[0105] In step S21, the residual calculation unit 212 calculates a corrected evaluation value by assigning different weights to the evaluation value of the first pixel of the difference image DI corresponding to a pixel whose information, based on at least one of the pixel information of the estimated image PI and the pixel information of the training image TI, shows a first brightness, and to the evaluation value of the second pixel of the difference image DI corresponding to a pixel whose information, based on at least one of the pixel information of the estimated image PI and the pixel information of the training image TI, shows a second brightness. The residual calculation unit 212 may assign a greater weight to the evaluation value of the second pixel than to the evaluation value of the first pixel.

[0106] The residual calculation unit 212 may obtain statistical values ​​for multiple pixels of the captured image CIs, indicating the degree of variation in pixel information among multiple captured image CIs, based on a comparison of multiple captured image CIs for substantially identical regions of the object 300. The residual calculation unit 212 then calculates the weighting of the evaluation value of the difference image based on the obtained statistical values. Specifically, as an example, the residual calculation unit 212 may obtain statistical values ​​for the evaluation value of multiple pixels in a difference image obtained by pre-calculating the information of each pixel in multiple captured image CIs. The residual calculation unit 212 then calculates the weighting of the evaluation value of the difference image DI based on the obtained statistical values. Alternatively, the residual calculation unit 212 may obtain statistical values ​​for each luminance of the luminance difference of multiple pixels in a difference image obtained by pre-calculating the luminance of each pixel in multiple captured image CIs. The residual calculation unit 212 then calculates the weighting of the luminance difference for each luminance in the difference image DI based on the obtained statistical values. The residual calculation unit 212 may also obtain statistical values ​​from a processing unit that performs various statistical processing to calculate statistical values.

[0107] In step S23, the residual calculation unit 212 calculates the residual of the difference image DI by performing arithmetic operations on the corrected evaluation value. The residual calculation unit 212 calculates the residual of the difference image DI by performing arithmetic operations on at least one of the average value of the corrected evaluation values ​​of multiple pixels and the maximum value of the corrected evaluation values ​​of multiple pixels. The residual calculation unit 212 may calculate and obtain the residual of the difference image DI based on the corrected evaluation value. The residual calculation unit 212 may use the corrected evaluation value as the residual of the difference image DI.

[0108] <Testing Method> Next, the inspection method according to Embodiment 1 will be described. Figure 18 is a flowchart illustrating the inspection method according to Embodiment 1. As shown in Figure 18, the inspection method of this embodiment comprises a step S100 of imaging the object 300 and a step S200 of performing information processing using the information processing method described above.

[0109] Next, the effects of this embodiment will be explained. The inspection device 1 of this embodiment determines that the training of the rendering model M10 is complete when the residual of the difference image DI satisfies predetermined conditions. At that time, the residual calculation unit 212 calculates the residual LS by assigning different weights to the evaluation value of the first pixel of the difference image DI corresponding to the pixel whose sample pixel information shows a first brightness, and to the evaluation value of the second pixel of the difference image DI corresponding to the pixel whose sample pixel information shows a second brightness. Therefore, the evaluation value of the first pixel, which has a large sample pixel information (brightness) and is prone to noise, can be underestimated. On the other hand, the evaluation value of the second pixel, which has a small sample pixel information (brightness) and is less prone to noise, can be overestimated. This makes it possible to equalize the degree of training in areas that are less susceptible to noise and areas that are prone to noise. Therefore, it is possible to train the rendering model M10 with the effects of noise suppressed. In this way, the information processing device 200 can improve its ability to handle areas prone to noise.

[0110] The residual calculation unit 212 calculates weights for statistical values ​​indicating the degree of variation in pixel information among multiple captured image CIs, based on a comparison of multiple captured image CIs for substantially identical regions of the object 300. Specifically, for example, the residual calculation unit 212 calculates weights for statistical values ​​of evaluation values ​​for multiple pixels in a differential captured image obtained by pre-difference-based imaging of the information of each pixel in multiple captured image CIs. This improves the accuracy of the corrected evaluation value, thereby improving the accuracy of the reference image RI output from the rendering model M10. Furthermore, the residual calculation unit 212 calculates weights for the luminance difference for each luminance in the differential image DI, based on statistical values ​​for each luminance of the luminance difference of multiple pixels in the differential captured image. Therefore, the accuracy of the corrected evaluation value can be further improved, thus further improving the accuracy of the reference image RI output from the rendering model M10.

[0111] <Variation> Figure 19 is a block diagram illustrating the configuration of an information processing device 201 according to a modified example of Embodiment 1. As shown in Figure 19, the information processing device 201 of this modified example further includes a processing unit 270 compared to the information processing device 200 of Embodiment 1 described above. The processing unit 270 performs various statistical processing to calculate statistical values. For example, the processing unit 270 may calculate statistical values ​​of evaluation values ​​for multiple pixels in a differential image obtained by pre-difference the information of each pixel of a plurality of captured image CIs. The processing unit 270 may also calculate F(x) and the standard deviation σ(x) for the differential image based on the statistical values ​​of evaluation values ​​for multiple pixels in the differential image obtained by pre-difference the brightness of each pixel of a plurality of captured image CIs. Furthermore, the processing unit 270 may calculate statistical values ​​for each brightness of the brightness difference for multiple pixels in a differential image obtained by pre-difference the brightness of each pixel of a plurality of captured image CIs. The processing unit 270 outputs the calculation results to the residual calculation unit 212. As a result, the residual calculation unit 212 can obtain various statistical values, etc.

[0112] In this modified version, the processing unit 270 calculates statistical values, etc. Therefore, the residual calculation unit 212 does not need to calculate statistical values, etc., thus reducing the processing load on the residual calculation unit 212. This improves the performance of the information processing device 201.

[0113] <Embodiment 2> Next, Embodiment 2 will be described. In the previously described embodiment, the residual calculation unit 212 calculates the weighting of the luminance differences for each luminance in the difference image DI based on statistical values ​​for each luminance of the luminance differences of multiple pixels in the difference image. In this embodiment, the residual calculation unit 212 obtains statistical values ​​for each pattern of the luminance differences of multiple pixels in the difference image. Then, the residual calculation unit 212 calculates the weighting of the luminance differences for each pattern in the difference image DI based on the obtained statistical values. The residual calculation unit 212 may obtain the statistical values ​​by calculating them, or it may obtain the statistical values ​​calculated by the processing unit 270.

[0114] Figure 20 is a graph illustrating the distribution of evaluation values ​​(difference values) of pixels in a difference image obtained by subtracting captured images in an information processing device 200 according to Embodiment 2. The horizontal axis represents the brightness of pixels in the captured image CI, and the vertical axis represents the evaluation value (difference value). Figure 20 shows the distribution of evaluation values ​​(difference values) for each pattern 311 of the object 300. As shown in Figure 20, the residual calculation unit 212 may acquire statistical values ​​of brightness differences for each pattern, such as pattern PN1, pattern PN2, and pattern PN3 shown in Figure 20. The residual calculation unit 212 may then calculate the weighting of the brightness differences for each pattern in the difference image D1 based on the acquired statistical values ​​of brightness differences for each pattern.

[0115] According to this embodiment, the residual calculation unit 212 calculates the weighting of the luminance difference for each pattern in the difference image DI based on statistical values ​​obtained for each pattern of luminance difference of multiple pixels in the difference image. Therefore, it is possible to improve the handling of patterns 311 corresponding to areas where noise is likely to occur. Other configurations and effects are included in the description of Embodiment 1.

[0116] <Embodiment 3> Next, a learning method according to Embodiment 3 will be described. Figure 21 is a flowchart illustrating a learning method using the learning unit 210 in the information processing device 201 according to Embodiment 3. As shown in Figure 21, the learning method of this embodiment further includes step S11a of classifying pixels into either the first group of pixels or the second group of pixels, compared to the learning method of Embodiment 1 shown in Figure 16.

[0117] In step S11a, the processing unit 270 classifies multiple pixels of the captured image CI into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured image CIs for substantially identical regions of the object 300. Here, the processing unit 270 defines the second group of pixels as those with less variation in pixel information among multiple captured image CIs for substantially identical regions of the object 300 than the first group of pixels. The processing unit 270 may further divide the degree of variation into three or more groups, such as a third group of pixels in addition to the first and second group of pixels.

[0118] Thus, in this embodiment, when comparing multiple captured image CIs, images are divided into those with large and small variations in pixel information (e.g., brightness) among the captured image CIs. The magnitude of the variation may change depending on the brightness. Also, the magnitude of the variation may change depending on the pattern 311. Therefore, the processing unit 270 may divide the images into a first group or a second group based on the magnitude of the variation which depends on the brightness, or it may divide the images into a first group or a second group based on the magnitude of the variation which depends on the pattern.

[0119] In step S12, the residual calculation unit 212 calculates the residual based on the difference image DI obtained by differentiating the information of each pixel in the estimated image PI with the information of each pixel in the training image TI. In this embodiment, the residual calculation unit 212 calculates a corrected evaluation value by assigning different weights to the evaluation value of the first pixel corresponding to the pixels of the first group of pixels in the difference image DI and the evaluation value of the second pixel corresponding to the pixels of the second group of pixels in the difference image DI. Here, the residual calculation unit 212 may calculate the corrected evaluation value by assigning weights according to the degree of variability. For example, the residual calculation unit 212 may assign weights using the variability calculated by the processing unit 270 based on a comparison of multiple captured images CI. That is, the residual calculation unit 212 may assign different weights according to the degree of variability by standardizing the evaluation value of the pixels in the difference image DI corresponding to the pixels at common positions of pixels in multiple captured images CI for substantially the same region of the object 300, based on the standard deviation σ or mean value of the brightness at those positions. Alternatively, the residual calculation unit 212 may perform different weighting according to the degree of variation by standardizing the evaluation value of the pixels in the difference image DI corresponding to the pixels belonging to a common pattern, based on the standard deviation σ or mean value of the brightness of pixels belonging to a common pattern of multiple captured images CI for substantially identical regions of the object 300. Other steps and configurations are included in the above-described embodiments 1 and 2 and modifications.

[0120] According to this embodiment, since the correction evaluation value is calculated according to the variation in pixel information, the accuracy of the correction evaluation value can be further improved.

[0121] <Embodiment 4> Next, a learning method according to Embodiment 4 will be described. In this embodiment, as in the previous step S11a, the processing unit 270 divides the pixels into either the first group of pixels or the second group of pixels.

[0122] On the other hand, in this embodiment, in step S12 for calculating residuals, the residual calculation unit 212 calculates a first residual for the first pixel corresponding to the first group of pixels in the difference image DI and a second residual for the second pixel corresponding to the second group of pixels in the difference image DI, based on the difference image DI obtained by subtracting the information of each pixel in the estimated image PI from the information of each pixel in the training image TI.

[0123] Then, in the determination step 13, the determination unit 213 sets a predetermined second condition applied to the second residual to be stricter than a predetermined first condition applied to the first residual, and determines that the model training is complete when the second residual satisfies the predetermined second condition and the first residual satisfies the predetermined first condition. Here, a stricter condition may include setting a threshold for the second residual to be smaller than the threshold used for the first residual. A stricter condition may also include applying a correction to the second residual so that the residual is overestimated, even though the threshold is the same. Furthermore, a stricter condition may also include applying a correction to the first residual so that the residual is underestimated. Other steps and configurations are included in the above-described embodiments 1 to 3 and modifications.

[0124] According to this embodiment, the completion of model training is determined by stringent conditions, thereby improving the accuracy of the model.

[0125] <Embodiment 5> Next, a learning method according to Embodiment 5 will be described. Figure 22 is a flowchart illustrating a learning method using the learning unit 210 in the information processing device 201 according to Embodiment 5. As shown in Figure 22, the learning method of this embodiment comprises a step S11a in which pixels are divided into either a first group of pixels or a second group of pixels, and a step S15 in which a model is trained.

[0126] In step S11a, the processing unit 270 divides multiple pixels of the captured image CI into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured image CIs for substantially identical regions of the object 300. Here, the processing unit 270 defines the second group of pixels as those for which the information variation of pixels among multiple captured image CIs for substantially identical regions of the object 300 is smaller than that of the first group of pixels.

[0127] In step S15, the training unit 214 trains a rendering model M10 that outputs a reference image RI to be compared with the captured image CI based on the design data D10, using the captured image CI of the object 300 and an image based on the design data D10 of the object 300 as training data. In step S15, the training data may include captured image CIs containing pixels of the first group and captured image CIs containing pixels of the second group. Also, the number of captured image CIs containing pixels of the first group may be greater than the number of captured image CIs containing pixels of the second group.

[0128] According to this embodiment, it is possible to learn more areas where residuals are likely to remain.

[0129] While embodiments of this disclosure have been described above, this disclosure includes appropriate modifications that do not impair its purpose and advantages, and is not limited by the embodiments described above. Furthermore, combinations of the configurations of Embodiments 1 and 2 are also within the scope of the technical concept of this disclosure. In addition, the following learning program for causing a computer to execute the learning method of the embodiments is also within the scope of the technical concept of this disclosure.

[0130] (Note 1) The process involves comparing the estimated image output from the training model with a training image that includes an image of the object, and calculating the residual. The step of determining that training of the model is complete when the residual satisfies predetermined conditions, Equipped with, In the step of calculating the residual, The residual is calculated based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image. A corrected evaluation value is calculated by applying different weights to the evaluation value of the first pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a first brightness, and to the evaluation value of the second pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a second brightness lower than the first brightness. The correction evaluation value is arithmetically processed to calculate the residual of the difference image. A learning program that instructs a computer to execute a command. (Note 2) In the step of calculating the residual, The weighting of the evaluation value of the second pixel is made greater than the weighting of the evaluation value of the first pixel. The learning program described in Appendix 1, which causes a computer to perform the following actions. (Note 3) In the step of calculating the residual, The residual of the difference image is calculated by performing an arithmetic operation on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels. The learning program described in Appendix 1, which causes a computer to perform the following actions. (Note 4) In the step of calculating the residual, Based on a comparison of multiple captured images of substantially identical regions of the object, statistical values ​​indicating the degree of variation in pixel information among the multiple captured images are obtained for multiple pixels of the captured images. Based on the acquired statistical values, the weighting of the evaluation value of the difference image is calculated. The learning program described in Appendix 1, which causes a computer to perform the following actions. (Note 5) The process involves comparing the estimated image output from the training model with the training image, which includes an image of the target object, and calculating the residual. The step of determining that training of the model is complete when the residual satisfies predetermined conditions, The steps include: comparing multiple captured images of substantially identical regions of the object, and classifying multiple pixels of the captured images into either a first group of pixels or a second group of pixels; Equipped with, The pixels of the second group are pixels that exhibit less variation in pixel information among multiple captured images of substantially the same region of the object than the pixels of the first group. In the step of calculating the residual, The residual is calculated based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image. A corrected evaluation value is calculated by applying different weights to the evaluation value of the first pixel corresponding to the pixels of the first group in the difference image and the evaluation value of the second pixel corresponding to the pixels of the second group in the difference image. The correction evaluation value is arithmetically processed to calculate the residual of the difference image. The learning program described in Appendix 1, which causes a computer to perform the following actions. (Note 6) The process involves comparing the estimated image output from the training model with the training image, which includes an image of the target object, and calculating the residual. The step of determining that training of the model is complete when the residual satisfies predetermined conditions, The steps include: dividing multiple pixels of the captured images into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured images of substantially identical regions of the object; Equipped with, The pixels of the second group are pixels that exhibit less variation in pixel information among multiple captured images of substantially the same region of the object than the pixels of the first group. In the step of calculating the residual, Based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image, a first residual for the first pixel corresponding to the first group of pixels in the difference image and a second residual for the second pixel corresponding to the second group of pixels in the difference image are calculated. In the step of determining that the training of the model is complete when the residual satisfies predetermined conditions, A predetermined second condition applied to the second residual is set to be stricter than a predetermined first condition applied to the first residual, and the training of the model is determined to be complete when the second residual satisfies the predetermined second condition and the first residual satisfies the predetermined first condition. The learning program described in Appendix 1, which causes a computer to perform the following actions. (Note 7) The steps include: dividing multiple pixels of a captured image into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured images of substantially identical regions of the object; The steps include training a model that outputs a reference image to be compared with the captured image based on the design data, using the captured image of the object and an image based on the design data of the object as training data, Equipped with, The pixels of the second group are pixels that exhibit less variation in pixel information among multiple captured images of substantially the same region of the object than the pixels of the first group. In the step of training the aforementioned model, The training data includes the captured image containing the first group of pixels and the captured image containing the second group of pixels, wherein the number of captured images containing the first group of pixels is greater than the number of captured images containing the second group of pixels. The learning program described in Appendix 1, which causes a computer to perform the following actions.

[0131] Furthermore, the learning program described above includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments when loaded into a computer. The learning program may be stored in a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The learning program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals. [Explanation of Symbols]

[0132] 1. Inspection device 100, 100a Imaging device 110, 110a illumination light source 120, 120a illumination optics 130 lens 130a Mirror 140 stages 150 lenses 160, 160a detection optics 170 detectors 200, 201 Information Processing Devices 210 Learning Department 211 Estimated Image Generation Unit 212 Residual Calculation Section 213 Judgment section 214 Training Department 220 Image acquisition unit 230 Reference Image Generation Unit 240 Evaluation Department 250 Learning memory unit 260 Control Unit 270 Processing Unit 300 objects 310 EUV Mask 311 patterns CI acquired images D10 Design Data DI difference image H1, H2, H3, H4 range KS coefficient LOS Loss Function L10 illumination light L20 transmitted light L30 illumination light L40 reflected light LS residual M10 Rendering Model MMR memory NW Network PI Estimated Image PRC Processor RI Reference Image STR storage TI Teacher Image UI User Interface VA, VB, VC, VD reference values

Claims

1. A learning unit that trains the model, An image acquisition unit that acquires an image of the target object, A reference image generation unit generates a reference image based on the design data and model of the object, An evaluation unit that evaluates the object based on a comparison between the reference image and the captured image, Equipped with, The aforementioned learning unit, A residual calculation unit calculates residuals by comparing the estimated image output from the model during the training process with the training image including the captured image. A determination unit that determines that training of the model is complete when the residual satisfies predetermined conditions, It has, The residual calculation unit is, The residual is calculated based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image. A corrected evaluation value is calculated by applying different weights to the evaluation value of the first pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a first brightness, and to the evaluation value of the second pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a second brightness lower than the first brightness. The correction evaluation value is arithmetically processed to calculate the residual of the difference image. Information processing device.

2. The residual calculation unit makes the weighting of the second pixel's evaluation value greater than the weighting of the first pixel's evaluation value. The information processing apparatus according to claim 1.

3. The residual calculation unit calculates the residual of the difference image by performing arithmetic operations on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels. The information processing apparatus according to claim 1.

4. The residual calculation unit is, Based on a comparison of multiple captured images of substantially identical regions of the object, statistical values ​​indicating the degree of variation in pixel information among the multiple captured images are obtained for multiple pixels of the captured images. Based on the acquired statistical values, the weighting of the evaluation value of the difference image is calculated. The information processing apparatus according to claim 1.

5. An imaging device for imaging the aforementioned object, An information processing device according to any one of claims 1 to 4, An inspection device equipped with the following features.

6. Steps to train the model, The steps include: acquiring an image of the target object, The steps include generating a reference image based on the design data and model of the object, A step of evaluating the object based on a comparison between the reference image and the captured image, Equipped with, The step of training the aforementioned model is: A step of calculating residuals by comparing the estimated image output from the model during the training process with the training image including the captured image, The step of determining that training of the model is complete when the residual satisfies predetermined conditions, Equipped with, In the step of calculating the residual, The residual is calculated based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image. A corrected evaluation value is calculated by applying different weights to the evaluation value of the first pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a first brightness, and to the evaluation value of the second pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a second brightness lower than the first brightness. The correction evaluation value is arithmetically processed to calculate the residual of the difference image. Information processing methods.

7. In the step of calculating the residual, The weighting of the evaluation value of the second pixel is made greater than the weighting of the evaluation value of the first pixel. The information processing method according to claim 6.

8. In the step of calculating the residual, The residual of the difference image is calculated by performing an arithmetic operation on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels. The information processing method according to claim 6.

9. In the step of calculating the residual, Based on a comparison of multiple captured images of substantially identical regions of the object, statistical values ​​indicating the degree of variation in pixel information among the multiple captured images are obtained for multiple pixels of the captured images. Based on the acquired statistical values, the weighting of the evaluation value of the difference image is calculated. The information processing method according to claim 6.

10. The steps include: imaging the target object, A step of performing information processing using the information processing method described in any one of claims 6 to 9, A testing method equipped with [a specific feature / feature].

11. The process involves comparing the estimated image output from the training model with the training image, which includes an image of the target object, and calculating the residual. The step of determining that training of the model is complete when the residual satisfies predetermined conditions, Equipped with, In the step of calculating the residual, The residual is calculated based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image. A corrected evaluation value is calculated by applying different weights to the evaluation value of the first pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a first brightness, and to the evaluation value of the second pixel of the difference image corresponding to a pixel whose pixel information, based on at least one of the pixel information of the estimated image and the pixel information of the training image, shows a second brightness lower than the first brightness. The correction evaluation value is arithmetically processed to calculate the residual of the difference image. Learning methods.

12. In the step of calculating the residual, The weighting of the evaluation value of the second pixel is made greater than the weighting of the evaluation value of the first pixel. The learning method according to claim 11.

13. In the step of calculating the residual, The residual of the difference image is calculated by performing an arithmetic operation on at least one of the average value of the correction evaluation values ​​of the plurality of pixels and the maximum value of the correction evaluation values ​​of the plurality of pixels. The learning method according to claim 11.

14. In the step of calculating the residual, Based on a comparison of multiple captured images of substantially identical regions of the object, statistical values ​​indicating the degree of variation in pixel information among the multiple captured images are obtained for multiple pixels of the captured images. Based on the acquired statistical values, the weighting of the evaluation value of the difference image is calculated. The learning method according to claim 11.

15. The process involves comparing the estimated image output from the training model with the training image, which includes an image of the target object, and calculating the residual. The step of determining that training of the model is complete when the residual satisfies predetermined conditions, The steps include: dividing multiple pixels of the captured images into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured images of substantially identical regions of the object; Equipped with, The pixels of the second group are pixels that exhibit less variation in pixel information among multiple captured images of substantially the same region of the object than the pixels of the first group. In the step of calculating the residual, The residual is calculated based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image. A corrected evaluation value is calculated by applying different weights to the evaluation value of the first pixel corresponding to the pixels of the first group in the difference image and the evaluation value of the second pixel corresponding to the pixels of the second group in the difference image. The correction evaluation value is arithmetically processed to calculate the residual of the difference image. Learning methods.

16. The process involves comparing the estimated image output from the training model with the training image, which includes an image of the target object, and calculating the residual. The step of determining that training of the model is complete when the residual satisfies predetermined conditions, The steps include: dividing multiple pixels of the captured images into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured images of substantially identical regions of the object; Equipped with, The pixels of the second group are pixels that exhibit less variation in pixel information among multiple captured images of substantially the same region of the object than the pixels of the first group. In the step of calculating the residual, Based on the difference image obtained by subtracting the information of each pixel in the estimated image from the information of each pixel in the training image, a first residual for the first pixel corresponding to the first group of pixels in the difference image and a second residual for the second pixel corresponding to the second group of pixels in the difference image are calculated. In the step of determining that the training of the model is complete when the residual satisfies predetermined conditions, A predetermined second condition applied to the second residual is set to be stricter than a predetermined first condition applied to the first residual, and the training of the model is determined to be complete when the second residual satisfies the predetermined second condition and the first residual satisfies the predetermined first condition. Learning methods.

17. The steps include: dividing multiple pixels of the captured images into either a first group of pixels or a second group of pixels, based on a comparison of multiple captured images of substantially identical regions of the object; The steps include training a model that outputs a reference image to be compared with the captured image based on the design data, using the captured image of the object and an image based on the design data of the object as training data, Equipped with, The pixels of the second group are pixels that exhibit less variation in pixel information among multiple captured images of substantially the same region of the object than the pixels of the first group. In the step of training the aforementioned model, The training data includes the captured image containing the first group of pixels and the captured image containing the second group of pixels, wherein the number of captured images containing the first group of pixels is greater than the number of captured images containing the second group of pixels. Learning methods.

Images