Image processing method, image processing apparatus, and learning method

Abstract

This invention provides a novel technology that can contribute to the realization of inspections using reference images that take into account the acquisition position of the image being compared. [Solution] The image processing method comprises the steps of: acquiring an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target taken by a detector having a predetermined imaging range; acquiring position data indicating the imaging position which is the position of the inspection image in the predetermined imaging range; generating a reference image based on the design information of the inspection target; and inspecting the inspection target area by comparing the inspection image and the reference image, wherein in the step of generating the reference image, different reference images are generated for areas that show a common structure in the design information by using the position data.

Description

Technical Field

[0001] The present disclosure relates to an image processing method, an image processing apparatus, and a learning method.

Background Art

[0002] For the inspection of an object such as a photomask manufactured based on design information, so-called die-to-database (DDB) inspection is known, in which an imaging image of the object is compared with a reference image generated from the design information of the object. In this regard, Patent Document 1 discloses a technique for generating a reference image using a machine learning model. ​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0006] An image processing method according to one aspect of the present disclosure comprises the steps of: acquiring an inspection image which is an image of an inspection target area of ​​an object to be inspected that is included in an image of the object to be inspected captured by a detector having a predetermined imaging range; acquiring position data indicating the position of the inspection image within the predetermined imaging range; generating a reference image based on design information of the object to be inspected; and inspecting the inspection target area by comparing the inspection image and the reference image, wherein in the step of generating the reference image, different reference images are generated for areas that show a common structure in the design information by using the position data.

[0007] In the image processing method described above, the step of generating the reference image may be performed by inputting the design image based on the design information and the positional data for the inspection image into a pre-trained machine learning model to generate the reference image.

[0008] In the image processing method described above, the machine learning model may be a model trained using training data which is a set of training images, which includes a training image included in an image of a training sample captured by the detector having a predetermined imaging range, a sample design image which is an image of the training sample drawn according to the design information of the training sample, and position data indicating the imaging position which is the position of the training image in the predetermined imaging range.

[0009] In the image processing method described above, in the step of generating the reference image, the design image based on the design information may be modified to a design image that reflects the position data by optical simulation using the position data of the inspection image, and the reference image may be generated by inputting the modified design image into a pre-trained machine learning model.

[0010] In the image processing method described above, the machine learning model may be a model trained using training data which is a pair of training images included in images of training samples captured by the detector having a predetermined imaging range, and sample design images which are images of the training samples drawn according to the design information of the training samples.

[0011] In the image processing method described above, in the step of generating the reference image, the reference image may be generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the position data for the inspection image.

[0012] In the image processing method described above, each of the plurality of machine learning models is a model trained using training data which is a pair of a training image included in an image of a training sample captured by the detector having a predetermined imaging range and a sample design image based on the design information of the training sample. The training data used for training may have different intervals for each machine learning model in which the imaging position, which is the position of the training image in the predetermined imaging range, belongs.

[0013] In the image processing method described above, the object to be imaged by the detector is illuminated by critical illumination, the detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction, and the position data may indicate the imaging position of the inspection image in the first direction.

[0014] In the image processing method described above, the position data used is a position image which is a partial image extracted from a gradient image having the same width as the image captured by the detector, and the relative position of the position image with respect to the gradient image may correspond to the imaging position of the inspection image.

[0015] In the image processing method described above, the detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction, and the gradient image may be an image gradient in the first direction.

[0016] In the image processing method described above, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing may be used as the position data, and the relative position of the subsequence with respect to the sequence may correspond to the imaging position of the inspection image.

[0017] In the image processing method described above, a subsequence of a sequence that is broadly monotonically increasing or broadly monotonically decreasing is used as the position data, the sequence contains a constant value continuously in the middle of the sequence, and the relative position of the subsequence with respect to the sequence may correspond to the imaging position of the inspection image.

[0018] An image processing apparatus according to one aspect of the present disclosure includes: an image acquisition unit that acquires an inspection image which is an image of an inspection target area of ​​an object to be inspected that is included in an image of the object to be inspected captured by a detector having a predetermined imaging range; a position data acquisition unit that acquires position data indicating the position of the inspection image in the predetermined imaging range; a reference image generation unit that generates a reference image based on design information of the object to be inspected; and an inspection unit that inspects the inspection target area by comparing the inspection image and the reference image, wherein the reference image generation unit generates different reference images for areas that show a common structure in the design information by using the position data.

[0019] In the image processing apparatus described above, the reference image generation unit may generate the reference image by inputting the design image based on the design information and the position data for the inspection image into a pre-trained machine learning model.

[0020] In the above-described image processing apparatus, the machine learning model may be a model learned using learning data that is a set of a learning image included in an image obtained by imaging a learning sample with the detector having the predetermined imaging range, a sample design image that is an image of the learning sample drawn according to the design information of the learning sample, and position data indicating an imaging position that is the position of the learning image in the predetermined imaging range.

[0021] In the above-described image processing apparatus, the reference image generation unit may generate the reference image by correcting a design image based on the design information to a design image in which the position data is reflected by optical simulation using the position data for the inspection image, and inputting the corrected design image into a machine learning model that has been learned in advance.

[0022] In the above-described image processing apparatus, the machine learning model may be a model learned using learning data that is a set of a learning image included in an image obtained by imaging a learning sample with the detector having the predetermined imaging range and a sample design image that is an image of the learning sample drawn according to the design information of the learning sample.

[0023] In the above-described image processing apparatus, the reference image generation unit may generate the reference image by inputting a design image based on the design information into a machine learning model selected based on the position data for the inspection image among a plurality of pre-learned machine learning models.

[0024] In the above-described image processing apparatus, each of the plurality of machine learning models is a model learned using learning data that is a set of a learning image included in an image obtained by imaging a learning sample with the detector having the predetermined imaging range and a sample design image based on the design information of the learning sample, and the imaging position belonging to the section where the imaging position of the learning image in the predetermined imaging range may be different for each of the machine learning models.

[0025] In the above image processing apparatus, the imaging target of the detector is illuminated by critical illumination, the detector is a TDI sensor having imaging elements arranged in a first direction and a second direction and integrating each charge from the plurality of imaging elements arranged in the second direction, and the position data may indicate the imaging position of the inspection image in the first direction.

[0026] In the above image processing apparatus, as the position data, a position image which is a partial image cut out from a gradation image having the same width as the image captured by the detector is used, and the relative position of the position image with respect to the gradation image may correspond to the imaging position of the inspection image.

[0027] In the above image processing apparatus, the detector is a TDI sensor having imaging elements arranged in a first direction and a second direction and integrating each charge from the plurality of imaging elements arranged in the second direction, and the gradation image may be an image gradated in the first direction.

[0028] In the above image processing apparatus, as the position data, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing is used, and the relative position of the subsequence with respect to the sequence may correspond to the imaging position of the inspection image.

[0029] In the above image processing apparatus, as the position data, a subsequence of a sequence that is monotonically increasing or monotonically decreasing in a broad sense is used, the sequence continuously includes a constant value at the center of the sequence, and the relative position of the subsequence with respect to the sequence may correspond to the imaging position of the inspection image.

[0030] A learning method according to one aspect of the present disclosure comprises the steps of: acquiring learning data which is a set of a learning image included in an image of a learning sample captured by a detector having a predetermined imaging range, a sample design image based on design information of the learning sample, and position data indicating the imaging position which is the position of the learning image in the predetermined imaging range; and generating a machine learning model which outputs a reference image by performing machine learning using the learning data, with a target design image and position data indicating the imaging position which is the position of an inspection image in the predetermined imaging range as input, wherein the target design image is an image of the inspection target area of ​​an object to be inspected drawn according to the design information of the object to be inspected, the inspection image is an image of the inspection target area of ​​the object to be inspected captured in an image of the object to be inspected by the detector, and the reference image is an image that is compared with the inspection image in order to inspect the inspection target area.

[0031] A learning method according to one aspect of the present disclosure comprises the steps of: acquiring at least: first learning data which is a pair of a first learning image included in a first section of an image of a learning sample captured by a detector having a predetermined imaging range and a sample design image based on design information of the learning sample; second learning data which is a pair of a second learning image included in a second section of the image of the learning sample captured by the detector and the sample design image; generating a first machine learning model which takes a target design image as input and outputs a first reference image by performing machine learning using the first learning data; and generating a second machine learning model which takes the target design image as input and outputs a second reference image by performing machine learning using the second learning data, wherein the target design image is an image of the area to be inspected of an object to be inspected drawn according to the design information of the object to be inspected; the first reference image and the second reference image are images which are compared with an inspection image in order to inspect the area to be inspected; and the inspection image is an image of the area to be inspected of an object to be inspected which is included in an image of the object to be inspected captured by the detector. [Effects of the Invention]

[0032] This disclosure provides a novel image processing method and the like that can contribute to realizing inspection using a reference image that takes into account the imaging position of the image being compared. [Brief explanation of the drawing]

[0033] [Figure 1] This is a schematic diagram showing the configuration of the inspection system according to the embodiment. [Figure 2] This is a block diagram showing an example of the configuration of an image processing apparatus according to an embodiment. [Figure 3] This is a schematic diagram illustrating the correspondence between inspection images and positional data. [Figure 4] This graph shows an example of pixel values ​​in a gradient image. [Figure 5] This is a schematic diagram illustrating the generation of a reference image by the reference image generation unit according to Embodiment 1. [Figure 6] This flowchart shows an example of the inspection operation flow in the image processing device according to the embodiment. [Figure 7] This is a schematic diagram illustrating the generation of a reference image by the reference image generation unit according to Embodiment 2. [Figure 8] This is a schematic diagram illustrating the generation of a reference image by the reference image generation unit according to Embodiment 3. [Figure 9] This is a schematic diagram showing an example of the division of the detector's imaging range. [Figure 10] This is a block diagram showing an example of the configuration of a computer that implements the processing of an image processing apparatus according to the embodiment. [Modes for carrying out the invention]

[0034] The specific configuration of this embodiment will be described below with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant explanations have been omitted as necessary for clarity of explanation. Furthermore, each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated not only with one specific embodiment but also with one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one of the drawings can be combined with features or steps shown in one or more other drawings to create embodiments that are not explicitly shown or described. Not all features or steps shown in any one of the drawings to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0035] <Embodiment 1> The inspection system according to Embodiment 1 will now be described. Figure 1 is a schematic diagram showing the configuration of the inspection system according to the embodiment. The inspection system 1 according to this embodiment includes an imaging device 100 and an image processing device 200, and is used for inspecting samples such as photomasks used in semiconductor manufacturing processes. As shown in Figure 1, the inspection system 1 is configured as a device that inspects a sample 90, which is the object to be inspected, by irradiating it with illumination light and imaging the object to be inspected.

[0036] In particular, in this embodiment, the inspection system 1 is used to perform die-to-database inspection. More specifically, in the inspection system 1, the inspection of the object to be inspected is performed by the image processing device 200 comparing a reference image generated by the image processing device 200 with an image of the object to be inspected captured by the imaging device 100. Here, the reference image is a good product image generated based on the design information of the object to be inspected.

[0037] The following will first describe the imaging device 100, and then the image processing device 200 in detail. The imaging device 100 may also be referred to as an optical device.

[0038] The sample 90 to be inspected by inspection system 1 is, for example, an EUV (Extreme Ultraviolet) mask, and the imaging device 100 irradiates the sample 90 with EUV light. The sample 90 is not limited to an EUV mask, but may also be various photomasks designed for light with wavelengths longer or shorter than EUV light, or various materials with fine patterns formed on them, such as semiconductor wafers with circuit patterns formed on them.

[0039] The imaging device 100 has an illumination optical system 10 and a detection optical system 20. The illumination optical system 10 has a light source 11, an ellipsoidal mirror 12, an ellipsoidal mirror 13, and a recessed mirror 14. The detection optical system 20 has a perforated concave mirror 21, a convex mirror 22, and a detector 23. The perforated concave mirror 21 and the convex mirror 22 constitute a Schwarzschild magnification optical system.

[0040] The light source 11 emits EUV light at 13.5 nm, the same wavelength as the exposure wavelength of the sample 90 which is an EUV mask, as illumination light L11. The illumination light L11 is not limited to EUV light and may be light of other wavelengths depending on the sample 90. The illumination light L11 emitted from the light source 11 is reflected by the ellipsoidal mirror 12. The illumination light L11 reflected by the ellipsoidal mirror 12 is focused at a focal point IF1 at a position conjugate to the upper surface 91 of the sample 90, and then spreads out as it enters a reflecting mirror such as the ellipsoidal mirror 13.

[0041] The illumination light L11 incident on the ellipsoidal mirror 13 is reflected by the ellipsoidal mirror 13. The illumination light L11 reflected by the ellipsoidal mirror 13 is focused and incident on the recessed mirror 14. In other words, the ellipsoidal mirror 13 causes the illumination light L11 to be incident on the recessed mirror 14 as focused light. The recessed mirror 14 is positioned directly above the sample 90. The illumination light L11 that is incident on the recessed mirror 14 and reflected is incident on the sample 90. That is, the recessed mirror 14 causes the illumination light L11 to be incident on the sample 90.

[0042] The ellipsoidal mirror 13 is designed and positioned to focus the illumination light L11 onto the sample 90. The illumination optical system 10 is installed so that when the illumination light L11 illuminates the sample 90, the image of the light source 11 (the image of a bright spot) is projected onto the upper surface 91 of the sample 90. Therefore, the illumination optical system 10 provides critical illumination. In this way, the illumination optical system 10 illuminates the object to be inspected (the object to be imaged) using critical illumination provided by the illumination light L11 generated by the light source 11.

[0043] The sample 90 is placed on the stage 92. Here, the plane parallel to the top surface 91 of the sample 90 is defined as the XY plane, and the direction perpendicular to the XY plane is defined as the Z direction. The illumination light L11 is incident on the sample 90 from a direction inclined with respect to the Z direction. That is, the illumination light L11 is incident at an oblique angle to illuminate the sample 90.

[0044] Stage 92 is an XYZ-driven stage. By moving Stage 92 in the XY direction, a desired area of ​​the sample 90 can be illuminated. Furthermore, by moving Stage 92 in the Z direction, focus adjustment can be performed. Stage 92 may be rotatable around at least one of the XYZ axes.

[0045] Illumination light L11 from light source 11 illuminates the inspection area of ​​sample 90. The inspection area illuminated by illumination light L11 is, for example, 0.5 mm square. Light, such as reflected light L12, generated from sample 90 based on an incident direction inclined with respect to the Z direction, is incident on the perforated concave mirror 21. A hole 21a is provided in the center of the perforated concave mirror 21. The light generated from sample 90 based on the incident illumination light L11 will hereafter be referred to as reflected light L12, but it may also be diffracted light, scattered light, fluorescence, etc.

[0046] The reflected light L12 reflected by the perforated concave mirror 21 is incident on the convex mirror 22. The convex mirror 22 reflects the reflected light L12 incident from the perforated concave mirror 21 toward the hole 21a of the perforated concave mirror 21. The reflected light L12 that has passed through the hole 21a is detected by the detector 23. The detector 23 is a detector that includes a TDI (Time Delay Integration) sensor and acquires image data of the sample 90 to be inspected. More specifically, the detector 23 is a TDI sensor that has image sensors arranged in a first direction and a second direction and integrates the charges from each of the multiple image sensors arranged in the second direction. Here, the first direction is, for example, the X direction, and the second direction is, for example, the Y direction. This TDI sensor integrates the charges of a row of multiple image sensors arranged in the second direction (i.e., multiple image sensors that are in the same position in the first direction) by transferring charge in the second direction (Y direction). This acquires one-dimensional image data for the first direction. The detector 23 acquires multiple one-dimensional image data by having multiple rows of image sensors arranged in the second direction in the first direction. Two-dimensional image data is generated by combining these multiple one-dimensional image data. The image sensor is, for example, a CCD (Charge Coupled Device), but is not limited to a CCD.

[0047] In this manner, the detection optical system 20 collects the reflected light L12 from the sample 90 illuminated by the illumination light L11, and the detector 23 detects the collected reflected light L12 to acquire image data of the sample 90. Multiple one-dimensional image data of the sample 90 acquired by the detector 23 are output to the image processing device 200 and processed as two-dimensional image data.

[0048] The image processing device 200 is connected to the detection optical system 20 by wire or wireless connection. The image processing device 200 receives two-dimensional image data consisting of multiple one-dimensional image data of the object to be inspected from the detector 23 in the detection optical system 20.

[0049] Incidentally, the inventors have found that the imaging result may vary depending on the position within the imaging range of the detector 23 at which imaging is performed. This is particularly noticeable in devices using a critical illumination optical system. Ideally, the intensity of the illumination light L11 should be uniform and constant within the imaging range of the detector 23 on the upper surface 91 of the sample 90. However, the intensity of the illumination light may be lower near both ends of the imaging range compared to other areas (near the center). For this reason, the imaging results near both ends of the imaging range of the detector 23 will differ from those outside the imaging range, even if the object being imaged is the same. Furthermore, the imaging results near the right end of the imaging range of the detector 23 will also differ from those near the left end. Therefore, in this embodiment, the influence of the imaging position is reduced by performing processing that focuses on the imaging position using the image processing device 200. Note that the imaging range of the detector 23 can also be said to be the imaging field of view of the detector 23. Also, the imaging position refers to the position within the imaging field of view of the detector 23 (field of view position). The image processing device 200 will be described below.

[0050] Figure 2 is a block diagram showing an example of the configuration of an image processing device 200. As shown in Figure 2, the image processing device 200 includes an image acquisition unit 201, a position data acquisition unit 202, a design image generation unit 203, a reference image generation unit 204, an inspection unit 205, a learning data acquisition unit 206, a model learning unit 207, a model storage unit 208, and a design information storage unit 209. In the example shown in Figure 2, the image processing device 200 includes components for generating a machine learning model used for inspecting an object to be inspected and components for utilizing the machine learning model, but the components for generating the machine learning model and the components for utilizing the machine learning model may belong to separate image processing devices. The image processing device may also be called an inspection device, etc. Furthermore, an image processing device that includes components for generating a machine learning model may be called a learning device.

[0051] The image acquisition unit 201 acquires an inspection image, which is an image of the inspection target area of ​​the object to be inspected. More specifically, the inspection image acquired by the image acquisition unit 201 is an image included in an image of the object to be inspected captured by the detector 23 having a predetermined imaging range. More specifically, the inspection image acquired by the image acquisition unit 201 can also be said to be a partial image extracted from an image of the object to be inspected captured by the detector 23 having a predetermined imaging range. In this disclosure, images other than inspection images (for example, training images, position images, etc., described later) are also described as partial images extracted from a specific image, but the images described in this way are just examples of images included in that specific image. In this embodiment, as described above, an inspection image is acquired by illuminating the inspection target area of ​​the object to be inspected with critical illumination and capturing the image. The inspection target area is, for example, a part of the surface of the object to be inspected. In this embodiment, the imaging range determined by the image sensors arranged in a line in the X direction of the detector 23 may correspond to the predetermined imaging range described above. In other words, in this embodiment, the imaging range may refer to the imaging range of the detector 23 in the X direction.

[0052] The position data acquisition unit 202 acquires position data indicating the imaging position of the inspection image acquired by the image acquisition unit 201. This position data indicates the imaging position of the inspection image within the predetermined imaging range described above. In other words, the position data indicates which position within the predetermined imaging range the acquired inspection image corresponds to. More specifically, the position data indicates the imaging position of the inspection image in the X direction. In this embodiment, as an example, the position data acquisition unit 202 acquires a gradient image having pixel values ​​representing the imaging position as position data. For this reason, in the following description, the position data will also be referred to as the position image.

[0053] Here, the inspection image and position data as an image described above will be explained with reference to the figure. Figure 3 is a schematic diagram showing the correspondence between the inspection image and position data (position image). Figure 3 also shows the detector 23, the two-dimensional image 901 obtained by imaging with the detector 23, and the gradient image 902, which is a two-dimensional image for obtaining the position image 902a. The two-dimensional image 901 and the gradient image 902 are two-dimensional images composed of pixels arranged in two orthogonal directions. As shown in Figure 3, the two-dimensional image 901 is obtained by scanning in the Y direction with the image sensor array 231 arranged in a line in the X direction of the detector 23. More specifically, as described above, since the detector 23 is a TDI sensor, multiple image sensor arrays 231 are arranged in the Y direction. The two-dimensional image 901 is an image consisting of M pixels (where M is a natural number) in the X direction and N pixels (where N is a natural number) in the Y direction, and is a two-dimensional image of the area to be inspected. The gradient image 902 is a two-dimensional image with a gradient in the X direction, consisting of M pixels in the X direction and N pixels in the Y direction. The group of pixels in the gradient image 902 arranged in the X direction corresponds to the imaging range of the detector 23 in the X direction. The pixel value of each pixel in the gradient image 902 depends on the position of that pixel in the X direction, but not on the position of that pixel in the Y direction. Therefore, the pixel value of the pixels in the gradient image 902 can indicate the position in the X direction.

[0054] The inspection image 901a acquired by the image acquisition unit 201 is, for example, a partial image extracted from the two-dimensional image 901, as shown in Figure 3, and consists of m pixels in the X direction (where m is a natural number) and n pixels in the Y direction (where n is a natural number). Here, m and n are the same as the size of the reference image generated by the reference image generation unit 204, which will be described later. Similarly, the position image 902a acquired by the position data acquisition unit 202 is, for example, a partial image extracted from the gradient image 902, as shown in Figure 3, and consists of m pixels in the X direction and n pixels in the Y direction. As shown in Figure 3, the relative position of the inspection image 901a with respect to the two-dimensional image 901 is the same as the relative position of the position image 902a with respect to the gradient image 902. In this embodiment, the image acquisition unit 201 and the position data acquisition unit 202 acquire such images, respectively. Therefore, the position image 902a represents the imaging position of the inspection image 901a. That is, the position image 902a represents the position within the X-direction imaging range of the detector 23 at which the inspection image 901a was captured. In the example shown in Figure 3, the gradient image 902 has the same size as the two-dimensional image 901 in both the X and Y directions, but the size in the Y direction may be different. For example, the size of the gradient image 902 in the Y direction only needs to be large enough to extract the position image 902a, and for example, the gradient image 902 may be an image consisting of n pixels in the Y direction. Thus, in this embodiment, the position data used is a position image, which is a partial image extracted from a gradient image that has the same width as the image captured by the detector 23. Furthermore, the relative position of the position image with respect to the gradient image corresponds to the relative position of the inspection image with respect to the original image from which the inspection image was extracted. In other words, the relative position of the position image with respect to the gradient image corresponds to the imaging position of the inspection image. Although the gradient image 902 described above is a gradient image only in the X direction, any image that has a gradient in at least the X direction is acceptable, and it may also have a gradient in the Y direction.

[0055] The position data (position image) acquired by the position data acquisition unit 202 is a gradient image in the first direction (X direction). That is, the position data is data indicating the imaging position in the first direction (X direction). The image acquisition unit 201 acquires the inspection image captured using the detector 23, as described above. As described above, the detector 23 is a TDI sensor that integrates the charges from multiple image sensors arranged in the second direction (Y direction). Therefore, in this embodiment, the position data acquisition unit 202 can acquire appropriate information to consider the difference in the relative position of the inspection image with respect to the source image from which the inspection image is extracted. This is because, as described above, in the second direction, the charge is integrated by the TDI sensor, so the effect of the difference in the relative position of the inspection image with respect to the source image is mitigated in the second direction. In contrast, such mitigation cannot be expected in the first direction. Therefore, when using a TDI sensor as the detector 23, it is preferable to acquire an image with a gradient in the first direction (X direction), that is, position data indicating the imaging position in the first direction (X direction), in order to acquire an appropriate reference image in the reference image generation unit 204 described later.

[0056] Here, a specific example of the gradient image 902 will be described. In this embodiment, as an example, the value of the pixel at the -X edge of the gradient image 902 is -1, and as the X coordinate of the pixel increases, the pixel value gradually changes from -1 to +1, and the value of the pixel at the +X edge of the gradient image 902 is +1. Figure 4 is a graph showing an example of the pixel values ​​of the gradient image 902. In the graph shown in Figure 4, the X coordinate of 0 represents, for example, the coordinate of the leftmost pixel of the gradient image 902, and corresponds to the left edge of the imaging range of the detector 23. M For example, this represents the coordinates of the rightmost pixel in the gradient image 902, and corresponds to the rightmost edge of the imaging range of the detector 23. Therefore, X MThe value of is specifically, for example, M. The pixel value of each pixel in the gradient image 902 may be represented by a monotonically increasing sequence in which the pixel value changes linearly according to the position in the X direction, as shown in graph Ga (solid line graph). However, it is not limited to this, and the pixel value of each pixel in the gradient image 902 may also be represented by a monotonically increasing sequence in which the pixel value changes nonlinearly according to the position in the X direction, as shown in graph Gb (dashed line cubic function graph). Note that in graphs Ga and Gb, the pixel value increases as the coordinate value in the X direction of the pixel increases, but the pixel value may also decrease as the coordinate value in the X direction of the pixel increases. In other words, the pixel value of each pixel in the gradient image 902 may be represented by a monotonically decreasing sequence in which the pixel value changes linearly or nonlinearly according to the position in the X direction.

[0057] Incidentally, graphs Ga and Gb represent a so-called strictly monotonically increasing sequence. Here, strictly monotonically increasing means that when the pixel value of a pixel with an X coordinate value of x1 is p1 and the pixel value of a pixel with an X coordinate value of x2 is p2, if x1 < x2, then p1 < p2 holds for the monotonically increasing case. Similarly, strictly monotonically decreasing means that if x1 < x2, then p1 > p2 holds for the monotonically decreasing case. In contrast, broadly monotonically increasing means that if x1 < x2, then p1 ≤ p2 holds for the monotonically increasing case. Also, broadly monotonically decreasing means that if x1 < x2, then p1 ≥ p2 holds for the monotonically decreasing case. The pixel value of each pixel in the gradation image 902 may be represented by a strictly monotonically increasing or strictly monotonically decreasing sequence, or may be represented by a broadly monotonically increasing or broadly monotonically decreasing sequence. For example, the pixel value of each pixel in the gradation image 902 may be represented by a broadly monotonically increasing sequence like graph Gc (the dashed graph). In graph Gc, the pixel values of pixels whose positions in the X direction are near the center of the whole are constant, and the pixel values of pixels whose positions in the X direction are outside the vicinity of the center increase monotonically according to the positions in the X direction. As described above, the images captured near both ends of the imaging range of the detector 23 are greatly affected by the imaging position on the imaging result. In other words, it is not necessarily important where the image is captured near the center of the imaging range of the detector 23. For this reason, like graph Gc, the pixel values of pixels near the center among the pixel groups arranged in the X direction may be constant. Note that in graph Gc, the pixel values of pixels other than those near the center among the pixel groups arranged in the X direction increase linearly, but they may increase non-linearly. Also, although graph Gc shows a graph of a broadly monotonically increasing sequence, the pixel value of each pixel in the gradation image 902 may be represented by a sequence in which the pixel value broadly monotonically decreases according to the position in the X direction.

[0058] As can be seen from the above explanation, it is not always necessary to use an image as position data; a subsequence of a strictly monotonically increasing or strictly monotonically decreasing sequence (see graph Ga or Gb) may be used. The relative position of this subsequence corresponds to the relative position of the examination image with respect to the source image from which the examination image was extracted (especially the relative position in the X direction), i.e., the imaging position of the examination image. Alternatively, a subsequence of a sequence (see graph Gc) that contains a constant value continuously in the middle and is broadly monotonically increasing or broadly monotonically decreasing may be used as position data. In this case as well, the relative position of the subsequence corresponds to the relative position of the examination image with respect to the source image from which the examination image was extracted (especially the relative position in the X direction), i.e., the imaging position of the examination image. Alternatively, the X-direction coordinate values ​​representing the imaging position may be used as position data.

[0059] The design image generation unit 203 generates a design image, which is an image drawn according to the design information of the object to be inspected. More specifically, the design image generation unit 203 generates a design image for the inspection target area of ​​the object to be inspected (particularly the area corresponding to the inspection image 901a). Specifically, the design image generation unit 203 generates, for example, an m × n pixel design image according to the design information of the object to be inspected stored in the design information storage unit 209. The design information storage unit 209 stores design information for any sample, including the object to be inspected. The design information may be, for example, vector data indicating a pattern formed on the sample. For example, the design image generation unit 203 performs rasterization based on the design information to generate a binary image. Then, the design image generation unit 203 pixels this binary image to generate a grayscale image having a predetermined number of gradations. This grayscale image is the design image. In this embodiment, the design image generation unit 203 generates a grayscale image obtained by pixelating a binary image as the design image, but it may also generate a binary image as the design image. Furthermore, if the design information storage unit 209 stores a design image in place of or together with the design information, the image processing device 200 does not need to have a design image generation unit 203. In other words, in this case, the image processing device 200 only needs to use the stored design image and does not need to generate a design image from the design information. Sometimes, the design information and the design image are not distinguished and are simply referred to as design information.

[0060] The reference image generation unit 204 generates a reference image from the design image. In this embodiment, the reference image generation unit 204 generates a reference image from the design image generated by the design image generation unit 203. However, as described above, if it is possible to obtain the design image without generating it, the reference image generation unit 204 does not necessarily have to use the design image generated by the design image generation unit 203. The reference image is an image that is compared with the inspection image in order to inspect the inspection target area of ​​the object to be inspected.

[0061] In particular, in this embodiment, the reference image generation unit 204 generates different reference images for a first inspection image and a second inspection image, which are inspection images with different imaging positions within a predetermined imaging range, by using the position data acquired by the position data acquisition unit 202. That is, even if the design images are the same, the reference image generation unit 204 generates different reference images if the imaging positions at the detector 23 are different. In other words, the reference image generation unit 204 generates different reference images for regions showing a common structure in the design information, if the imaging positions at the detector 23 are different.

[0062] Figure 5 is a schematic diagram showing the generation of a reference image by the reference image generation unit 204 according to this embodiment. Specifically, as shown in Figure 5, in this embodiment, the reference image generation unit 204 generates a reference image 913 by inputting a design image 911 and position data 912 (position image) for the inspection image into a pre-trained machine learning model 910. That is, the reference image generation unit 204 generates a reference image using a machine learning model that has been pre-trained to output a reference image by taking a design image and position data indicating the imaging position of the inspection image within a predetermined imaging range as input. This machine learning model 910 can also be described as a model that reflects the influence of the characteristics of the imaging device 100 or the characteristics of the manufacturing process of the object to be inspected (e.g., lithography process) on the image captured by the imaging device 100, and the influence of differences in imaging position on the image captured, on the image input to the model. The reference image generation unit 204 uses a machine learning model that has been pre-trained by the model learning unit 207. The model learning by the model learning unit 207 will be described later.

[0063] The inspection unit 205 inspects for abnormalities in the inspection area of ​​the object to be inspected by comparing the inspection image with a reference image. The inspection unit 205 compares the inspection image acquired by the image acquisition unit 201 with the reference image generated by the reference image generation unit 204. For example, the inspection unit 205 calculates the difference in gradation values ​​(luminance) between the reference image and the inspection image and compares the difference value with a threshold. The inspection unit 205 detects pattern abnormalities, defects, etc., based on the comparison result between the difference value and the threshold. That is, a location where a pattern abnormality occurs is, for example, a location where foreign matter is attached, and the difference value will be larger than the threshold. The inspection unit 205 outputs the inspection results. For example, the inspection unit 205 outputs inspection results indicating the presence or absence of abnormalities. The inspection unit 205 may also output the abnormal location and its position coordinates in association. The inspection unit 205 may display the inspection results on a display or transmit them to another device. The inspection unit 205 may also compare images using M × N pixels as shown in Figure 3. In this case, the image acquisition unit 201 sequentially extracts inspection images 901a from the M×N pixel two-dimensional image 901 (see Figure 3) until the entire area of ​​the two-dimensional image 901 is covered. The position data acquisition unit 202 also sequentially extracts position images 902a corresponding to the extracted inspection images 901a from the gradient image 902 (see Figure 3). Furthermore, the design image generation unit 203 generates a design image for each inspection image 901a. Then, the reference image generation unit 204 generates a reference image for each inspection image 901a. That is, the reference image generation unit 204 repeatedly generates an m×n pixel reference image corresponding to the inspection image 901a using the design image corresponding to the inspection image 901a and the position data (position image) corresponding to the inspection image 901a. Subsequently, the inspection unit 205 compares the M×N pixel two-dimensional image 901 with an M×N pixel reference image formed by concatenating multiple m×n pixel reference images.

[0064] Next, a flowchart illustrating the operation flow of the image processing device 200 described above is shown. Figure 6 is a flowchart of an example of the inspection operation flow in the image processing device 200. The operation flow for inspecting the object to be inspected will be explained below with reference to Figure 6.

[0065] In step S100, the image acquisition unit 201 acquires an inspection image of the object to be inspected. Next, in step S101, the position data acquisition unit 202 acquires position data (position image) indicating the imaging position of the inspection image acquired in step S100. Next, in step S102, the reference image generation unit 204 generates a reference image using the design image and the position data. Prior to this step, if necessary, the design image generation unit 203 generates a design image from the design information. After step S102, in step S103, the inspection unit 205 inspects the object to be inspected by comparing the inspection image and the reference image.

[0066] Next, the machine learning model used by the reference image generation unit 204 will be described. In this embodiment, a deep learning model is used as the machine learning model, as an example.

[0067] The learning data acquisition unit 206 acquires learning data to be used for machine learning of the model used by the reference image generation unit 204. The learning data acquisition unit 206 may acquire learning data input from other devices, or it may acquire learning data by reading learning data stored in a storage device such as the memory 502 of the image processing device 200, which will be described later. The learning data acquired by the learning data acquisition unit 206 consists of a set of data: a learning image, which is a partial image extracted from an image of a learning sample captured by a detector having a predetermined imaging range; a sample design image, which is an image of the learning sample drawn according to the design information of the learning sample; and position data indicating the imaging position of the learning image in the predetermined imaging range. The learning sample is, for example, a sample manufactured through the same manufacturing process as the object to be inspected. The learning sample may be a sample on which a pattern used only for learning has been formed (i.e., a sample on which the pattern formed is different from that of the object to be inspected), or the object to be inspected may be used as the learning sample.

[0068] The training images are captured by the detector 23. Therefore, the training data acquisition unit 206 may acquire the training images via the image acquisition unit 201. The training images are, like the inspection images 901a (see Figure 3), for example, m × n pixel two-dimensional images extracted from the two-dimensional image obtained by the detector 23.

[0069] Furthermore, in this embodiment, the training position data is a position image, and, similar to the position image 902a (see Figure 3), is, for example, an m x n pixel two-dimensional image extracted from a two-dimensional image (gradient image 902). Note that the relative position of the training image with respect to the source two-dimensional image is the same as the relative position of the training position image with respect to the source two-dimensional image (gradient image 902).

[0070] The sample design image is a design image for the region shown in the training image, and is an image generated from design information in the same way as the design image used during inspection. Specifically in this embodiment, the sample design image is a grayscale image obtained by pixelating a binary image generated by rasterizing based on the design information. Therefore, the training data acquisition unit 206 may acquire the sample design image via the design image generation unit 203. For this reason, the design information storage unit 209 may store the design information of the training sample.

[0071] The model learning unit 207 generates a machine learning model by performing machine learning using the learning data acquired by the learning data acquisition unit 206. Therefore, the model learning unit 207 generates a machine learning model by performing learning processing using learning data which is a set of learning images, which are partial images extracted from images of a learning sample captured by a detector having a predetermined imaging range; sample design images, which are images of the learning sample drawn according to the design information of the learning sample; and position data indicating the imaging position of the learning image within the predetermined imaging range. This machine learning model is the model used by the reference image generation unit 204 described above. That is, the machine learning model generated by the model learning unit 207 is a model that takes a design image of the object to be inspected and position data indicating the imaging position of the inspection image within the predetermined imaging range as input and outputs a reference image. The trained model generated by the machine learning processing of the model learning unit 207 is stored in the model storage unit 208. Then, the reference image generation unit 204 generates a reference image using the trained model stored in the model storage unit 208. In other words, the trained model generated by the model training unit 207 is used as a computer program module to make the computer function in order to generate a reference image.

[0072] Embodiment 1 has been described above. In this embodiment, a reference image is generated considering the imaging position of the inspection image. Therefore, a more appropriate reference image can be generated. The position data used in this embodiment only needs to indicate the imaging position of the image, and does not necessarily have to be a position image extracted from a gradient image.

[0073] <Embodiment 2> Next, Embodiment 2 will be described. This embodiment differs from Embodiment 1 in its method of generating a reference image using a design image and position data. The following describes the configuration or operation that differs from Embodiment 1, and explanations that overlap with Embodiment 1 will be omitted as appropriate.

[0074] Figure 7 is a schematic diagram showing the generation of a reference image by the reference image generation unit 204 according to Embodiment 2. As shown in Figure 7, in this embodiment, the reference image generation unit 204 modifies the design image 922 into a design image 924 that reflects the position data 923 by optical simulation 920 using position data 923 for the inspection image. The design image 922 is an image of the inspection target area drawn according to the design information of the object to be inspected, and is, for example, an image generated by the design image generation unit 203. After modifying the design image 922 into the design image 924, the reference image generation unit 204 generates a reference image 925 by inputting the modified design image 924 into a pre-trained machine learning model 921. The optical simulation 920 is a simulator (software) that simulates an image captured based on the design image 922, using the optical design of the imaging device 100 (shape or arrangement of mirrors and lenses, magnification of lenses, etc.) and position data as parameters. Known software can be used as the simulator to realize the optical simulation 920.

[0075] As described above, in this embodiment, an optical simulation 920 using positional data is performed, so the positional data is reflected in the modified design image 924. For this reason, the machine learning model 921 in this embodiment does not require positional data as input, unlike the machine learning model used in Embodiment 1. In other words, in this embodiment, the reference image generation unit 204 generates a reference image using a machine learning model that has been pre-trained to output a reference image with the design image as input.

[0076] The machine learning model 921 in this embodiment is a model trained using training data consisting of a set of training images described in Embodiment 1 and a sample design image described in Embodiment 1. The model learning unit 207 in this embodiment generates the machine learning model 921 using such training data. Thus, unlike Embodiment 1, this embodiment uses a machine learning model 921 that has been trained without considering positional data. This machine learning model 921 can be said to be a model that reflects the influence of the characteristics of the imaging device 100 or the characteristics of the manufacturing process of the object to be inspected (e.g., lithography process) on the image captured by the imaging device 100 to the image input to the model. As mentioned above, the influence of differences in imaging position on the image captured by the imaging device 100 is reflected in the design image 924 by optical simulation 920. In this embodiment as well, the reference image generation unit 204 generates different reference images if the imaging position at the detector 23 is different, even if the design images are the same.

[0077] Embodiment 2 has been described above. In this embodiment as well, a reference image is generated considering the imaging position of the inspection image. Therefore, a more appropriate reference image can be generated. In this embodiment as well, the position data only needs to indicate the imaging position of the image, and does not necessarily have to be a position image extracted from a gradient image.

[0078] <Embodiment 3> Next, Embodiment 3 will be described. This embodiment also differs from Embodiment 1 in its method for generating a reference image using design images and positional data. The following describes configurations or operations that differ from Embodiment 1, and explanations that overlap with Embodiment 1 will be omitted as appropriate.

[0079] Figure 8 is a schematic diagram showing the generation of a reference image by the reference image generation unit 204 according to Embodiment 3. As shown in Figure 8, in this embodiment, the reference image generation unit 204 generates a reference image 933 by inputting a design image 932 to a machine learning model selected from among a plurality of pre-trained machine learning models based on position data 931 for the inspection image. In this embodiment, as an example, the reference image generation unit 204 uses three machine learning models 930a, 930b, and 930c based on the position data 931. The selection of models based on the position data 931 will be explained in detail below with reference to Figure 9.

[0080] Figure 9 is a schematic diagram showing an example of the division of the imaging range of the detector 23. In Figure 9, the X coordinate 0 corresponds to, for example, the left edge of the imaging range of the detector 23, and the X coordinate X M For example, this corresponds to the right edge of the imaging range of detector 23. Therefore, X M The value of is specifically, for example, M. In this embodiment, the imaging range of the detector 23 is classified into three sections. In the example shown in Figure 9, for example, the first section 951a of the imaging range is a predetermined partial imaging range near the left edge of the imaging range of the detector 23, the second section 951b of the imaging range is a predetermined partial imaging range near the center of the imaging range of the detector 23, and the third section 951c of the imaging range is a predetermined partial imaging range near the right edge of the imaging range of the detector 23.

[0081] In this embodiment, the machine learning model 930a shown in Figure 8 is a model used when the imaging position indicated by the position data 931 used to generate the reference image belongs to the first interval 951a described above. Similarly, the machine learning model 930b is a model used when the imaging position indicated by the position data 931 used to generate the reference image belongs to the second interval 951b described above, and the machine learning model 930c is a model used when the imaging position indicated by the position data 931 used to generate the reference image belongs to the third interval 951c described above.

[0082] Machine learning model 930a is a model pre-trained using first training data, which is a pair of a first training image, which is a partial image cut out from a first interval of an image captured by a detector 23 having a predetermined imaging range, and a sample design image, which is an image of the training sample drawn according to the design information of the training sample. Here, the "partial image cut out from the first interval" is, for example, a partial image whose imaging position belongs to the first interval 951a within the predetermined imaging range. Machine learning model 930b is a model pre-trained using second training data, which is a pair of a second training image, which is a partial image cut out from a second interval of an image captured by a detector 23 having a predetermined imaging range, and a sample design image, which is an image of the training sample drawn according to the design information of the training sample. Here, the "partial image cut out from the second interval" is, for example, a partial image whose imaging position belongs to the second interval 951b within the predetermined imaging range. Similarly, the machine learning model 930c is a model pre-trained using third training data, which is a pair of a third training image, which is a partial image extracted from a third section of an image captured by a detector 23 having a predetermined imaging range, and a sample design image, which is an image of the training sample drawn according to the design information of the training sample. Here, the "partial image extracted from the third section" mentioned above is, for example, a partial image whose imaging position belongs to the third section 951c within the predetermined imaging range. For this reason, in this embodiment, the training data acquisition unit 206 acquires the first training data, the second training data, and the third training data. The model learning unit 207 then generates a machine learning model 930a, which takes the design image as input and outputs a first reference image, by performing machine learning using the first training data. Similarly, the model learning unit 207 generates a machine learning model 930b that takes a design image as input and outputs a second reference image by performing machine learning using the second training data, and generates a machine learning model 930c that takes a design image as input and outputs a third reference image by performing machine learning using the third training data.

[0083] Thus, in this embodiment, each of the multiple machine learning models is a model trained using training data consisting of a training image, which is a partial image extracted from an image of a training sample captured by a detector having a predetermined imaging range, and a sample design image, which is an image of the training sample drawn according to the design information of the training sample. However, the training data used for training differs for each machine learning model in the interval to which the imaging position of the training image within the predetermined imaging range belongs. In this embodiment, three models are used interchangeably, but the reference image generation unit 204 only needs to use at least two models interchangeably.

[0084] The reference image generation unit 204 generates a reference image using a machine learning model selected from the machine learning models generated in advance according to the position data (i.e., the imaging position). The reference image generation unit 204 determines which of the three intervals described above the position data 931 acquired by the position data acquisition unit 202 for generating the reference image belongs to. Then, the reference image generation unit 204 generates a reference image using the machine learning model from machine learning models 930a to 930c that corresponds to the determined interval. In this embodiment as well, even if the design image is the same, the reference image generation unit 204 generates different reference images if the imaging position at the detector 23 is different.

[0085] Embodiment 3 has been described above. In this embodiment as well, a reference image is generated considering the imaging position of the inspection image. Therefore, a more appropriate reference image can be generated. In this embodiment as well, the position data only needs to indicate the imaging position of the image, and does not necessarily have to be a position image extracted from a gradient image.

[0086] Although embodiments have been described above, the above-mentioned functions (processing) of the image processing device 200 may also be realized by a computer 500 having, for example, the following configuration.

[0087] Figure 10 is a block diagram showing an example configuration of a computer 500 that implements the processing of the image processing device 200. As shown in Figure 10, the computer 500 includes an input / output interface 501, memory 502, and a processor 503.

[0088] The input / output interface 501 is an interface for connecting to other devices (for example, the imaging device 100).

[0089] Memory 502 is composed of, for example, a combination of volatile memory and non-volatile memory. Memory 502 is used to store software (computer programs) containing one or more instructions executed by the processor 503, and data used for various processes. The model storage unit 208 and the design information storage unit 209 can be implemented by, for example, memory 502, but may also be implemented by any storage device other than memory 502.

[0090] The processor 503 performs the above-described processing of the image processing device 200 by reading and executing software (computer programs) from the memory 502. The processor 503 may be, for example, a microprocessor, an MPU (Micro Processor Unit), a CPU (Central Processing Unit), or a GPU (Graphics Processing Unit). The processor 503 may include multiple processors.

[0091] The program is included in the computer program product. The program also includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments when loaded into the computer. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals.

[0092] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention.

[0093] Furthermore, some or all of the above embodiments may also be described as follows, but are not limited to the following. (Note 1) A step of acquiring an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target taken with a detector having a predetermined imaging range, A step of acquiring position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range, A step of generating a reference image based on the design information of the object to be inspected, The process involves comparing the inspection image with the reference image to inspect the area to be inspected. Equipped with, In the process of generating the reference image, the position data is used to generate different reference images for regions that show a common structure in the design information. Image processing methods. (Note 2) In the process of generating the reference image, the design image based on the design information and the positional data for the inspection image are input to a pre-trained machine learning model to generate the reference image. The image processing method described in Appendix 1. (Note 3) The machine learning model is a model trained using training data which is a set of training images, which includes a training image contained in an image of a training sample captured by the detector having a predetermined imaging range, a sample design image which is an image of the training sample drawn according to the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. The image processing method described in Appendix 2. (Note 4) In the process of generating the aforementioned reference image, By performing an optical simulation using the positional data for the inspection image, the design image based on the design information is modified to a design image that reflects the positional data. The reference image is generated by inputting the modified design image into a pre-trained machine learning model. The image processing method described in Appendix 1. (Note 5) The machine learning model is a model trained using training data which consists of a training image included in an image of a training sample captured by the detector having a predetermined imaging range, and a sample design image which is an image of the training sample drawn according to the design information of the training sample. The image processing method described in Appendix 4. (Note 6) In the process of generating the reference image, the reference image is generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the position data for the inspection image. The image processing method described in Appendix 1. (Note 7) Each of the aforementioned plurality of machine learning models is a model trained using training data which is a pair of training images included in images captured by the detector having a predetermined imaging range and sample design images based on the design information of the training samples. The training data used for learning is such that the interval to which the imaging position, which is the position of the training image within the predetermined imaging range, belongs differs for each machine learning model. The image processing method described in Appendix 6. (Note 8) The object to be imaged by the aforementioned detector is illuminated by critical illumination. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The position data indicates the imaging position of the inspection image in the first direction. The image processing method described in any one of the appendices 1 to 7. (Note 9) As the position data, a position image is used, which is a partial image extracted from a gradient image having the same width as the image captured by the detector. The relative position of the position image with respect to the gradient image corresponds to the imaging position of the inspection image. The image processing method described in any one of the appendices 1 to 8. (Note 10) The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The aforementioned gradient image is an image that is gradient in the first direction. The image processing method described in Appendix 9. (Note 11) As the position data, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing is used. The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The image processing method described in any one of the appendices 1 to 10. (Note 12) As the position data, a subsequence of a sequence that is broadly monotonically increasing or broadly monotonically decreasing is used. The aforementioned sequence contains a constant value consecutively in the middle of the sequence, The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The image processing method described in any one of the appendices 1 to 10. (Note 13) An image acquisition unit acquires an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target taken by a detector having a predetermined imaging range, A position data acquisition unit acquires position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range. A reference image generation unit that generates a reference image based on the design information of the object to be inspected, By comparing the inspection image with the reference image, the inspection unit inspects the area to be inspected. Equipped with, The reference image generation unit uses the position data to generate different reference images for regions that exhibit a common structure in the design information. Image processing device. (Note 14) The reference image generation unit generates the reference image by inputting the design image based on the design information and the position data for the inspection image into a pre-trained machine learning model. The image processing apparatus described in Appendix 13. (Note 15) The machine learning model is a model trained using training data which is a set of training images, which includes a training image contained in an image of a training sample captured by the detector having a predetermined imaging range, a sample design image which is an image of the training sample drawn according to the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. The image processing apparatus described in Appendix 14. (Note 16) The aforementioned reference image generation unit, By performing an optical simulation using the positional data for the inspection image, the design image based on the design information is modified to a design image that reflects the positional data. The reference image is generated by inputting the modified design image into a pre-trained machine learning model. The image processing apparatus described in Appendix 13. (Note 17) The machine learning model is a model trained using training data which consists of a training image included in an image of a training sample captured by the detector having a predetermined imaging range, and a sample design image which is an image of the training sample drawn according to the design information of the training sample. The image processing apparatus described in Appendix 16. (Note 18) The reference image generation unit generates the reference image by inputting the design image based on the design information to a machine learning model selected from among a plurality of pre-trained machine learning models based on the position data for the inspection image. The image processing apparatus described in Appendix 13. (Note 19) Each of the aforementioned plurality of machine learning models is a model trained using training data which is a pair of training images included in images captured by the detector having a predetermined imaging range and sample design images based on the design information of the training samples. The training data used for learning is such that the interval to which the imaging position, which is the position of the training image within the predetermined imaging range, belongs differs for each machine learning model. The image processing apparatus described in Appendix 18. (Note 20) The object to be imaged by the aforementioned detector is illuminated by critical illumination. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The position data indicates the imaging position of the inspection image in the first direction. An image processing apparatus as described in any one of the appendices 13 to 19. (Note 21) As the position data, a position image is used, which is a partial image extracted from a gradient image having the same width as the image captured by the detector. The relative position of the position image with respect to the gradient image corresponds to the imaging position of the inspection image. An image processing apparatus as described in any one of the appendices 13 to 20. (Note 22) The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The aforementioned gradient image is an image that is gradient in the first direction. The image processing apparatus described in Appendix 21. (Note 23) As the position data, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing is used. The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. An image processing apparatus as described in any one of the appendices 13 to 22. (Note 24) As the position data, a subsequence of a sequence that is broadly monotonically increasing or broadly monotonically decreasing is used. The aforementioned sequence contains a constant value consecutively in the middle of the sequence, The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. An image processing apparatus as described in any one of the appendices 13 to 22. (Note 25) A step of acquiring training data which is a set of training images, which includes a training image contained in an image of a training sample captured by a detector having a predetermined imaging range, a sample design image based on the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. A machine learning model is generated by performing machine learning using the training data, which takes a target design image and position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range, as inputs and outputs a reference image. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The inspection image is an image of the inspection target area of ​​the inspection target, which is included in the image of the inspection target captured by the detector. The aforementioned reference image is an image that is compared with the inspection image in order to inspect the area to be inspected. Learning methods. (Note 26) First training data is a pair consisting of a first training image included in a first section of an image captured of a training sample with a detector having a predetermined imaging range, and a sample design image based on the design information of the training sample. The second training data is a pair of the second training image, which is included in the second section of the image captured by the detector of the training sample, and the sample design image. A process to obtain at least, The process involves generating a first machine learning model that takes a target design image as input and outputs a first reference image by performing machine learning using the first training data, and generating a second machine learning model that takes the target design image as input and outputs a second reference image by performing machine learning using the second training data. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The first and second reference images are images that are compared with the inspection image in order to inspect the area to be inspected. The inspection image is an image of the inspection target area of ​​the inspection target, which is included in the image of the inspection target captured by the detector. Learning methods. (Note 27) The steps include: acquiring an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target taken with a detector having a predetermined imaging range; A step of acquiring position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range, The steps include generating a reference image based on the design information of the object to be inspected, The step of inspecting the area to be inspected by comparing the inspection image with the reference image. Have the computer run it, In the step of generating the reference image, the position data is used to generate different reference images for regions that show a common structure in the design information. program. (Note 28) In the step of generating the reference image, the design image based on the design information and the positional data for the inspection image are input to a pre-trained machine learning model to generate the reference image. The program described in Appendix 27. (Note 29) The machine learning model is a model trained using training data which is a set of training images, which includes a training image contained in an image of a training sample captured by the detector having a predetermined imaging range, a sample design image which is an image of the training sample drawn according to the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. The program described in Appendix 28. (Note 30) In the step of generating the aforementioned reference image, By performing an optical simulation using the positional data for the inspection image, the design image based on the design information is modified to a design image that reflects the positional data. The reference image is generated by inputting the modified design image into a pre-trained machine learning model. The program described in Appendix 27. (Note 31) The machine learning model is a model trained using training data which consists of a training image included in an image of a training sample captured by the detector having a predetermined imaging range, and a sample design image which is an image of the training sample drawn according to the design information of the training sample. The program described in Appendix 30. (Note 32) In the step of generating the reference image, the reference image is generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the position data for the inspection image. The program described in Appendix 27. (Note 33) Each of the aforementioned plurality of machine learning models is a model trained using training data which is a pair of training images included in images captured by the detector having a predetermined imaging range and sample design images based on the design information of the training samples. The training data used for learning is such that the interval to which the imaging position, which is the position of the training image within the predetermined imaging range, belongs differs for each machine learning model. The program described in Appendix 32. (Note 34) The object to be imaged by the aforementioned detector is illuminated by critical illumination. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The position data indicates the imaging position of the inspection image in the first direction. The program described in any one of the items in Appendix 27 to 33. (Note 35) As the position data, a position image is used, which is a partial image extracted from a gradient image having the same width as the image captured by the detector. The relative position of the position image with respect to the gradient image corresponds to the imaging position of the inspection image. The program described in any one of the items in Appendix 27 to 34. (Note 36) The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The aforementioned gradient image is an image that is gradient in the first direction. The program described in Appendix 35. (Note 37) As the position data, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing is used. The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The program described in any one of the items in Appendix 27 to 36. (Note 38) As the position data, a subsequence of a sequence that is broadly monotonically increasing or broadly monotonically decreasing is used. The aforementioned sequence contains a constant value consecutively in the middle of the sequence, The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The program described in any one of the items in Appendix 27 to 36. (Note 39) A step of acquiring an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target taken with a detector having a predetermined imaging range, A step of acquiring position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range, A step of generating a reference image based on the design information of the object to be inspected, The process involves comparing the inspection image with the reference image to inspect the area to be inspected. Equipped with, In the step of generating the reference image, by using the position data, different reference images are generated for the first and second inspection images, which are inspection images whose imaging positions within the predetermined imaging range are different from each other. Image processing methods. [Explanation of symbols]

[0094] 1. Inspection System 10 Illumination optical system 11 Light source 12, 13 Ellipsoidal mirror 14 Mirror 20 detection optics 21 concave mirror 21a hole 22 Convex mirror 23 Detectors 90 samples 91 Top surface 92 stages 100 Imaging device 200 Image Processing Devices 201 Image Acquisition Unit 202 Location data acquisition unit 203 Design Image Generation Unit 204 Reference Image Generation Unit 205 Inspection Department 206 Training Data Acquisition Unit 207 Model Learning Department 208 Model Memory Unit 209 Design information storage section 231 Image sensor array 500 Computers 501 Input / Output Interface 502 memory 503 Processor 901 Two-dimensional image 901a Inspection image 902 Gradient Image 902a Location image 910 Machine Learning Models 911 design image 912 Location data 913 Reference Image 920 Optical Simulation 921 Machine Learning Models 922 Design Image 923 Location data 924 Design Images 925 Reference Image 930a, 930b, 930c Machine Learning Models 931 Location data 932 Design Image 933 Reference Image Sections 951a, 951b, and 951c Ga, Gb, Gc graph IF1 Focus point L11 Illumination Light L12 reflected light

Claims

1. A step of acquiring an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target taken with a detector having a predetermined imaging range, A step of acquiring position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range, A step of generating a reference image based on the design information of the object to be inspected, The process involves comparing the inspection image with the reference image to inspect the area to be inspected. Equipped with, In the process of generating the reference image, the position data is used to generate different reference images for regions that show a common structure in the design information. Image processing methods.

2. In the process of generating the reference image, the design image based on the design information and the positional data for the inspection image are input to a pre-trained machine learning model to generate the reference image. The image processing method according to claim 1.

3. The machine learning model is a model trained using training data which is a set of training images, which includes a training image contained in an image of a training sample captured by the detector having a predetermined imaging range, a sample design image which is an image of the training sample drawn according to the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. The image processing method according to claim 2.

4. In the process of generating the aforementioned reference image, By performing an optical simulation using the positional data for the inspection image, the design image based on the design information is modified to a design image that reflects the positional data. The modified design image is input into a pre-trained machine learning model to generate the reference image. The image processing method according to claim 1.

5. The machine learning model is a model trained using training data which consists of a training image included in an image of a training sample captured by the detector having a predetermined imaging range, and a sample design image which is an image of the training sample drawn according to the design information of the training sample. The image processing method according to claim 4.

6. In the process of generating the reference image, the design image based on the design information is input to a machine learning model selected from among a plurality of pre-trained machine learning models based on the position data for the inspection image, thereby generating the reference image. The image processing method according to claim 1.

7. Each of the aforementioned plurality of machine learning models is a model trained using training data which is a pair of training images included in images of training samples captured by the detector having a predetermined imaging range and sample design images based on the design information of the training samples. The training data used for learning is such that the interval to which the imaging position, which is the position of the training image within the predetermined imaging range, belongs differs for each machine learning model. The image processing method according to claim 6.

8. The object to be imaged by the aforementioned detector is illuminated by critical illumination. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The position data indicates the imaging position of the inspection image in the first direction. The image processing method according to any one of claims 1 to 7.

9. As the position data, a position image is used, which is a partial image extracted from a gradient image having the same width as the image captured by the detector. The relative position of the position image with respect to the gradient image corresponds to the imaging position of the inspection image. The image processing method according to claim 1.

10. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The aforementioned gradient image is an image that is gradient in the first direction. The image processing method according to claim 9.

11. As the position data, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing is used. The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The image processing method according to claim 1.

12. As the position data, a subsequence of a sequence that is broadly monotonically increasing or broadly monotonically decreasing is used. The aforementioned sequence contains a constant value consecutively in the middle of the sequence, The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The image processing method according to claim 1.

13. An image acquisition unit acquires an inspection image which is an image of the inspection target area of ​​the inspection target included in an image of the inspection target object captured by a detector having a predetermined imaging range, A position data acquisition unit acquires position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range. A reference image generation unit that generates a reference image based on the design information of the object to be inspected, By comparing the inspection image with the reference image, the inspection unit inspects the area to be inspected. Equipped with, The reference image generation unit uses the position data to generate different reference images for regions that exhibit a common structure in the design information. Image processing device.

14. The reference image generation unit generates the reference image by inputting the design image based on the design information and the position data for the inspection image into a pre-trained machine learning model. The image processing apparatus according to claim 13.

15. The machine learning model is a model trained using training data which is a set of training images, which includes a training image contained in an image of a training sample captured by the detector having a predetermined imaging range, a sample design image which is an image of the training sample drawn according to the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. The image processing apparatus according to claim 14.

16. The aforementioned reference image generation unit, By performing an optical simulation using the positional data for the inspection image, the design image based on the design information is modified to a design image that reflects the positional data. The modified design image is input into a pre-trained machine learning model to generate the reference image. The image processing apparatus according to claim 13.

17. The machine learning model is a model trained using training data which consists of a training image included in an image of a training sample captured by the detector having a predetermined imaging range, and a sample design image which is an image of the training sample drawn according to the design information of the training sample. The image processing apparatus according to claim 16.

18. The reference image generation unit generates the reference image by inputting the design image based on the design information to a machine learning model selected from among a plurality of pre-trained machine learning models based on the position data for the inspection image. The image processing apparatus according to claim 13.

19. Each of the aforementioned plurality of machine learning models is a model trained using training data which is a pair of training images included in images of training samples captured by the detector having a predetermined imaging range and sample design images based on the design information of the training samples. The training data used for learning is such that the interval to which the imaging position, which is the position of the training image within the predetermined imaging range, belongs differs for each machine learning model. The image processing apparatus according to claim 18.

20. The object to be imaged by the aforementioned detector is illuminated by critical illumination. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The position data indicates the imaging position of the inspection image in the first direction. The image processing apparatus according to any one of claims 13 to 19.

21. As the position data, a position image is used, which is a partial image extracted from a gradient image having the same width as the image captured by the detector. The relative position of the position image with respect to the gradient image corresponds to the imaging position of the inspection image. The image processing apparatus according to claim 13.

22. The detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The aforementioned gradient image is an image that is gradient in the first direction. The image processing apparatus according to claim 21.

23. As the position data, a subsequence of a sequence that is strictly monotonically increasing or strictly monotonically decreasing is used. The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The image processing apparatus according to claim 13.

24. As the position data, a subsequence of a sequence that is broadly monotonically increasing or broadly monotonically decreasing is used. The aforementioned sequence contains a constant value consecutively in the middle of the sequence, The relative position of the sub-sequence with respect to the sequence corresponds to the imaging position of the inspection image. The image processing apparatus according to claim 13.

25. A step of acquiring training data which is a set of training images, which includes a training image contained in an image of a training sample captured by a detector having a predetermined imaging range, a sample design image based on the design information of the training sample, and position data indicating the imaging position which is the position of the training image within the predetermined imaging range. A machine learning model is generated by performing machine learning using the training data, which takes a target design image and position data indicating the imaging position, which is the position of the inspection image within the predetermined imaging range, as inputs and outputs a reference image. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The inspection image is an image of the inspection target area of ​​the inspection target, which is included in the image of the inspection target captured by the detector. The aforementioned reference image is an image that is compared with the inspection image in order to inspect the area to be inspected. Learning methods.

26. First training data is a pair consisting of a first training image included in a first section of an image captured of a training sample with a detector having a predetermined imaging range, and a sample design image based on the design information of the training sample. The second training data is a pair of the second training image, which is included in the second section of the image captured by the detector of the training sample, and the sample design image. A process to obtain at least, The process involves generating a first machine learning model that takes a target design image as input and outputs a first reference image by performing machine learning using the first training data, and generating a second machine learning model that takes the target design image as input and outputs a second reference image by performing machine learning using the second training data. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The first reference image and the second reference image are images that are compared with the inspection image in order to inspect the area to be inspected. The inspection image is an image of the inspection target area of ​​the inspection target, which is included in the image of the inspection target captured by the detector. Learning methods.

Images