Image processing method, image processing apparatus, and learning method

Abstract

This invention provides a novel technology that can contribute to the realization of inspection using reference images that take into account bright spot fluctuations in devices employing critical illumination optics. [Solution] The image processing method comprises the steps of: acquiring an inspection image, which is an image taken by illuminating the inspection target area of ​​the object to be inspected with critical illumination; acquiring illumination profile information indicating the illumination profile at the time of imaging; generating a reference image based on the design information of the object to be inspected; and inspecting the inspection target area by comparing the inspection image and the reference image. In the step of generating the reference image, different reference images are generated for areas that show a common structure in the design information by using the illumination profile information.

Description

Technical Field

[0001] The present disclosure relates to an image processing method, an image processing apparatus, and a learning method.

Background Art

[0002] For inspecting an object such as a photomask manufactured based on design information, so-called die-to-database (DDB) inspection is known, in which an imaging image of the object is compared with a reference image generated from the design information of the object. In this regard, Patent Document 1 discloses a technique for generating a reference image using a model learned by associating a process variation amount with a learning imaging image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, there are various optical systems of the apparatus used for imaging an object, and there are also various parameters representing the state of the apparatus. Using all such parameters in the configuration of an image generation model is not preferable from the viewpoint of an increase in processing load or the like. Therefore, it is preferable to select parameters used for configuring an image generation model that can output a reference image considering the state of the apparatus according to the characteristics of the apparatus.

[0005] Here, the inventor has found that in an apparatus using a critical illumination optical system for imaging an object, the luminance distribution of the imaging image varies due to the variation in the position of bright spots. For this reason, a model capable of generating a reference image considering this is required.

[0006] This disclosure is made against the background of the circumstances described above and provides a novel image processing method and the like that can contribute to the realization of inspection using a reference image that takes into account bright spot fluctuations in a device using a critical illumination optical system. [Means for solving the problem]

[0007] An image processing method according to one aspect of the present disclosure includes the steps of: acquiring an inspection image, which is an image taken by illuminating an inspection target area of ​​an object to be inspected with critical illumination; acquiring illumination profile information indicating the illumination profile at the time of imaging; generating a reference image based on design information of the object to be inspected; and inspecting the inspection target area by comparing the inspection image and the reference image, wherein in the step of generating the reference image, different reference images are generated for areas that show a common structure in the design information by using the illumination profile information.

[0008] In the image processing method described above, the step of generating the reference image may be performed by inputting the design image based on the design information and the lighting profile information for the inspection image into a pre-trained machine learning model to generate the reference image.

[0009] In the image processing method described above, the machine learning model may be a model trained using training data which is a set of training images, which are images of training samples captured with critical illumination; sample design images, which are images of the training samples drawn according to the design information of the training samples; and illumination profile information, which indicates the illumination profile at the time the training images were captured.

[0010] In the image processing method described above, in the step of generating the reference image, the design image based on the design information may be modified to a design image that reflects the lighting profile information by performing an optical simulation using the lighting profile information for the inspection image, and the reference image may be generated by inputting the modified design image into a pre-trained machine learning model.

[0011] In the image processing method described above, the machine learning model may be a model trained using training data which consists of a training image, which is an image of a training sample captured with critical illumination, and a sample design image, which is an image of the training sample drawn according to the design information of the training sample.

[0012] In the image processing method described above, in the step of generating the reference image, the reference image may be generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information for the inspection image.

[0013] In the image processing method described above, each of the multiple machine learning models is a model trained using training data which is a pair of training images, which are images captured by illuminating a training sample with critical illumination, and sample design images based on the design information of the training sample. The characteristics of the illumination profile of the critical illumination used in the training data may differ for each machine learning model.

[0014] In the image processing method described above, the step of acquiring the inspection image involves acquiring the inspection image captured using a first detector, the first detector being a TDI sensor having image sensors arranged in a first direction and a second direction and integrating the charges from a plurality of image sensors arranged in the second direction, and the illumination profile information may indicate the luminance intensity distribution of the illumination light in the first direction.

[0015] In the image processing method described above, in the step of acquiring the inspection image, the inspection image captured using the first detector is acquired, and the illumination profile information may be an image captured by the second detector of an image obtained by imaging light from a light source.

[0016] In the image processing method described above, the illumination profile information may be an image captured by the second detector, which is obtained by imaging a portion of the illumination light reaching the object to be imaged by the first detector from the light source.

[0017] An image processing apparatus according to one aspect of the present disclosure includes an image acquisition unit that acquires an inspection image, which is an image taken by illuminating an inspection target area of ​​an object to be inspected with critical illumination; a profile acquisition unit that acquires illumination profile information indicating the illumination profile at the time of imaging; a reference image generation unit that generates a reference image based on design information of the object to be inspected; and an inspection unit that inspects the inspection target area by comparing the inspection image and the reference image, wherein the reference image generation unit generates different reference images for areas that show a common structure in the design information by using the illumination profile information.

[0018] In the image processing apparatus described above, the reference image generation unit may generate the reference image by inputting the design image based on the design information and the lighting profile information for the inspection image into a pre-trained machine learning model.

[0019] In the image processing apparatus described above, the machine learning model may be a model trained using training data which is a set of training images, which are images of a training sample captured with critical illumination; sample design images, which are images of the training sample drawn according to the design information of the training sample; and illumination profile information, which indicates the illumination profile at the time the training image was captured.

[0020] In the above-described image processing apparatus, the reference image generation unit may generate the reference image by correcting a design image based on the design information to a design image in which the illumination profile information is reflected by optical simulation using the illumination profile information for the inspection image, and inputting the corrected design image into a machine learning model that has been pre-learned.

[0021] In the above-described image processing apparatus, the machine learning model may be a model learned using learning data that is a pair of a learning image, which is an image obtained by illuminating a learning sample by critical illumination and imaging it, and a sample design image, which is an image of the learning sample drawn according to the design information of the learning sample.

[0022] In the above-described image processing apparatus, the reference image generation unit may generate the reference image by inputting a design image based on the design information into a machine learning model selected based on the illumination profile information for the inspection image among a plurality of pre-learned machine learning models.

[0023] In the above-described image processing apparatus, each of the plurality of machine learning models is a model learned using learning data that is a pair of a learning image, which is an image obtained by illuminating a learning sample by critical illumination and imaging it, and a sample design image based on the design information of the learning sample, and the learning data used for learning may have different characteristics of the illumination profile of the critical illumination for each of the machine learning models.

[0024] In the above-described image processing apparatus, the image acquisition unit acquires the inspection image captured using a first detector, the first detector is a TDI sensor having imaging elements arranged in a first direction and a second direction and integrating each charge from the plurality of imaging elements arranged in the second direction, and the illumination profile information may indicate the luminance intensity distribution of the illumination light in the first direction.

[0025] In the above-described image processing apparatus, the image acquisition unit acquires the inspection image captured using the first detector, and the illumination profile information may be an image captured by the second detector of an image obtained by imaging light from a light source.

[0026] In the above-described image processing apparatus, the illumination profile information may be an image captured by the second detector of an image obtained by imaging a part of the illumination light that reaches the imaging target of the first detector from the light source.

[0027] A learning method according to an aspect of the present disclosure includes a step of acquiring learning data, which is a set of a learning image that is an image of a learning sample illuminated and imaged by critical illumination, a sample design image based on the design information of the learning sample, and illumination profile information indicating the illumination profile at the time of imaging the learning image; and a step of generating a machine learning model that outputs a reference image by inputting a target design image and illumination profile information indicating the illumination profile at the time of imaging an inspection image. The target design image is an image of an inspection target area of the inspection target object drawn according to the design information of the inspection target object, the inspection image is an image of the inspection target area of the inspection target object illuminated and imaged by critical illumination, and the reference image is an image to be compared with the inspection image to inspect the inspection target area.

[0028] A learning method according to one aspect of the present disclosure includes at least the steps of acquiring first learning data, which is a set of a first learning image, an image captured by illuminating a learning sample with critical illumination, wherein the illumination profile of a region has a first feature, and a sample design image based on design information of the learning sample; and second learning data, which is a set of a second learning image, an image captured by illuminating the learning sample with critical illumination, wherein the illumination profile of a region has a second feature, and the sample design image; and a first machine learning model that takes a target design image as input and outputs a first reference image. The process includes generating a Dell by performing machine learning using the first training data, and generating a second machine learning model that takes the target design image as input and outputs a second reference image by performing machine learning using the second training data, wherein the target design image is an image of the area to be inspected of the object to be inspected drawn according to the design information of the object to be inspected, the first reference image and the second reference image are images that are compared with an inspection image in order to inspect the area to be inspected, and the inspection image is an image taken by illuminating the area to be inspected of the object to be inspected with critical illumination. [Effects of the Invention]

[0029] This disclosure provides a novel image processing method and the like that can contribute to realizing inspection using a reference image that takes into account bright spot fluctuations in a device using a critical illumination optical system. [Brief explanation of the drawing]

[0030] [Figure 1] This is a schematic diagram showing the configuration of the inspection system according to the embodiment. [Figure 2] This is a block diagram showing an example of the configuration of an image processing apparatus according to an embodiment. [Figure 3] This is a schematic diagram illustrating the correspondence between inspection images and lighting profile information. [Figure 4] This is a schematic diagram illustrating the generation of a reference image by the reference image generation unit according to Embodiment 1. [Figure 5] This flowchart shows an example of the inspection operation flow in the image processing device according to the embodiment. [Figure 6] This is a schematic diagram illustrating the generation of a reference image by the reference image generation unit according to Embodiment 2. [Figure 7] This is a schematic diagram illustrating the generation of a reference image by the reference image generation unit according to Embodiment 3. [Figure 8] This graph shows an example of the brightness intensity distribution obtained by the second detector. [Figure 9] This is a schematic diagram showing an example of the configuration of an imaging device according to a modified example. [Figure 10] This is a block diagram showing an example of the configuration of a computer that implements the processing of an image processing apparatus according to the embodiment. [Modes for carrying out the invention]

[0031] The specific configuration of this embodiment will be described below with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant explanations have been omitted as necessary for clarity of explanation. Furthermore, each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated not only with one specific embodiment but also with one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one of the drawings can be combined with features or steps shown in one or more other drawings to create embodiments that are not explicitly shown or described. Not all features or steps shown in any one of the drawings to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0032] <Embodiment 1> The inspection system according to Embodiment 1 will now be described. Figure 1 is a schematic diagram showing the configuration of the inspection system according to the embodiment. The inspection system 1 according to this embodiment includes an imaging device 100 and an image processing device 200, and is used for inspecting samples such as photomasks used in semiconductor manufacturing processes. As shown in Figure 1, the inspection system 1 is configured as a device that inspects a sample 90, which is the object to be inspected, by irradiating it with illumination light and imaging the object to be inspected.

[0033] In particular, in this embodiment, the inspection system 1 is used to perform die-to-database inspection. More specifically, in the inspection system 1, the inspection of the object to be inspected is performed by the image processing device 200 comparing a reference image generated by the image processing device 200 with an image of the object to be inspected captured by the imaging device 100. Here, the reference image is a good product image generated based on the design information of the object to be inspected.

[0034] The following will first describe the imaging device 100, and then the image processing device 200 in detail. The imaging device 100 may also be referred to as an optical device.

[0035] The sample 90 to be inspected by inspection system 1 is, for example, an EUV (Extreme Ultraviolet) mask, and the imaging device 100 irradiates the sample 90 with EUV light. The sample 90 is not limited to an EUV mask, but may also be various photomasks designed for light with wavelengths longer or shorter than EUV light, or various materials with fine patterns formed on them, such as semiconductor wafers with circuit patterns formed on them.

[0036] The imaging device 100 includes an illumination optical system 10, a detection optical system 20, and a monitor unit 30. The illumination optical system 10 includes a light source 11, an ellipsoidal mirror 12, an ellipsoidal mirror 13, and a recessed mirror 14. The detection optical system 20 includes a perforated concave mirror 21, a convex mirror 22, and a first detector 23. The perforated concave mirror 21 and the convex mirror 22 constitute a Schwarzschild magnification optical system. The monitor unit 30 includes a cut mirror 31, a concave mirror 32, and a second detector 33.

[0037] The light source 11 emits EUV light at 13.5 nm, the same wavelength as the exposure wavelength of the sample 90 which is an EUV mask, as illumination light L11. The illumination light L11 is not limited to EUV light and may be light of other wavelengths depending on the sample 90. The illumination light L11 emitted from the light source 11 is reflected by the ellipsoidal mirror 12. The illumination light L11 reflected by the ellipsoidal mirror 12 is focused at a focal point IF1 at a position conjugate to the upper surface 91 of the sample 90, and then spreads out as it enters a reflecting mirror such as the ellipsoidal mirror 13.

[0038] The illumination light L11 incident on the ellipsoidal mirror 13 is reflected by the ellipsoidal mirror 13. The illumination light L11 reflected by the ellipsoidal mirror 13 is focused and incident on the recessed mirror 14. In other words, the ellipsoidal mirror 13 causes the illumination light L11 to be incident on the recessed mirror 14 as focused light. The recessed mirror 14 is positioned directly above the sample 90. The illumination light L11 that is incident on the recessed mirror 14 and reflected is incident on the sample 90. That is, the recessed mirror 14 causes the illumination light L11 to be incident on the sample 90.

[0039] The ellipsoidal mirror 13 is designed and positioned to focus the illumination light L11 onto the sample 90. The illumination optical system 10 is installed so that when the illumination light L11 illuminates the sample 90, the image of the light source 11 (the image of a bright spot) is projected onto the upper surface 91 of the sample 90. Therefore, the illumination optical system 10 provides critical illumination. In this way, the illumination optical system 10 illuminates the object to be inspected using critical illumination provided by the illumination light L11 generated by the light source 11.

[0040] The sample 90 is placed on the stage 92. Here, the plane parallel to the top surface 91 of the sample 90 is defined as the XY plane, and the direction perpendicular to the XY plane is defined as the Z direction. The illumination light L11 is incident on the sample 90 from a direction inclined with respect to the Z direction. That is, the illumination light L11 is incident at an oblique angle to illuminate the sample 90.

[0041] Stage 92 is an XYZ-driven stage. By moving Stage 92 in the XY direction, a desired area of ​​the sample 90 can be illuminated. Furthermore, by moving Stage 92 in the Z direction, focus adjustment can be performed. Stage 92 may be rotatable around at least one of the XYZ axes.

[0042] Illumination light L11 from light source 11 illuminates the inspection area of ​​sample 90. The inspection area illuminated by illumination light L11 is, for example, 0.5 mm square. Light, such as reflected light L12, generated from sample 90 based on an incident direction inclined with respect to the Z direction, is incident on the perforated concave mirror 21. A hole 21a is provided in the center of the perforated concave mirror 21. The light generated from sample 90 based on the incident illumination light L11 will hereafter be referred to as reflected light L12, but it may also be diffracted light, scattered light, fluorescence, etc.

[0043] The reflected light L12 reflected by the perforated concave mirror 21 is incident on the convex mirror 22. The convex mirror 22 reflects the reflected light L12 incident from the perforated concave mirror 21 toward the hole 21a of the perforated concave mirror 21. The reflected light L12 that has passed through the hole 21a is detected by the first detector 23. The first detector 23 is a detector that includes a TDI (Time Delay Integration) sensor and acquires image data of the sample 90 to be inspected. More specifically, the first detector 23 is a TDI sensor that has image sensors arranged in a first direction and a second direction and integrates the charges from each of the multiple image sensors arranged in the second direction. Here, the first direction is, for example, the X direction, and the second direction is, for example, the Y direction. This TDI sensor integrates the charges of a row of multiple image sensors arranged in the second direction (i.e., multiple image sensors that are in the same position in the first direction) by transferring charge in the second direction (Y direction). This acquires one-dimensional image data for the first direction. The first detector 23 acquires multiple one-dimensional image data by having multiple rows of image sensors arranged in the second direction in the first direction. Two-dimensional image data is generated by combining these multiple one-dimensional image data. The image sensor is, for example, a CCD (Charge Coupled Device), but is not limited to a CCD.

[0044] In this manner, the detection optical system 20 collects the reflected light L12 from the sample 90 illuminated by the illumination light L11, and the first detector 23 detects the collected reflected light L12 to acquire image data of the sample 90. Multiple one-dimensional image data of the sample 90 acquired by the first detector 23 are output to the image processing device 200 and processed as two-dimensional image data.

[0045] As shown in Figure 1, the cut mirror 31 of the monitor unit 30 is positioned between the ellipsoidal mirror 13 and the recessed mirror 14, and extracts a portion of the illumination light L11 between the ellipsoidal mirror 13 and the recessed mirror 14. The cut mirror 31 reflects a portion of the illumination light L11 beam, cutting out a small portion of it. This portion of the beam is, for example, the upper part of the beam.

[0046] In the cross-sectional area of ​​the section perpendicular to the optical axis of the illumination light L11 at the position where the cut mirror 31 is placed, the cross-sectional area of ​​a portion reflected by the cut mirror 31 is smaller than the cross-sectional area of ​​the illumination light L11 other than that portion.

[0047] For example, if the cross-sectional area of ​​the section perpendicular to the optical axis of the illumination light L11 at the position where the cut mirror 31 is placed is 100, then some of the cross-sectional areas are approximately 1. The extraction angle of the illumination light L11 extracted from the light source 11 in the direction perpendicular to the optical axis is, for example, ±7°. The range used as illumination light L11 for the sample 90 is, for example, ±6°. For use in the monitor unit 30, the upper part of the illumination light L11 beam, for example, in the range of 1°, is extracted by the cut mirror 31. Even if the upper part of the beam is extracted in this way, the amount of illumination light L11 on the sample 90 does not decrease significantly.

[0048] The cut mirror 31 is positioned, for example, near the pupil in the illumination optical system 10. By extracting the illumination light L11 with the cut mirror 31 at a position near the pupil in the illumination optical system 10, a good correlation can be obtained between the image data acquired by the first detector 23 and the image data acquired by the second detector 33. Even if the numerical aperture (NA) for the first detector 23 and the NA for the second detector 33 are different, and the point spread function (PSF) is different, the difference in NA does not affect this embodiment because the plasma size is sufficiently large compared to the PSF size.

[0049] The illumination light L11 reflected by the cut mirror 31 is focused at a focal point, then spreads out as it enters the concave mirror 32.

[0050] Illumination light L11 that enters the concave mirror 32 and is reflected by the concave mirror 32 is detected by the second detector 33. The second detector 33 is a detector that includes a TDI sensor and acquires image data representing the intensity distribution of the illumination light L11. More specifically, the second detector 33, like the first detector 23, has image sensors arranged in a first direction and a second direction, and is a TDI sensor that integrates the charges from multiple image sensors arranged in the second direction. Here, the first direction is, for example, the X direction, and the second direction is, for example, the Y direction. This TDI sensor integrates the charges of multiple image sensors in a row arranged in the second direction (i.e., multiple image sensors that are in the same position in the first direction) by transferring charge in the second direction (Y direction). As a result, one-dimensional image data for the first direction is acquired. The second detector 33 acquires multiple one-dimensional image data by having multiple such rows of image sensors arranged in the second direction in the first direction. The one-dimensional image data acquired by the second detector 33 shows the intensity distribution of the illumination light L11. Two-dimensional image data is generated by combining multiple one-dimensional image data. The image sensor is, for example, a CCD, but is not limited to a CCD.

[0051] For example, the optical system of the monitor unit 30 may be arranged so that the image of the light source 11 of the illumination light L11 (the image of the bright spot) is imaged onto the second detector 33. In this case, the first detector 23 and the second detector 33 are conjugate. As a result, the monitor unit 30 can acquire image data showing the brightness intensity distribution of the illumination light L11 detected by illuminating the second detector 33 with a portion of the illumination light L11 using critical illumination. The monitor unit 30 outputs the acquired image data of the brightness intensity distribution of the illumination light L11 to the image processing device 200.

[0052] The image processing device 200 is connected to the detection optical system 20 and the monitor unit 30 by wire or wireless connection. The image processing device 200 receives two-dimensional image data consisting of multiple one-dimensional image data of the object to be inspected from the first detector 23 in the detection optical system 20. The image processing device 200 also receives two-dimensional image data consisting of multiple one-dimensional image data of the brightness intensity distribution of the illumination light L11 from the second detector 33 in the monitor unit 30.

[0053] Incidentally, the inventors have found that when a critical illumination optical system is used for imaging, fluctuations in the position of the bright spot of the light source have a particularly significant effect on the brightness distribution of the captured image. Ideally, the intensity of the illumination light L11 in the imaging range on the upper surface 91 of the sample 90 should be uniform and constant. However, in reality, the intensity of the illumination light L11 on the upper surface 91 of the sample 90 may differ depending on the imaging position or the time of imaging. Therefore, in this embodiment, the above-mentioned effects are reduced by performing processing that focuses on the illumination profile during imaging using the image processing device 200. The image processing device 200 will be described below.

[0054] Figure 2 is a block diagram showing an example of the configuration of an image processing device 200. As shown in Figure 2, the image processing device 200 includes an image acquisition unit 201, a profile acquisition unit 202, a design image generation unit 203, a reference image generation unit 204, an inspection unit 205, a training data acquisition unit 206, a model training unit 207, a model storage unit 208, and a design information storage unit 209. In the example shown in Figure 2, the image processing device 200 includes components for generating a machine learning model used for inspecting an object to be inspected and components for utilizing the machine learning model, but the components for generating the machine learning model and the components for utilizing the machine learning model may belong to separate image processing devices. The image processing device may also be called an inspection device, etc. Furthermore, an image processing device that includes components for generating a machine learning model may be called a training device.

[0055] The image acquisition unit 201 acquires an inspection image, which is an image taken by illuminating the object to be inspected with critical illumination. More specifically, the inspection image is an image of the inspection area of ​​the object to be inspected. The inspection area is, for example, a part of the surface of the object to be inspected. In this embodiment, as an example, the image acquisition unit 201 acquires a two-dimensional image of the object to be inspected from the first detector 23 as an inspection image. That is, the image acquisition unit 201 acquires an inspection image taken using the first detector 23.

[0056] The profile acquisition unit 202 acquires illumination profile information that shows the illumination profile of the object to be inspected during imaging. The illumination profile represents the state of illumination used when imaging the object to be imaged, and more specifically, it represents the distribution of brightness intensity on the surface of the object to be imaged. In this embodiment, as an example, the profile acquisition unit 202 acquires the two-dimensional image from the second detector 33 described above as illumination profile information. For this reason, in the following description, illumination profile information will also be referred to as a profile image.

[0057] Here, the inspection image and the illumination profile information as an image will be explained with reference to the figure. Figure 3 is a schematic diagram showing the correspondence between the inspection image and the illumination profile information (profile image). Figure 3 also shows the first detector 23, the two-dimensional image 901 obtained by imaging with the first detector 23, and the second detector 33, and the two-dimensional image 902 obtained by imaging with the second detector 33. As shown in Figure 3, the two-dimensional image 901 is obtained by scanning in the Y direction with the image sensor array 231 arranged in a line in the X direction of the first detector 23. Similarly, the two-dimensional image 902 is obtained by scanning in the Y direction with the image sensor array 331 arranged in a line in the X direction of the second detector 33. More specifically, as mentioned above, since the first detector 23 and the second detector 33 are TDI sensors, multiple image sensor arrays 231 or 331 are arranged in the Y direction. Two-dimensional image 901 is an image consisting of M pixels in the X direction (where M is a natural number) and N pixels in the Y direction (where N is a natural number), and is a two-dimensional image of the area to be inspected. Two-dimensional image 902 is an image consisting of M pixels in the X direction and N pixels in the Y direction, and is a two-dimensional image showing the illumination profile at the time of acquisition of two-dimensional image 901.

[0058] The inspection image 901a acquired by the image acquisition unit 201 is, for example, a partial image extracted from the two-dimensional image 901, as shown in Figure 3, and consists of m pixels in the X direction (where m is a natural number) and n pixels in the Y direction (where n is a natural number). Here, m and n are the same as the size of the reference image generated by the reference image generation unit 204, which will be described later. Similarly, the profile image 902a acquired by the profile acquisition unit 202 is, for example, a partial image extracted from the two-dimensional image 902, as shown in Figure 3, and consists of m pixels in the X direction and n pixels in the Y direction. As shown in Figure 3, the relative position of the inspection image 901a with respect to the two-dimensional image 901 is the same as the relative position of the profile image 902a with respect to the two-dimensional image 902. In this embodiment, the image acquisition unit 201 and the profile acquisition unit 202 acquire such images, respectively. In this embodiment, the inspection image 901a acquired by the image acquisition unit 201 is described as a partial image of the two-dimensional image 901, but the image acquisition unit 201 may acquire the two-dimensional image 901 as an inspection image. In this case, the profile acquisition unit 202 may acquire the two-dimensional image 902 as illumination profile information.

[0059] The illumination profile information (profile image) acquired by the profile acquisition unit 202 is illumination profile information that shows the luminance intensity distribution of the illumination light in the first direction (X direction). The image acquisition unit 201 acquires an inspection image captured using the first detector 23, as described above. As described above, the first detector 23 is a TDI sensor that integrates the charges from multiple image sensors arranged in the second direction (Y direction). Therefore, in this embodiment, the profile acquisition unit 202 can acquire appropriate information to take into account fluctuations in the illumination light. This is because, as described above, the charge is integrated by the TDI sensor in the second direction, so the effect of fluctuations in the illumination light is mitigated in the second direction. For example, even if the light source 11 fluctuates, the effect of the fluctuation of the light source 11 is mitigated in the second direction due to the integration of the charges of the image sensors arranged in the second direction. In contrast, such mitigation cannot be expected in the first direction. Therefore, when using a TDI sensor as the first detector 23, it is preferable to acquire illumination profile information showing the luminance intensity distribution of illumination light in the first direction (X direction) in order to acquire an appropriate reference image in the reference image generation unit 204 described later.

[0060] The design image generation unit 203 generates a design image, which is an image drawn according to the design information of the object to be inspected. More specifically, the design image generation unit 203 generates a design image for the inspection target area of ​​the object to be inspected (particularly the area corresponding to the inspection image 901a). Specifically, the design image generation unit 203 generates, for example, an m × n pixel design image according to the design information of the object to be inspected stored in the design information storage unit 209. The design information storage unit 209 stores design information for any sample, including the object to be inspected. The design information may be, for example, vector data indicating a pattern formed on the sample. For example, the design image generation unit 203 performs rasterization based on the design information to generate a binary image. Then, the design image generation unit 203 pixels this binary image to generate a grayscale image having a predetermined number of gradations. This grayscale image is the design image. In this embodiment, the design image generation unit 203 generates a grayscale image obtained by pixelating a binary image as the design image, but it may also generate a binary image as the design image. Furthermore, if the design information storage unit 209 stores a design image in place of or together with the design information, the image processing device 200 does not need to have a design image generation unit 203. In other words, in this case, the image processing device 200 only needs to use the stored design image and does not need to generate a design image from the design information. Sometimes, the design information and the design image are not distinguished and are simply referred to as design information.

[0061] The reference image generation unit 204 generates a reference image from the design image. In this embodiment, the reference image generation unit 204 generates a reference image from the design image generated by the design image generation unit 203. However, as described above, if it is possible to obtain the design image without generating it, the reference image generation unit 204 does not necessarily have to use the design image generated by the design image generation unit 203. The reference image is an image that is compared with the inspection image in order to inspect the inspection target area of ​​the object to be inspected.

[0062] In particular, in this embodiment, the reference image generation unit 204 generates different reference images for the first and second inspection images, which are inspection images with different illumination profiles at the time of imaging, by using the illumination profile information acquired by the profile acquisition unit 202. That is, the reference image generation unit 204 generates different reference images if the illumination profile information is different, even if the design images are the same. In other words, the reference image generation unit 204 generates different reference images if the illumination profile information is different for regions that show a common structure in the design information.

[0063] Figure 4 is a schematic diagram showing the generation of a reference image by the reference image generation unit 204 according to this embodiment. Specifically, as shown in Figure 4, in this embodiment, the reference image generation unit 204 generates a reference image 913 by inputting a design image 911 and lighting profile information 912 (profile image) for the inspection image into a pre-trained machine learning model 910. In other words, the reference image generation unit 204 generates a reference image using a machine learning model that has been pre-trained to output a reference image with a design image and lighting profile information as input. This machine learning model 910 can also be described as a model that reflects the influence of the characteristics of the imaging device 100 or the characteristics of the manufacturing process of the object to be inspected (e.g., lithography process) on the image captured by the imaging device 100, and the influence of differences in the lighting profile at the time of shooting on the image captured, onto the image input to the model. The reference image generation unit 204 uses a machine learning model that has been pre-trained by the model learning unit 207. The model learning by the model learning unit 207 will be described later.

[0064] The inspection unit 205 inspects for abnormalities in the inspection area of ​​the object to be inspected by comparing the inspection image with a reference image. The inspection unit 205 compares the inspection image acquired by the image acquisition unit 201 with the reference image generated by the reference image generation unit 204. For example, the inspection unit 205 calculates the difference in gradation values ​​(luminance) between the reference image and the inspection image and compares the difference value with a threshold. The inspection unit 205 detects pattern abnormalities, defects, etc., based on the comparison result between the difference value and the threshold. That is, a location where a pattern abnormality occurs is, for example, a location where foreign matter is attached, and the difference value will be larger than the threshold. The inspection unit 205 outputs the inspection results. For example, the inspection unit 205 outputs inspection results indicating the presence or absence of abnormalities. The inspection unit 205 may also output the abnormal location and its position coordinates in association. The inspection unit 205 may display the inspection results on a display or transmit them to another device. The inspection unit 205 may also compare images using M × N pixels as shown in Figure 3. In this case, the image acquisition unit 201 sequentially extracts inspection images 901a from the M×N pixel two-dimensional image 901 (see Figure 3) until the entire area of ​​the two-dimensional image 901 is covered. The profile acquisition unit 202 also sequentially extracts profile images 902a corresponding to the extracted inspection images 901a from the two-dimensional image 902 (see Figure 3). Furthermore, the design image generation unit 203 generates a design image for each inspection image 901a. Then, the reference image generation unit 204 generates a reference image for each inspection image 901a. That is, the reference image generation unit 204 repeatedly generates an m×n pixel reference image corresponding to the inspection image 901a using the design image corresponding to the inspection image 901a and the profile information (profile image) corresponding to the inspection image 901a. Subsequently, the inspection unit 205 compares the M×N pixel two-dimensional image 901 with an M×N pixel reference image formed by concatenating multiple m×n pixel reference images.

[0065] Next, a flowchart illustrating the operation flow of the image processing device 200 described above is shown. Figure 5 is a flowchart of an example of the inspection operation flow in the image processing device 200. The operation flow for inspecting the object to be inspected will be explained below with reference to Figure 5.

[0066] In step S100, the image acquisition unit 201 acquires an inspection image of the object to be inspected. Next, in step S101, the profile acquisition unit 202 acquires illumination profile information (profile image) that shows the illumination profile at the time of imaging of the inspection image acquired in step S100. Next, in step S102, the reference image generation unit 204 generates a reference image using the design image and the profile information. Prior to this step, if necessary, the design image generation unit 203 generates a design image from the design information. After step S102, in step S103, the inspection unit 205 inspects the object to be inspected by comparing the inspection image and the reference image.

[0067] Next, the machine learning model used by the reference image generation unit 204 will be described. In this embodiment, a deep learning model is used as the machine learning model, as an example.

[0068] The learning data acquisition unit 206 acquires learning data to be used for machine learning of the model used by the reference image generation unit 204. The learning data acquisition unit 206 may acquire learning data input from other devices, or it may acquire learning data by reading learning data stored in a storage device such as the memory 502 of the image processing device 200, which will be described later. The learning data acquired by the learning data acquisition unit 206 consists of a set of data: a learning image, which is an image captured by illuminating a learning sample with critical illumination; a sample design image, which is an image of the learning sample drawn according to the design information of the learning sample; and learning profile information, which is illumination profile information indicating the illumination profile at the time the learning image was captured. The learning sample is, for example, a sample manufactured through the same manufacturing process as the object to be inspected. The learning sample may be a sample on which a pattern used only for learning has been formed (i.e., a sample on which the pattern formed is different from that of the object to be inspected), or the object to be inspected may be used as the learning sample.

[0069] The training image is captured by the first detector 23. Therefore, the training data acquisition unit 206 may acquire the training image via the image acquisition unit 201. The training image is, like the inspection image 901a (see Figure 3), a two-dimensional image of m × n pixels extracted from a two-dimensional image obtained by the first detector 23.

[0070] Furthermore, in this embodiment, the learning profile information is a profile image, which is an image captured by the second detector 33. Therefore, the learning data acquisition unit 206 may acquire the learning profile information via the profile acquisition unit 202. The learning profile information is, like the profile image 902a (see Figure 3), for example, an m × n pixel two-dimensional image extracted from a two-dimensional image obtained by the second detector 33. Note that the relative position of the learning image with respect to the original two-dimensional image is the same as the relative position of the learning profile information (profile image) with respect to the original two-dimensional image.

[0071] The sample design image is a design image for the region shown in the training image, and is an image generated from design information in the same way as the design image used during inspection. Specifically in this embodiment, the sample design image is a grayscale image obtained by pixelating a binary image generated by rasterizing based on the design information. Therefore, the training data acquisition unit 206 may acquire the sample design image via the design image generation unit 203. For this reason, the design information storage unit 209 may store the design information of the training sample.

[0072] The model learning unit 207 generates a machine learning model by performing machine learning using the learning data acquired by the learning data acquisition unit 206. Therefore, the model learning unit 207 generates a machine learning model by performing learning processing using learning data which consists of a set of learning images (images taken with critical illumination of a learning sample), sample design images (images of the learning sample drawn according to the design information of the learning sample), and illumination profile information (showing the illumination profile at the time of acquisition of the learning image). This machine learning model is the model used by the reference image generation unit 204 described above. In other words, the machine learning model generated by the model learning unit 207 is a model that takes a design image of the object to be inspected and illumination profile information (showing the illumination profile at the time of acquisition of the inspection image) as inputs and outputs a reference image. The trained model generated by the machine learning processing of the model learning unit 207 is stored in the model storage unit 208. The reference image generation unit 204 then generates a reference image using the trained model stored in the model storage unit 208. In other words, the trained model generated by the model learning unit 207 is used as a computer program module to make the computer function to generate a reference image.

[0073] Embodiment 1 has been described above. In this embodiment, a reference image is generated considering the illumination profile during imaging. Therefore, a more appropriate reference image can be generated. In particular, in this embodiment, the reference image is generated with attention to the illumination profile, which is a parameter suitable for inspections using an imaging device with a critical illumination optical system. Therefore, a reference image suitable for inspections using an imaging device with a critical illumination optical system can be generated. Note that the illumination profile information used in this embodiment only needs to show the illumination profile at the time of imaging of the inspection image, and does not necessarily have to be a profile image acquired by the second detector 33.

[0074] <Embodiment 2> Next, Embodiment 2 will be described. This embodiment differs from Embodiment 1 in its method of generating a reference image using a design image and lighting profile information. The following describes the configuration or operation that differs from Embodiment 1, and descriptions that overlap with Embodiment 1 will be omitted as appropriate.

[0075] Figure 6 is a schematic diagram showing the generation of a reference image by the reference image generation unit 204 according to Embodiment 2. As shown in Figure 6, in this embodiment, the reference image generation unit 204 modifies the design image 922 into a design image 924 that reflects the illumination profile information 923 by performing an optical simulation 920 using illumination profile information 923 for the inspection image. The design image 922 is an image of the inspection target area drawn according to the design information of the object to be inspected, and is, for example, an image generated by the design image generation unit 203. After modifying the design image 922 into the design image 924, the reference image generation unit 204 generates a reference image 925 by inputting the modified design image 924 into a pre-trained machine learning model 921. The optical simulation 920 is a simulator (software) that simulates an image captured based on the design image 922, using the optical design of the imaging device 100 (shape or arrangement of mirrors and lenses, magnification of lenses, etc.) and illumination profile information as parameters. Known software can be used as the simulator to realize the optical simulation 920.

[0076] As described above, in this embodiment, an optical simulation 920 using lighting profile information is performed, so the lighting profile information is reflected in the modified design image 924. For this reason, the machine learning model 921 in this embodiment does not require lighting profile information as input, unlike the machine learning model used in Embodiment 1. In other words, in this embodiment, the reference image generation unit 204 generates a reference image using a machine learning model that has been pre-trained to output a reference image with the design image as input.

[0077] The machine learning model 921 in this embodiment is a model trained using training data consisting of a set of training images described in Embodiment 1 and a sample design image described in Embodiment 1. The model learning unit 207 in this embodiment generates the machine learning model 921 using such training data. Thus, unlike Embodiment 1, this embodiment uses a machine learning model 921 that has been trained without considering the lighting profile. This machine learning model 921 can be said to be a model that reflects the influence of the characteristics of the imaging device 100 or the characteristics of the manufacturing process of the object to be inspected (e.g., lithography process) on the image captured by the imaging device 100 to the image input to the model. As mentioned above, the influence of differences in the lighting profile during shooting on the image captured by the imaging device 100 is reflected in the design image 924 by the optical simulation 920. In this embodiment as well, the reference image generation unit 204 generates different reference images if the lighting profile information is different, even if the design images are the same.

[0078] Embodiment 2 has been described above. In this embodiment as well, a reference image is generated considering the illumination profile during imaging. Therefore, a more appropriate reference image can be generated. In this embodiment as well, the illumination profile information only needs to show the illumination profile at the time of imaging of the inspection image, and does not necessarily have to be a profile image acquired by the second detector 33.

[0079] <Embodiment 3> Next, Embodiment 3 will be described. This embodiment also differs from Embodiment 1 in its method for generating a reference image using the design image and lighting profile information. The following describes the configuration or operation that differs from Embodiment 1, and explanations that overlap with Embodiment 1 will be omitted as appropriate.

[0080] Figure 7 is a schematic diagram showing the generation of a reference image by the reference image generation unit 204 according to Embodiment 3. As shown in Figure 7, in this embodiment, the reference image generation unit 204 generates a reference image 933 by inputting a design image 932 to a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information 931 for the inspection image. In this embodiment, as an example, the reference image generation unit 204 uses three machine learning models 930a, 930b, and 930c based on the lighting profile information 931. The selection of models based on the lighting profile information 931 will be explained in detail below with reference to Figure 8.

[0081] Figure 8 is a graph showing an example of the luminance intensity distribution obtained by the second detector 33. The graph in Figure 8 shows the luminance intensity at each imaging position within the imaging range of the second detector 33, and this graph also corresponds to the luminance intensity at each imaging position within the imaging range of the first detector 23. In this embodiment, the illumination profile information is classified into three patterns. The first pattern of illumination profile information has the characteristic that the luminance intensity increases as the coordinate value of the imaging position increases, as shown in the illumination profile information 931a in Figure 8. The second pattern of illumination profile information has the characteristic that the luminance intensity is constant regardless of the coordinate value of the imaging position, as shown in the illumination profile information 931b in Figure 8. Hereinafter, "constant" means approximately constant, meaning that the variation in luminance intensity due to the coordinate value of the imaging position is within a predetermined allowable variation range. The third pattern of illumination profile information has the characteristic that the luminance intensity decreases as the coordinate value of the imaging position increases, as shown in the illumination profile information 931c in Figure 8.

[0082] In this embodiment, the machine learning model 930a shown in Figure 7 is a model used when the lighting profile information 931 used to generate the reference image belongs to the first pattern described above. Similarly, the machine learning model 930b is a model used when the lighting profile information 931 used to generate the reference image belongs to the second pattern described above, and the machine learning model 930c is a model used when the lighting profile information 931 used to generate the reference image belongs to the third pattern described above.

[0083] Machine learning model 930a is a model pre-trained using first training data, which consists of a first training image, an image captured by illuminating a training sample with critical illumination and having an illumination profile belonging to a first pattern, and a sample design image, an image of the training sample drawn according to the design information of the training sample. The first training image is an image captured by illuminating a training sample with critical illumination and can also be described as an image of the region in which the illumination profile has the first feature. Machine learning model 930b is a model pre-trained using second training data, which consists of a second training image, an image captured by illuminating a training sample with critical illumination and having an illumination profile belonging to a second pattern, and a sample design image, an image of the training sample drawn according to the design information of the training sample. The second training image is an image captured by illuminating a training sample with critical illumination and can also be described as an image of the region in which the illumination profile has the second feature. Similarly, the machine learning model 930c is a model pre-trained using third training data, which is a pair of a third training image, an image captured by illuminating a training sample with critical illumination and having an illumination profile belonging to the third pattern, and a sample design image, an image of the training sample drawn according to the design information of the training sample. The third training image is an image captured by illuminating a training sample with critical illumination and can also be described as an image of a region in which the illumination profile has the third feature. Therefore, in this embodiment, the training data acquisition unit 206 acquires the first training data, the second training data, and the third training data. The model learning unit 207 then generates a machine learning model 930a, which takes the design image as input and outputs a first reference image, by performing machine learning using the first training data. Similarly, the model learning unit 207 generates a machine learning model 930b that takes a design image as input and outputs a second reference image by performing machine learning using the second training data, and generates a machine learning model 930c that takes a design image as input and outputs a third reference image by performing machine learning using the third training data.

[0084] Thus, in this embodiment, each of the multiple machine learning models is a model trained using training data consisting of a training image, which is an image of a training sample captured under critical illumination, and a sample design image, which is an image of the training sample drawn according to the design information of the training sample. However, the illumination profile of the critical illumination used in the training data differs for each machine learning model. In this embodiment, three models are used interchangeably, but the reference image generation unit 204 only needs to use at least two models interchangeably.

[0085] The reference image generation unit 204 generates a reference image using a machine learning model selected from the pre-generated machine learning models according to the lighting profile information. As explained with reference to Figure 3, the lighting profile information used by the reference image generation unit 204 to generate one reference image is a part of the luminance intensity distribution shown in Figure 8, such as the lighting profile information 931a, 931b, and 931c in Figure 8. Therefore, the reference image generation unit 204 determines which of the three patterns described above the lighting profile information 931 acquired by the profile acquisition unit 202 to generate the reference image belongs to. Then, the reference image generation unit 204 generates a reference image using the machine learning model from machine learning models 930a to 930c that corresponds to the determined pattern. In this embodiment as well, the reference image generation unit 204 generates different reference images if the lighting profile information is different, even if the design images are the same.

[0086] Embodiment 3 has been described above. In this embodiment as well, a reference image is generated considering the illumination profile during imaging. Therefore, a more appropriate reference image can be generated. The illumination profile information used in this embodiment only needs to show the illumination profile at the time of imaging of the inspection image, and does not necessarily have to be a profile image acquired by the second detector 33.

[0087] <Variation> Figure 1 above shows an example configuration of an imaging device 100 having a first detector 23 and a second detector 33. However, the imaging device 100 of the inspection system 1 may be replaced with an imaging device 100a having the following configuration. Figure 9 is a schematic diagram showing an example configuration of the imaging device 100a according to a modified example. The optical system of the imaging device 100a will be described below with reference to Figure 9.

[0088] As shown in Figure 9, in the imaging device 100a, the first light L21 from the light source 11a reaches the sample 90a via the mirror 51, homogenizer 52, and mirror 53, illuminating the sample 90a. Here, the first light L21 emitted from the light source 11a within the angular range θ1 is collected by the mirror 51 for the illumination of the sample 90a. The light from the sample 90a is detected by the first detector 23 via the mirror 54. On the other hand, the second light L22 from the light source 11a is detected by the second detector 33 via the mirror 55. The second light L22 is illumination light having a different optical path from the first light L21. Here, the second light L22 emitted from the light source 11a within the angular range θ2 is collected by the mirror 55. The angular ranges θ1 and θ2 described above do not overlap in space. The angular ranges θ1 and θ2 may have symmetry with respect to the symmetry axis 56 of the light source 11a.

[0089] In the configuration shown in Figure 1, the image captured by the second detector 33, which is obtained by imaging a portion of the illumination light reaching the imaging target (upper surface 91 of the sample 90) from the light source 11 to the first detector 23, is used as illumination profile information. That is, in the configuration shown in Figure 1, a portion of the illumination light is extracted by the cut mirror 31 and observed by the second detector 33. In contrast, according to the modified imaging device 100a shown in Figure 9, the image captured by the second detector 33, which is obtained by imaging illumination light from a different optical path than the illumination light reaching the imaging target of the first detector 23, is used as illumination profile information. Thus, when acquiring illumination profile information with the second detector 33, the illumination profile information only needs to be an image captured by the second detector 33 of an image obtained by imaging light from the light source, and it is not limited to whether the light imaged for the second detector 33 is a portion of the illumination light reaching the imaging target of the first detector 23 from the light source.

[0090] Although embodiments and modified examples have been described above, the above-mentioned functions (processing) of the image processing device 200 may be realized by a computer 500 having, for example, the following configuration.

[0091] Figure 10 is a block diagram showing an example configuration of a computer 500 that implements the processing of the image processing device 200. As shown in Figure 10, the computer 500 includes an input / output interface 501, memory 502, and a processor 503.

[0092] The input / output interface 501 is an interface for connecting to other devices (for example, the imaging device 100).

[0093] Memory 502 is composed of, for example, a combination of volatile memory and non-volatile memory. Memory 502 is used to store software (computer programs) containing one or more instructions executed by the processor 503, and data used for various processes. The model storage unit 208 and the design information storage unit 209 can be implemented by, for example, memory 502, but may also be implemented by any storage device other than memory 502.

[0094] The processor 503 performs the above-described processing of the image processing device 200 by reading and executing software (computer programs) from the memory 502. The processor 503 may be, for example, a microprocessor, an MPU (Micro Processor Unit), a CPU (Central Processing Unit), or a GPU (Graphics Processing Unit). The processor 503 may include multiple processors.

[0095] The program is included in the computer program product. The program also includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments when loaded into the computer. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals.

[0096] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention.

[0097] Furthermore, some or all of the above embodiments may also be described as follows, but are not limited to the following. (Note 1) A process to acquire an inspection image, which is an image captured by illuminating the area to be inspected of the object to be inspected with critical illumination, A process to acquire lighting profile information that shows the lighting profile during imaging, A step of generating a reference image based on the design information of the object to be inspected, The process involves comparing the inspection image with the reference image to inspect the area to be inspected. Equipped with, In the process of generating the reference image, the lighting profile information is used to generate different reference images for regions that show a common structure in the design information. Image processing methods. (Note 2) In the process of generating the reference image, the design image based on the design information and the lighting profile information for the inspection image are input to a pre-trained machine learning model to generate the reference image. The image processing method described in Appendix 1. (Note 3) The machine learning model is a model trained using training data which consists of training images, which are images of training samples captured under critical illumination; sample design images, which are images of the training samples drawn according to the design information of the training samples; and illumination profile information, which indicates the illumination profile at the time the training images were captured. The image processing method described in Appendix 2. (Note 4) In the process of generating the aforementioned reference image, By performing an optical simulation using the illumination profile information for the inspection image, the design image based on the design information is modified to a design image that reflects the illumination profile information. The modified design image is input into a pre-trained machine learning model to generate the reference image. The image processing method described in Appendix 1. (Note 5) The aforementioned machine learning model is a model trained using training data consisting of training images, which are images of training samples captured under critical illumination, and sample design images, which are images of the training samples drawn according to the design information of the training samples. The image processing method described in Appendix 4. (Note 6) In the process of generating the reference image, the reference image is generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information for the inspection image. The image processing method described in Appendix 1. (Note 7) Each of the aforementioned machine learning models is a model trained using training data consisting of a training image, which is an image captured by illuminating a training sample with critical illumination, and a sample design image based on the design information of the training sample. The training data used for learning is such that the characteristics of the illumination profile of the critical illumination differ for each machine learning model. The image processing method described in Appendix 6. (Note 8) In the step of acquiring the inspection image, the inspection image captured using the first detector is acquired. The first detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The illumination profile information indicates the luminance intensity distribution of the illumination light in the first direction. The image processing method described in any one of the appendices 1 to 7. (Note 9) In the step of acquiring the inspection image, the inspection image captured using the first detector is acquired. The illumination profile information is an image captured by a second detector, which is an image obtained by imaging the light from the light source. The image processing method described in any one of the appendices 1 to 8. (Note 10) The illumination profile information is an image captured by the second detector, which is obtained by imaging a portion of the illumination light reaching the object being imaged by the first detector from the light source. The image processing method described in Appendix 9. (Note 11) An image acquisition unit acquires an inspection image, which is an image captured by illuminating the inspection area of ​​the object to be inspected with critical illumination. A profile acquisition unit that acquires illumination profile information indicating the illumination profile during imaging, A reference image generation unit that generates a reference image based on the design information of the object to be inspected, By comparing the inspection image with the reference image, the inspection unit inspects the area to be inspected. Equipped with, The reference image generation unit uses the lighting profile information to generate different reference images for regions that exhibit a common structure in the design information. Image processing device. (Note 12) The reference image generation unit generates the reference image by inputting the design image based on the design information and the lighting profile information for the inspection image into a pre-trained machine learning model. The image processing device described in Appendix 11. (Note 13) The machine learning model is a model trained using training data which consists of training images, which are images of training samples captured under critical illumination; sample design images, which are images of the training samples drawn according to the design information of the training samples; and illumination profile information, which indicates the illumination profile at the time the training images were captured. The image processing apparatus described in Appendix 12. (Note 14) The aforementioned reference image generation unit, By performing an optical simulation using the illumination profile information for the inspection image, the design image based on the design information is modified to a design image that reflects the illumination profile information. The modified design image is input into a pre-trained machine learning model to generate the reference image. The image processing device described in Appendix 11. (Note 15) The aforementioned machine learning model is a model trained using training data consisting of training images, which are images of training samples captured under critical illumination, and sample design images, which are images of the training samples drawn according to the design information of the training samples. The image processing apparatus described in Appendix 14. (Note 16) The reference image generation unit generates the reference image by inputting the design image based on the design information to a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information for the inspection image. The image processing device described in Appendix 11. (Note 17) Each of the aforementioned machine learning models is a model trained using training data consisting of a training image, which is an image captured by illuminating a training sample with critical illumination, and a sample design image based on the design information of the training sample. The training data used for learning is such that the characteristics of the illumination profile of the critical illumination differ for each machine learning model. The image processing apparatus described in Appendix 16. (Note 18) The image acquisition unit acquires the inspection image captured using the first detector, The first detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The illumination profile information indicates the luminance intensity distribution of the illumination light in the first direction. An image processing apparatus as described in any one of the appendices 11 to 17. (Note 19) The image acquisition unit acquires the inspection image captured using the first detector, The illumination profile information is an image captured by a second detector, which is an image obtained by imaging the light from the light source. An image processing apparatus as described in any one of the appendices 11 to 18. (Note 20) The illumination profile information is an image captured by the second detector, which is obtained by imaging a portion of the illumination light reaching the object being imaged by the first detector from the light source. The image processing apparatus described in Appendix 19. (Note 21) A step of acquiring training data which is a set of training data consisting of a training image, which is an image taken by illuminating a training sample with critical illumination; a sample design image based on the design information of the training sample; and illumination profile information indicating the illumination profile at the time of acquisition of the training image. The process involves generating a machine learning model that outputs a reference image by performing machine learning using the training data, taking a target design image and lighting profile information indicating the lighting profile during imaging of the inspection image as inputs. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The aforementioned inspection image is an image captured by illuminating the inspection area of ​​the object to be inspected with critical illumination. The aforementioned reference image is an image that is compared with the inspection image in order to inspect the area to be inspected. Learning methods. (Note 22) A first training data set is an image captured by illuminating a training sample with critical illumination, wherein the illumination profile is a set of a first training image of a region having a first feature and a sample design image based on the design information of the training sample. A second training data set which is a pair of a second training image, in which the illumination profile is a region having a second characteristic, and the sample design image, obtained by illuminating the training sample with critical illumination and capturing an image, and A process to obtain at least, The process involves generating a first machine learning model that takes a target design image as input and outputs a first reference image by performing machine learning using the first training data, and generating a second machine learning model that takes the target design image as input and outputs a second reference image by performing machine learning using the second training data. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The first and second reference images are images that are compared with the inspection image in order to inspect the area to be inspected. The inspection image is an image captured by illuminating the inspection area of ​​the object to be inspected with critical illumination. Learning methods. (Note 23) The steps include: acquiring an inspection image, which is an image captured by illuminating the area to be inspected of the object to be inspected with critical illumination; A step to obtain lighting profile information that shows the lighting profile during imaging, The steps include generating a reference image based on the design information of the object to be inspected, The step of inspecting the area to be inspected by comparing the inspection image with the reference image. Have the computer run it, In the step of generating the reference image, the lighting profile information is used to generate different reference images for regions that show a common structure in the design information. program. (Note 24) In the step of generating the reference image, the design image based on the design information and the lighting profile information for the inspection image are input to a pre-trained machine learning model to generate the reference image. The program described in Appendix 23. (Note 25) The machine learning model is a model trained using training data which consists of training images, which are images of training samples captured under critical illumination; sample design images, which are images of the training samples drawn according to the design information of the training samples; and illumination profile information, which indicates the illumination profile at the time the training images were captured. The program described in Appendix 24. (Note 26) In the step of generating the aforementioned reference image, By performing an optical simulation using the illumination profile information for the inspection image, the design image based on the design information is modified to a design image that reflects the illumination profile information. The modified design image is input into a pre-trained machine learning model to generate the reference image. The program described in Appendix 23. (Note 27) The aforementioned machine learning model is a model trained using training data consisting of training images, which are images of training samples captured under critical illumination, and sample design images, which are images of the training samples drawn according to the design information of the training samples. The program described in Appendix 26. (Note 28) In the step of generating the reference image, the reference image is generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information for the inspection image. The program described in Appendix 23. (Note 29) Each of the aforementioned machine learning models is a model trained using training data consisting of a training image, which is an image captured by illuminating a training sample with critical illumination, and a sample design image based on the design information of the training sample. The training data used for learning is such that the characteristics of the illumination profile of the critical illumination differ for each machine learning model. The program described in Appendix 28. (Note 30) In the step of acquiring the inspection image, the inspection image captured using the first detector is acquired. The first detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The illumination profile information indicates the luminance intensity distribution of the illumination light in the first direction. The program described in any one of the items in Appendix 23 to 29. (Note 31) In the step of acquiring the inspection image, the inspection image captured using the first detector is acquired. The illumination profile information is an image captured by a second detector, which is an image obtained by imaging the light from the light source. The program described in any one of the items in Appendix 23 to 30. (Note 32) The illumination profile information is an image captured by the second detector, which is obtained by imaging a portion of the illumination light reaching the object being imaged by the first detector from the light source. The program described in Appendix 31. (Note 33) A process to acquire an inspection image, which is an image captured by illuminating the area to be inspected of the object to be inspected with critical illumination, A process to acquire lighting profile information that shows the lighting profile during imaging, A step of generating a reference image from a design image which is an image of the area to be inspected drawn according to the design information of the object to be inspected, The process involves comparing the inspection image with the reference image to inspect the area to be inspected. Equipped with, In the process of generating the reference image, by using the illumination profile information, different reference images are generated for the first and second inspection images, which are inspection images with different illumination profiles at the time of imaging. Image processing methods. [Explanation of Symbols]

[0098] 1. Inspection System 10 Illumination optical system 11, 11a light source 12, 13 Ellipsoidal mirror 14 Mirror 20 detection optics 21 concave mirror 21a hole 22 Convex mirror 23 First detector 30 Monitor section 31 Cut Mirror 32 concave mirror 33. Second detector 51 Miller 52 Homogenizers 53, 54, 55 Miller 56 Axis of Symmetry 90 samples 90a Sample 91 Top surface 92 stages 100, 100a Imaging device 200 Image Processing Devices 201 Image Acquisition Unit 202 Profile Acquisition Unit 203 Design Image Generation Unit 204 Reference Image Generation Unit 205 Inspection Department 206 Training Data Acquisition Unit 207 Model Learning Department 208 Model Memory Unit 209 Design information storage section 231 Image sensor array 331 Image sensor array 500 Computers 501 Input / Output Interface 502 memory 503 Processor 901 Two-dimensional image 901a Inspection image 902 Two-dimensional images 902a profile image 910 Machine Learning Models 911 design image 912 Lighting Profile Information 913 Reference Image 920 Optical Simulation 921 Machine Learning Models 922 Design Image 923 Lighting Profile Information 924 Design Images 925 Reference Image 930a, 930b, 930c Machine Learning Models 931 Lighting Profile Information 931a, 931b, 931c Lighting Profile Information 932 Design Image 933 Reference Image L11 Illumination Light L12 reflected light L21 First Light L22 Second Light

Claims

1. A process to acquire an inspection image, which is an image captured by illuminating the area to be inspected of the object to be inspected with critical illumination, A process to acquire lighting profile information that shows the lighting profile during imaging, A step of generating a reference image based on the design information of the object to be inspected, The process involves comparing the inspection image with the reference image to inspect the area to be inspected. Equipped with, In the process of generating the reference image, the lighting profile information is used to generate different reference images for regions that show a common structure in the design information. Image processing methods.

2. In the process of generating the reference image, the design image based on the design information and the lighting profile information for the inspection image are input to a pre-trained machine learning model to generate the reference image. The image processing method according to claim 1.

3. The machine learning model is a model trained using training data which consists of training images, which are images of training samples captured under critical illumination; sample design images, which are images of the training samples drawn according to the design information of the training samples; and illumination profile information, which indicates the illumination profile at the time the training images were captured. The image processing method according to claim 2.

4. In the process of generating the aforementioned reference image, By performing an optical simulation using the illumination profile information for the inspection image, the design image based on the design information is modified to a design image that reflects the illumination profile information. The modified design image is input into a pre-trained machine learning model to generate the reference image. The image processing method according to claim 1.

5. The aforementioned machine learning model is a model trained using training data consisting of training images, which are images of training samples captured under critical illumination, and sample design images, which are images of the training samples drawn according to the design information of the training samples. The image processing method according to claim 4.

6. In the process of generating the reference image, the reference image is generated by inputting the design image based on the design information into a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information for the inspection image. The image processing method according to claim 1.

7. Each of the aforementioned machine learning models is a model trained using training data consisting of a training image, which is an image captured by illuminating a training sample with critical illumination, and a sample design image based on the design information of the training sample. The training data used for learning is such that the characteristics of the illumination profile of the critical illumination differ for each machine learning model. The image processing method according to claim 6.

8. In the step of acquiring the inspection image, the inspection image captured using the first detector is acquired. The first detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The illumination profile information indicates the luminance intensity distribution of the illumination light in the first direction. The image processing method according to any one of claims 1 to 7.

9. In the step of acquiring the inspection image, the inspection image captured using the first detector is acquired. The illumination profile information is an image captured by a second detector, which is an image obtained by imaging the light from the light source. The image processing method according to claim 1.

10. The illumination profile information is an image captured by the second detector, which is obtained by imaging a portion of the illumination light reaching the object being imaged by the first detector from the light source. The image processing method according to claim 9.

11. An image acquisition unit acquires an inspection image, which is an image captured by illuminating the inspection area of ​​the object to be inspected with critical illumination. A profile acquisition unit that acquires illumination profile information indicating the illumination profile during imaging, A reference image generation unit that generates a reference image based on the design information of the object to be inspected, By comparing the inspection image with the reference image, the inspection unit inspects the area to be inspected. Equipped with, The reference image generation unit uses the lighting profile information to generate different reference images for regions that exhibit a common structure in the design information. Image processing device.

12. The reference image generation unit generates the reference image by inputting the design image based on the design information and the lighting profile information for the inspection image into a pre-trained machine learning model. The image processing apparatus according to claim 11.

13. The machine learning model is a model trained using training data which consists of training images, which are images of training samples captured under critical illumination; sample design images, which are images of the training samples drawn according to the design information of the training samples; and illumination profile information, which indicates the illumination profile at the time the training images were captured. The image processing apparatus according to claim 12.

14. The aforementioned reference image generation unit, By performing an optical simulation using the illumination profile information for the inspection image, the design image based on the design information is modified to a design image that reflects the illumination profile information. The modified design image is input into a pre-trained machine learning model to generate the reference image. The image processing apparatus according to claim 11.

15. The aforementioned machine learning model is a model trained using training data consisting of training images, which are images of training samples captured under critical illumination, and sample design images, which are images of the training samples drawn according to the design information of the training samples. The image processing apparatus according to claim 14.

16. The reference image generation unit generates the reference image by inputting the design image based on the design information to a machine learning model selected from among a plurality of pre-trained machine learning models based on the lighting profile information for the inspection image. The image processing apparatus according to claim 11.

17. Each of the aforementioned machine learning models is a model trained using training data consisting of a training image, which is an image captured by illuminating a training sample with critical illumination, and a sample design image based on the design information of the training sample. The training data used for learning is such that the characteristics of the illumination profile of the critical illumination differ for each machine learning model. The image processing apparatus according to claim 16.

18. The image acquisition unit acquires the inspection image captured using the first detector, The first detector is a TDI sensor having image sensors arranged in a first direction and a second direction, and integrating the charges from each of the plurality of image sensors arranged in the second direction. The illumination profile information indicates the luminance intensity distribution of the illumination light in the first direction. The image processing apparatus according to any one of claims 11 to 17.

19. The image acquisition unit acquires the inspection image captured using the first detector, The illumination profile information is an image captured by a second detector, which is an image obtained by imaging the light from the light source. The image processing apparatus according to claim 11.

20. The illumination profile information is an image captured by the second detector, which is obtained by imaging a portion of the illumination light reaching the object being imaged by the first detector from the light source. The image processing apparatus according to claim 19.

21. A step of acquiring training data which is a set of training data consisting of a training image, which is an image taken by illuminating a training sample with critical illumination; a sample design image based on the design information of the training sample; and illumination profile information indicating the illumination profile at the time of acquisition of the training image. The process involves generating a machine learning model that outputs a reference image by performing machine learning using the training data, taking a target design image and lighting profile information indicating the lighting profile during imaging of the inspection image as inputs. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The aforementioned inspection image is an image captured by illuminating the inspection area of ​​the object to be inspected with critical illumination. The aforementioned reference image is an image that is compared with the inspection image in order to inspect the area to be inspected. Learning methods.

22. A first training data set is an image captured by illuminating a training sample with critical illumination, wherein the illumination profile is a set of a first training image of a region having a first characteristic and a sample design image based on the design information of the training sample. A second training data set which is a pair of a second training image, in which the illumination profile is a region having a second characteristic, and the sample design image, obtained by illuminating the training sample with critical illumination and capturing an image, and A process to obtain at least, The process involves generating a first machine learning model that takes a target design image as input and outputs a first reference image by performing machine learning using the first training data, and generating a second machine learning model that takes the target design image as input and outputs a second reference image by performing machine learning using the second training data. Equipped with, The aforementioned design image is an image of the inspection area of ​​the object to be inspected, drawn according to the design information of the object to be inspected. The first reference image and the second reference image are images that are compared with the inspection image in order to inspect the area to be inspected. The inspection image is an image captured by illuminating the inspection area of ​​the object to be inspected with critical illumination. Learning methods.

Images