Image processing system and image processing method

Abstract

In order to achieve image synthesis without compromising surface information of an SE image and shadow information of a BSE image, the present disclosure proposes image processing technology in which: data of a first image-quality image (low image-quality image) are applied to a trained model; structural feature quantities and material feature quantities of a second image-quality image (high image-quality image) corresponding to the first image-quality image are estimated; at least one piece of shadow data is calculated on the basis of the structural feature quantities and synthesizing parameters; at least one piece of gradation data is calculated on the basis of the material feature quantities and the synthesizing parameters; a synthesized image is generated from the at least one piece of shadow data and the at least one piece of gradation data; and the synthesized image is output as a predicted result of the second image-quality image (see fig. 8).

Description

Image processing system and image processing method

[0001] The present disclosure relates to an image processing system and an image processing method.

[0002] A known method for reviewing defects in semiconductor circuit patterns is to have the user observe the details of the defects using images of the sample being reviewed. In defect reviews of semiconductor circuit patterns, a scanning electron microscope is sometimes used to acquire the images.

[0003] Conventionally, known methods for improving the visibility of semiconductor circuit patterns and defects on the circuit patterns include contrast enhancement processing using histogram equalization or the like, and image quality improvement using image processing such as noise reduction processing and super-resolution processing. As an example of a conventional method, Patent Document 1 discloses a method for converting a low-quality image into a high-quality image using a trained neural network, and a method for training the neural network.

[0004] In order to improve the visibility of circuit patterns and defects on the circuit patterns, it is necessary to adjust the direction and intensity of the shading of the circuit patterns, as well as the brightness according to the material properties.

[0005] Furthermore, in order to observe the details of the defect structure or the details of the unevenness of a circuit pattern, it is common practice to acquire a plurality of shadow images with different shadow information and a plurality of directly above images with almost no shadow information using a scanning electron microscope, and then combine them. In such a general composite image generation method, for example, when a shadow image is acquired as a backscattered electron image (BSE image) and a directly above image is acquired as a secondary electron image (SE image), the SE image, which has excellent detailed information about the sample surface, is combined with the unevenness information (shadow information) of the BSE image to create a composite image for defect review, by adding the SE image and the BSE image together at an arbitrary mixing ratio.

[0006] Japanese Patent Application Laid-Open No. 2020-144489

[0007] However, the image quality improvement method disclosed in Patent Document 1 often applies processing uniformly to the entire image, which can result in excessive emphasis on shadows, reduction in the sharpness of edges, and the occurrence of artifacts.

[0008] Furthermore, as described above, in a typical composite image generation method, a composite image is generated by adding together signals from an SE image and a BSE image. In the composite image, the surface information of the SE image and the shading information of the BSE image (each amount of information) are reduced compared to the respective information amounts before synthesis, depending on an arbitrary mixing ratio. As a result, the surface information and shading information in the composite image are impaired compared to the respective images before synthesis. In view of this situation, the present disclosure provides an image synthesis technique in which the surface information of the SE image and the shading information of the BSE image are not impaired.

[0009] (i) In order to solve the above problem, the present disclosure provides an image processing method in which a computer generates a trained model for converting a first quality image into a second quality image of higher quality than the first quality image by training a machine learning model using a plurality of images of a target sample acquired under different imaging conditions, the method comprising: receiving at least one machine learning model, a plurality of first quality images, and a plurality of second quality images of higher quality than the first quality images; applying data of the plurality of first quality images to the at least one machine learning model to estimate structural features and material features of the second quality images corresponding to the plurality of first quality images; calculating at least one first shading data based on the structural features; calculating at least one second shading data from the plurality of second quality images; comparing the at least one first shading data with the at least one second shading data to obtain a first comparison result; calculating at least one first gradation data based on the material feature; and calculating at least one second gradation data from the plurality of second quality images. The present invention proposes an image processing method including: comparing the at least one first gradation data with the at least one second gradation data to obtain a second comparison result; and updating parameters of the at least one machine learning model based on the first comparison result and the second comparison result.

[0010] (ii) The present disclosure also proposes an image processing method in which a computer applies a first quality image of a target sample to a trained model, thereby predicting and outputting a second quality image of higher quality than the first quality image, the image processing method including: accepting a plurality of first quality images and synthesis parameters; applying data of the first quality images to the trained model to estimate structural features and material features of the second quality image corresponding to the first quality images; calculating at least one piece of shading data based on the structural features and the synthesis parameters; calculating at least one piece of gradation data based on the material features and the synthesis parameters; generating a composite image from the at least one piece of shading data and the at least one piece of gradation data, and outputting the composite image as a predicted result of the second quality image.

[0011] (iii) Further related features will become apparent from the description of this specification and the accompanying drawings. Furthermore, the aspects of the present disclosure are achieved and realized by the elements and combinations of various elements, as well as the aspects of the following detailed description and the appended claims. The description of this specification is merely exemplary and does not limit the scope or application of the claims of the present disclosure in any way.

[0012] According to the technology of the present disclosure, it is possible to acquire a composite image (high-quality predicted image) in which the surface information of the SE image and the shadow information of the BSE image are not lost.

[0013] 2 is a diagram showing an example of the configuration of a scanning electron microscope (SEM) 10, which is an example of an image generation tool for acquiring a semiconductor pattern image to be subjected to image processing in an image processing system according to this embodiment. It is a diagram showing an example in which a lower detector 117 is configured with four shadow image detectors 201 to 204. It is a diagram showing BSEs 302A and 302B emitted from the irradiation position when an electron beam 107 is irradiated onto a sample 301 in the scanning electron microscope 10 illustrated in FIG. 1. It is a diagram showing an example of multiple SEM images of a sample (pillar circuit pattern) acquired using the scanning electron microscope 10 of FIG. 1 and the shadow image detector of FIG. 2. It is a diagram showing lower detectors 501A and 501B having a two-stage configuration, which are a modified example of the lower detector 117. It is a diagram showing an example of the configuration of an image processing system 60 that performs learning processing to create a machine learning model according to this embodiment. It is a flowchart for explaining the procedure of the learning processing according to this embodiment. It is a diagram showing an example of the configuration of an image processing system that performs image synthesis processing to generate a synthesized high-quality image according to this embodiment. 11 is a flowchart illustrating the procedure of image synthesis processing according to the present embodiment. 22 is a diagram illustrating an example of a case where the image synthesis processing according to the present embodiment is applied to an SEM image having a particle defect. 23 is a diagram illustrating an example of the configuration of a GUI screen 1100 displayed on the screen of a display used by a user when executing image synthesis processing in the image processing system 80 according to the present embodiment.

[0014] This embodiment proposes an image processing technique that allows the brightness and shading of sample images acquired under different detection conditions to be independently manipulated when the images are combined.

[0015] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be designated by the same numerals. Note that the accompanying drawings show embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

[0016] Although the present embodiment has been described in sufficient detail to enable those skilled in the art to practice the present disclosure, it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0017] 1 is a diagram showing an example of the configuration of a scanning electron microscope (SEM) 10, which is an example of an image generation tool for acquiring a semiconductor pattern image to be subjected to image processing in an image processing system according to this embodiment. Note that the image generation tool to which this embodiment is applied is not limited to a scanning electron microscope, and for example, a focused ion beam (FIB) device that generates an image based on scanning an ion beam can also be used as the image generation tool.

[0018] The scanning electron microscope 10 illustrated in FIG. 1 includes an imaging unit 101 , a computer system 102 , a signal processing unit 103 , an input / output unit 104 , and a storage unit 105 .

[0019] The imaging unit 101 includes an electron gun 106 that irradiates an electron beam 107, a focusing lens 108 that focuses the electron beam 107, and a focusing lens 109 that further focuses the electron beam 107 that has passed through the focusing lens 108. The imaging unit 101 further includes a deflector 110 that deflects the electron beam 107, and an objective lens 111 that controls the height at which the electron beam 107 is focused.

[0020] The electron beam 107 that has passed through the optical system of the imaging unit 101 is irradiated onto a sample 112 placed on a sample stage 113. Secondary electrons (SE) 114 emitted from the sample 112 by irradiation with the electron beam 107 are mainly detected by a secondary electron detector (upper detector) 115. Backscattered electrons (BSE) 116 emitted from the sample 112 are mainly detected by a backscattered electron detector (lower detector) 117.

[0021] The computer system 102 controls the imaging unit 101. The signal processing unit 103 generates SEM images (SE images, BSE images) based on the outputs of the upper detector 115 and the lower detector 117. When storing detection signals in a frame memory in synchronization with the scanning of a scanning deflector (not shown), the signal processing unit 103 stores the detection signals at positions corresponding to the scanning positions of the frame memory, thereby generating a signal profile (one-dimensional information) and an SEM image (two-dimensional information). The memory unit 105 also functions as a non-transitory recording medium that stores computer programs that control the operation of this system. The input / output unit 104 inputs various instructions from an operator (via various switches, a keyboard, a touch panel, etc.) and outputs the generated signal profile, SEM image, etc. to a display (not shown) or the like.

[0022] 2 to 5, a method for acquiring SEM images under different detection conditions using the scanning electron microscope 10 illustrated in Fig. 1 will be described. Specifically, in order to observe the details of the structural characteristics of a sample (such as layer structure or surface unevenness), an SEM image (shade image) having shading information according to the structural characteristics of the sample and a method for arranging a detector for acquiring the shade image will be described.

[0023] (i) Example of the configuration of the lower detector 117 Fig. 2 is a diagram showing an example in which the lower detector 117 is configured with four shadow image detectors 201 to 204. When the electron beam 107 is irradiated onto the sample, the shadow image detectors 201 to 204 acquire BSEs of different radiation directions among the BSEs emitted from the sample, and generate an SEM image from each acquired signal.

[0024] (ii) Formation of Shadow Images FIG. 3 is a diagram showing BSEs 302A and 302B emitted from the irradiation position when the electron beam 107 is irradiated onto the sample 301 in the scanning electron microscope 10 illustrated in FIG. 1 . As shown in FIG. 3 , the sample 301 has irregularities on its surface. When BSEs 302A are acquired by the shadow image detector 202 and BSEs 302B are acquired by the shadow image detector 203, the number of BSEs acquired by the shadow image detector 202 is smaller than the number of BSEs acquired by the shadow image detector 203 because BSEs 302A are blocked by the convex portions of the sample. In such a case, the irradiation position in FIG. 3 appears as a shadow on the SEM image generated from the signal acquired by the shadow image detector 202. On the other hand, on the SEM image generated by the shadow image detector 203, the irradiation position in FIG. 3 does not appear as a shadow, but instead has a brightness corresponding to the material properties of the sample.

[0025] (iii) Example of Shadow Images Figure 4 is a diagram showing an example of a plurality of SEM images acquired of a sample (pillar circuit pattern) using the scanning electron microscope 10 of Figure 1 and the shadow image detector of Figure 2. Shadow images 401 to 404 are SEM images generated from the acquired signals of the shadow image detectors 201 to 204, respectively, and have different shadow information according to the structural characteristics of the sample.

[0026] The directly above image 405 is an SEM image (BSE image) with almost no shading information, which is obtained by combining the signals acquired by the shadow image detectors 201 and 204. The directly above image 406 is an SEM image (SE image) generated from the signals acquired by the upper detector 115. Examples of shading information in this embodiment include the direction, density, and size (area) of the shading on the SEM image.

[0027] 2, the lower detector 117 is configured with four shadow image detectors 201 to 204, but it may be configured with any number of shadow image detectors, for example, two or three, and there is no limit to the number of shadow image detectors. However, if the lower detector 117 is configured with one detector, the vectors of the BSE emitted from the sample are combined, and shadow information due to differences in the emission direction is lost. Therefore, it is desirable that the lower detector 117 in this embodiment be configured with two or more shadow image detectors.

[0028] Furthermore, in this embodiment, the lower detector 117 is configured with a plurality of detectors, but the upper detector 115 may also be configured with a plurality of detectors in the same manner.

[0029] (iv) Modification of the lower detector 117 Figure 5 shows a modification of the lower detector 117, showing lower detectors 501A and 501B with a two-stage configuration. The lower detectors 501A and 501B are at different distances from the sample 112. The lower detectors 501A and 501B respectively acquire BSEs 502A and 502B, which are emitted from the sample and have different emission angles with respect to the traveling direction of the electron beam 107. The emission angles of the BSEs emitted from the sample 112 vary depending on the unevenness of the surface of the sample 112. Furthermore, the number of emitted BSEs also varies depending on the emission angle, and the number of BSEs acquired by each detector differs. Therefore, when the lower detector 501A and the lower detector 501B are each composed of four shadow detectors as shown in Figure 2, a difference occurs in the shadow information between the shadow image acquired by the shadow image detector 201 of detector 501A and the shadow image acquired by the shadow image detector 201 of the lower detector 501B.

[0030] In the following explanation, image processing will be described when either the lower detector 117 is placed as the lower detector 501A or 501B, but both the lower detector 501A and the lower detector 501B may be provided, or the distance between the lower detector 117 and the sample 112 may be mechanically changeable.

[0031] <Image Processing System with Learning Processing Function> With reference to FIGS. 6 and 7 , a learning process will be described for the image processing system of this embodiment, which uses a plurality of low-quality SEM images (low-quality images) and high-quality SEM images (high-quality images) acquired with the scanning electron microscope and detector configuration exemplified in FIGS. 1 and 2 to create a machine learning model that can estimate, from the plurality of low-quality images, feature quantities (structural feature quantities) that indicate the structural characteristics and feature quantities (material feature quantities) that indicate the material characteristics of a sample in a high-quality image.

[0032] 6 is a diagram showing an example of the configuration of an image processing system 60 that performs learning processing to create a machine learning model in this embodiment. The image processing system 60 is composed of a computer system 600 that receives a machine learning model 601, a low-quality image 602, and a high-quality image 606 as input, and outputs structural features 604, material features 605, and an updated model 610. The main functions of the image processing system 60 are executed by the computer system 600.

[0033] The computer system 600 is composed of one or more computer subsystems, each including one or more CPUs. The one or more computer subsystems can implement the processes described below in software using one or more processors, or can implement part or all of the processes in hardware, such as electronic circuits.

[0034] The computer system 600 includes, for example, a feature prediction unit 603, a shadow comparison unit 607, a brightness comparison unit 608, and a model update unit 609. The feature prediction unit 603, the shadow comparison unit 607, the brightness comparison unit 608, and the model update unit 609 may be realized virtually by software, or may be realized by hardware such as an electronic circuit.

[0035] The computer system 600 receives a machine learning model 601, a low-quality image 602, and a high-quality image 606. The feature prediction unit 603 uses the machine learning model 601 to estimate structural feature values ​​604 and material feature values ​​605 of the sample 112 in the high-quality image 606 from the low-quality image 602. The shadow comparison unit 607 calculates shadow data of the sample from the structural feature values ​​604 and from the high-quality image 606, and compares the two. The brightness comparison unit 608 calculates gradation data of the sample from the material feature values ​​605 and from the high-quality image 606, and compares the two. The model update unit 609 updates the parameters of the machine learning model 601 according to the comparison results of the shadow comparison unit 607 and the brightness comparison unit 608, and outputs (saves) an updated model 610.

[0036] The machine learning model 601 may be configured with two models: one model that estimates structural features from the low-quality image 602, and the other model that estimates material features 605 from the low-quality image 602, or it may be configured with one model that estimates both structural features and material features from the low-quality image 602. In this embodiment, a convolutional neural network (CNN) model is applied as the machine learning model 601. In this case, the parameters of the machine learning model 601 saved in the model update unit 609 include filter weights of the convolutional layer that constitutes the CNN model.

[0037] The structural feature 604 and material feature 605 estimated by the feature prediction unit 503 may be recorded in a physical memory (not shown) provided in the computer system 600, or may be recorded in a storage device (not shown) such as a hard disk in a binary file format, an image file format, or the like.

[0038] The low-quality images 602 are a plurality of SEM images acquired under low-quality imaging conditions using a scanning electron microscope 10 configured as illustrated in FIGS. 1 and 2 , with variations in SEM images (SE images and BSE images) resulting from differences in detector acquisition signals as illustrated in FIG. 4 . On the other hand, the high-quality images 606 are a plurality of SEM images acquired under high-quality imaging conditions using a scanning electron microscope 10 configured as illustrated in FIGS. 1 and 2 , with variations in images resulting from differences in detector acquisition signals as illustrated in FIG. 4 . The difference between the low-quality and high-quality imaging conditions is, for example, the frame accumulation number and resolution of the SEM images. In an SEM image, the more the frame accumulation number is increased, the more noise is reduced, and the higher the pixel resolution, the more the resolution improves. In this embodiment, the high-quality image 606 has the same frame accumulation number and pixel resolution as the low-quality image 602, but the other is larger (higher), or both are larger (higher). Another method for acquiring the high-quality image 606 is to improve visibility by changing the acceleration voltage in the scanning electron microscope 10. The file formats of the low-quality image 602 and the high-quality image 606 may be image file formats such as TIFF, GIF, PNG, or a binary file format.

[0039] <Learning Process of Machine Learning Model> (i) Overview of the Learning Process In the learning process of the machine learning model, the low-quality image 602 and the high-quality image 606 are a pair acquired at the same position on the sample and with the same FoV. As described above, the low-quality image 602 and the high-quality image 606 are multiple SEM images with variations, as shown in FIG. 4 , resulting from differences in detector placement or acquired signals. Pairs of the low-quality image 602 and the high-quality image 606 captured at multiple different positions on the sample are used as a learning dataset. The structural features and material features estimated from any of the low-quality images are compared with the paired high-quality image in the shade comparison unit 607 and the brightness comparison unit 608. To improve the generalization performance of the machine learning model 601, it is preferable that the learning dataset contains a large number of images, and that it includes a wide variety of circuit pattern shapes.

[0040] (ii) Low-Quality Image 602 One method of inputting the low-quality image 602 to the CNN model, which is the machine learning model 601, is to read each of the shadow image and the direct-on image contained in the low-quality image 602 as a three-dimensional array (height, width, channel), combine the three-dimensional arrays in the channel direction, and input the combined array to the CNN model. Here, height indicates the number of pixels in the vertical direction of the image, and width indicates the number of pixels in the horizontal direction of the image. Note that the channel indicates the number of types of color information, etc. For example, when color information is represented in RGB, the number of channels is three, and when color information is represented in grayscale, the number of channels is one. Furthermore, when BSE images and SE images are stored in the same array, the number of channels is set to two. This is because BSE images and SE images are stored in each channel.

[0041] (iii) Structural Feature 604 The structural feature 604 is a feature that indicates the structural characteristics of the sample on the corresponding high-resolution image, estimated from the low-resolution image 602 using the machine learning model 601. Examples of the structural feature 604 include a normal map, a bump map, a height map, and a displacement map that represent the layer structure or surface unevenness of the sample. Information input to the CNN model to predict the structural feature 604 is, for example, SEM images 401 to 404 shown in FIG. 4. These SEM images have different shading information according to the structural characteristics of the sample 112.

[0042] In this embodiment, the structural feature 604 output by the CNN model is a normal map of the sample on the high-resolution image corresponding to the input low-resolution image. Here, the normal map is, for example, a three-dimensional unit vector quantity representing the dominant direction of the amount of electrons emitted from a position when the electron beam 107 is irradiated onto the sample 112. It includes information on the unevenness of the sample surface as well as shadows cast on the underlying pattern surface due to shielding by adjacent pattern structures (e.g., multilayer structures). More specifically, the normal map indicates the direction in which light irradiated from above onto the sample 112 is reflected, and thus can be considered information representing the direction of the shadow cast when light strikes the sample 112. By calculating the dot product of this normal map and the vector quantity representing the detector direction (the respective directions from the lower detector 201 to the lower detector 204 relative to the electron beam irradiation position), a shadow map representing the attenuation rate of brightness due to shadows (a decimal number between 0 and 1) at each position on the sample can be obtained.

[0043] (iv) Material Feature 605 The material feature 605 is a feature that indicates the material characteristics of the sample in the corresponding high-quality image 606, estimated from the low-quality image 602 using the machine learning model 601. An example of the material feature 605 is a predicted image of the direct-on image of the high-quality image 606. This predicted image may be an image that includes imaging noise of the direct-on image of the high-quality image 606, or may be an image that is predicted so as to remove the imaging noise. Information input to the CNN model to predict the material feature 605 is, for example, an SEM image (direct-on image 405) that is a composite image of SEM images 401 to 404, and an SEM image (direct-on image 406) that is a direct-on image of the sample 112, as shown in FIG. 4 . The output material feature 605 is a predicted image of the direct-on image of the high-quality image corresponding to the input low-quality image, and is composed of a predicted BSE image and a predicted SE image.

[0044] (v) Example of Comparison Method in Shade Comparison Unit 607 The shade comparison unit 607 first calculates a shade map (each value in the map ranges from 0 to 1) corresponding to each shaded image in the high-quality image 606 from the dot product of a normal map estimated as a structural feature 604 from the low-quality image 602 and three-dimensional unit vector quantities representing the position of each detector that acquired the multiple shaded images in the high-quality image 606.

[0045] Next, the shadow comparison unit 607 multiplies the above-mentioned shadow map by a direct image created by adding together multiple shadow images contained in the high-quality image 606 to create a predicted image of the shadow image, and calculates the difference (such as the mean square error) between this and the corresponding shadow image in the high-quality image 606. This is performed a number of times for all shadow images contained in the high-quality image 606, and the sum of the calculated differences is taken as the loss value.

[0046] (vi) Example of Comparison Method in Brightness Comparison Unit 608 The brightness comparison unit 608 compares the predicted BSE image and predicted SE image estimated as material feature values ​​605 from the low-quality image 602 with the directly above images (ground truth images) of the corresponding high-quality image 606, and determines the difference between them (such as the mean square error) as the loss value.

[0047] (vii) Model Update Unit 609 The model update unit 609 uses an optimizer such as Adam or SGD to update the parameters (such as the weights of the CNN filter) of the machine learning model 601 so as to reduce the loss values ​​calculated by the shadow comparison unit 607 and the brightness comparison unit 608. The updated model 610 is then saved in a predetermined file format.

[0048] (viii) Example of a learning model composed of two CNN models The machine learning model 601 can be composed of two CNN models: a first CNN model that estimates a normal map, and a second CNN model that estimates a predicted image of a directly above image. An example of the input / output format of each model will be described below.

[0049] The input of the first CNN model is, for example, a three-dimensional array obtained by reading multiple shadow images contained in the low-quality image 602 as three-dimensional arrays (height, width, channel) and combining them in the channel direction. From this three-dimensional array, the first CNN model predicts a normal map of the sample on the high-quality image 606 as a three-dimensional array. In this case, the height and width of the three-dimensional array of the normal map are equal to the height and width of the high-quality image 606, and the number of channels in the array is three, with each channel corresponding to the x, y, and z components of the normal vector.

[0050] The input of the second CNN model is, for example, a three-dimensional array obtained by reading multiple direct-above images (BSE image, SE image) contained in the low-quality image 602 as a three-dimensional array (height, width, channel) and combining them in the channel direction. The second CNN model predicts a predicted image of the direct-above image of the high-quality image 606 from this three-dimensional array as a three-dimensional array. At this time, the height and width of the predicted three-dimensional array are equal to the height and width of the high-quality image 606, the number of channels of the predicted three-dimensional array is two, and a predicted BSE image and a predicted SE image are stored in each channel. Note that the input / output formats of the first CNN model and the second CNN model described here are merely examples and are not limited to the above formats.

[0051] 7 is a flowchart illustrating the procedure of the learning process in this embodiment. Note that the subject of operation of each step is the computer system 600. As an example, the computer system 600 can be configured to load a program for implementing the learning process (learning process program) from a storage device (not shown; this may be an internal memory of the computer system 600 or an external storage device) and execute each step.

[0052] (i) Step S701: The computer system 600 receives an input of a machine learning model 601 (e.g., a CNN model) and sets the machine learning model 601 in the feature prediction unit 603. Note that the input machine learning model 601 may be initialized using an initial value of He or an initial value of Xivier, which are known methods for initializing CNN weights, or a model created in advance using a procedure similar to this learning process may be used.

[0053] (ii) Step S702: The computer system 600 receives as learning data the input of a low-quality image 602 and a high-quality image 606. The low-quality image 602 and the high-quality image 606 are images acquired by the image generation tool (scanning electron microscope 10) under different imaging conditions, and may be temporarily stored in a storage device (not shown) and then input to the computer system 600 from there, or may be input to the computer system 600 directly from the image generation tool.

[0054] (iii) Step S703 In the computer system 600, the feature prediction unit 603 uses the machine learning model 601 to estimate the structural feature 604 and the material feature 605 from the low-quality image 602.

[0055] (iv) Step S704 In the shade comparison unit 607, the computer system 600 compares the shade data calculated from the structural feature 604 with the shade data calculated from the high-quality image 606, and also compares the gradation data calculated from the material feature 605 with the gradation data calculated from the high-quality image 606.

[0056] (v) Step S705: The computer system 600 determines whether to continue the learning process based on the comparison results of the shadow data and the gradation data. If it is determined that the learning process should be continued (Yes in step S705), the process proceeds to step 706. If it is determined that the learning process should not be continued (No in step S705), the process proceeds to step 707.

[0057] The determination of whether to continue the learning process can be made based on the loss value calculated as the comparison result of step S704. For example, if the loss value is equal to or greater than a reference value (threshold) specified in advance by the user, the learning process continues, and if it is equal to or less than the reference value, the process proceeds to step S707. As an alternative method, the determination may be made based on the number of times the model parameters are updated, regardless of the comparison result of step S704. In this case, in step S705, if the number of times the model parameters are updated is equal to or less than the number specified in advance by the user, the learning process continues, and if the specified number of times is reached, the process proceeds to step S707.

[0058] (vi) Step 706: The computer system 600 updates the parameters of the machine learning model 601 based on the comparison results of the shading data and the gradation data. The parameter update can be performed using a general backpropagation algorithm. Thereafter, the process proceeds to step S702, and steps S702 to S705 are executed again.

[0059] (vii) Step S707 The computer system 600 saves the machine learning model 601 having the parameters at that time, and ends the learning process.

[0060] <Generation of Composite High-Quality Image> (i) Overview of Generation of Composite High-Quality Image Referring to FIGS. 8 to 11 , the image synthesis process for creating a composite high-quality image in the image processing system 80 of this embodiment will be described. The image processing system 80 estimates feature quantities (structural feature quantities) indicating structural characteristics and feature quantities (material feature quantities) indicating material characteristics of the sample 112 in the high-quality image from the multiple low-quality images using multiple low-quality images acquired by the upper detector 115 and the lower detector 117 of the scanning electron microscope 10 illustrated in FIGS. 1 and 2 , a trained model created by the training process illustrated in FIGS. 6 and 7 , and synthesis parameters input by the user. Next, the image processing system 80 calculates shading data based on the structural feature quantities and the synthesis parameters. The image processing system 80 also calculates gradation data based on the material feature quantities and the synthesis parameters. The image processing system 80 then performs image synthesis based on the shading data and gradation data to generate a composite high-quality image.

[0061] (ii) Example of the Configuration of Image Processing System 80 Fig. 8 is a diagram showing an example of the configuration of an image processing system that executes image synthesis processing to generate a high-quality synthesized image according to this embodiment. The functions of the image processing system 80 are realized by a computer system 800.

[0062] The computer system 800 is composed of one or more computer subsystems, each including one or more CPUs. The one or more computer subsystems can implement the processes described below in software using one or more processors, or can implement part or all of the processes in hardware, such as electronic circuits.

[0063] As an example, the computer system 800 includes a feature prediction unit 603, a shadow calculation unit 803, a brightness calculation unit 804, and an image synthesis unit 805. The feature prediction unit 603, the shadow calculation unit 803, the brightness calculation unit 804, and the image synthesis unit 805 may be realized virtually by software, or may be realized by hardware such as an electronic circuit. Furthermore, the computer systems 600 and 800 may be the same computer system, or may be independent computer systems.

[0064] The computer system 800 receives a trained model 801, a low-quality image 602, and synthesis parameters 802. The feature prediction unit 603 uses the trained model 801 to estimate, from the low-quality image 602, a structural feature 604 and a material feature 605 of the sample 112 in the high-quality image corresponding to the low-quality image 602.

[0065] A shading calculation unit 803 calculates shading data based on the structural feature amount 604 and the synthesis parameters 802. A brightness calculation unit 804 calculates gradation data based on the material feature amount 605 and the synthesis parameters 802.

[0066] The image synthesis unit 805 generates a synthesized high-quality image 806 based on the shadow data calculated by the shadow calculation unit 803 and the gradation data calculated by the brightness calculation unit 804, and outputs (stores) it.

[0067] The trained model 801 is a model that has learned the correspondence between the low-quality image 602 and the high-quality image 606 in the training process described with reference to Figures 6 and 7. Therefore, the trained model 801 can estimate, from the low-quality image 602 that is the target of image synthesis, the structural feature quantity 604 and material feature quantity 605 of the sample in the corresponding high-quality image.

[0068] The trained model 801 estimates structural features and material features based on the correspondence between the low-quality images and high-quality images used in the training process. Therefore, it is desirable that the low-quality images 602 input in the image synthesis process have the same imaging conditions (number of frames, pixel resolution, etc.) as the low-quality images used in the training process.

[0069] Furthermore, through the above-mentioned learning process, the trained model 801 can estimate a normal map of the sample on the corresponding high-resolution image from a combination of multiple shadow images contained in the low-resolution image 602.Furthermore, the trained model 801 can estimate predicted images (predicted BSE image, predicted SE image) of the corresponding direct-above image of the high-resolution image from the direct-above image contained in the low-resolution image 602 (SEM image (direct-above image 405) generated by synthesizing the BSE image 404 from the BSE image 401, and SEM image (direct-above image 406) corresponding to the SE image).

[0070] The synthesis parameters 802 are two or more types of parameters that specify the shading and brightness of the synthesized high-quality image 806. The synthesis parameters 802 are parameters that are input by the user via a GUI (described later), and include a three-dimensional unit vector that indicates the direction of the detector that acquires the signal, and a mixing ratio of the predicted BSE image and the predicted SE image.

[0071] An example of a method for calculating shading data (shading map) from the structural feature 604 and the synthesis parameters 802 in the shading calculation unit 803 is to calculate the dot product of the normal map estimated as the structural feature 604 and the three-dimensional unit vector in the detector direction specified by the synthesis parameters 802.

[0072] Furthermore, an example of a method for calculating gradation data (luminance map) from the material feature 605 and the synthesis parameters 802 in the luminance calculation unit 804 is a method in which the predicted BSE image and the predicted SE image estimated as the material feature 605 are synthesized at a mixture ratio specified by the synthesis parameters 802.

[0073] Furthermore, the image synthesis unit 805 can generate a synthesized high-quality image 806 from the shadow map calculated by the shadow calculation unit 803 and the brightness map calculated by the brightness calculation unit 804 by multiplying the shadow map by the brightness map. This allows the brightness map to be synthesized with the shadow information (intensity and direction) contained in the shadow map.

[0074] The file format for saving the composite high-quality image 806 may be an image file format such as TIFF, GIF, or PNG, or may be a binary file format.

[0075] 9 is a flowchart illustrating the procedure of the image synthesis processing according to this embodiment. Note that the subject of operation of each step is the computer system 800. As an example, the computer system 800 can be configured to load a program for realizing the image synthesis processing (image synthesis program) from a storage device (not shown; this may be an internal memory of the computer system 800 or an external storage device) and execute each step.

[0076] (i) Step S901 The computer system 800 receives input of the trained model 801 created by the procedure illustrated in FIG. 7 and the low-quality image 602, and sets the trained model 801 in the feature prediction unit 603.

[0077] (ii) Step S902 The computer system 800 uses the trained model 801 in the feature prediction unit 603 to estimate the structural feature 604 and the material feature 605 from the low-quality image 602.

[0078] To estimate the structural feature 604, shading information corresponding to the structural characteristics in the low-quality image 602 is used. For example, this is SEM image 401 to SEM image 404 shown in FIG. 4. Furthermore, to estimate the material feature 605, a directly above image of the sample 112 in the low-quality image 602 is used. For example, this is SEM image 405, which is a composite image of SEM images 401 to 404, and SEM image 406, which is a directly above image of the sample 112, shown in FIG. 4. This information is the same as the information used to train the machine learning model.

[0079] (iii) Step S903: The computer system 800 receives synthesis parameters input by the user via a GUI (see FIG. 11).

[0080] (iv) Step S904 The computer system 800 calculates shadow data in accordance with the structural feature amount 604 and the synthesis parameters, and calculates gradation data in accordance with the material feature amount 605 and the synthesis parameters.

[0081] (v) Step S905 The computer system 800 generates a composite high-quality image based on the shadow data and gradation data calculated in step S904.

[0082] (vi) Step S906: The computer system 800 determines whether to continue the image synthesis process based on the synthesized high-quality image generated in step S905. If it is determined that the image synthesis process should be continued (Yes in step S906), the process proceeds to step S903. In this case, in step 903, the computer system 800 accepts new synthesis parameters input by the user and executes the processes of steps 904 to S906 again. On the other hand, if it is determined that the image synthesis process should be terminated (No in step S906: for example, if the desired image has been obtained), the process proceeds to S907.

[0083] Whether or not to continue the image synthesis process can be determined by, for example, the user visually evaluating the generated high-quality synthesized image on the GUI screen, and determining that the image synthesis process is complete if an image of the desired quality has been obtained (by inputting an instruction to complete the image synthesis process into the computer system 800).The user evaluates the created high-quality synthesized image from the perspective of the visibility of circuit patterns and defects, and determines whether the image satisfies their requirements.

[0084] (vii) Step S907 The computer system 800 stores the synthesized high-quality image in a storage device (not shown), and the image synthesis process ends.

[0085] <Application Example of Image Combining Processing> Fig. 10 is a diagram showing an example in which the image combining processing according to this embodiment is applied to an SEM image having a particle defect. In Fig. 10, a shadow image 1001 represents a low-quality image or a high-quality image acquired, for example, by the shadow image detector 202 in the scanning electron microscope 10 illustrated in Figs. 1 and 2. A direct-above image 1002 represents a low-quality image or a high-quality image acquired, for example, by the upper detector 115 in the scanning electron microscope 10. A shadow map 1003 represents a shadow map calculated from a normal map estimated as the structural feature 604 and a three-dimensional unit vector in the detector direction specified by the combination parameter 802. A combined high-quality image 1004 represents an image obtained by combining the shadow map 1004 with a brightness map, which is a predicted SE image estimated from the direct-above image 1002 of the low-quality image.

[0086] When creating a composite image for defect review, depending on the material of the sample and the type of defect, it may be desirable to combine the surface information (material information) of the direct-above image 1002 and the shading information (unevenness information) of the shaded image 1001. As described above, when combining images, the direct-above image 1002 and the shaded image 1001 are generally combined at an arbitrary blending ratio. However, by adding the two images together, the surface information of the direct-above image 1002 and the shading information of the shaded image 1001 are averaged (by multiplying by the blending ratio), resulting in a reduction in the amount of information compared to before combination. On the other hand, by applying the image combination processing according to this embodiment, it is possible to combine only the shaded map representing the shading information of the shaded image 1001 with the brightness map representing the surface information of the direct-above image 1002, thereby creating a composite image that combines the surface information of the direct-above image 1002 and the shading information of the shaded image 1001. In other words, it is possible to prevent the loss of both the surface information and the shading information during combination.

[0087] 11 is a diagram showing an example of the configuration of a GUI screen 1100 that is displayed on the screen of a display used by a user when executing image synthesis processing in the image processing system 80 according to this embodiment. On the GUI screen 1100, the user can adjust synthesis parameters 802 that set brightness and shading while checking the created synthesized high-quality image.

[0088] The GUI screen 1100 is configured by, for example, a composite image display unit 1101, a shading operation unit 1102, and a brightness operation unit 1103. In this embodiment, when adjusting the composite parameters 802 on the GUI screen 1100, a normal map, a predicted BSE image, and a predicted SE image are estimated in advance using a low-quality image 602 of the sample to be composited and a trained model 801. From these normal map, predicted BSE image, and predicted SE image, a composite high-quality image 806 is generated according to the composite parameters 802 set by the shading operation unit 1102 and the brightness operation unit 1103.

[0089] The user sets the detector direction (orientation) using a circular slider on the screen using the shadow operation unit 1102. The shadow calculation unit 803 generates a shadow map (shadow data) by multiplying a three-dimensional unit vector calculated from the detector direction set by the user by the normal map.

[0090] The user also inputs a mixing ratio α of the predicted BSE image and the predicted SE image using a brightness operation unit 1103. The brightness calculation unit 804 generates a brightness map (grayscale data) by calculating a weighted average of the predicted BSE image and the predicted SE image according to the mixing ratio α input by the user. The composite image display unit 1101 displays a composite high-quality image obtained by multiplying the above-mentioned shadow map and the above-mentioned brightness map.

[0091] <Modifications> The technology of the present disclosure is not limited to the above-described embodiment and implementation examples, and includes various modifications. Examples of modifications include the following.

[0092] (i) In this embodiment, the plurality of low-quality images and the plurality of high-quality images have variations of the SEM image exemplified in Fig. 4 due to differences in the positions of detectors such as the shadow image detector 201 to the shadow image detector 204 illustrated in Fig. 2 and differences in the types of signals (BSE, SE) acquired. Note that another method for acquiring SEM images with a plurality of variations is, for example, a method such as tilt SEM, in which the tilt angle of the sample stage 113 or the irradiation angle of the electron beam 107 with respect to the sample 112 is changed to acquire SEM images with variations in the appearance of the sample depending on the relative relationship of the irradiation angle on the sample.

[0093] (ii) In the above embodiment, as a method for acquiring a shadow image, a method for acquiring BSEs using a plurality of shadow image detectors as exemplified in Fig. 2 and generating an SEM image has been described. However, the present invention is not limited to this method, and a shadow image generated from SEs or a signal in which SEs and BSEs are mixed may be used in a configuration similar to or different from the configuration of the scanning electron microscope 10 and the upper detector 115 and the lower detector 117 exemplified in Figs. 1 and 2.

[0094] (iii) To realize the technology of the present disclosure, it is not necessary to have all of the constituent elements of the above-described embodiments. For example, it is possible to replace part of the configuration shown in one drawing with the configuration shown in another drawing, or it is also possible to add the configuration shown in another drawing to the configuration of one drawing. Furthermore, it is possible to add, delete, or replace part of the configuration of this embodiment with other configurations.

[0095] <Summary> This embodiment proposes a learning process that trains a machine learning model to generate a trained model, and a synthetic high-quality image generation process that applies a low-quality image to the trained model to generate a high-quality image.

[0096] (i) Regarding the Learning Process This embodiment proposes generating a trained model for converting a first quality image into a second quality image by training a machine learning model using a first quality image (low quality image) and a second quality image (high quality image) of the target sample 112. More specifically, the computer system 600 receives at least one machine learning model 601, a plurality of low quality images 602, and a plurality of high quality images 606 (high quality images acquired at the same position and with the same FoV as the low quality images 602 on the sample 112). Next, the computer system 600 applies data of the plurality of low quality images 602 to the machine learning model 601, and estimates structural features 604 and material features 605 of a high quality image 606 corresponding to the plurality of low quality images 602. Furthermore, the computer system 600 causes the shade comparison unit 607 to calculate at least one piece of first shade data based on the structural feature 604, and calculate at least one piece of second shade data from the multiple high-quality images 606, and compare these (obtain a first comparison result). The computer system 600 also causes the brightness comparison unit 608 to calculate at least one piece of first gradation data based on the material feature, and calculate at least one piece of second gradation data from the multiple high-quality images 606, and compare these (obtain a second comparison result). The computer system 600 then updates the parameters of the machine learning model based on the first and second comparison results. This makes it possible to generate a trained model for predicting high-quality images from low-quality images.

[0097] Here, the low-quality image and the high-quality image each include a secondary electron image (SE image) and a backscattered electron image (BSE image) acquired by irradiating a target sample with a charged particle beam in the charged particle beam device 10 (see FIG. 4 ). The structural feature 604 is estimated by applying the backscattered electron images 401 to 404 of the low-quality image 602 to a machine learning model 601. The structural feature 604 may include, for example, at least one of a normal map, a bump map, a height map, or a displacement map, which represent the layer structure or surface unevenness of the target sample 112. Furthermore, the material feature 605 is estimated by applying the secondary electron image 406 of the low-quality image 602 and a composite image 405 of the BSE image to the machine learning model 601. The material feature 605 may include, for example, a brightness map, which represents, as a gradation, differences in acquired signals due to the material of the target sample 112.

[0098] The learning process can be repeated until a desired result (a result with high estimation accuracy of a high-quality image) is obtained. Specifically, the computer system 600 completes learning of the machine learning model based on whether the first comparison result (difference value of shading data: loss value) and the second comparison result (difference value of brightness: loss value) are equal to or less than a predetermined threshold, and designates the model at that point as a trained model. If the loss value is greater than the threshold, the learning process is executed again based on the backpropagation method.

[0099] (ii) Composite Image Generation Process: Use of Trained Model This embodiment further proposes predicting and outputting a high-quality image (second-quality image) by applying a low-quality image (first-quality image) of the target sample 112 to the trained model 801. More specifically, the computer system 800 receives multiple low-quality images 602 (including SE images and BSE images) and synthesis parameters 802, applies the data of the low-quality images 602 to the trained model 801, and estimates the structural feature 604 and material feature 605 of the high-quality image corresponding to the low-quality image. Next, the computer system 800 calculates shading data based on the structural feature 604 and the synthesis parameters 802, and calculates gradation data based on the material feature 605 and the synthesis parameters 802. The computer system 800 then generates a composite image from the shading data and gradation data, and outputs the composite image as a prediction result of a high-quality image (composite high-quality image 806). In this way, the shadow information and brightness information are handled independently to generate the synthesized high-quality image 806, so that it is possible to predict (synthesize) a high-quality image without losing the surface information of the SE image and the shadow information of the BSE.

[0100] The structural feature 604 is estimated by applying the backscattered electron images 401 to 404 of the low-quality image 602 to the trained model 801. The structural feature 604 can include, for example, at least one of a normal map, a bump map, a height map, or a displacement map, which represent the layer structure or surface unevenness of the target sample 112. Furthermore, the material feature 605 is estimated by applying the secondary electron image 406 of the low-quality image 602 and the composite image 405 of the BSE image to the trained model 801. The material feature 605 can include, for example, a brightness map that represents, as a gradation, differences in acquired signals due to the material of the target sample 112.

[0101] Furthermore, the synthesis parameters 802 include parameters that specify shading and brightness in the synthesized high-quality image 806. For example, the parameter that specifies shading is a three-dimensional unit vector that represents the direction of the detectors (lower detectors 201 to 204) that acquire the signals of the backscattered electron image. Also, the parameter that specifies brightness is a mixing ratio α of the shading data and the grayscale data.

[0102] The computer system 800 receives the user's confirmation of the composite high-quality image and determines whether to execute the composite image generation process again. Specifically, the computer system 800 receives, in response to the user's input, composite parameters including parameter values ​​different from those used in the previous composite image generation process, and executes the composite image generation process again using the re-accepted composite parameters (see FIG. 9 ).

[0103] (iii) The functions described in the present embodiment can also be realized by software program code. In this case, a storage medium on which the program code is recorded is provided to a system or device, and the computer (or CPU or MPU) of the system or device reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the above-described embodiment, and the program code itself and the storage medium on which it is stored constitute the present disclosure. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0104] Furthermore, an operating system (OS) running on a computer may perform some or all of the actual processing based on instructions in the program code, and the functions of the above-described embodiments may be realized by this processing.Furthermore, after the program code is read from a storage medium and written to a memory on the computer, a CPU of the computer may perform some or all of the actual processing based on instructions in the program code, and the functions of the above-described embodiments may be realized by this processing.

[0105] Furthermore, the program code of the software that realizes the functions of the embodiment may be distributed via a network and stored in a storage means such as a hard disk or memory of the system or device, or in a storage medium such as a CD-RW or CD-R, so that when used, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage means or storage medium.

[0106] The processes and techniques described herein are not inherently related to any particular device and can be implemented by a combination of components. Various types of general-purpose devices can also be added. A dedicated device may be constructed to perform the functions of the present embodiment. Various functions can also be formed by appropriately combining multiple components disclosed in the present embodiment.

[0107] Although specific embodiments are described in this disclosure, they are in all respects for the purpose of explanation (understanding the technology of the present disclosure) and not for the purpose of limitation. Those skilled in the art will recognize that there are many combinations of hardware, software, and firmware suitable for implementing the technology of the present disclosure. For example, the software described can be implemented in a wide variety of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, Java (registered trademark), etc.

[0108] Furthermore, in the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0109] In addition, other implementations of the present disclosure will be apparent to those skilled in the art from consideration of the present embodiments. The specification and examples are exemplary only, with the scope and spirit of the technology of the present disclosure being indicated by the following claims.

[0110] 10 Scanning electron microscope 60, 80 Image processing system 101 Imaging unit 107 Electron beam 301, 112 Sample 113 Sample stage 114 Secondary electrons (SE) 115 Secondary electron detector (upper detector) 116 Backscattered electrons (BSE) 117 Backscattered electron detector (lower detector) 201 to 204 Shadow image detector (lower detector) 401 to 404 Shadow image (backscattered electron image) 405 Directly above image (composite image of BSE) 406, 1002 Directly above image (SE image) 102, 600, 800 Computer system 601 Machine learning model 602 Low-quality image 603 Feature prediction unit 604 Structural feature 605 Material feature 606 High-quality image 607 Shadow comparison unit 608 Brightness comparison unit 609 Model update unit 610 Updated model 801 Trained model 802 Synthesis parameters 803 Shading calculation unit 804 Brightness calculation unit 805 Image synthesis unit 806 Synthesized high-quality image 1001 Shading image 1003 Shading map 1004 Synthesized high-quality image

Claims

1. An image processing system that generates a trained model for converting a first-quality image into a second-quality image with higher image quality than the first-quality image by training a machine learning model using multiple images acquired under different imaging conditions for a target sample, A storage device that holds a learning processing program for generating the aforementioned trained model, The system includes a computer that reads the learning processing program from the memory device and performs a learning process to generate the trained model, The aforementioned computer, A process that accepts at least one machine learning model, multiple first-quality images, and multiple second-quality images of higher quality than the first-quality images, A process of applying the data of the plurality of first image quality images to at least one machine learning model and estimating the structural features and material features of the second image quality image corresponding to the plurality of first image quality images, A process for calculating at least one first shaded data based on the aforementioned structural features, A process for calculating at least one second shading data from the plurality of second image quality images, A first comparison process that compares the at least one first shaded data with the at least one second shaded data, A process for calculating at least one first grayscale data based on the aforementioned material characteristics, A process for calculating at least one second grayscale data from the plurality of second image quality images, A second comparison process that compares the at least one first grayscale data with the at least one second grayscale data, A process to update the parameters of at least one machine learning model based on the first comparison result from the first comparison process and the second comparison result from the second comparison process, An image processing system that performs the following.

2. In claim 1, An image processing system in which each of the plurality of first image quality images and the plurality of second image quality images includes at least one secondary electron image and at least one backscattered electron image, each obtained by irradiating the target sample with a charged particle beam in a charged particle beam apparatus.

3. In claim 2, The computer is an image processing system that applies the at least one backscattered electron image of the first image quality to the at least one machine learning model to estimate the structural features of the second image quality.

4. In claim 2, The computer is an image processing system that applies the at least one secondary electronic image of the first image quality to the at least one machine learning model to estimate the material features of the second image quality.

5. In claim 1, An image processing system in which the plurality of first image quality images and the plurality of second image quality images each include at least one shaded image.

6. In claim 1, An image processing system in which the structural features include at least one of a normal map, bump map, height map, or displacement map that represents the layer structure or surface irregularities of the target sample.

7. In claim 1, The aforementioned material feature quantity includes a brightness map that represents the difference in acquired signals due to the material of the target sample as grayscale, in an image processing system.

8. In claim 1, The computer further performs an image processing system which completes the training of at least one machine learning model based on whether the first comparison result and the second comparison result are below a predetermined threshold, and sets the model at that point as the trained model.

9. An image processing system that predicts and outputs a second image with higher image quality than the first image by applying a first image quality of a target sample to a trained model, A storage device that holds an image synthesis program for synthesizing the second image quality using the trained model, The system includes a computer that reads the image synthesis program from the storage device and performs an image synthesis process to synthesize the second image quality, The aforementioned computer, A process that accepts multiple first-quality images and synthesis parameters. The process involves applying the data of the first image quality to the trained model and estimating the structural and material features of the second image quality corresponding to the first image quality. A process for calculating at least one shaded data based on the structural features and the synthesis parameters, A process for calculating at least one grayscale data based on the material characteristics and the synthesis parameters, A process that generates a composite image from the at least one shading data and the at least one grayscale data, and outputs the composite image as the prediction result of the second image quality image, An image processing system that performs the following.

10. In claim 9, An image processing system in which each of the plurality of first image quality images includes at least one secondary electron image and at least one backscattered electron image, obtained by irradiating the target sample with a charged particle beam in a charged particle beam apparatus.

11. In claim 10, The computer is an image processing system that applies the at least one backscattered electron image of the first image quality to the trained model to estimate the structural features, and applies the at least one secondary electron image of the first image quality to the trained model to estimate the material features of the second image quality.

12. In claim 10, The image processing system includes, as the synthesis parameters, parameters that specify shading and brightness in the synthesized image.

13. In claim 12, The parameter specifying the shading is a three-dimensional unit vector representing the direction of the detector that acquires the signal of the backscattered electron image. An image processing system in which the parameter for specifying the brightness is the mixing ratio of the at least one shading data and the at least one grayscale data.

14. In claim 9, The computer is an image processing system that receives the user's confirmation result of the composite image and determines whether to execute the composite image generation process again.

15. In claim 14, The computer is an image processing system that accepts new synthesis parameters, each containing different parameter values ​​from those used in the previous synthesis image generation process, and then re-executes the synthesis image generation process using the newly accepted synthesis parameters.

16. An image processing method in which a computer trains a machine learning model using multiple images of a target sample acquired under different imaging conditions to generate a trained model for converting a first image quality image into a second image quality image with higher resolution than the first image quality image, It accepts at least one machine learning model, multiple first-resolution images, and multiple second-resolution images of higher resolution than the first-resolution images. Applying the data of the plurality of first image quality images to the at least one machine learning model, and estimating the structural features and material features of the second image quality images corresponding to the plurality of first image quality images, Based on the aforementioned structural features, at least one first shaded data point is calculated, To calculate at least one second shading data from the aforementioned plurality of second image quality images, The first comparison result is obtained by comparing the at least one first shaded data and the at least one second shaded data. Based on the aforementioned material characteristics, at least one first grayscale data is calculated, To calculate at least one second grayscale data from the plurality of second image quality images, The process involves comparing the at least one first grayscale data with the at least one second grayscale data to obtain a second comparison result. Based on the first comparison result and the second comparison result, the parameters of the at least one machine learning model are updated. Image processing methods, including those mentioned above.

17. An image processing method in which a computer applies a first image quality of a target sample to a trained model to predict and output a second image quality that is of higher quality than the first image quality, It accepts multiple first-quality images and synthesis parameters. Applying the data of the first image quality to the trained model, and estimating the structural and material features of the second image quality corresponding to the first image quality, To calculate at least one shaded data based on the structural features and the synthesis parameters, Calculating at least one grayscale data based on the material characteristics and the synthesis parameters, A composite image is generated from the at least one shading data and the at least one grayscale data, and the composite image is output as the prediction result of the second image quality image. Image processing methods, including those mentioned above.