Computer system, sample observation method, and program

Abstract

Provided are techniques that, regarding technology such as a sample observation device, make it possible to implement suitable image quality quantification. A computer system in a sample observation device for observing a sample is provided, the computer system comprising at least one processor and at least one memory, wherein the processor determines a plurality of evaluating partial regions Si with respect to an image 201 obtained by imaging a sample, and calculates an image quality evaluation value Ei for each evaluating partial region Si on the basis of the plurality of evaluating partial regions Si, to thereby calculate a plurality of image quality evaluation values Ei for performing evaluation regarding a plurality of different image qualities.

Description

Technical Field

[0001] The present disclosure relates to sample observation technology, and relates to a sample observation apparatus and the like having a function of observing defects, abnormalities, etc. (sometimes collectively referred to as defects) and circuit patterns in a sample such as a semiconductor wafer.

Background Art

[0002] In the manufacture of semiconductor wafers, it is important to quickly start up the manufacturing process and promptly shift to a high-yield mass production system in order to ensure profitability. For this purpose, various inspection devices, observation devices, measurement devices, etc. are introduced into the production line. The semiconductor wafer as a sample is inspected for defects, for example, in an inspection device. The inspection device outputs coordinate information indicating the position and part of the defect in the sample as defect candidate coordinates. The output defect candidate coordinates are supplied to a sample observation apparatus which is an observation device for observing the defect.

[0003] The sample observation apparatus captures an image of a defect candidate on the wafer surface with high resolution based on the defect candidate coordinates and outputs the captured image. As the sample observation apparatus, an observation apparatus using an optical microscope or an observation apparatus using a scanning electron microscope (SEM) is widely used. These apparatuses are, in other words, charged particle beam apparatuses having a mechanism for irradiating a charged particle beam, and have a function of performing imaging, measurement, observation, evaluation, inspection, etc. (sometimes collectively referred to as observation) of a sample.

[0004] In semiconductor wafer mass production lines, automation of observation work using observation devices is desired. For automation in mass production lines, observation devices may be equipped with functions for automatic defect image acquisition, or in other words, automatic defect review (ADR), which automatically collects images of defect locations within the sample, and automatic defect image classification, or in other words, automatic defect classification (ADC), which automatically classifies the collected defect images. These functions make it possible to automatically obtain classified defect images.

[0005] The defect candidate coordinates output by the inspection device contain errors. Because there is a difference between the coordinate system of the defect candidate coordinates in the inspection device and the coordinate system in the observation device, the resulting discrepancy may cause the defect candidate to not be found even when the defect candidate coordinates are imaged by the observation device. Therefore, in ADR, the observation device images a wide field of view centered on the defect candidate coordinates and detects the defect candidate area from the image obtained from this imaging. The observation device then images the obtained defect candidate area at high magnification and high resolution and outputs the captured image as the observation image.

[0006] Furthermore, observation images of semiconductor wafers may undergo image quality enhancement (in other words, high-resolution processing) using an image quality enhancement engine to improve visibility.

[0007] As an example of prior art, Japanese Patent Publication No. 2012-142299 (Patent Document 1) can be cited. Patent Document 1 describes a technique for improving the accuracy of image processing by obtaining high-quality images with reduced noise components from images of a sample obtained by imaging with a scanning charged particle microscope. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2012-142299 [Overview of the project] [Problems that the invention aims to solve]

[0009] Patent Document 1 describes a technical example of a process for improving the image quality of images taken of semiconductor wafers. Patent Document 1 describes calculating an image degradation function based on imaging conditions and sample information, and obtaining a high-quality image by performing an image restoration process using the degradation function.

[0010] Traditionally, the quality of observation images obtained using imaging functions and image enhancement engines has been judged by the user's visual evaluation. This has resulted in challenges such as increased workload for users and ambiguity in human judgment criteria.

[0011] Therefore, if the image quality of images captured from wafers can be quantified in observation devices and other systems, these problems can be solved.

[0012] However, when quantifying image quality, the required image quality, or in other words, the evaluation values ​​or parameter values ​​for image quality, are not necessarily singular but may consist of multiple factors. For example, the image quality required for suitable observation or inspection of a wafer circuit pattern may involve multiple elements or parameter values, such as sharpness, defect visibility, and noise level (or, in other words, the degree of noise suppression).

[0013] Conventional methods for quantifying image quality do not consider the multiple elements and parameter values ​​of such image quality for images captured by microscopes such as SEMs (in other words, charged particle beam devices and imaging devices). In images of wafers, it is thought that the optimal region for calculating image quality evaluation values ​​may differ depending on the elements and parameter values ​​of such image quality.

[0014] The purpose of this disclosure is to provide a technology that enables suitable image quality quantification with respect to the above-mentioned sample observation device and other technologies. [Means for solving the problem]

[0015] A typical embodiment of this disclosure has the following configuration. The computer system of one embodiment is a computer system in a sample observation apparatus for observing a sample, the computer system having one or more processors and one or more memories, the processor determines a plurality of evaluation sub-regions for an image captured of the sample, and calculates a plurality of image quality evaluation values ​​for evaluating a plurality of different image quality values ​​by calculating an image quality evaluation value for each evaluation sub-region based on the plurality of evaluation sub-regions. [Effects of the Invention]

[0016] According to a representative embodiment of this disclosure, suitable image quality quantification can be achieved with respect to the above-mentioned sample observation device and other technologies. Other issues, configurations, and effects will be shown in the embodiments for carrying out the invention. [Brief explanation of the drawing]

[0017] [Figure 1] A diagram showing the configuration of the system including the sample observation device of Embodiment 1. [Figure 2] A diagram showing the processing flow and functional block configuration of the sample observation apparatus and method of Embodiment 1. [Figure 3] This figure shows an example of image quality evaluation value information and image quality evaluation result data in Embodiment 1. [Figure 4] A diagram showing an example of image quality evaluation values ​​in Embodiment 1. [Figure 5] A diagram showing an example of image quality evaluation values ​​in Embodiment 1. [Figure 6] A diagram illustrating step S6 of the setup in Embodiment 1. [Figure 7] A diagram showing an example of the processing in steps S2 and S3 in Embodiment 1. [Figure 8] This figure shows an example of the correspondence between the evaluation subregion Si and the image quality evaluation value Ei in Embodiment 1. [Figure 9]Diagram showing the case where the evaluation partial region Si overlaps or does not overlap in Embodiment 1. [Figure 10] Diagram showing a detailed processing example of step S2 in Embodiment 1. [Figure 11] Diagram showing a processing example of classification (step S21) using design data in Embodiment 1. [Figure 12] Diagram showing a processing example of classification (step S21) using imaging conditions in Embodiment 1. [Figure 13] Diagram showing a processing example of determining the evaluation partial region Si based on classification (step S22) in Embodiment 1. [Figure 14] Diagram showing an example of the evaluation partial region Si for each image quality evaluation value Ei in Embodiment 1. [Figure 15] Diagram showing an example of the classification criteria for the classification divided regions in Embodiment 1. [Figure 16] Diagram showing a processing example and a GUI example for determining the internal parameters of step S63 in Embodiment 1. [Figure 17] Diagram showing the internal parameters of the image quality evaluation value Ei and a data example of the elemental image quality evaluation values in Embodiment 1. [Figure 18] ] Diagram showing a processing example of calculating the image quality evaluation value using the elemental image quality evaluation values in Embodiment 1. [Figure 19] Diagram showing an example of the machine learning model used in step S3 in Embodiment 1. [Figure 20] Diagram showing a processing example of calculating the comprehensive evaluation value C in step S3 in Embodiment 1. [Figure 21] Diagram showing a flow and a functional block configuration including an image quality improvement process in Embodiment 1. [Figure 22] Diagram showing the determination of the parameter P of the image quality improvement process using the region target value in Embodiment 1. [Figure 23] Diagram showing the determination of the parameter P of the image quality improvement process using the comprehensive evaluation value C in Embodiment 1. [Figure 24] Diagram showing the determination of the parameter P when using a rule - based image quality improvement engine in Embodiment 1. [Figure 25] This figure illustrates the determination of the parameter P when using a machine learning-type image quality enhancement engine in Embodiment 1. [Figure 26] This figure shows an example of a GUI screen related to image quality improvement processing in Embodiment 1. [Figure 27] This figure shows an example of a GUI screen related to the correction of the evaluation subregion in Embodiment 1. [Figure 28] A diagram showing a flow in Embodiment 1, including steps such as correction of the evaluation subregion. [Figure 29] This figure shows an example of a GUI screen related to the benchmark steps in Embodiment 1. [Figure 30] This figure shows an example of the defect detection step in Embodiment 1. [Figure 31] This figure shows an example of the processing steps for shape measurement in Embodiment 1. [Figure 32] A diagram showing a flowchart in Embodiment 1, including the step of determining imaging conditions using image quality evaluation values. [Figure 33] This figure shows an example of the processing steps for monitoring the device using image quality evaluation values ​​in Embodiment 1. [Modes for carrying out the invention]

[0018] The embodiments of this disclosure will be described in detail below with reference to the drawings. In the drawings, the same parts are generally denoted by the same reference numerals, and repeated descriptions are omitted. In the drawings, the representation of components may not show their actual location, size, shape, extent, etc., in order to facilitate understanding of the invention.

[0019] In explanations, when describing program-based processing, the focus may sometimes be on the program, functions, or processing units. However, the core hardware component is the processor, or a controller, device, computer, or system composed of such a processor. The computer, using its processor, executes processing according to the program read into memory, utilizing resources such as memory and communication interfaces as appropriate. This realizes the specified functions and processing units. The processor is composed of semiconductor devices such as CPUs / MPUs and GPUs. Processing is not limited to software program processing; it can also be implemented using dedicated circuits. FPGAs, ASICs, CPLDs, etc., can be used as dedicated circuits.

[0020] The program may be pre-installed as data on the target computer, or it may be distributed as data to the target computer from the program source. The program source may be a program distribution server on a communication network, or a non-transient computer-readable storage medium, such as a memory card or disk. The program may consist of multiple modules. The computer system may consist of multiple devices. The computer system may consist of a client-server system, a cloud computing system, an IoT system, etc. Various types of data and information are composed of structures such as tables and lists, but are not limited to these. Representations such as identification information, identifiers, IDs, names, and numbers are interchangeable.

[0021] <Embodiment 1> The sample observation apparatus and method of Embodiment 1 will be described using Figures 1 to 33. The sample observation method of Embodiment 1 is a method having steps performed by the sample observation apparatus of Embodiment 1. The sample observation apparatus 1 of Embodiment 1 shown in Figure 1, etc., is an apparatus that has the function of observing a semiconductor wafer 10, which is a sample 10.

[0022] The sample observation device 1 of Embodiment 1 uses an optical microscope or SEM as a microscope or imaging device to capture an image of the wafer 10, which is the sample 10. Embodiment 1 will be described in the case where the sample observation device 1 is equipped with an SEM 2 (Figure 1). The sample observation device 1 of Embodiment 1 is equipped with a computer system 3 (in other words, a controller or control device) that controls the SEM 2. Embodiment 1 will be described in the case where the computer system 3 calculates the image quality evaluation value Ei, etc.

[0023] The computer system 3 of the sample observation device 1 includes an imaging mechanism that controls the SEM 2 to acquire an image 201 (Figure 2), which is an image captured (in other words, an observation image), and an observation mechanism that observes the image 201, such as a defect detection mechanism that detects defective areas in the observation image. Each mechanism is implemented by a control processor 102 (Figure 1), etc.

[0024] The sample observation method of Embodiment 1, shown in Figure 2, etc., includes the steps of: step S1, obtaining an image 201 by imaging a semiconductor wafer 10, which is the sample 10, with a SEM2 (Figure 1); and step S5, observing the semiconductor wafer 10, which is the sample 10, using the captured image 201. The sample observation method of Embodiment 1 includes step S3, which calculates a plurality of image quality evaluation values ​​Ei related to the captured image 201 for the purpose of observation. The sample observation method of Embodiment 1 includes step S2, which determines an evaluation partial region Si for calculating each image quality evaluation value Ei from the captured image 201 obtained by the SEM2 (Figure 1). Then, using the evaluation partial region Si, the sample observation method of Embodiment 1 calculates a plurality of image quality evaluation values ​​Ei as a plurality of elements constituting the image quality of the image 201 in step S3. Based on the image quality evaluation values ​​Ei obtained as a result of steps S2 and S3, it is possible to select an image 201 suitable for observation (step S5), that is, an image 201 with the highest possible image quality, from the captured image 201 as the observation image.

[0025] Embodiment 1 describes an example where the sample observation apparatus 1 and method are used to observe and detect defects in a semiconductor wafer 10 as a sample 10. Step S5 in Figure 2 is a predetermined processing operation, which involves performing an observation operation on the observation image, but is not limited to an observation operation. Furthermore, Embodiment 1 describes a case where the sample observation apparatus 1 refers to defect detection information 8 (including the aforementioned defect candidate coordinates) generated and output by an external defect inspection apparatus 5 (Figure 1), and uses those defect candidate coordinates as the imaging position to capture an image 201 (Figure 2).

[0026] Furthermore, Embodiment 1 describes an example in which SEM2 (Figure 1) is used as the microscope or imaging device for the sample observation device 1. However, it is not limited to this, and other types of microscopes and imaging devices can also be applied, such as optical microscopes and charged particle beam devices that use charged particles such as ions. Processing in the system including the sample observation device 1 is performed using an appropriate image that matches the type of microscope / imaging device used and imaging information (e.g., imaging position, imaging magnification, etc.).

[0027] Furthermore, in Embodiment 1, the image 201 (Figure 2) captured by the SEM2 is basically described as an image captured from a direction perpendicular to the wafer top surface, in other words, a top-view image. However, it is not limited to this, and images captured from a direction oblique to the vertical direction of the wafer top surface, in other words, a tilt direction, can also be similarly applied to this image 201.

[0028] [Sample observation device] Figure 1 shows the configuration of a system including the sample observation device 1 of Embodiment 1. The sample observation device 1 is broadly composed of an SEM2 (scanning electron microscope) and a computer system 3. The SEM2 is an example of an imaging device or microscope, in other words, a charged particle beam device. The computer system 3 is a higher-level control device for the SEM2, in other words, a controller. A specific example of the sample observation device 1 is a review SEM with the aforementioned ADR function. The computer system 3 is connected to the SEM2 via connection lines (in other words, signal lines, communication lines), etc. The computer system 3 is a device that has functions such as controlling the SEM2. The sample observation device 1 has at least the function of controlling the SEM2 to acquire images captured by the SEM2, and the function of observing the sample 10 based on the captured images.

[0029] The sample observation device 1 is equipped with the necessary functional blocks and various devices, but the drawing only illustrates some of the essential elements. In other words, the entire system, including the sample observation device 1 shown in Figure 1, is configured as a semiconductor inspection system, etc.

[0030] External devices such as a defect inspection device 5 and a defect classification device 6 are connected to the communication network 9 (e.g., LAN) as external devices to the sample observation device 1. In this example, defect detection information 8 is created in advance by the external defect inspection device 5 as a result of inspecting the wafer, which is the sample 10. The defect detection information 8 is information that includes the coordinates of candidate defects. The defect detection information 8 output from the defect inspection device 5 is stored in an external storage device 4 in advance, for example. However, the defect detection information 8 may also be stored in a database such as a server on the communication network 9. When observing, the computer system 3 reads and refers to the defect detection information 8 from the external storage device 4.

[0031] The defect classification device 6 is a device or system having the aforementioned ADC function. The defect classification device 6 performs ADC processing based on the data and information from the observation processing results by the ADR function of the sample observation device 1, and obtains the results of classifying defects and defect images. Note that the configuration is not limited to the example shown in Figure 1, and a configuration in which the defect classification device 6 is integrated with the sample observation device 1 is also possible.

[0032] Figure 1 shows a case where the higher-level control unit of SEM2 is composed of one computer system 3, but the higher-level control unit may be composed of multiple computer systems, such as multiple server devices.

[0033] Computer system 3 includes a control processor 102, a memory processor 103, an arithmetic processor 104, an input / output interface 105, a communication interface 107, a user interface control processor 106, and the like. These components are connected to a bus 114, enabling them to communicate and input / output with each other.

[0034] The control processor 102 controls the entire system. The control processor 102 is configured, for example, with a hardware circuit or a processor such as a CPU, MPU, or GPU. If the control processor 102 has a processor such as a CPU, the processor executes processing according to the program read from the memory processor 103. The control processor 102 implements various functions, for example, based on program processing. The arithmetic processor 104 is configured, for example, with a processor such as a CPU, MPU, or GPU and memory such as ROM or RAM. The arithmetic processor 104 performs calculations according to the program read from the memory processor 103. Note that the control processor 102 and the arithmetic processor 104 may be configured as a single unit.

[0035] The memory processor 103 is composed of a device that stores various information and data, including programs, and can be composed of a storage medium device equipped with, for example, a magnetic disk or semiconductor memory. The memory processor 103 may store data such as programs read from an external storage device 4 or a communication network 9. The memory processor 103 may also store defect detection information 8 read from an external storage device 4 or the like. The memory processor 103 may also store image data acquired from the SEM 2.

[0036] The input / output interface 105 is a device that implements interfaces with input devices, output devices, and an external storage device 4, and performs input and output of data and information to and from these devices. The computer system 3 is connected to, for example, the external storage device 4 through the input / output interface 105. Various programs and data may be stored in the external storage device 4. Image data and processing result information may also be stored in the external storage device 4.

[0037] The communication interface 107 is a device equipped with a communication interface compatible with a communication network 9 such as a LAN, and sends and receives data and information to and from the communication network 9. The computer system 3 is connected to the communication network 9 through the communication interface 107. The computer system 3 can connect to and communicate with external systems and devices via the communication network 9. For example, a defect inspection device 5 and a defect classification device 6 are connected to the communication network 9. Other examples of external devices include a database server and a manufacturing execution system (MES). The computer system 3 may also refer to sample design data (in other words, sample information) and manufacturing process information from external devices. Examples of design data include CAD data. Examples of manufacturing processes include etching and deposition.

[0038] The user interface control processor 106 is the part that provides and controls a user interface, including a graphical user interface (GUI), for inputting and outputting information and data with the user, or in other words, the operator. A user terminal 7 (in other words, a client terminal) may be connected to the computer system 3 as an input / output terminal through the user interface control processor 106. The user terminal 7 may be a device connected to a communication network 9. The user terminal 7 and other input / output devices may be integrated into the computer system 3. The user interface control processor 106 provides the user terminal 7 with data for a GUI-compatible screen (e.g., a web page). Other input / output devices, such as display devices, audio output devices, and operation devices, may be connected to the input / output interface 105 or the user interface control processor 106. The user interface control processor 106 may be integrated with a control processor 102, etc.

[0039] The configuration is not limited to the computer system 3 shown in Figure 1; it may have one or more computer systems. Although the computer system 3 in Figure 1 has multiple processors, any configuration with one or more processors is acceptable.

[0040] The user operates the user terminal 7 or other input / output devices to input information, such as instructions and settings, to the sample observation device 1, particularly the computer system 3, and to confirm information displayed on the screen, for example. The user terminal 7 may be, for example, a general-purpose PC. The user terminal 7 may have a built-in keyboard, mouse, display, etc., or these may be externally connected. The user terminal 7 may also be a remote terminal connected to a communication network 9, such as the Internet. The user interface control processor 106 creates screen data with a GUI and provides it to the user terminal 7 via communication, and the user terminal 7 displays the screen on its display.

[0041] The system including the sample observation device 1 may take the following form. The computer system 3 may be configured as a server in a client-server system, cloud computing system, IoT system, etc. The user terminal 7 may be configured as a client computer to this server. In the client-server system configuration, the user operates, for example, the user terminal 7, which is a client computer, and the user terminal 7 sends a request to the computer system 3, which is the server. The server receives the request and performs processing according to the request (e.g., imaging or observation processing). The server sends data of a screen (e.g., a web page) including the requested processing results, etc., as a response to the user terminal 7. The user terminal 7 receives the response data and displays the screen (e.g., a web page) on its display. The user can then check the processing results, etc., on that screen.

[0042] Furthermore, for example, computer system 3 or an external device may perform machine learning. Machine learning may require many computing resources. In this case, processing related to machine learning may be performed on a server group such as a cloud computing system. Alternatively, the functions may be shared between the server group and the client computer. When performing machine learning, computer system 3 trains a learning model using training images, etc., in the learning phase. Through training, the parameters of the learning model are adjusted. For example, a CNN (Convolutional Neural Network) can be applied as the learning model. In the estimation phase, computer system 3 inputs the target image into the trained learning model and obtains judgment result information, etc., as output, which is the result of estimation by the learning model. Note that the business operator and computer system that perform model learning and training in the learning phase and the business operator and computer system that perform estimation using the trained model in the estimation phase may be different entities.

[0043] [SEM] In Figure 1, the SEM2 is housed in a housing 101 and includes a stage 109, an electron gun 110, an electron lens (not shown), a deflector 112, and a detector 111, among other components. The stage 109 is, in other words, a sample stage on which the semiconductor wafer, which is the sample 10, is placed and held. The stage 109 is a mechanism that allows for movement in horizontal directions (X,Y directions in the figure) and vertical directions (Z direction), as well as rotation and tilting. The electron gun 110 irradiates the sample 10 on the stage 109 with an electron beam b1, in other words, a charged particle beam b1. The electron lens (not shown) focuses the electron beam b1 onto the surface of the sample 10. The deflector 112 scans the electron beam b1 on the surface of the sample 10. The detector 111 detects electrons and particles b2, such as secondary electrons and backscattered electrons, generated from the sample 10 by the irradiation of the electron beam b1, as electrical signals. In other words, the detector 111 detects the state of the surface of the sample 10 as an image. In this example, the detector 111 has multiple detectors at multiple locations.

[0044] In this example, the SEM control function in computer system 3 controls elements of SEM2 such as the stage 109, electron gun 110, deflector 112, and detector 111. Note that elements such as drive circuits for operating mechanisms like the stage 109 are not shown in the diagram.

[0045] SEM2 images the wafer 10, which is the sample 10, according to the set imaging conditions based on control from the computer system 3. The image signal detected as an electrical signal by the detector 111 of SEM2 is supplied to the computer system 3 through a connecting line. The detector 111 may have circuits such as an analog-to-digital conversion circuit inside or downstream of it. The computer system 3 acquires an image (image 201 in Figure 2) by processing the image signal supplied from the detector 111 with a control processor 102 or an arithmetic processor 104, etc. When imaging with SEM2, multiple images may be acquired by multiple detectors 111 equipped in SEM2. The computer system 3 may generate image 201 by processing the multiple acquired images. The computer system 3 stores the image data and information in a memory processor 103, etc. The computer system 3 performs observation processing on the image, for example, determining and detecting defective areas, and stores the processing result information in the memory processor 103, etc.

[0046] The sample observation device 1 in Figure 1 is equipped with a SEM 2 that images the sample 10 based on the defect candidate coordinates of the defect detection information 8 from the defect inspection device 5. Alternatively, the sample observation device 1 may perform similar operations using an external microscope or imaging device. The computer system 3 may control the external microscope or imaging device or acquire images from the external microscope or imaging device. This system may include multiple types of microscopes or imaging devices, such as an SEM and an optical microscope. The computer system 3 may appropriately select and use different microscopes or imaging devices. The overall system as shown in Figure 1 is not limited to a single sample observation device 1 configuration, but may also include one or more microscopes / imaging devices that capture images of the sample 10, and one or more computer systems or sample observation devices that perform processing such as observation of the captured images.

[0047] [Basic Processing Flow] The sample observation apparatus 1 and method of Embodiment 1 will be described using Figure 2. Figure 2 shows the basic processing flow performed by the computer system 3 (Figure 1) in the sample observation apparatus 1 and method of Embodiment 1, and has steps S1 to S7. The sample observation method of Embodiment 1 is a method for observing a sample 10 and has steps S2 and S3 for calculating the image quality evaluation value Ei. The computer system 3 in the sample observation apparatus 1 of Embodiment 1 performs processing such as calculating the image quality evaluation value Ei. The processing flow in Figure 2 also corresponds to the basic functional block configuration of the computer system 3 in Figure 1.

[0048] [Calculation of image quality evaluation value for observed images] Figure 2 illustrates an example of calculating the image quality evaluation value Ei in the sample observation apparatus 1 and method of Embodiment 1. In Figure 2, one of the features of the sample observation method of Embodiment 1 is that it comprises a region determination step (step S2) for determining a partial region Si for evaluation from an image 201 (also called an SEM image) of a semiconductor wafer 10, which is a sample 10, captured by SEM2, and an image quality evaluation value calculation step (step S3) for calculating a different image quality evaluation value Ei for each partial region Si for evaluation.

[0049] The following are challenges in calculating image quality evaluation values ​​for SEM images: Image features related to the image quality evaluation value of an SEM image are often located in specific regions of the SEM image. Therefore, if the entire SEM image is used as the calculation region for the image quality evaluation value, image features in regions that are not related to the image quality evaluation value, or have only a slight connection to it, will also affect the calculated image quality evaluation value. For example, when evaluating sharpness as an aspect of SEM image quality, we only want to evaluate the change in brightness near the edges of the circuit pattern on the wafer. If the entire SEM image is set as the calculation region, noise generated in the flat areas of the circuit pattern will affect the image quality evaluation value, i.e., the sharpness.

[0050] Therefore, in the sample observation apparatus and method of Embodiment 1, when there are multiple image quality evaluation values ​​Ei (in other words, parameter values, evaluation item values) as elements constituting image quality for the captured image 201, a specific region suitable for calculating each image quality evaluation value Ei is determined as the evaluation partial region Si (step S2). Then, the sample observation apparatus and method of Embodiment 1 calculates the image quality evaluation value Ei using the evaluation partial region Si (step S3).

[0051] In Figure 2, first, in step S1, which is the imaging step, the computer system 3 controls the imaging by the SEM 2 to acquire an image 201 (in other words, an image, target image, SEM image, etc.) for observation of the sample 10, which is the wafer 10. At this time, the image 201 is acquired using imaging conditions 205 (Figure 6), which will be described later.

[0052] Next, in step S2, which is a region determination step, the computer system 3 performs a region determination process on the captured image 201, thereby determining the evaluation subregions Si (i=1~N, N: number of subregions) for each of the multiple image quality evaluation values ​​Ei to be applied to the image 201. In this step S2, in addition to the image 201, imaging conditions 205 described later, design data 206 (Figure 6), and manufacturing process information may also be used as input information.

[0053] Next, in step S3, which is the image quality evaluation value calculation step, the computer system 3 uses the evaluation subregion Si corresponding to each image quality evaluation value Ei to be applied to the image 201 obtained in step S2 to calculate each different image quality evaluation value Ei (i=1~N, N: number of image quality evaluation values) for each evaluation subregion Si.

[0054] Next, in step S4, the computer system 3 stores the image quality evaluation value Ei calculated in step S3 in a memory resource (for example, the memory processor 103 in Figure 1) and displays it to the user (Figure 1). This display can be provided, for example, by the user interface control processor 106 in Figure 1, to the user of the user terminal 7 with a GUI screen (described later) containing information such as the image quality evaluation value Ei. The user can check the image quality evaluation value Ei on this screen. The result of step S4 is output as an image quality evaluation result 204 including the image quality evaluation value Ei.

[0055] Next, in step S5, the computer system 3 uses the image quality evaluation result 204, which includes the image quality evaluation value Ei calculated in step S3, to perform predetermined processing operations such as observation, generate processing result information, and save it. In one example, in the case of observation, the computer system 3 selects an image 201 that is judged to have high image quality based on the image quality evaluation value Ei, uses it as the observation image, and performs the observation operation. After step S5, this flow ends.

[0056] Furthermore, step S6 is a step in which the method, or in other words, the set value, for the processing operations related to steps S2 and S3 is determined; in other words, it is a setting step and a method determination step. This step S6 includes determining / setting the method for determining the evaluation sub-region Si in step S2 and the method for calculating the image quality evaluation value Ei in step S3. In step S6, the computer system 3 determines the method and set value based on the user setting information 202. The computer system 3 determines control parameter values, etc., for the processing of each step (S21, S22, S31, S32) using a method selected from several methods, and applies them as set values ​​to the processing of each step. The content of the determination in step S6 may be set in advance as part of the design of this system, or it may be determined according to the user setting information 202 set by the user.

[0057] If user configuration information 202 is used, in step S7, the user (Figure 1) has already made settings on the GUI screen provided by the computer system 3, such as selecting the application method and control parameter values, and the user configuration information 202 is saved in memory resources. Steps S6 and S7 may be combined into one.

[0058] In step S6, the computer system 3 may, similar to step S2, input the target image 201, the imaging conditions 204 described later, the design data 205, etc. (Figure 6) to determine the method and settings for each step. Step S6 may be performed before the execution of steps S2 and S3, or after the user has confirmed the results of the execution of steps S2 and S3.

[0059] The details of the flow in Figure 2 are as follows. Step S2 specifically consists of two steps: Step S21 and Step S22. Step S21 is a segmentation and classification step that generates a classification segmentation region Rj obtained by segmenting and classifying image 201. Step S22 is an evaluation sub-region determination step that determines an evaluation sub-region Si using the classification segmentation region Rj, which is the classification result.

[0060] Step S3 specifically includes two steps: Step S31 and Step S32. Step S31 is the step of calculating the image quality evaluation value Ei for each evaluation subregion Si using the evaluation subregion Si. Step S32 is the step of calculating the overall evaluation value C by combining multiple image quality evaluation values ​​Ei.

[0061] In step S2, the evaluation subregion Si may be calculated using rule-based image processing or machine learning. Also, in step S3, the image quality evaluation value Ei may be calculated using rule-based image processing or machine learning.

[0062] [Image quality evaluated using the image quality evaluation value Ei (1)] The image quality evaluation value Ei and the image quality evaluated using the image quality evaluation value Ei will be described. Embodiment 1 of this disclosure has multiple different image quality values ​​that are evaluated using multiple different image quality evaluation values ​​Ei for an image 201 captured by SEM2. The image quality evaluation value Ei and image quality, in other words, the elements, evaluation items, and parameter values ​​of image quality are as follows.

[0063] Figure 3(A) shows the image quality evaluation value Ei and the image quality setting information / data, namely image quality evaluation value information 203, in tabular format. The computer system 3 stores the image quality evaluation value information 203 in memory resources in advance based on the settings. Figure 2 shows that the image quality evaluation value information 203 is used in the setting step S6. The image quality evaluation value information 203 may also be set in the user setting step S7. The set image quality evaluation value information 203 is referenced in steps S2 and S3.

[0064] In the table of image quality evaluation value information 203 in Figure 3(A), the columns are "Image Quality Evaluation Value (Ei)", "Image Quality", and "Evaluate / Do Not Evaluate". The "Image Quality Evaluation Value (Ei)" column is the identification information of the image quality evaluation value. The "Image Quality" column is the image quality evaluated by the image quality evaluation value Ei, corresponding to the image quality evaluation value Ei, in other words, the image quality name and image quality description. The "Evaluate / Do Not Evaluate" column indicates whether or not the image quality of the image quality evaluation value Ei in that row is evaluated for the target image 201, in other words, whether or not the evaluation item / parameter is applied. "Evaluate" (applied) is indicated by a circle.

[0065] Depending on the target image 201 (Figure 2), it is possible to appropriately set whether to "evaluate / not evaluate" each image quality evaluation value Ei. This setting may be made by the user via a GUI screen (step S7 of the user settings in Figure 2). Alternatively, the system may determine it automatically (step S6). For example, the computer system 3 may automatically select an image quality evaluation value Ei used in a past process (semiconductor device manufacturing process) that handled a similar pattern to the semiconductor pattern seen in image 201 and apply it to the image 201. The computer system 3 may appropriately refer to manufacturing process information, past performance information, and statistical information related to the sample 10.

[0066] Note that in Figure 3(A), only the parameters of the image quality evaluation value Ei are shown for illustrative purposes; the actual parameter values ​​are stored in memory resources in association with the target image 201. Figure 3(B) shows an example of the image quality evaluation result 204 (Figure 2) data in tabular format. In this image quality evaluation result 204, the parameters (evaluation items) and values ​​of the image quality evaluation value Ei are stored for each target image 201. The parameters include, for example, E1 to E9, similar to (A), and the overall evaluation value C calculated in step S32.

[0067] In Embodiment 1, two or more of the above-mentioned multiple image quality evaluation values ​​Ei are used for each image 201 (Figure 2), or for each predetermined object (for example, the same semiconductor wafer 10). The example in Figure 3 shows a setting in which each image quality is evaluated using image quality evaluation values ​​E1, E2, E3, E5, and E9 for a given image 201.

[0068] In the example shown in Figure 3(A), the multiple image quality evaluation values ​​Ei have nine types of parameter values, E1 to E9, as shown below. (E1): Image sharpness (E2): Visibility of defects (E3): Degree of noise suppression (E4): Interlayer contrast (E5): Dark area emphasis level (E6): Degree of shape preservation (degree of preservation of measured length values) (E7): Roughness preservation (E8): Degree of ringing suppression (E9): Image naturalness

[0069] The following is an overview of each image quality evaluation value Ei and the image quality evaluated using it.

[0070] Image sharpness (E1) represents the ease with which areas with different structures or materials can be distinguished, and is generally calculated as the degree of brightness change at the boundaries (edges) of areas with different structures or materials.

[0071] The visibility of a defect in (E2) refers to how easily the defect area can be distinguished from the surrounding area, and includes, for example, the contrast between the defect area and the surrounding area, the contrast within the defect area, and the intensity of the shadow caused by the defect.

[0072] The noise suppression level (E3) indicates the degree of noise generated due to imaging and other factors.

[0073] The interlayer contrast in (E4) represents the contrast between different layers in a semiconductor pattern.

[0074] The (E5) dark area enhancement level represents the contrast in dark areas such as the lower layers of the semiconductor pattern, as well as the contrast between dark areas and defect areas.

[0075] The shape preservation (or length preservation) in (E6) represents the degree of similarity between the length values ​​in a reference image showing a similar pattern and the length values ​​in the image being evaluated. The higher this evaluation value, the better the shape of the pattern in the sample is preserved.

[0076] The roughness preservation score (E7) represents the degree of similarity between the roughness of the pattern in a reference image showing a similar pattern and the roughness of the pattern in the image being evaluated. A higher score indicates better preservation of the roughness of the sample pattern.

[0077] The ringing suppression level of (E8) indicates the degree of ringing (referring to wave-like artifacts) occurring in the image.

[0078] The naturalness of the image in (E9) represents the degree to which a person perceives little discomfort when viewing the image visually.

[0079] [Image quality evaluated using the image quality evaluation value Ei (2)] Let's provide some supplementary explanations regarding the definitions of each image quality evaluation value Ei in Figure 3. Figures 4 and 5 are schematic diagrams illustrating the definitions of each image quality evaluation value Ei.

[0080] (1) Figures 4(1a) and (1b) are explanatory diagrams for "sharpness" as the image quality evaluation value E1, where (1a) is an example image and (1b) shows the luminance profile of the A and B lines in the image. The A and B lines are lines that cross the boundary (edge) 403 between certain vertical line pattern regions 401 and 402. The computer system 3 calculates the gradient 404 of the luminance profile of the boundary 403 of the pattern region as the sharpness value.

[0081] (2) Figures 4(2a) and (2b) are explanatory diagrams for "defect visibility" as an image quality evaluation value E2. Image (2a) is an example of an image when defect visibility is relatively low, and image (2b) is an example of an image when defect visibility is relatively high. The computer system 3 calculates the contrast between the defect region 421 and the surrounding area of ​​the defect region 421 as a value for defect visibility.

[0082] (3) Figures 4(3a) and (3b) are explanatory diagrams for "noise suppression degree" as the image quality evaluation value E3. Image (3a) is an example of an image when the noise suppression degree is relatively small, and image (3b) is an example of an image when the noise suppression degree is relatively large. The computer system 3 calculates the noise amount as the value of the noise suppression degree, for example, the noise generated in the wiring upper surface area 431 and the dark area area 432. The smaller the noise amount, the greater the noise suppression degree.

[0083] (4) Figures 4(4a) and (4b) are explanatory diagrams for "interlayer contrast" as the image quality evaluation value E4. Image (4a) is an example of an image when the interlayer contrast is relatively small, and image (4b) is an example of an image when the interlayer contrast is relatively large. The computer system 3 calculates the value of interlayer contrast, for example, the difference in average brightness between the wiring upper surface area 441 and the dark area (lower layer area) 442. The larger the brightness difference, the greater the interlayer contrast.

[0084] (5) Figures 4(5a) and (5b) are explanatory diagrams for "dark area enhancement degree" as the image quality evaluation value E5. Image (5a) is an example of an image when the dark area enhancement degree is relatively small, and image (5b) is an example of an image when the dark area enhancement degree is relatively large. The computer system 3 calculates the value of the dark area enhancement degree, for example, the contrast within the dark area region 451, or the contrast between the dark area region 451 and the defective area 452.

[0085] (6) Figures (6a), (6b), and (6c) in Figure 5 are explanatory diagrams for "shape preservation" as an image quality evaluation value E6, and show example images. Image (6a) is an example of a reference image, image (6b) is an example of an image with relatively low shape preservation, and image (6c) is an example of an image with relatively high shape preservation. Shape preservation is a value that evaluates whether the shape of the pattern captured in the reference image is preserved in the observation image that is the subject of image quality evaluation. This evaluation value can be calculated by comparing the edges of the reference image with the edges of the image to be evaluated. For example, a certain edge 461 in the reference image (6a) is compared with the corresponding edge 462 in the image (6b). Also, a certain edge 461 in the reference image (6a) is compared with the corresponding edge 463 in the image (6c). In image (6b), the change in the shape of edge 462 is relatively large compared to edge 461, but in image (6c), the change in the shape of edge 463 is relatively small compared to edge 461.

[0086] In this example, we showed a case where the shape of the wiring pattern changes, but in calculating the degree of shape preservation, we evaluate not only whether the shape of the wiring pattern is preserved, but also whether the shape of the defects (defect area 464) is preserved. In this example, we showed a case where the shape of part of the pattern is not preserved, but even in cases where the width of the entire pattern is reduced while the shape of the edges remains unchanged, the position of the edges changes, so the measured length value of the pattern changes, and the degree of shape preservation decreases.

[0087] (7) Figures (7a), (7b), and (7c) in Figure 5 are explanatory diagrams for "roughness preservation" as the image quality evaluation value E7, and show examples of images. Image (7a) is an example of a reference image, image (7b) is an example of an image when the roughness preservation is relatively small, and image (7c) is an example of an image when the roughness preservation is relatively large. During the manufacturing process of semiconductor patterns, fluctuations occur in the shape of the edge of the pattern. Computer system 3 evaluates whether the roughness of the edge of the pattern captured in the reference image is preserved in the observation image that is the subject of image quality evaluation, as the value of roughness preservation. In this example, edge 471 of a certain pattern in the reference image (7a) has fluctuations and roughness. In contrast, in image (7b), the corresponding edge 472 is close to a straight line, and in image (7c), the corresponding edge 473 has fluctuations and roughness.

[0088] (8) Figures 5(8a) and (8b) are explanatory diagrams for the "ringing suppression degree" as the image quality evaluation value E8, and show example images. Image (8a) is an example of an image when the ringing suppression degree is relatively small, and image (8b) is an example of an image when the ringing suppression degree is relatively large. Observed images show wavy artifacts (called ringing) around the edges of the sample pattern due to the effects of imaging conditions and image quality improvement processing. Computer system 3 evaluates whether or not ringing is present in the image to be evaluated as the ringing suppression degree value. At this time, similar to the shape preservation degree and roughness preservation degree described above, computer system 3 can calculate the ringing suppression degree by preparing a reference image and, for example, detecting brightness changes that were not present in the reference image. In image (8a), for example, ringing (shown by multiple vertical lines) occurs in the transverse direction around a certain edge 481. In contrast, in image (8b), ringing is suppressed at the corresponding edge 482.

[0089] (9) As an image quality evaluation value of E9, "image naturalness" represents the degree to which a person perceives a lack of unnaturalness when viewing the image. For example, normal observation images contain noise, albeit to varying degrees, so an image from which all noise has been completely removed may have a lower image naturalness.

[0090] [Setup Step (S6)] Using Figure 6, we will explain the details of step S6 of the setting (method determination). Step S6, in more detail, has, for example, five steps, S61, S62, S63, S64, and S65, as shown in Figure 6.

[0091] In step S61, the computer system 3 sets the classification method for the classification partition region Rj in step S21 of Figure 2. In step S61, the computer system 3 sets the classification method for the classification partition region Rj, for example, as shown in Figure 10, which will be described later.

[0092] In step S62, the computer system 3 sets the method for determining the evaluation subregion Si in step S22 of Figure 2.

[0093] For steps S21 and S22, the methods can be broadly categorized into rule-based or machine learning methods.

[0094] In step S63, the computer system 3 determines the method for calculating the image quality evaluation value Ei in step S31 of Figure 2.

[0095] In step S64, the computer system 3 determines the method for calculating the overall evaluation value C in step S32 of Figure 2.

[0096] The methods used in steps S31 and S32 can be broadly categorized into rule-based or machine learning methods.

[0097] In addition to the steps described above, step S65 may also be used. The image quality evaluated using the image quality evaluation value Ei is not limited to the nine image quality evaluation values ​​E1 to E9 shown in Figure 3; it is also possible to define and add new image quality evaluation values. In that case, in step S65, the computer system 3 adds a new image quality evaluation value Ei as needed by the user. For example, the computer system 3 adds a new image quality evaluation value Ei in response to user operations on the GUI screen and sets the information of the added image quality evaluation value Ei in the image quality evaluation value information 203. More specifically, the added image quality evaluation value Ei corresponds to the calculation method, definition information, program, etc., for calculating that image quality evaluation value Ei. The added image quality evaluation value Ei then becomes a candidate for the image quality evaluation value Ei to be applied to the image 201, and it becomes possible to select "evaluate / do not evaluate" in Figure 3. The added image quality evaluation value Ei may also be a variation of the image quality evaluation values ​​E1 to E9 in Figure 3. For example, with respect to a certain image quality evaluation value Ei, if other calculation formulas are possible in the definition for calculating that value, these other formulas may be set as variations. For example, two types of image quality evaluation values, Eia and Eib, including an additional one, may be set.

[0098] Examples of imaging conditions 205 include various parameter values ​​during imaging by SEM2 (Figure 1), such as acceleration voltage, probe current, beam aperture angle, beam tilt angle, and focal position. Examples of design data 206, or sample information, include the shape of the semiconductor wafer 10, which is the sample 10 (Figure 1), particularly information such as height according to its horizontal position, and material information.

[0099] [Determination of evaluation region Si and calculation of image quality evaluation value Ei] One challenge in evaluating image quality is that when the entire image is used as the calculation domain for image quality evaluation, image features in areas unrelated to image quality also affect the evaluation value, making it impossible, difficult, or resulting in a decrease in the quality of quantification.

[0100] In contrast, in Embodiment 1, from the target image 201 (Figure 2), only the region containing image features related to the image quality evaluation value Ei corresponding to the image quality to be evaluated is determined as the evaluation subregion Si (step S2). This makes it possible to calculate the image quality evaluation value Ei for each image quality using the appropriate evaluation subregion Si, enabling image quality quantification that takes multiple image quality levels into consideration.

[0101] Figure 7 shows an example of the process for determining the evaluation subregion Si and calculating the image quality evaluation value Ei, corresponding to steps S2 and S3 in Figure 2. In step S701, the computer system 3 obtains and references information on two or more image quality evaluation values ​​Ei to be applied to the image quality evaluation from the image quality evaluation value information 203 as shown in Figure 3, for the input target image 201. Here, the multiple (N) image quality evaluation values ​​Ei to be applied are represented as E1 to EN.

[0102] Next, in step S702, which corresponds to step S2, the computer system 3 determines multiple (N) evaluation subregions Si (S1 to SN) corresponding to multiple (N) image quality evaluation values ​​Ei (E1 to EN) to be applied. That is, the computer system 3 determines the evaluation subregion Si for each image quality evaluation value Ei. Here, the multiple (N) evaluation subregions Si are represented by S1 to SN.

[0103] In this example, as shown in Figure 7, there is a one-to-one correspondence between multiple image quality evaluation values ​​Ei and multiple evaluation subregions Si. The evaluation subregions Si consist of one or more regions, i=1 to N, where N is the number of subregions. The image quality evaluation value Ei consists of one or more parameter values, i=1 to N, where N is the number of parameters for the evaluation item. In this example, the number of evaluation subregions Si N and the number of parameters N for the image quality evaluation value Ei are the same, but are not limited to this.

[0104] Next, in step S703, which corresponds to step S3, the computer system 3 calculates the image quality evaluation value Ei for each evaluation subregion Si. The calculation results are stored as image quality evaluation result 204.

[0105] Figure 8 shows an example of a modified case where the correspondence between the evaluation subregion Si and the image quality evaluation value Ei is not one-to-one. Any correspondence is possible. (A) is the case where the correspondence between Si and Ei includes one-to-many relationships. For example, using the evaluation subregion S1, image quality evaluation values ​​E1, E2, and E3 are calculated, respectively. (B) is the case where the correspondence between Si and Ei includes many-to-one relationships. For example, using the evaluation subregions S1, S2, and S3, image quality evaluation value E1 is calculated.

[0106] Figure 9 is an explanatory diagram illustrating the cases where the evaluation subregions Si for each image quality evaluation value Ei do not overlap and when they do overlap. Each evaluation subregion Si may or may not overlap, and the method is applicable in both cases. Example (A) shows the case where the evaluation subregion Sa for a certain image quality evaluation value Ea and the evaluation subregion Sb for a certain image quality evaluation value Eb do not overlap. In this example, the diagonal pattern area in image 901 is the evaluation subregion Sa, and the white area in image 902 is the evaluation subregion Sb; these do not overlap.

[0107] Example (B) shows cases where the evaluation sub-region Sa for a certain image quality evaluation value Ea and the evaluation sub-region Sc for a certain image quality evaluation value Ec overlap and are identical, or where the evaluation sub-region Sa for a certain image quality evaluation value Ea and the evaluation sub-region Sd for a certain image quality evaluation value Ed partially overlap. In this example, the diagonal pattern area in image 911 is the evaluation sub-region Sa, and the diagonal pattern area in image 912 is the evaluation sub-region Sb, and these are identical. Also, the area indicated by the dashed frame in image 913 is the evaluation sub-region Sd, and this evaluation sub-region Sd partially overlaps with the evaluation sub-region Sa. For example, the evaluation sub-region Sd is an area that is like an expanded version of the width of the evaluation sub-region Sa.

[0108] In determining the evaluation subregion Si (step S22 in Figure 2), different image quality evaluation values ​​Ei, such as E1 and E2, are values ​​of different image quality evaluation items (parameters). Therefore, the evaluation subregions Si (e.g., S1 and S2), which are the regions for calculating these image quality evaluation values ​​Ei, are often different regions. For example, as shown in Figure 9(A), the evaluation subregion Sa for one image quality evaluation value Ea and the evaluation subregion Sb for another image quality evaluation value Eb are separate regions that do not overlap. However, each evaluation subregion Si associated with each image quality evaluation value Ei may be the same region or partially overlapping regions, as shown in Figure 9(B).

[0109] [Domain determination step (S2)] The details of step S2 in region determination shown in Figure 2 are explained below. In step S21, the computer system 3 classifies each of the classification division regions Rj (j=1 to Nr) obtained by dividing the image 201. Here, the classification division regions are denoted as Rj, with j=1 to Nr, and Nr being the number of division regions. In step S22, the computer system 3 determines the evaluation subregions Si (S1 to SN) based on the classification division regions Rj and their classification results.

[0110] One challenge in image quality evaluation is that, as shown in the examples of sharpness (E1) and defect visibility (E2) in Figure 3, calculating image quality evaluation values ​​(Ei) that focus on the structure of semiconductor devices is difficult when there is no information about the structure (circuit patterns, etc.) and the location of that structure in the captured image. For example, the image quality evaluation value E1 related to sharpness represents the degree of brightness change, such as the edges of circuit patterns, as a structure. However, without information about the structure and location in the captured image 201, information from areas with large brightness changes, such as noise in flat areas, which do not contribute to the evaluation of sharpness, is included in the calculation of the image quality evaluation value. This reduces the accuracy of the image quality evaluation value.

[0111] Therefore, in Embodiment 1, in step S21, the computer system 3 divides the image 201 into regions such that each region represents an element of the semiconductor device structure, and then classifies each divided region, i.e., the classification divided region Rj, to determine which structural element it is. Then, in step S22, the computer system 3 uses the classification divided region Rj and the classification result information to determine the region suitable for calculating each image quality evaluation value Ei as the evaluation sub-region Si.

[0112] Figure 10 shows a detailed example of step S2. First, the computer system 3 performs steps S1001 and S1002 on the image 201 to be evaluated, corresponding to the division and classification step S21. In step S1001, the computer system 3 divides the entire image 201 into regions according to the structural elements of the semiconductor device found in the image 201, and obtains divided regions in the state before classification. In step S1002, the computer system 3 classifies each divided region in the image 201 according to its structural elements, so that each divided region in the image 201 becomes a structural element of a semiconductor device, and determines the classification divided region Rj, which is the divided region in the classified state.

[0113] In steps S1001 and S1002, the input may include not only an image 201 of the sample 10, but also semiconductor pattern design data 206, imaging conditions 205, and user classification information 207 corresponding to the image 201. The design data 206 contains information such as the position and shape of the circuit pattern on the wafer 10, which is the sample 10. The imaging conditions 205 are information such as conditions for controlling the irradiation of the electron beam b1 of the SEM2 in Figure 1 and conditions related to detection. The user classification information 207 is information for dividing and classifying into classification division regions Rj, which is set by the user by viewing and operating the captured image on the GUI screen (for example, step S7 in Figure 6).

[0114] In step S2, you may use rule-based segmentation methods such as the Watershed method, or machine learning-based segmentation methods such as CNNs. Alternatively, in step S2, you may use semantic segmentation methods such as FCN (Fully Convolutional Network) to perform region division and region classification simultaneously.

[0115] The division and classification step S21 (S1001, S1002 in Figure 10) yields the classification result 1010 of the classification division region Rj. In this example, the classification result 1010 includes, but is not limited to, a defect region 1011, a lower layer region 1012, an edge region 1013, and a wiring top surface region 1014 as the classification division region Rj, as shown in the figure. These regions correspond to structural elements of the semiconductor device and are each associated with the classification division region Rj. An example of the association, identification, and classification of the classification division region Ri in the data information of the classification result 1010 is as follows: R1 = defect region 1011, R2 = lower layer region 1012, R3 = edge region 1013, R4 = wiring top surface region 1014.

[0116] Defect region 1011 is a region where a defect is roughly suspected, and is separate from the defect observation result itself in step S5. Lower layer region 1012 is a region on the lower side in terms of imaging direction and semiconductor device structure, in other words, a dark area. Wiring upper surface region 1014 is a region on the upper side in terms of imaging direction and semiconductor device structure, and in this example, it is a vertical line pattern region. Edge region 1013 is a region that constitutes the edge of wiring upper surface region 1014.

[0117] Subsequently, in step S1003, which corresponds to step S22, the computer system 3 determines the evaluation partial region Si(1020) by extracting an evaluation partial region Si corresponding to the image quality evaluation value Ei based on the classification result 1010. In the illustrated evaluation partial region 1020, the illustrated white regions in each image G1 to GN represent the regions used to calculate the image quality evaluation value Ei, which correspond to the evaluation partial region Si, while the illustrated black regions do not correspond to the evaluation partial region Si and represent regions not used to calculate the image quality evaluation value Ei. In the calculation of the image quality evaluation value Ei in the later step S31, the computer system 3 uses only the white regions, which are the evaluation partial region Si, from image 201 to calculate the image quality evaluation value Ei, and does not use the black regions.

[0118] For example, the evaluation sub-region S1 for the image quality evaluation value E1 "Sharpness" is constructed using the extracted classification sub-region R3 = edge region 1013 (shown by a diagonal pattern). More specifically, the evaluation sub-region S1 is constructed by expanding the edge region 1013 (R3). Also, for example, the evaluation sub-region S5 for the image quality evaluation value E5 "Shadow Enhancement Degree" is constructed using the extracted classification sub-region R2 = lower layer region 1012 (shown by a diagonal pattern).

[0119] The evaluation subregion Si output as a result of step S2 may be manually modified by the user after being saved to memory resources. As will be described later, the computer system 3 displays the evaluation subregion Si on a GUI screen, for example, allowing the user to view and verify it, make any necessary modifications, and then confirm and save it.

[0120] [Classification method for classification-based partitioned regions Rj using design data] Using Figure 11, a detailed example of the classification method for the classification division region Rj is explained, using the semiconductor pattern design data 206 corresponding to image 201, and following the division and classification step S21 in Figure 2 (steps S1001 and S1002 in Figure 10).

[0121] One of the features of Embodiment 1 (particularly the embodiment shown in Figure 11) is that step S21 of the division and classification takes the semiconductor circuit design data 206 corresponding to image 201 as input and performs classification for each classification division region Rj.

[0122] One challenge was that it was sometimes impossible or difficult to classify the images being evaluated into semiconductor structural elements. In contrast, in Embodiment 1 (Figure 11), the semiconductor design data 206 is used as the classification criterion for the structural elements in the classification of the classification division region Rj. This makes it possible to classify the classification division region Rj.

[0123] Figure 11 shows an example of processing for classification using design data 206. First, the computer system 3 inputs the semiconductor design data 206 corresponding to the image 201 to be evaluated. In this example, the design data 206 shows that the white area 1103 corresponds to the top surface wiring area, and the black area 1104 corresponds to the dark area. The design data 206 in this example includes the areas of structural elements corresponding to the circuit patterns in image 201. The design data 206 includes not only the wiring patterns visible in image 201, but also other wiring patterns. Therefore, in step S1101, the computer system 3 inputs image 201 and design data 206, matches them, and obtains the matched design data 206B. The matched design data 206B is the data from the design data 206 that corresponds to the wiring patterns visible in image 201.

[0124] Subsequently, in step S1102, which is a classification step, the computer system 3 classifies the classification division region Rj of the image 201 based on the matched design data 206B and obtains the classification result 1110. The defective region 1111 in the classification result 1110 can be determined by extracting regions with a large difference between the average brightness of the top surface wiring region and the average brightness of the dark region.

[0125] [Classification method for classification division regions Rj using imaging conditions] Using Figure 12, a detailed example of the classification method for the classification division region Rj is explained using the imaging conditions 205 of the image 201 to be evaluated, and the division and classification step S21 (steps S1001 and S1002 in Figure 10).

[0126] One of the features of Embodiment 1 (particularly the embodiment shown in Figure 12) is that the classification step S21 takes the imaging conditions 205 of the image 201 as input and performs classification for each classification division region Rj.

[0127] One challenge was that it was sometimes impossible or difficult to classify the images to be evaluated into semiconductor structural elements. In contrast, in Embodiment 1, the imaging conditions 205 of the image 201 to be evaluated are used as the classification criteria for the structural elements in the classification of the classification division region Rj. This makes it possible to classify the classification division region Rj.

[0128] Figure 12(A) shows an example of processing for classification using imaging conditions 205. In step S1201, which corresponds to step S21 of the segmentation and classification, the computer system 3 takes the image 201 to be evaluated and the imaging conditions 205 used when the image 201 was captured as input, and classifies the classification segmentation region Rj of the image 201 by semiconductor structure, and obtains the classification result 1210.

[0129] In step S1201 of this classification, the computer system 3 pre-learns, for example, the relationship between the acceleration voltage, which is one of the imaging conditions 205, the brightness value of the image 201, and the material of the semiconductor structure. Based on the learning model 1200, it estimates the material of the structure as an output from the input acceleration voltage and brightness value, and classifies the classification division region Rj based on the estimation result.

[0130] Figure 12(B) shows an example of the pre-learning phase. In learning step S1202, the computer system 3 constructs and trains a learning model 1200 using the learning imaging conditions 1221, learning image data 1222, and learning semiconductor structure information 1223 (including material information of the semiconductor structure). Through training, for example, the parameter values ​​of the learning model 1200 are adjusted to be suitable.

[0131] Furthermore, the defective regions 1211 in the classification result 1210 can be determined, for example, by extracting regions where the material cannot be estimated.

[0132] [Steps to determine Si using Rj] Using Figure 13, we will explain in detail step S22 (Figure 2), in which the evaluation subregion Si is determined using the classification subregion Rj. One of the features of Embodiment 1 is that in step S2 (S21, S22) in Figure 2, and especially in Figure 13, the evaluation subregion Si for each image quality evaluation value Ei is generated by combining, dividing, expanding, contracting, or a combination thereof of the classification subregion Rj.

[0133] One challenge is that, depending on the image quality evaluation value Ei, the evaluation subregion Si, which is suitable for calculating that value, and the classification subregion Rj, which is divided and classified according to the semiconductor structure, do not necessarily coincide. In such cases, simply calculating the image quality evaluation value Ei for each classification subregion Rj does not allow for the evaluation of the desired image quality; in other words, it is difficult to improve the accuracy of the image quality evaluation. For example, the image quality evaluation value E2 shown in Figure 3, "visibility of defects," represents the ease of distinguishing between the defect region and the surrounding region, as shown in Figure 4 (2a) and (2b). Therefore, for this image quality "visibility of defects," it is preferable to calculate the image quality evaluation value E2 in a region that includes not only the defect region but also the surrounding region.

[0134] In contrast, in Embodiment 1, an evaluation subregion Si, which is suitable for calculating the image quality evaluation value Ei, is generated by applying one selected process, or a combination of selected processes, from processes such as combining, dividing, expanding, and contracting the classification division region Rj. This makes it possible to calculate the image quality evaluation value Ei even in regions that include multiple classification division regions Rj, or in regions that are only a part of the classification division region Rj.

[0135] Figure 13 shows an example of a process for generating an evaluation subregion Si by performing operations such as combining, dividing, expanding, and contracting the classification subregion Rj. The input classification result 1010 is the same as in Figure 10. First, we take an example of setting the evaluation subregion Si when the image quality evaluation value Ei is the image quality evaluation value E2 "visibility of defects". When calculating the image quality evaluation value E2 "visibility of defects", the contrast between the defect region and the surrounding region affects the evaluation value. Therefore, not only the defect region but also the surrounding region adjacent to the defect region must be included in the evaluation subregion Si.

[0136] Therefore, in step S1301, the computer system 3 selects a classification division region Rj from the classification result 1010 according to the image quality evaluation value Ei (E2). The computer system 3 selects the defective region 1011 (R1) and the dark region 1012 (R2), which is adjacent to the defective region 1011 (R1), as the classification division region Rj. In Figure 13, the classification division region Rj selected here is also shown as the hatching pattern region in image 1304. Then, in step S1302, the computer system 3 combines the selected defective region 1011 (R1) and region 1012 (R2) to generate an evaluation sub-region S2 for the image quality evaluation value E2 of "visibility of defects". The white area in the generated image 1305 is the evaluation sub-region S2.

[0137] Image 1311 shows another example of calculating the evaluation subregion S2 for the image quality evaluation value E2. In this example, when taking the surrounding region for the defective region 1011(R1), instead of using the entire adjacent dark region 1012(R2), only a portion of the adjacent dark region 1012(R2) that is close to the defective region 1011(R1) is used, and in this example only the region 1312 shown as a rectangle is used.

[0138] As another example, when the image quality evaluation value Ei is the "sharpness" of the image quality evaluation value E1 in Figure 3, the determination of the evaluation subregion S1 is as follows: When the computer system 3 calculates the image quality evaluation value E1 for "sharpness," it calculates the degree of brightness change at the boundary (in other words, edge, etc.) of areas with different semiconductor structures and materials. In calculating the degree of brightness change, it is necessary to look at the amount of change when the brightness of the edge area is compared with the brightness of the surrounding area. Therefore, the evaluation subregion S1 for the image quality evaluation value E1 needs to include not only the edge area but also the area surrounding the edge area.

[0139] Therefore, the computer system 3 selects a classification division region Rj from the classification results 1010 according to the image quality evaluation value E1. As the classification division region Rj, the computer system 3 first selects region 1013 (R3), which is an edge region. The region selected here is also shown as the hatching pattern region in image 1307. In step S1303, the computer system 3 generates an evaluation sub-region S1 related to the image quality evaluation value E1, which is related to "sharpness," by expanding region 1013 (R3), which is an edge region. In this example, as shown as the expansion direction 1309 in image 1307, the computer system 3 expands each region 1013 (R3), which is an edge of the vertical line pattern extending in the y-axis direction, in both the left and right directions of the x-direction, indicated by the arrows perpendicular to it. The white area in the generated image 1308 is the evaluation sub-region S1.

[0140] For other image quality evaluation values ​​Ei, evaluation subregions Si can also be generated in the same way as in the example above, by processing using the classification subregion Rj based on the definition of processing for each image quality evaluation value Ei. Furthermore, depending on the image quality evaluation value Ei, evaluation subregions Si can be generated not only by combining or expanding the classification subregion Rj in the example above, but also by dividing, contracting, or a combination thereof. As an example of dividing the classification subregion Rj, the wiring top surface region 1014 (R4) may be divided into multiple regions according to their relationship to the edge region 1013 (R3), such as whether they are far or near. As an example of contracting the classification subregion Rj, the wiring top surface region 1104 (R4) may be contracted so that the region close to the edge region 1103 (R3) is removed.

[0141] The method described above for generating evaluation subregions Si from classification subregions Rj for each image quality evaluation value Ei may be set by the user as a rule-based generation method or as a machine learning generation method in step S62 of Figure 6.

[0142] Figure 14 shows examples of the generation results of evaluation subregions Si (S1 to S8) for each image quality evaluation value Ei (E1 to E8), including examples of other image quality evaluation values ​​Ei. In each image, the white areas represent the evaluation subregions Si. For example, in the case of image quality evaluation value E3 "Degree of noise suppression", the computer system 3 generates evaluation subregion S3 from the aforementioned wiring top surface region 1014 (R4) and dark region 1012 (R2). Also, in the case of image quality evaluation value E4 "Interlayer contrast", the computer system 3 generates evaluation subregion S4 from the aforementioned wiring top surface region 1014 (R4) and dark region 1012 (R2). Also, in the case of image quality evaluation value E5 "Degree of dark area enhancement", the computer system 3 generates evaluation subregion S5 from the aforementioned wiring top surface region 1014 (R4) and defect region 1011 (R1).

[0143] Furthermore, for example, in the case of image quality evaluation value E6 "Degree of shape preservation", the computer system 3 generates an evaluation sub-region S6 so that it can evaluate whether the shape of the defect region is preserved not only in the area surrounding the edge of the pattern, but also in other areas. Region 1406 is an area whose width has been expanded to some extent so as to include the shape (e.g., ellipse) of the defect region 1011(R1). Also, in the case of image quality evaluation value E7 "Degree of roughness preservation", the computer system 3 generates an evaluation sub-region S7 from the aforementioned edge region 1013(R3). Also, in the case of image quality evaluation value E8 "Degree of ringing suppression", the computer system 3 generates an evaluation sub-region S8 from the area around the edge (e.g., the area below a certain distance from the edge region), excluding the edge region.

[0144] [Classification criteria for the classification segmentation area Rj] Using Figure 15, the classification criteria for the classification division region Rj related to step S21 in Figure 2 will be explained. In Embodiment 1, one of the features of the classification division region Rj is that it includes at least one of the following regions in terms of type and classification: wiring top surface region, edge region, dark region, and defect region. These classification division regions Rj correspond to the defect region 1011, dark region 1012, edge region 1013, and wiring top surface region 1014 in the example of Figure 10.

[0145] One challenge is that the appearance differs significantly from image to image, making it difficult to define classification criteria for the classification division region Rj used to determine the evaluation region Si. In contrast, Embodiment 1 focuses on the structural information of the semiconductor and classifies the classification division region Rj to include at least one of the following regions: the wiring surface region, the edge region, the dark region, and the defect region. For example, as mentioned above, the defect region 1011 is classified as the first classification division region R1, the dark region 1012 as the second classification division region R2, the edge region 1013 as the third classification division region R3, and the wiring surface region 1014 as the fourth classification division region R4.

[0146] Figure 15 is an explanatory diagram of the above classification criteria. The wiring top surface region is the area occupied by the wiring pattern at the highest position in the vertical direction (Z direction) relative to the wafer surface in a top-view image taken from above the wafer surface. The dark area is the area occupied by the wiring pattern or background at the lowest position in the vertical direction (Z direction) relative to the wafer surface in a top-view image taken from above the wafer surface. The edge area is the boundary area between the wiring top surface region and the dark area. The defect area is a general area where defects such as unintended patterns or foreign objects are suspected.

[0147] In the example shown in Figure 15, image 1501 on the left is the same image as image 201 mentioned above. The result of classifying image 1501 into multiple classification division regions Rj is shown in classification result 1502. In classification result 1502, the region within image 1501 is classified into four types of classification division regions Rj (e.g., R1 to R4): the wiring top surface region 1503, the edge region 1504, the dark area region 1505, and the defect region 1506. Note that, as shown in the example, a certain type of classification division region Rj may have multiple regions at distant locations.

[0148] In another example, image 1507, shown on the right, is an image of a pattern with the same structure as image 1501, but with a different appearance. The results of classifying image 1507 are shown in classification result 1508. Similar to classification result 1502, classification result 1508 also classifies the areas within image 1507 into four classification division regions Rj: the wiring top surface region 1509, the edge region 1510, the dark area region 1511, and the defect region 1512.

[0149] As shown in the example above, by focusing on semiconductor structure information, a unified classification becomes possible for images that show the same structure but have different appearances. Here, the semiconductor structure information may be the structural information obtained from image analysis of the captured image 201, or it may be the structural information obtained from the design data 206.

[0150] In the first embodiment, the image was classified into four types of regions: the wiring top surface region, the edge region, the dark region, and the defect region, but the method is not limited to these. For example, if the wiring has a structure of three or more layers, other regions such as the wiring middle layer region may be added to the classification results. Also, the edge region may be set not only as the boundary between the wiring top surface region and the dark region, but also as the boundary between the wiring top region and the wiring middle layer region, or the boundary between the wiring middle layer region and the dark region, or as the boundary of each layer region.

[0151] [Method for determining the internal parameters involved in the calculation of Ei] Using Figure 16, we will explain how to determine the internal parameter Qi_m involved in the calculation of the image quality evaluation value Ei in step S31 in step S63 of Figure 6.

[0152] Embodiment 1 has an interface into which user evaluations regarding image quality are input, and includes a step S63 in which a method for calculating the image quality evaluation value Ei is determined using this user evaluation. One of the features of this step S63 is that the computer system 3 calculates the image quality evaluation value Ei using internal parameters Qi_m (m=1~Nm), and determines the internal parameters Qi_m so that the image quality evaluation value Ei matches the user evaluation, or brings them as close as possible. The internal parameters Qi_m are parameters that constitute the image quality evaluation value Ei, in other words, parameters that constitute the processing in the method for calculating the image quality evaluation value Ei. m=1~Nm, where Nm is the number of internal parameters.

[0153] One challenge is that, in evaluating image quality, the criteria for good or bad image quality differ among users, and the calculated image quality evaluation value Ei, which is designed in advance, may not match the user's evaluation. In response to this, Embodiment 1 focuses on the fact that the image quality evaluation value Ei changes depending on the internal parameter Qi_m used when calculating the image quality evaluation value Ei. In Embodiment 1, the internal parameter Qi_m is determined so that the calculated image quality evaluation value Ei matches the user's evaluation. This eliminates the discrepancy between the calculated image quality evaluation value Ei and the user's evaluation.

[0154] Figure 16 is an explanatory diagram for determining the internal parameter Q described above. (A) shows an example of the process, and (B) shows an example of the GUI screen corresponding to (A). In (A), the computer system 3 inputs a group of images 1600 to be evaluated. The group of images 1600 consists of multiple images of the same location on the wafer, but with different image quality. In step S1601, the computer system 3 provides the user with a GUI 1601, and the user inputs an evaluation of the image quality using the GUI 1601.

[0155] In Embodiment 1, as shown in (B), the GUI screen 1601 is provided as a GUI 1601 into which user evaluations regarding image quality are input. The computer system 3 displays an evaluation sub-region 1601A for calculating the target image quality evaluation value Ei, and multiple images with different image quality (user evaluation images) 1601B on the GUI screen 1601. The evaluation sub-region 1601A corresponds to the evaluation sub-region Si determined as in step S2 above, targeting the images of the image group 1600. In this example, as shown by the white area, the wiring top surface area and the dark area are the calculation areas for the image quality evaluation value Ei. The user evaluation images 1601B display, for example, three images with different image quality (#1, #2, #3) corresponding to the image group 1600.

[0156] The user evaluation image 1601B is provided with an image quality evaluation interface 1601C. The user visually inspects the user evaluation image 1601B on the GUI screen 1601, makes a subjective evaluation, and inputs the image quality ranking of each image into the image quality evaluation interface 1601C. In other words, the user judges which images in the user evaluation image 1601B have relatively high or low image quality and inputs the ranking. The image quality evaluation interface 1601C allows the user to select the ranking, for example, using a list box. However, it is not limited to this; the image quality evaluation interface 1601C may also be an interface where the user drags the images to arrange them in order of image quality. After the user has finished inputting the image quality rankings, they press the evaluation result output button 1601D. As a result, the computer system 3 saves and outputs the user evaluation results, including the image quality rankings set by the user, as user settings 1605 (user settings information 202 in Figure 2). Step S1601, the user evaluation step on GUI screen 1601, corresponds to step S7 of the user settings in Figure 2.

[0157] After the user settings 1605 are saved and output, the computer system 3, in step S1602, calculates an image quality evaluation value Ei for each image of the user evaluation image 1601B (corresponding image group 1600) displayed to the user on the GUI screen 1601. At this time, the computer system 3 calculates the target image quality evaluation value Ei for each image using pre-prepared initial internal parameters 1607.

[0158] Next, in step S1603, the computer system 3 compares the image quality evaluation value Ei obtained in step S1602 with the image quality ranking, which is the user evaluation result in user setting 1605. In this comparison, the computer system 3 checks whether the relative magnitudes of the image quality evaluation values ​​Ei for each image match the image quality ranking set by the user. If they do not match, it is determined that adjustment of the internal parameters is necessary.

[0159] Subsequently, in step S1604, the computer system 3 uses the obtained comparison results to determine the internal parameter Qi_m of the image quality evaluation value Ei. At this time, the computer system 3 determines the value of the internal parameter Qi_m such that the relative magnitudes of the image quality evaluation values ​​Ei of the user evaluation images 1601B displayed to the user match the image quality ranking set by the user, and that there is a significant difference in the image quality evaluation value Ei of each of these images.

[0160] Examples of internal parameters Qi_m for the image quality evaluation value Ei include the following: When the image quality evaluation value Ei is calculated using rule-based processing, for example, when the sharpness of edges in an image (image quality evaluation value E1) is calculated using the brightness gradient, the internal parameter Qi_m could be the filter size of the Sobel filter. When the image quality evaluation value Ei is calculated using a machine learning model, it could be model parameters such as the weights between network nodes (in other words, the coupling coefficients) and biases.

[0161] Figure 17(A) shows an example of the data structure for the internal parameter Qi_m of the image quality evaluation value Ei. Such internal parameter Qi data and information are managed, for example, by being included in the aforementioned image quality evaluation value information 203 (Figure 3). The table in Figure 17(A) has columns for the image quality evaluation value Ei and the internal parameter Qi_m. For example, the image quality evaluation value E1 has multiple internal parameters Qi_m, specifically internal parameters Q1_1 to Q1_Nm. Each internal parameter Qi has a parameter value assigned to it. Furthermore, each internal parameter Qi may also have settings indicating whether it is used or not (in other words, on / off).

[0162] [Elements related to Ei calculation: Image quality evaluation values] Figure 18 will be used to explain the "elemental image quality evaluation values" involved in the calculation of the image quality evaluation value Ei. One feature of Embodiment 1 is that the image quality evaluation value Ei may be expressed as a combination of elemental image quality evaluation values ​​Di_k (k=1~Nd). Nd is the number of elemental image quality evaluation values. Another feature of Embodiment 1 is that the internal parameter Qi_m of the image quality evaluation value Ei may be a parameter that specifies the combination method of the elemental image quality evaluation values ​​Di_k.

[0163] One challenge was that it was difficult to change the elements of the image features considered in calculating the image quality evaluation value Ei, making it difficult to adapt to variations in the semiconductor structure being evaluated. In contrast, in Embodiment 1, the image quality evaluation value Ei is expressed as a combination of elemental image quality evaluation values ​​Di_k. This makes it possible to change the image quality evaluation value Ei in accordance with variations in the semiconductor structure and their changes, enabling appropriate image quality quantification.

[0164] Figure 18 is an explanatory diagram regarding the element image quality evaluation value Di. The processing example in Figure 18 can be applied to step S3 (Figure 2). Furthermore, this processing example can be applied to each image quality evaluation value Ei. Assume that an image to be evaluated 1801 and an evaluation sub-region 1802 corresponding to the image 1801 for calculating the image quality evaluation value Ei are given. In this case, in step S1801, the computer system 3 takes the image 1801 and the evaluation sub-region 1802 as input and calculates multiple (Nd) element image quality evaluation values ​​Di_k related to the target image quality evaluation value Ei. Then, in step S1802, the computer system 3 takes the Nd element image quality evaluation values ​​Di_k and an internal parameter (Qi_m) 1804 related to the target image quality evaluation value Ei as input and calculates the image quality evaluation value Ei constructed by combining these Nd element image quality evaluation values ​​Di_k. This calculation is expressed as Ei = H(Di_k, Qi_m), where H is a function of Di. The internal parameter (Qi_m)1804 referenced in step S1802 is information that specifies how the element image quality evaluation value Di_k is combined. As a result of step S1802, an image quality evaluation result 204 including the image quality evaluation value Ei is obtained.

[0165] Figure 17(B) shows an example of the data structure for elemental image quality evaluation values ​​Di_k of image quality evaluation value Ei. Such data and information for elemental image quality evaluation values ​​Di_k are managed, for example, by being included in the image quality evaluation value information 203 (Figure 3) mentioned above. The table in Figure 17(B) has columns for image quality evaluation value Ei and elemental image quality evaluation values ​​Di_k. For example, image quality evaluation value E1 has multiple elemental image quality evaluation values ​​Di_k, specifically elemental image quality evaluation values ​​D1_1 to D1_Nk. Parameter values ​​are set for each elemental image quality evaluation value Di_k. Furthermore, each elemental image quality evaluation value Di_k may also have settings indicating whether it is used or not (in other words, on / off).

[0166] If one of the internal parameters Qi_m of the image quality evaluation value Ei is a parameter that specifies the combination method of the element image quality evaluation values ​​Di_k, then it will be as follows. For example, one of the internal parameters Q1_m of the image quality evaluation value E1 is a parameter that specifies the combination method of the element image quality evaluation values ​​D1_k. An example of a combination method is to use all of each element image quality evaluation value D1_k (k=1 to Nk). In that case, the one internal parameter Q1_m that specifies that combination method is information that indicates the use of all of the element image quality evaluation values ​​D1_k (k=1 to Nk). For example, if the use of each element image quality evaluation value D1_k is represented as on (1) / off (0), then the value of that internal parameter Q1_m will be {1,1,……,1}.

[0167] The internal parameter specifying the combination method may also be the information specifying the function in step S1802 of Figure 18. Alternatively, setting information may be provided separately from the internal parameter, and in step S63 of Figure 6, for example, the calculation method for the image quality evaluation value Ei, including the combination method or function of the element image quality evaluation value D1_k, may be set.

[0168] A specific example of the above element image quality evaluation value Di is as follows. When calculating the image quality evaluation value E3 "Degree of Noise Suppression" in Figure 3 as the image quality evaluation value Ei, we will explain how to obtain the element image quality evaluation values ​​D3_k (for example, D3_1, D3_2) that constitute the image quality evaluation value E3. First, in order to calculate the amount of brightness change due to noise in the target image 1801, the computer system 3 calculates the local average value of brightness between the target pixel and surrounding pixels in the evaluation sub-region 1802, treating each pixel of the image 1801 as a target pixel, and calculates the difference between the brightness of the target pixel and the local average value. The computer system 3 calculates the difference obtained by the above procedure for all pixels, thereby obtaining the brightness change due to noise as the first element, element image quality evaluation value D3_1.

[0169] On the other hand, the impression that the noise in image 1801 gives to the user is affected not only by the brightness changes caused by the noise, but also by the contrast of the image. Therefore, computer system 3 calculates the contrast of image 1801 as another second element, the element image quality evaluation value D3_2.

[0170] The image quality evaluation value E3, "Degree of Noise Suppression," is constructed by combining two elements, for example, the element image quality evaluation value D3_1 ("Brightness Change Due to Noise") and the element image quality evaluation value D3_2 ("Image Contrast"). This combination method can be specified by one of the internal parameters Qi_m. This allows the computer system 3 to calculate a suitable image quality evaluation value E3, "Degree of Noise Suppression," which takes into account both brightness change due to noise and image contrast. In the above example, "Brightness Change Due to Noise" (the difference above) and image contrast were used to calculate the "Degree of Noise Suppression," but the image quality evaluation value Ei can be calculated using other element image quality evaluation values ​​Di, not limited to these.

[0171] [Calculation of Ei using machine learning] Referring to Figure 19, a method for calculating the image quality evaluation value Ei using machine learning will be described. One of the features of Embodiment 1 is that step S3 of the image quality evaluation value calculation in Figure 2 may be performed to calculate the image quality evaluation value Ei using a machine learning model.

[0172] One challenge was that image quality, which is determined subjectively by human perception such as naturalness, is difficult to evaluate quantitatively because it is difficult to determine the image features relevant to calculating the evaluation value in advance. In contrast, in Embodiment 1, when machine learning is used in step S3, the image quality evaluation value Ei is calculated using a machine learning model that has been trained on human subjectivity as training data. This makes it possible to calculate the evaluation value without determining the image features in advance.

[0173] Figure 19 shows an example of a machine learning model to be applied to step S3, specifically step S31, in Figure 2. For example, known deep learning methods can be applied to implement this machine learning. Specifically, this model can utilize a convolutional neural network (CNN). The example in Figure 19 shows a machine learning model using a CNN with a three-layer structure. In this model, g represents the input image, and F(g) is the output of the model's estimation result. F1(g) and F2(g) represent intermediate data between the input and the estimation result. The intermediate data F1(g) and F2(g) and the estimation result F(g) are calculated using equations 1 to 3 below.

[0174] Equation 1: F1(g)=max(0,W1*g+B1) Equation 2: F2(g)=max(0,W2*F1(g)+B2) Equation 3: F(g)=average(W3*F2(g)+B3)

[0175] Here, * indicates a convolution operation, W1 represents n1 filters of size u0×f1×f1, u0 represents the number of channels in the input image, and f1 represents the size of the spatial filter. By convolving the input image g with u0×f1×f1 filters nu1 times, a nu1-dimensional feature map is obtained. B1 is an n1-dimensional vector and represents the bias components corresponding to the n1 filters. Similarly, W2 represents nu2 filters of size nu1×f2×f2, B2 is an n2-dimensional vector, W3 represents one filter of size nu2×f3×f3, and B3 is a 1-dimensional vector. u0 is a value determined by the number of channels in the image being evaluated. Also, f1, f2, nu1, and nu2 are values ​​determined by the user before the training sequence, for example, f1=9, f2=5, nu1=128, and nu2=64.

[0176] In the example of Embodiment 1, the estimation result F(g) obtained by Equation 3 is used as the image quality evaluation value Ei. Note that the evaluation subregion Si may be used as a mask for F1, F2, and F. Furthermore, other configurations may be used, not limited to the above CNN configuration example. For example, the number of layers may be changed, a network with four or more layers may be used, or a configuration with skip connections may be used. Also, although the example in Equation 3 shows the calculation of F(g) using the average value, the maximum or minimum value may also be calculated.

[0177] [Calculation of overall evaluation value C (Step S32)] Using Figure 20, a method for calculating an overall evaluation value C from multiple image quality evaluation values ​​Ei in step S32 of Figure 2 will be explained. One of the features of Embodiment 1 is that in step S32, an overall evaluation value C composed of combinations of image quality evaluation values ​​Ei is calculated.

[0178] Figure 20 is an explanatory diagram regarding the calculation of the overall evaluation value C. As a result of the processing shown in Figure 7 (particularly step S703), the computer system 3 obtains information on multiple image quality evaluation values ​​Ei for the target image 201. These multiple image quality evaluation values ​​Ei are the multiple image quality evaluation values ​​Ei necessary to calculate the overall evaluation value C. Which image quality evaluation values ​​Ei to use to calculate the overall evaluation value C is set as part of the calculation method in step S64 of Figure 6.

[0179] Next, in step S2001, the computer system 3 calculates an overall evaluation value C, which is constructed by combining the above-mentioned multiple image quality evaluation values ​​Ei, according to the method for calculating the overall evaluation value C 2005. The method for calculating the overall evaluation value C 2005 is the setting value in step S64 of Figure 6, and is information that specifies, for example, which of the multiple image quality evaluation values ​​Ei (i=1~N) to use, and how to combine the multiple image quality evaluation values ​​Ei to be used. The overall evaluation value C can be expressed, for example, as a function of the image quality evaluation values ​​Ei. The calculated overall evaluation value C is saved together with the associated image quality evaluation values ​​Ei so that it is included in the image quality evaluation result 204.

[0180] The overall evaluation value C is calculated, for example, using equation 4 below. Below, Fc is a function that takes the image quality evaluation value Ei as an argument. In this case, the calculation method 2005 for the overall evaluation value C becomes a parameter related to the function Fc.

[0181] Equation 4: C = Fc(Ei)

[0182] The following is a specific example of how the overall evaluation value C is calculated. For example, generally, high sharpness is desirable in observational images. However, if imaging or image quality enhancement processing is applied to increase sharpness, there is a risk that ringing will be amplified. In other words, there is a trade-off relationship between sharpness and ringing. The sharpness mentioned above is related to the image quality evaluation value E1 "Sharpness" in Figure 3, and the ringing mentioned above is related to the image quality evaluation value E8 "Ringing Suppression Degree" in Figure 3. Therefore, it may be difficult to make a comprehensive judgment about the quality of an image by looking only at the individual image quality evaluation values ​​Ei.

[0183] Therefore, in Embodiment 1, for example, a comprehensive evaluation value C is calculated by combining the image quality evaluation value E1 for sharpness and the image quality evaluation value E8 for the degree of ringing suppression. This comprehensive evaluation value C makes it possible to comprehensively observe and judge the image quality. Furthermore, by making the combination method of image quality evaluation value E1 and image quality evaluation value E8 variable using parameters, it becomes possible to set which image quality to prioritize as a priority. This makes it possible to calculate a comprehensive evaluation value C that is tailored to the user's preferences.

[0184] [Calculation of overall rating C: Priority] One feature of Embodiment 1 is that the overall evaluation value C may be calculated using the priority assigned and set with respect to the image quality evaluation value Ei.

[0185] One challenge is that when users want to prioritize a specific image quality evaluation value Ei among multiple image quality evaluation values ​​Ei, it can be difficult to perform a comprehensive evaluation that addresses this. In contrast, Embodiment 1 assigns and sets priorities to each image quality evaluation value Ei according to the image quality that the user prioritizes, and calculates a comprehensive evaluation value C based on the combination of these image quality evaluation values ​​Ei.

[0186] Figure 20 also shows the calculation of the overall evaluation value C using priority. The calculation method 2005 for the overall evaluation value C in Figure 20, which can be set in step S64 of Figure 6, allows setting not only the combination of which image quality evaluation values ​​Ei to use in calculating the overall evaluation value C, but also the priority of the image quality evaluation values ​​Ei to be used, and the calculation method for the overall evaluation value C using priority. Here, the priority of each image quality evaluation value Ei is denoted as ri (i=1~N).

[0187] As an example, suppose the overall evaluation value C is defined using a combination of E1 and E8. C = Fc(E1, E8). Each image quality evaluation value Ei(E1, E8) used is assigned a priority ri(r1, r8). For example, each image quality evaluation value Ei is assigned a weighting coefficient according to its priority ri. For example, weight w1 corresponds to priority r1, weight w8 corresponds to priority r8, etc. The formula for calculating the overall evaluation value C is the function Fc defined using the weights (in other words, coefficients) according to its priority ri. For example, C = Fc(E1, E8) = E1 × w1 + E8 × w8. In this example, a larger value of Ei indicates a higher evaluation, a higher priority ri indicates a larger weight wi, and a larger overall evaluation value C indicates a higher overall evaluation. Also, although this example uses weighted addition, it is not limited to this, and definitions using arithmetic operations are also possible. Similarly, the overall evaluation value C can be defined by calculations using multiple image quality evaluation values ​​Ei and their priorities ri.

[0188] For example, if a user wants to prioritize image quality evaluation value E1 "sharpness" in their evaluation, they should set the priority value r1 (corresponding weight w1) to a higher value. Similarly, if a user wants to prioritize image quality evaluation value E8 "ringing suppression degree" in their evaluation, they should set the priority value r8 (corresponding weight w8) to a higher value. User settings related to the calculation method 2005 for the overall evaluation value C, including the setting of priority ri as described above, can also be made on the GUI screen. Furthermore, if there are multiple calculation methods for the overall evaluation value C depending on the differences in image quality to be prioritized, as in the example above, these multiple calculation methods can be defined and set in advance, allowing the user to select and apply one of them during evaluation. Additionally, a priority ri item may be included in the image quality evaluation value information 203 in Figure 3.

[0189] [Example of using the image quality evaluation value Ei] The image quality evaluation value Ei or overall evaluation value C calculated in step S3 of Figure 2 can be used in various ways. Examples of how to use the image quality evaluation value Ei or overall evaluation value C are listed below. Details of each will be described later.

[0190] (Example 1) Adjustment and determination of parameter P for image quality enhancement processing (image quality enhancement engine) (Figure 21) (Example 2) Correction of the evaluation subregion (Figure 28) (Example 3) Benchmark (Comparison of image quality between images) (Figure 29) (Example 4) Defect detection (Figure 30) (Example 5) Shape measurement (Figure 31) (Example 6) Determination of imaging conditions (Figure 32) (Example 7) Device monitoring (Figure 33)

[0191] At least one function corresponding to the above-described usage example is implemented in the sample observation system and method of Embodiment 1. If multiple functions are implemented, the user can select and use the function through user settings or other means.

[0192] [Adjustment and determination of parameter P for image quality enhancement processing] Using Figure 21 and other figures, the adjustment and determination of the parameters P of the image quality improvement process (image quality improvement engine) using the image quality evaluation value Ei will be explained. One of the features of Embodiment 1 is that it may include the steps of determining the parameters P of the image quality improvement engine based on the image quality evaluation value Ei, and improving the image quality using the image quality improvement engine based on the determined parameters P.

[0193] One challenge was that, when adjusting the parameters of the image quality enhancement engine, it was not possible to quantify the image quality of the output image with a given parameter, making it difficult to judge the quality of the output image and thus difficult to adjust the parameters. In contrast, in Embodiment 1, by using the image quality evaluation value Ei, it becomes possible to quantify the image quality, and the quality of the image can be judged based on the magnitude of the image quality evaluation value Ei. Therefore, the parameters of the image quality enhancement engine are adjusted based on the image quality evaluation value Ei. This makes it possible to determine the parameters of the image quality enhancement engine that output images with superior quality.

[0194] First, Figure 21 shows an example of the overall flow and functional block configuration, including the image quality improvement process (image quality improvement engine) and the calculation of the image quality evaluation value Ei mentioned above. In particular, Figure 21 shows the adjustment and determination of the parameter P of the image quality improvement process (corresponding image quality improvement engine) using the image quality evaluation value Ei. First, as shown on the left, step S2101, in which the target image 201 is input and the image quality evaluation value Ei is calculated, is the same as steps S2 and S3 in Figure 2 mentioned above. As a result of step S2101, an image quality evaluation result 204 including the image quality evaluation value Ei is obtained.

[0195] If the computer system 3 wants to improve the image quality of the target image 201, in step S2100 it performs image quality improvement processing using an image quality improvement engine. As a result of step S2100, an image 2101, which is a high-quality image (in other words, an image with improved image quality), is obtained. The image quality improvement processing in step S2100 may include processes to improve resolution and S / N (signal-to-noise ratio). For example, in the image restoration process, a clear image with a high S / N ratio is estimated and obtained from the captured image in such a way that resolution degradation and noise superimposition are removed.

[0196] Regarding the image quality enhancement engine for the image quality enhancement processing in step S2100, the computer system 3 includes, for example, a rule-based image quality enhancement engine 2100A and a machine learning-based image quality enhancement engine 2100B. In step S2100, the computer system 3 performs image quality enhancement processing using the image quality enhancement engine selected in the settings for step S6 in Figure 6. This example shows a case where both types of image quality enhancement engines are provided, but the system is not limited to this, and depending on the system, it may be configured to provide only one type of image quality enhancement engine.

[0197] The image quality improvement process performed by the image quality improvement engine in step S2100 has one or more parameters that constitute the image quality improvement process, and these parameters (in other words, image quality improvement processing parameters) are denoted as P. Multiple parameters P are denoted as Pi (i=1 to Np), where Np is the number of parameters.

[0198] In step S2100, when using the rule-based image enhancement engine 2100A, the parameter P is an image processing parameter. When using the machine learning-based image enhancement engine 2100B, the parameter P is a machine learning model parameter, such as a CNN parameter.

[0199] During the image quality improvement process using the image quality improvement engine in step S2100, the image quality improvement process is initially performed according to the image quality improvement process setting information 2110, which includes the parameter P set in step S6 (setting step) in Figure 6.

[0200] Furthermore, the computer system 3 may use image 201 or the high-resolution image 2101 to calculate an image quality evaluation value Ei in step S2101, and then use the calculated image quality evaluation value Ei to adjust and determine the parameter P for the image quality improvement process in step S2102. In this example, the computer system 3 first obtains the high-resolution image 2101, then calculates an image quality evaluation value Ei for that image 2101, and adjusts and determines the parameter P in step S2102. As a result of step S2102, the adjusted parameter P2104 is obtained. In the adjustment in step S2102, the adjusted parameter P2104 is obtained based on the setting information 2110 for the parameter P. The setting information 2110, which includes the parameter P, may be updated by the adjusted parameter P2104.

[0201] If the adjusted parameter P2104 is obtained in step S2102, in step S2100, the input image 201 is subjected to image quality improvement processing using the adjusted parameter P2104, and as a result, a high-resolution image 2101 is obtained.

[0202] In Embodiment 1, the system has a function (particularly in step S2102) to adjust and determine the parameter P2104 of the image quality improvement process in step S2100, as shown in Figure 21, so that it becomes a more suitable value, that is, so that the image 2101 has a more suitable image quality.

[0203] For example, suppose that in step S2100, as a result of the image quality improvement processing by the rule-based image quality improvement engine 2100A, the image 2101 obtained is examined for an image quality evaluation value E1 related to edge sharpness, and a small value (i.e., an evaluation that the sharpness is not good) is output for that sharpness. In this case, in step S2102, the computer system 3 selects a parameter P for the image quality improvement processing that increases the number of image restoration processing steps for blur removal, or a parameter P that applies stronger edge enhancement processing, as an adjustment to the parameter P for the image quality improvement processing. Such adjusted parameter P is then applied to the image quality improvement processing in step S2100. After that, the computer system 3 performs the image quality improvement processing in step S2100 with the adjusted parameter P on the same image 201 (or any other image from later), and obtains an image 2101 with improved image quality (e.g., sharpness) compared to the previous image.

[0204] The computer system 3 repeats, as necessary, the image quality improvement step S2100 as shown in Figure 21, the image quality evaluation value Ei calculation step S2101, and the parameter P adjustment / determination step S2102. Through this cycle, the parameters P2104 for the image quality improvement process can be optimized so that the image quality 2101 desired by the user is obtained as a result of the image quality improvement process in step S2100.

[0205] In the above example, the computer system 3 calculates one or more image quality evaluation values ​​Ei in step S2101 and adjusts the parameter P using these image quality evaluation values ​​Ei, but it is not limited to this. In step S2101, the computer system 3 may also calculate an overall evaluation value C (for example, an overall evaluation value C that takes image quality improvement processing into account) in the same manner as in step S32 described above, and adjust the parameter P using this overall evaluation value C.

[0206] Examples of parameters P for image quality improvement processing (e.g., image restoration processing) according to the imaging conditions 205 of image 201 include chromatic aberration coefficients and spherical aberration coefficients.

[0207] [Adjustment and determination of parameter P using the target value of Ei] Using Figure 22, a method for using a target value (referred to as the region target value 2201) for the image quality evaluation value Ei will be explained in step S2102 of adjusting and determining the parameter P in Figure 21. One feature of Embodiment 1 is that the computer system 3 may individually set a region target value 2201 for some or all of the multiple image quality evaluation values ​​Ei, and adjust and determine the parameter P for the image quality improvement process so that the image quality evaluation value Ei approaches the region target value 2201.

[0208] One challenge is that even if an image with the highest possible image quality (where a larger Ei value indicates better image quality) is generated and output in the image enhancement step S2100, the image quality may not meet the user's expectations. For example, if the degree of noise suppression for noise present in an image is set to one of the image quality evaluation values ​​Ei (image quality evaluation value E3 in Figure 4), the image quality evaluation value Ei will be best for an image without noise. However, since images that users normally use for observation often contain noise, the image with the best image quality evaluation value Ei may be judged as unnatural. Therefore, in Embodiment 1, the computer system 3 individually sets target values ​​for some or all of the image quality evaluation values ​​Ei, and adjusts and determines the parameter P so that the image quality evaluation value Ei approaches the target values.

[0209] Figure 22 is an explanatory diagram illustrating the adjustment and determination of the parameters P for image quality improvement processing, using regional target values ​​which are target values ​​for the image quality evaluation value Ei, as a detailed processing example or modification of step S2102 in Figure 21. Specifically, step S2102 includes steps S2102A and S2102B. Here, the regional target value for each image quality evaluation value Ei is denoted as Ti.

[0210] First, as shown in Figure 21, the computer system 3 generates and outputs image 2101, which is the result of image quality improvement processing in step S2100, on the image 201 acquired for observation. The computer system 3 takes image 2101 as input and calculates the image quality evaluation value Ei of image 2101 in step S2101, obtaining the image quality evaluation result 204. Subsequently, in step 2102A of step S2102, the computer system 3 calculates the difference between the image quality evaluation value Ei and the pre-set region target value Ti2201. This difference is the difference for each image quality evaluation value Ei. Then, in step S2102B, the computer system 3 adjusts and determines the image quality improvement processing parameter P2104 so that the difference becomes smaller. Step S2102B may be an adjustment so that the difference becomes smaller than a threshold, or an adjustment so that the difference obtained this time is smaller than the difference obtained last time. Computer system 3 repeatedly performs this series of processes and saves and outputs parameters P that minimize the difference between the image quality evaluation value Ei and the region target value Ti2201.

[0211] Furthermore, Figure 22 shows an example of a region target value Ti in a table. This table has columns for image quality evaluation value Ei and region target value Ti. Such setting information for region target value Ti 2201 may be managed as part of the image quality evaluation value information 203 mentioned above.

[0212] Furthermore, Figure 22 illustrates an example of the configuration of the image quality improvement process performed by the image quality improvement engine 2100 in step S2100. In this example, the image quality improvement engine 2100 is configured to sequentially perform multiple processes (process 1 to process n) 2210, as shown in the flow diagram, and has parameters P (P1 to Pn) related to each process 2210. In step S2102, the computer system 3 may determine the values ​​of each of these parameters P.

[0213] Regarding the setting method for the region target value 2201, the user may directly input and set the region target value 2201 on the GUI screen as part of step S6 of the setting in Figure 6. Alternatively, a method in which the computer system 3 sets it automatically may be applied. One such method is to automatically set the region target value 2201 by using an image quality evaluation value Ei calculated from an image that meets the image quality desired by the user and has a similar appearance to the observation image 201.

[0214] The above example shows the case where a target value for the image quality evaluation value Ei is used. However, similarly, it is also possible to set a target value for the overall evaluation value C (overall target value) and determine the parameter P using the target value of the overall evaluation value C.

[0215] [Adjustment and determination of parameter P using the overall evaluation value C] Using Figure 23, we will explain how to adjust and determine the parameters P for image quality improvement processing in step S2102 of Figure 21 using the overall evaluation value C calculated in step S32 of Figure 2 (in detail in Figure 20). One of the features of Embodiment 1 is that step S2102 of Figure 21 may be adjusted and determined to improve the value of the overall evaluation value C.

[0216] One challenge is to determine the appropriate parameters P for image quality enhancement processing, even when considering multiple image quality evaluation values ​​Ei comprehensively, or when prioritizing a specific image quality evaluation value Ei among multiple image quality evaluation values ​​Ei. In response to this, Embodiment 1 adjusts and determines the parameters P for image quality enhancement processing so that the overall evaluation value C improves. In particular, the parameters P for image quality enhancement processing are adjusted and determined so that the overall evaluation value C calculated using the aforementioned priority of image quality evaluation values ​​Ei improves.

[0217] Figure 23 is an explanatory diagram regarding the adjustment and determination of parameters P for image quality improvement processing using the overall evaluation value C. Figure 23 is a modified flow chart of Figure 21. First, the computer system 3 generates and outputs an image 2101 with improved image quality from the image 201 captured for observation in step S2100. Next, in step S2301 (corresponding to steps S2 and S31 above), the computer system 3 calculates the image quality evaluation value Ei of the image 2101. Next, in step S2302 (corresponding to step S32 above), the computer system 3 calculates an overall evaluation value C, which is constructed by combining the above-calculated image quality evaluation value Ei, based on the calculation method 2005 (Figure 20) for the overall evaluation value C. The calculated overall evaluation value C, together with the image quality evaluation value Ei, is included in the image quality evaluation result 204.

[0218] Next, in step S2303, the computer system 3 adjusts and determines the parameters P2104 of the image quality improvement process (step S2100) so that the value of the overall evaluation value C improves. Step S2302 may involve adjusting the C value to be smaller than a threshold, or adjusting the C value obtained this time to be smaller than the C value obtained last time. By repeating this series of processes, the computer system 3 determines the parameters P2104 so that it can generate and output an image 2102 with the best possible image quality for the overall evaluation value C.

[0219] Furthermore, in the example in Figure 23, similar to the example in Figure 22, a target value of the overall evaluation value C, the overall target value 2305, may also be set, and in step S2303, the parameter P may be adjusted and determined so that the difference between the overall evaluation value C and the overall target value 2305 becomes small. In setting the overall target value 2305, similar to the domain target value 2201, the user may directly input and set the overall target value 2305 on the GUI screen. Alternatively, similar to the domain target value 2201, a method may be applied in which the computer system 3 automatically sets the overall target value 2305. In addition, in setting the overall target value 2305, an overall target value 2305 may be set by combining the domain target values ​​2201. If the overall target value 2305 is Tc, then Tc can be defined as a function Ft(Ti) ​​of the domain target value Ti.

[0220] [Rule-based image quality enhancement engine] Using Figure 24, we will explain the method when the image quality improvement engine (image quality improvement engine 2100 in Figure 21) in step S2100 of the image quality improvement process in Figure 21 is an image quality improvement engine 2100A composed of rule-based image processing. In this case, the parameter P2104 of the image quality improvement process is the image processing parameter of the image quality improvement engine 2100A.

[0221] One challenge was that when using a machine learning-based image quality enhancement engine to improve image quality, it was difficult to fine-tune the image quality of the output image using the trained model. In contrast, in Embodiment 1, when using the rule-based image quality enhancement engine 2100A, it becomes possible to fine-tune the image quality of the image 2102 after image quality enhancement processing.

[0222] Figure 24 is an explanatory diagram of the rule-based image enhancement engine 2100A. In this example, the computer system 3 obtains image 2102, which is the result of performing image enhancement processing on image 201, which was captured for observation, using the rule-based image enhancement engine 2100A. The image enhancement engine 2100A has n pre-designed processes 2210 (referred to as processes 1 to n) implemented (similar to Figure 22). Examples of these processes 2210 include shading correction processing, noise reduction processing, and image restoration processing.

[0223] As shown in the table in Figure 24, the computer system 3 sets the ON (perform) / OFF (do not perform) status and the value of the parameter P (P1~Pn) for each of the n processes 2210 in the image quality improvement process parameter P2104. In this example, for the target image 201, process 1, which is a filter process, is set to ON, and the filter size is set to 3x3 as the parameter P1 of process 1. On the other hand, process 2 is set to OFF. Therefore, with this parameter P setting, process 2 is not performed when the image quality improvement process in step S2100 is performed on the image 201.

[0224] As a variation, the following may also be used. In the example in Figure 24, the rule-based image quality improvement engine 2100A generates intermediate images 2401 between each of the multiple processes 2210. For example, a first intermediate image 2401 is generated as the output of process 1, and a second intermediate image 2401 is generated as the output of process 2. In the variation, as also shown in Figure 24, the computer system 3 may calculate the image quality evaluation value Ei for the intermediate images 2401 generated during the image quality improvement process of the image quality improvement engine 2100A. In addition, the image quality evaluation value Ei may be calculated for intermediate images 2401 generated by specific multiple processes. For example, the image quality evaluation value Ei may be calculated for intermediate images 2401 generated by executing only processes 2 and 3. Settings for this can also be set in step S6. For example, it is possible to set which intermediate images 2401 of which process 2210 are targeted and which image quality evaluation value Ei is calculated. The resulting image quality evaluation value Ei can be used to adjust and determine the parameter P, as described above.

[0225] [Machine learning-based image quality enhancement engine] Using Figure 25, we will explain the method when the image quality improvement engine in step S2100 of Figure 21 is an image quality improvement engine 2100B consisting of machine learning-type image processing provided by a CNN, for example. In this case, the parameters P2104 of the image quality improvement process are model parameters such as the weights (in other words, connection coefficients) between nodes of the CNN network and bias.

[0226] One challenge is that it can be difficult to pre-design the processing necessary for improving image quality. Because the appearance variations of the semiconductor patterns targeted in observation images are diverse, it is difficult to design a rule-based image quality improvement engine that can handle all observation images. In contrast, Embodiment 1 addresses this by using an image quality improvement engine 2100B, which is composed of machine learning-type image processing provided by a CNN, for example.

[0227] Figure 25 is an explanatory diagram of the machine learning-type image enhancement engine 2100B. In this example, the computer system 3 uses the machine learning-type image enhancement engine 2100B to perform image enhancement processing on the image 201 acquired for observation, and obtains the resulting image 2102. In this example, the machine learning-type image enhancement engine 2100B uses a CNN with a three-layer structure as the machine learning model 2500. This CNN model 2500 calculates intermediate data Z1(g), Z2(g) and an estimated result Z(g) from the input image g, and outputs the estimated result Z(g) as the high-resolution image 2102. The computer system 3 sets the model parameters of the image enhancement engine 2100B model 2500 in the image enhancement processing parameter P2104. In this example, as shown in the table, the bias of Z1(g) is set to 0.5 and the weight between Y1 and Y2 is set to 0.05 in parameter P2104.

[0228] Furthermore, the neural network configuration of the 2100B image quality enhancement engine is not limited to the example above; other configurations may be applied. For example, the number of layers in the CNN may be changed, a network with four or more layers may be used, or a configuration with skip connections may be adopted.

[0229] In the adjustment and determination of the parameter P2104 for the machine learning-type image quality enhancement engine 2100B (step S2102 in Figure 21), for example, the image quality evaluation value Ei or the overall evaluation value C is used as the loss function during training of model 2500. This allows for the efficient determination of model parameters that improve image quality.

[0230] [Correction of evaluation subregion] Next, the function of correcting the evaluation region Si will be explained. In step S4 of Figure 2, the computer system 3 displays the image quality evaluation value Ei to the user on the GUI screen, and then the user performs user settings, including region correction (referring to the correction of the evaluation region Si). The computer system 3 uses the user setting information 202, which includes region correction information obtained from the user settings, as input information for setting step S6 (especially steps S21 and S22). Then, the processing content of steps S21 and S22 is determined to be more favorable based on the settings in step S6. Through this cycle, the evaluation region Si determined in step S22 is optimized, and as a result, a more favorable image quality evaluation value Ei can be calculated.

[0231] Figure 28 shows a flow chart including the user setting step S7 for each function, which includes the function for correcting the evaluation subregion Si. The flow chart in Figure 28 is based on the flow chart in Figure 2. In the flow chart in Figure 28, based on the image quality evaluation value Ei obtained in the aforementioned steps S1, S2, and S3, the image quality evaluation value Ei is displayed on the GUI screen in step S4, and then a user setting step S7 is provided. In this step S7, the computer system 3 provides a GUI screen for user setting, and the user uses each setting function by performing input operations while viewing the GUI screen, and the result of the user setting is saved as user setting information 202.

[0232] Step S7 in Figure 28 includes, as an example of functionality, step S71 for setting the image quality evaluation value Ei, step S72 for correcting the evaluation sub-region Si, and step S73 for benchmarking (comparison of image quality between images), which will be described later. The user can use the function selected on the GUI screen. Step S71 corresponds to a function that allows the user to check and set the image quality evaluation information 203 as shown in Figure 3. This function includes a function to add and set the image quality evaluation value Ei (step S65 in Figure 6). Step S72 corresponds to the region correction function described above. The user setting information 202 (including the image quality evaluation information 203 in this case) from the result of step S7 is used for setting step S6 in Figure 6.

[0233] [GUI(1)] First, we will describe examples of GUIs related to user settings (step S7) and the display of image quality evaluation value Ei (step S4) in the sample observation system of Embodiment 1.

[0234] In Embodiment 1, the user is shown the target image 201, the evaluation region Si, and the image quality evaluation value Ei on a GUI screen. Furthermore, in Embodiment 1, in accordance with the region correction function, a GUI is provided on the GUI screen that allows the user to manually correct the evaluation region Si. Also in Embodiment 1, a GUI is provided that allows the user to set the aforementioned internal parameter Qi_m (Figure 16) and the region target value Ti (Figure 22).

[0235] Figure 26 shows an example of a GUI screen equipped with the above GUI. The screen in Figure 26 is an example of a GUI screen related to step S2101 (Figure 21) of the image quality improvement process and its associated functions. The screen in Figure 26 has an interface area (image list field) 2602, an interface area (observation image field) 2603, an interface area (image quality to be evaluated field) 2604, an interface area (image quality field) 2605, an interface area (partial evaluation area field) 2606, an interface area (internal parameter field) 2607, an interface area (image quality improvement parameter field) 2608, an interface area (image quality improvement image field) 2610, an interface area (evaluation value field) 2611, an interface area (target value field) 2612, an interface area (OK / NG field) 2613, etc.

[0236] The interface area (image list section) 2602 displays a list of images 201 (referred to here as observation images) acquired in imaging step S2 (Figure 2). The interface area (observation image section) 2603 displays the observation image with the ID selected in the list.

[0237] The interface area (evaluation target image quality section) 2604 displays a list of image quality evaluation values ​​Ei corresponding to the image quality to be evaluated. From this list, the user can determine the image quality evaluation value Ei corresponding to the image quality to be evaluated for the target image, for example, by selecting it using checkboxes. The image quality evaluation value Ei corresponding to the image quality selected with checkboxes can then be specified in interface area 2605. In interface area (image quality section) 2605, for example, when the user manipulates a list box, image quality evaluation values ​​Ei are displayed as options, and the user can select the desired image quality evaluation value Ei. The image quality evaluation value Ei of the image quality selected in interface area 2605 is then applied to interface areas 2603, 2610, 2606, etc.

[0238] The interface area (evaluation sub-area field) 2606 displays the evaluation sub-area Si (determined in step S22) used to calculate the image quality evaluation value Ei selected in the interface area 2605.

[0239] The interface area (internal parameter field) 2607 displays the internal parameter Qi_m of the selected image quality evaluation value Ei. In the interface area 2607, for example, the user can operate a list box to select the internal parameter Qi_m to display, and the parameter value of that internal parameter Qi_m (for example, "0.1") will be displayed.

[0240] In the interface area (image quality improvement parameter section) 2608, the user can manually adjust and set the parameters P2104 of the image quality improvement engine (step S2100 in Figure 21). In the interface area 2608, the parameters P2104 of the image quality improvement engine are displayed for each individual parameter P, for example, using sliders. The user can adjust the value of each parameter P (for example, #1 to #4) by changing the position of the slider. The default value of each parameter P is also represented by a gray slider.

[0241] When the user presses the interface area (image quality improvement processing execution button) 2615, image quality improvement processing (step S2100) is executed using the value of parameter P represented by the slider in interface area 2608. The image quality improved image 2102 generated and output by this image quality improvement processing is displayed in the interface area (image quality improved image section) 2610. The user can compare and confirm the observed image in interface area 2603, the image quality improved image in interface area 2610, and the evaluation sub-area Si in interface area 2606.

[0242] Furthermore, when the user presses the interface area (auto-adjust button) 2609, the computer system 3 performs an automatic adjustment of the parameter P2104 of the image quality enhancement engine (step S2100) and adjusts and determines its parameter value. The position of the slider for parameter P in the interface area 2608 is automatically changed according to the adjusted value of parameter P. The image 2102, which has been enhanced in image quality by the image quality enhancement engine using the automatically adjusted parameter P, is displayed in the interface area 2610.

[0243] The image quality evaluation value Ei for the image quality improvement image in interface area 2610, as specified in interface area 2605, is displayed in interface area (evaluation value column) 2611. In addition, the region target value Ti related to that image quality evaluation value Ei is displayed in interface area (target value column) 2612. The user checks the image quality improvement image in interface area 2610, the image quality evaluation value Ei in interface area 2611, and the region target value Ti in interface area 2612, and determines whether an image quality improvement image with satisfactory quality has been obtained (OK / NG). The user inputs or selects the result of this determination in interface area (OK / NG column) 2613.

[0244] Furthermore, in the interface area (design data section) 2614, you can specify the design data 206 (Figure 6) used in the classification of the classification division area Rj necessary to determine the evaluation sub-area Si (step S21 in Figure 2).

[0245] [GUI(2)] Figure 27 then shows an example of a GUI screen related to the function of correcting the evaluation subregion Si. The screen in Figure 27 has a GUI related to the correction of the evaluation subregion Si and includes interface areas 2716, 2717, 2718, 2719, 2720, 2721, 2724, etc. Note that the screen in Figure 27 may be integrated with the screen in Figure 26, or it may be a separate GUI to which it is transitioned.

[0246] The evaluation region Si may not be determined to the area expected by the user due to factors such as the structure of the semiconductor pattern and the size of the defect area in the image. In such cases, the evaluation region Si can be corrected (in other words, manually modified) based on the user's input using the screen shown in Figure 27.

[0247] The interface area (inspection image list section) 2716 displays a list of images 201 (observation images) acquired in imaging step S1 (Figure 2). The observation image with the ID selected by the user from this list is displayed in the interface area (observation image section) 2717. The interface area (evaluation target image quality section) 2718 displays a list of image quality evaluation values ​​Ei corresponding to the image quality to be evaluated. The interface area (image quality section) 2719 displays the image quality evaluation value Ei selected by the user, using the image quality evaluation value Ei selected in interface area 2718 as an option. Each of these sections has the same function as the corresponding sections in Figure 26.

[0248] The interface area (pre-correction evaluation area field) 2720 displays the pre-correction evaluation area Si. This field displays the evaluation area Si (determined in step S22) used to calculate the image quality evaluation value Ei selected in interface area 2719.

[0249] The user specifies the evaluation target image quality corresponding to the evaluation subregion Si that they want to correct in interface region 2719. Interface region 2720 displays the evaluation subregion Si used to calculate the image quality evaluation value Ei corresponding to the image quality selected in interface region 2719. In this example, this evaluation subregion Si has a location a1, indicated by a dashed line, within the vertical line pattern (white area), where two adjacent line patterns are connected horizontally, and this location is the area the user wants to correct. This location a1 corresponds to a location that appears to be a defect area in the observation image in interface region 2717.

[0250] Furthermore, the interface area (correction input field) 2721 initially displays the same evaluation sub-area Si as shown in interface area 2720, for correction input. In interface area 2721, the user inputs the boundary of the evaluation sub-area Si (white area) as correction information (in other words, manual correction information) using an input device such as a mouse, for example, as shown by the dotted line 2722. In this example, at location a1, the dotted line 2722 is input to draw a new boundary for the vertical line pattern so that the part where the vertical line patterns are connected horizontally is not included in the evaluation sub-area Si. The method of inputting correction information is not limited to this, and in other examples, methods such as coloring the pixel area to be included in the evaluation sub-area Si, or conversely, coloring the pixel area not to be included in the evaluation sub-area Si, may be used.

[0251] After entering the above correction information, the user presses the interface area (correction execution button) 2723. This causes the computer system 3 to use the correction information in the interface area 2721 to re-determine the evaluation subregion Si (step S22), and display the determined evaluation subregion Si in the interface area 2724. The corrected evaluation subregion Si is then displayed in the interface area (corrected evaluation subregion field) 2724.

[0252] If the user checks the corrected evaluation partial area Si in the interface area 2724 and determines as a result that the correction is successful, the user completes the correction by pressing the interface area (correction completion button) 2725. If the user determines that the correction is not successful, the user inputs correction information again using the interface area 2721 and performs the correction operation similarly.

[0253] In this embodiment, regarding the determination of a suitable evaluation partial area Si, an example where the user can manually correct the boundary of the evaluation partial area Si is shown, but it is not limited thereto, and the user may change the parameters of the segmentation method or the machine learning model in step S21 of region division / classification (step S61 in FIG. 6). Further, the user may change the method of generating the evaluation partial area Si from the classification division area Rj (step S62 in FIG. 6).

[0254] [Benchmark using the image quality evaluation value Ei] Using FIG. 29, the steps of the benchmark using the image quality evaluation value Ei (step S73 in FIG. 28) will be described. In Embodiment 1, regarding the image quality evaluation value Ei of the same image quality, based on the result of comparison between the image quality evaluation value Ei_1 of the first image and the image quality evaluation value Ei_2 of the second image, one of the features is to have the steps and functions of a benchmark for determining the superiority or inferiority of the image quality between the first image and the second image.

[0255] As a problem, conventionally, when determining the superiority or inferiority of the image quality of two images, since a person visually checks and determines the image quality, there is a point that the judgment criteria are ambiguous. In contrast, in Embodiment 1, by using the image quality evaluation value Ei, quantitative comparison of the image quality becomes possible.

[0256] In FIG. 28, in step S73 of the benchmark, the computer system 3 compares the specified first image and second image with the image quality evaluation value Ei of the specified image quality based on the image quality evaluation result 204 including the image quality evaluation value Ei, and outputs the comparison result to the user.

[0257] FIG. 29 shows an example of a GUI screen related to step S73 of the benchmark. In the screen of FIG. 29, an example of outputting two images and a comparison result is shown. The screen of FIG. 29 has GUIs such as a benchmark (or rather, a comparison result) 2900, a recommended image 2910, and an observed image selection 2920.

[0258] The benchmark 2900 is in tabular form, and for the first image 2901 and the second image 2902, which are two images for image quality comparison, their respective image quality evaluation values Ei (E1 to EN) and the comprehensive evaluation value C are displayed in parallel. The user can compare the two images and each image quality evaluation value Ei and the comprehensive evaluation value C in the benchmark 2900.

[0259] For example, for the image quality evaluation value E1 "sharpness" (FIG. 3), the E1 value of the second image 2902 is larger than the E1 value of the first image 2901. For example, it is defined that a larger E1 value indicates better image quality. Therefore, for the image quality "sharpness" evaluated by the image quality evaluation value E1, the user can judge that the second image 2902 is superior to the first image 2901 without relying solely on visual judgment of the two images. The same applies to other image quality evaluation values Ei.

[0260] Also, the comprehensive evaluation value C is similarly compared between the first image 2901 and the second image 2902. This makes it possible to judge which image is comprehensively superior in image quality for the entire image. For example, it is defined that a larger value of the comprehensive evaluation value C indicates comprehensively better image quality. In this example, since the comprehensive evaluation value C of the second image 2902 is higher, the user can judge that the second image 2902 is comprehensively superior in image quality.

[0261] Furthermore, the computer system 3 may have a function to automatically recommend to the user which of the two images is more suitable to use as an observation image, based on the comparison result 2900 in step S73 of the benchmark described above. For example, the computer system 3 may determine that, overall, the second image 2902 is more suitable as an observation image, based particularly on the overall evaluation value C in the comparison result 2900. Then, in the recommended image 2910 area, the computer system 3 displays whether the first image 2901 and the second image 2902 are recommended or not as observation images. In this case, the computer system 3 may, for example, determine that an image is "recommended" if its overall evaluation value C is above a threshold, or it may determine that the image with the highest overall evaluation value C among multiple images (in this case, two images) is "recommended".

[0262] When using the above recommendation function, the user checks the quality of the recommended image (e.g., the second image 2902) on the GUI screen's recommended image 2910 compared to other images (e.g., the first image 2901), and inputs the image to be used as the observation image based on the decision. For example, in the observation image selection area 2920, the user can select which of the two images to use as the observation image. The computer system 3 saves the image selected by the user on the GUI screen as the observation image as user setting information 202. Based on this user setting information 202, the specified image will be used as the observation image (the image used in the observation processing operation in step S5 of Figure 2).

[0263] Furthermore, the computer system 3 inputs user setting information 202, including the specification of the observation image, into step S63 of the setting step S6 in Figure 28. In step S63, the computer system 3 automatically determines the method for calculating the image quality evaluation value Ei in step S31 based on the user setting information 202. At that time, the computer system 3 determines the method for calculating the image quality evaluation value Ei so that the image quality evaluation value Ei is larger for the image with better image quality that has been designated as the observation image.

[0264] Furthermore, the computer system 3 inputs user setting information 202, including the specification of the observation image, into step S64 of step S6. In step S64, the computer system 3 automatically determines the method for calculating the overall evaluation value C in step S32 based on the user setting information 202. The computer system 3 determines the method for calculating the overall evaluation value C so that the overall evaluation value C is larger for the image with better image quality that has been designated as the observation image.

[0265] [Defect observation using image quality evaluation value Ei] In the example shown in Figure 2 above, step S5 illustrates the case where the image quality evaluation value Ei is used for defect observation of image 201. In step S5, the computer system 3, for example, if the image quality evaluation value Ei of image 201 is sufficiently high, for example above a threshold, uses image 201 as an observation image (in other words, an inspection image) and performs defect observation (in other words, defect inspection, defect detection). For example, an image 201 with a sufficiently high image quality evaluation value E2 "visibility of defects" is selected as an observation image. In that observation image, defects are determined and detected using a predetermined defect observation method (for example, a method of comparing with a reference image).

[0266] A detailed example of defect observation using the image quality evaluation value Ei (step S5) will be explained using Figure 30. One of the features of Embodiment 1 is that it includes a defect detection step in which defects in an image of a semiconductor wafer are detected based on the image quality evaluation value Ei.

[0267] One challenge was that when the image quality of the image targeted for defect detection changed, over-detection (in other words, false detection) or missed defects could occur. In response to this, Embodiment 1 performs defect detection processing based on the image quality evaluation value Ei.

[0268] Figure 30 shows step S3000 of defect detection using the image quality evaluation value Ei. Figure 30 shows an example of adjusting the sensitivity of defect detection based on the image quality evaluation value Ei. The computer system 3 uses the image quality evaluation value Ei (including the image quality evaluation result 204) obtained in steps S2 and S3 described above to perform the defect detection process in step S3000 and obtain the resulting defect detection result 3010.

[0269] One common defect detection method involves comparing an inspection image (in other words, a target image) that captures the location of the defect with a reference image that does not contain the defect, and then examining the difference between them to determine and detect the defect. The defect detection process in step S3000 of Figure 30 uses such a method.

[0270] First, the computer system 3 receives an inspection image 3001 in which the defect site is captured and a reference image 3002 that does not contain the defect. In step S3001, it performs a difference calculation on these images to obtain and output a difference detection result 3004.

[0271] The difference detection result 3004 detects differences that occurred in the defective area, but also detects differences that occurred due to the influence of noise and other factors. Therefore, in Embodiment 1 (Figure 30), the computer system 3 calculates the image quality evaluation value Ei3006 for the inspection image 3001 in step S3002, which corresponds to steps S2 and S3 of the image quality evaluation value calculation in Figure 2 described above.

[0272] Next, in step S3002, the computer system 3 adjusts the sensitivity of defect detection based on the image quality evaluation value Ei3006. This sensitivity is the sensitivity used in determining and detecting defective parts in the defect detection method used for defect detection processing. This sensitivity is not particularly limited, but examples include parameter values ​​such as the weight when determining defective parts based on differences, and the threshold when removing false information.

[0273] In this example, the defect detection sensitivity adjusted in step S3002 is used for false alarm removal in step S3004. In step S3004, the computer system 3 removes false alarms from the difference detection result 3004 before false alarm removal, according to the sensitivity, and outputs the result as the difference detection result 3009 after false alarm removal. Based on the difference detection result 3009, the defect detection result 3010 is generated and output. In this example, false alarms are removed from the area indicated in white, which is presumed to be a defect, included in the difference detection result 3004, and as a result, the defect area a1 indicated in white is detected in the difference detection result 3004.

[0274] In this embodiment, we have described a case where the sensitivity of defect detection is adjusted based on the image quality evaluation value Ei. However, instead of this, or in addition to this, it is also possible to change the feature quantities in defect detection based on the image quality evaluation value Ei, or to change the pre-processing and post-processing.

[0275] In this embodiment, a method using a reference image for defect detection has been described. However, the method is not limited to this; a method in which a reference image is estimated from the inspection image 3001 and compared with the estimated reference image may also be applied. Alternatively, a method in which only the inspection image 3001 is input to a machine learning model and the defect detection result is output as an estimation result by the model may also be applied.

[0276] [Shape measurement using image quality evaluation value Ei] Figure 31 illustrates the shape measurement using the image quality evaluation value Ei. One of the features of Embodiment 1 is that it includes a shape measurement step in which the shape (in other words, the measured length value, CD: Critical Dimension) in the image captured of the semiconductor wafer is measured based on the image quality evaluation value Ei.

[0277] One challenge was that if the image quality of the image being measured changed, the signal used for measurement would change, resulting in a change in the measured length value. In contrast, Embodiment 1 performs shape measurement based on the image quality evaluation value Ei.

[0278] Figure 31 shows step S3100 of shape measurement using the image quality evaluation value Ei. In this embodiment, step S3100 of shape measurement in Figure 31 shows an example of adjusting the length measurement threshold for image shape measurement based on the image quality evaluation value Ei obtained as a result of steps S2 and S3 in Figure 2 described above. The computer system 3 performs shape measurement by inputting the image 3101 to be measured and analyzing the signals of the measurement points in the image 3101. In this example, the case of measuring the wiring width between A and B in the image 3101 will be described.

[0279] First, in step S3101 of signal analysis, the computer system 3 analyzes the signal (or profile) between A and B in the image 3101 and generates and outputs a signal analysis result 3103.

[0280] Next, the computer system 3 calculates an image quality evaluation value Ei3105 in step S3102 corresponding to steps S2 and S3 for calculating the image quality evaluation value in Figure 2 described above for the image 3101. Next, the computer system 3 adjusts and determines the length measurement threshold in step S3103 based on the image quality evaluation value Ei3105. After that, the computer system 3 performs shape measurement on the signal analysis result 3103 using the determined length measurement threshold in step S3104, and as a result, generates and outputs a shape measurement result 3110 including a length measurement value 3108.

[0281] In this embodiment, an example of adjusting the length measurement threshold for shape measurement based on the image quality evaluation value Ei has been described. However, the present invention is not limited to this, and changes in feature amounts in shape measurement based on the image quality evaluation value Ei, changes in pre-processing and post-processing, etc. may be performed.

[0282] [Adjustment of imaging conditions using the image quality evaluation value Ei] The adjustment of imaging conditions using the image quality evaluation value Ei will be explained using Figure 32. One of the features of Embodiment 1 is that it includes a step of determining the imaging conditions for image 201 (imaging conditions 205 in Figure 6) based on the image quality evaluation value Ei. The computer system 3 adjusts and determines suitable imaging conditions 205 using the image quality evaluation value Ei. Alternatively, the computer system 3 may determine the imaging conditions 205 using a target value of the image quality evaluation value Ei (region target value Ti in Figure 22).

[0283] One challenge was that when adjusting the image acquisition conditions, it was necessary for a person to visually evaluate the image quality, which made the adjustment process time-consuming and laborious. In contrast, Embodiment 1 quantifies the image quality, eliminating the need for visual evaluation and reducing the time and effort required to adjust the acquisition conditions.

[0284] Figure 32 shows the adjustment of imaging conditions 205 using the image quality evaluation value Ei. Figure 32 is based on Figure 2 but differs in some parts. First, the computer system 3 calculates the image quality evaluation value Ei for the image 201 captured in step S1 based on certain imaging conditions 205 using the aforementioned steps S2 and S3, and obtains the image quality evaluation result 204. In step S4, the computer system 3 may display the image quality evaluation value Ei and imaging conditions 205 to the user.

[0285] Next, in step S3200, the computer system 3 adjusts and determines (in other words, updates) the imaging conditions 205 for image 201 based on the image quality evaluation value Ei of image 201. As a result, the updated imaging conditions 205B are obtained. These imaging conditions 205B become the new imaging conditions 205.

[0286] In step S3200, for example, if the image quality evaluation value E5 "degree of dark area enhancement" (Figure 3) is small, the computer system 3 changes the acceleration voltage, which is one of the parameters of the imaging condition 205, to a larger value in order to improve the visibility of the dark areas in the image 201.

[0287] Computer system 3 repeats the series of processes shown in Figure 32 using the updated imaging conditions 205B. This determines more suitable imaging conditions 205 to enable the acquisition of images 201 with better image quality. In other words, computer system 3 optimizes the imaging conditions 205, taking into account the image quality evaluation value Ei.

[0288] In adjusting and determining the imaging conditions 205 in Figure 32, similar to adjusting the parameter P of the image quality improvement engine (Figure 21) described above, a regional target value Ti may be set for some or all of the multiple image quality evaluation values ​​Ei, and the imaging conditions 205B may be determined so that the image quality evaluation value Ei approaches the regional target value Ti.

[0289] [Monitoring of sample observation device using image quality evaluation value Ei] Using Figure 33, we will explain the monitoring of the sample observation device using the image quality evaluation value Ei. One of the features of Embodiment 1 is that it includes a device monitoring step in which the sample observation device is monitored based on the image quality evaluation value Ei. The sample observation device to be monitored here is the sample observation device 1 of Embodiment 1 in Figure 1 (in other words, the device itself).

[0290] One challenge was that when using captured images to determine whether there was an abnormality in the sample observation device, it was necessary for a person to visually check and judge the image, which resulted in labor costs. In contrast, Embodiment 1 uses an image quality evaluation value Ei to quantitatively determine whether there is an abnormality in the sample observation device 1. This eliminates the need for visual inspection by a person, thereby reducing labor costs.

[0291] Figure 33 shows step S3300 of monitoring the sample observation device using the image quality evaluation value Ei. First, in step S3301, the computer system 3 inputs the image 201 captured by the SEM2 of the sample observation device 1 shown in Figure 1, and calculates the image quality evaluation value Ei3302 for that image 201 in step S3301, which corresponds to steps S2 and S3 in Figure 2. Then, in step S3302, the computer system 3 uses the image quality evaluation value Ei3302 and the normal range information 3303 for the image quality evaluation value Ei as input to determine whether there is an abnormality in the sample observation device 1, in other words, the device status. For example, if the computer system 3 determines that the device status is normal (in other words, no abnormality) if the image quality evaluation value Ei is within the normal range information 3303, it may determine that the device status is abnormal (in other words, abnormal) if it is outside the normal range.

[0292] The normal range information 3303 for image quality evaluation value Ei defines the range of values ​​considered normal for each image quality evaluation value Ei. This normal range information 3303 may be manually set by the user in user settings (step S7). Alternatively, this normal range information 3303 may be automatically set by the computer system 3. In that case, for example, the computer system 3 may set the normal range information 3303 using the statistical distribution of image quality evaluation value Ei of images captured by the sample observation device 1 under normal conditions.

[0293] If the result of step S3302 determines that an abnormality has occurred in the device (abnormality present) (YES), the process proceeds to step S3303. If the result determines that no abnormality has occurred (no abnormality) (NO), the process proceeds to step S3304. In step S3303, the computer system 3 performs an abnormality cause analysis related to the above abnormality and outputs an abnormality cause 3306 (in other words, the abnormality cause analysis result) as a result.

[0294] For example, if the image quality evaluation value E1 "Sharpness" (Figure 3) of image 201 is found to be abnormal, the computer system 3 will determine and output in step S3303 that there is an abnormality (in other words, an abnormality factor) in the electrical axis of the SEM2 (Figure 1) related to the sharpness of image 201. The electrical axis refers to the trajectory of the electron beam emitted from the electron gun 110.

[0295] In step S3305, the computer system 3 displays the device status (abnormal) and the abnormality factor 3306, for example, on a GUI screen, which the user can then review. Based on the abnormality factor 3306, the user can take corrective action, such as adjusting the position of the part of the sample observation device 1 associated with the abnormality factor 3306, or performing maintenance or replacement.

[0296] Meanwhile, in step S3304, the computer system 3 outputs normal information 3307, which indicates that there is no abnormality in the device status.

[0297] [Effects of Embodiment 1, etc.] As described above, the sample observation apparatus and method of Embodiment 1 enable the quantification of the image quality of the observed image. According to Embodiment 1, a suitable evaluation subregion Si can be used for each image quality evaluation value Ei corresponding to various image quality levels, thereby enabling optimal image quality quantification. According to Embodiment 1, various effects can be achieved through various functions (for example, adjusting the parameters of the image enhancement processing described above) using the quantified image quality evaluation value Ei.

[0298] Although embodiments of this disclosure have been specifically described above, the invention is not limited to the embodiments described above, and various modifications are possible without departing from the gist of the invention. Each embodiment allows for the addition, deletion, and replacement of components, except for essential components. Unless otherwise specified, each component may be singular or plural. Combinations of each embodiment are also possible. [Explanation of Symbols]

[0299] 1...Sample observation device, 2...SEM (Scanning Electron Microscope), 3...Computer system, 10...Sample.

Claims

1. A computer system in a sample observation device for observing a sample, The computer system has one or more processors and one or more memories. The aforementioned processor, Multiple evaluation regions are determined from the image captured of the aforementioned sample. Based on the aforementioned multiple evaluation sub-regions, an image quality evaluation value is calculated for each evaluation sub-region, thereby calculating multiple image quality evaluation values ​​for evaluating multiple different image qualities. Computer system.

2. In the computer system described in claim 1, The aforementioned multiple image quality evaluation values ​​include at least two of the following: image sharpness, defect visibility, noise suppression degree, interlayer contrast, dark area enhancement degree, shape preservation degree, roughness preservation degree, ringing suppression degree, and image naturalness. Computer system.

3. In the computer system described in claim 1, The aforementioned processor, Classification is performed for each of the classification regions obtained by dividing the aforementioned image. Based on the classification results of the classification division region, the evaluation sub-region is determined. Computer system.

4. In the computer system described in claim 3, The processor performs the classification using sample information representing the structure of the semiconductor device, which is the sample. Computer system.

5. In the computer system described in claim 3, The processor performs the classification using the image acquisition conditions of the image. Computer system.

6. In the computer system described in claim 3, The processor generates the evaluation subregion by combining, dividing, expanding, contracting, or a combination thereof the classification subregions. Computer system.

7. In the computer system described in claim 3, The classification of the aforementioned classification division region includes at least two of the following in the semiconductor device sample: the wiring top surface region, edge region, dark region, and defect region. Computer system.

8. In the computer system described in claim 1, The processor calculates an overall evaluation value which is composed of the multiple image quality evaluation values. Computer system.

9. In the computer system described in claim 1, The aforementioned image quality evaluation value is calculated using internal parameters based on the settings. The processor, based on the input of a user's evaluation of the image quality, determines a method for calculating the image quality evaluation value using the internal parameters such that the image quality evaluation value matches or approaches the user's evaluation. Computer system.

10. In the computer system described in claim 1, The aforementioned image quality evaluation value is defined by a combination of multiple elemental image quality evaluation values ​​based on the settings. Computer system.

11. In the computer system described in claim 1, The processor determines the parameter values ​​for image quality improvement processing of the image based on the image quality evaluation value. Computer system.

12. In the computer system according to claim 11, The aforementioned processor, A target value is individually set for one or more of the aforementioned multiple image quality evaluation values. The parameter values ​​of the image quality improvement process are adjusted so that the image quality evaluation value approaches the target value. Computer system.

13. In the computer system according to claim 11, The aforementioned processor, A comprehensive evaluation value is calculated by combining the aforementioned multiple image quality evaluation values. The parameter values ​​of the image quality improvement process are adjusted so that the overall evaluation value improves. Computer system.

14. In the computer system described in claim 1, The aforementioned processor, The aforementioned image, the evaluation portion region, and the image quality evaluation value are displayed on the screen. Based on user input to the aforementioned screen, the evaluation sub-region is corrected. Computer system.

15. In the computer system described in claim 1, The aforementioned processor, As the aforementioned images, a benchmark was performed to compare the image quality of the first image and the second image. In the benchmark, the image quality evaluation value of the first image and the image quality evaluation value of the second image are compared, and the comparison result is displayed on the screen. Computer system.

16. In the computer system described in claim 1, The processor calculates an image quality evaluation value for the image of the semiconductor wafer, which is the sample, selects an image for defect observation based on the image quality evaluation value, and performs defect observation on the selected image. Computer system.

17. In the computer system described in claim 1, The aforementioned processor, The image quality evaluation value is calculated for the image obtained by imaging the semiconductor wafer, which is the sample. Based on the aforementioned image quality evaluation value, the imaging conditions when capturing the image are adjusted. The image is captured using the adjusted imaging conditions. Computer system.

18. In the computer system described in claim 1, The processor, as part of monitoring the status of the sample observation device, determines whether an abnormality has occurred in the sample observation device based on the image quality evaluation value. Computer system.

19. Method corresponding to claim 1 A method for observing a sample in a sample observation device for observing a sample, The computer system in the sample observation apparatus has one or more processors and one or more memories. The steps performed by the aforementioned processor include: The steps include determining multiple evaluation regions from an image of the aforementioned sample, The steps include calculating multiple image quality evaluation values ​​for evaluating different image quality by calculating an image quality evaluation value for each evaluation sub-region based on the aforementioned multiple evaluation sub-regions, A sample observation method having the following characteristics.

20. Program corresponding to claim 1 A program for causing a computer system to execute the sample observation method described in claim 19.

Images