Image processing method, shape inspection method, image processing system, and shape inspection system

Abstract

This invention provides an image processing method, a shape inspection method, an image processing system, and a shape inspection system. The image processing method uses a system equipped with an input receiving unit, an estimation unit, and an output unit to acquire data for an estimated photographic image used when comparing an estimated photographic image obtained based on reference data of a sample with an actual photographic image of the sample. The method includes: an input step, where the input receiving unit receives reference data, process information of the sample, and learned model data; an estimation step, where the estimation unit uses the reference data, process information, and model data to calculate a photographic image statistic representing the probability distribution of the possible values ​​that the data in the photographic image can take; and an output step, where the output unit outputs the photographic image statistic, and the estimated photographic image can be generated based on the photographic image statistic. Therefore, when comparing a simulated image estimated based on design data with an actually photographed image, the estimation time can be shortened, and the comparison can be performed in real time.

Description

Technical Field

[0001] This invention relates to image processing methods, shape inspection methods, image processing systems, and shape inspection systems. Background Technology

[0002] Currently, in order to conduct evaluations (such as defect inspections) and dimensional measurements using image data, the design data and captured images are compared for items that are the objects of evaluation or dimensional measurement. An example of such an item is a semiconductor circuit.

[0003] In the inspection and measurement of semiconductor circuits (hereinafter referred to as "circuits"), the design data of the circuit and the captured image data (hereinafter referred to as "captured images") are compared to align their positions. This process is called pattern matching.

[0004] By aligning the design data and the positions of the captured images, it is possible to specify measurement points and evaluate the degree of deviation of the circuit shape from the design data. Shape deformations exist in the circuit due to various conditions set during the manufacturing process. Furthermore, differences in image quality (contrast variations, image noise, etc.) occur in the captured images of the circuit due to various conditions set during the shooting process. In addition, even under the same conditions, the shape of the circuit and the image quality can vary due to fluctuations.

[0005] For example, in pattern matching, when design data is used directly as a template image, positional alignment becomes difficult due to the difference between the circuit shape in the design data and the circuit shape in the captured image. Therefore, in the template image, it is preferable to use data that closely resembles the circuit shape in the captured image, rather than using the design data directly.

[0006] Patent Document 1 discloses a computer installation method for generating simulated images based on design information, comprising: a step of determining features of design information of an object by inputting design information into two or more encoder layers of a generation model; and a step of generating one or more simulated images by inputting the determined features into two or more decoder layers of the generation model. Here, the simulated image represents the design information expressed by an image of an object generated by an image system. Patent Document 1 discloses that the generation model can be replaced by a convolutional neural network (CNN).

[0007] Patent Document 2 discloses a pattern inspection system that uses an image of an object pattern of an electronic device and data used to manufacture the object pattern to inspect the image of the object pattern using a recognizer constructed through machine learning. The pattern inspection system stores multiple pattern images of the electronic device and pattern data used to manufacture the electronic device. Based on the stored pattern data and pattern images, it selects a learning pattern image for machine learning from multiple pattern images. This saves the time of creating the truth value of the learning data, reduces the amount of learning data, and shortens the learning time.

[0008] Prior art literature

[0009] Patent documents

[0010] Patent Document 1: US Patent No. 9965901

[0011] Patent Document 2: Japanese Patent Application Publication No. 2020-35282 Summary of the Invention

[0012] The problem that the invention aims to solve

[0013] According to the method disclosed in Patent Document 1, when applied to the circuit pattern of the object to be inspected, although a circuit pattern as a simulated image can be obtained, since the input is only design data, the differences in conditions (hereinafter also referred to as "process information") such as manufacturing processes and photographing processes cannot be clearly specified. In order to obtain the differences in conditions, it is necessary to prepare a dataset including photographed images of circuits manufactured or photographed under these conditions, and to learn the mathematical model used for simulation according to the conditions.

[0014] To understand the impact of process information on circuits and their captured images, multiple simulations had to be performed under specific conditions. Previous simulators used methods such as Monte Carlo simulation, which was time-consuming. Furthermore, commercially available semiconductor circuit process simulations are divided into processes such as photolithography, etching, and imaging. To comprehensively grasp the relationships between parameters across these processes, multi-level simulators were required.

[0015] However, simulating manufacturing or filming processes using methods like Monte Carlo simulations requires lengthy calculations, resulting in significant time consumption even in a single trial. To handle multiple conditions and parameters, such calculations need to be run multiple times, which, even with multiple simulators, requires substantial computational time and costs, making it impractical.

[0016] The pattern inspection system disclosed in Patent Document 2 can reduce the amount of learning data during machine learning, thereby shortening the learning time. However, when using the obtained learning data in actual inspection, it is believed that the data processing method needs to be further improved.

[0017] The purpose of this invention is to shorten the time required for comparison when comparing simulated images estimated based on design data with actual captured images, and to perform the comparison in real time.

[0018] Methods for solving problems

[0019] The image processing method of the present invention uses a system having an input receiving unit, an estimation unit, and an output unit to acquire data of an estimated photographed image used when comparing an estimated photographed image obtained based on reference data of a sample with an actual photographed image of the sample. The image processing method includes: an input step in which the input receiving unit receives reference data, process information of the sample, and learned model data; an estimation step in which the estimation unit uses the reference data, process information, and model data to calculate a photographed image statistic representing the probability distribution of values ​​that the data of the photographed image can take; and an output step in which the output unit outputs the photographed image statistic, and the estimated photographed image can be generated based on the photographed image statistic.

[0020] Invention Effects

[0021] According to the present invention, when comparing simulated images estimated based on design data with actually captured images, the time required for the estimation can be shortened, and the comparison can be performed in real time. Attached Figure Description

[0022] Figure 1A This is an example of a photograph obtained based on design data and process information.

[0023] Figure 1B This is another example of a photograph obtained based on design data and process information.

[0024] Figure 2 This is a structural diagram illustrating an embodiment of the image processing system.

[0025] Figure 3A This is a structural diagram illustrating the flow of data processed in the image processing system involved in the embodiment.

[0026] Figure 3B This is a structural diagram illustrating the flow of data processed in the image processing system involved in the embodiment.

[0027] Figure 4 This is a flowchart illustrating an example of the learning process involved in the embodiment.

[0028] Figure 5 This is a structural diagram showing the shape inspection system.

[0029] Figure 6A This is a schematic diagram illustrating an example of transforming a design data image into feature quantities.

[0030] Figure 6B This is a schematic diagram illustrating an example of the combination of characteristic quantities and process information.

[0031] Figure 7A This is a schematic diagram illustrating an example of the input format in the embodiment.

[0032] Figure 7B This is a schematic diagram illustrating an example of the combination form in the embodiment.

[0033] Figure 8A This is an example diagram showing a design data image.

[0034] Figure 8B It is shown that... Figure 8A The image is an example of the captured image corresponding to the design data image 801.

[0035] Figure 8C It is shown that... Figure 8A The image is an example of the captured image corresponding to the design data image 801.

[0036] Figure 8D It is shown that... Figure 8A The image is an example of the captured image corresponding to the design data image 801.

[0037] Figure 9 This is a graph illustrating an example of how the statistics of captured images are represented.

[0038] Figure 10A This is a diagram showing an example of a design data image.

[0039] Figure 10B This is a diagram showing an example of a captured image.

[0040] Figure 11 This is a block diagram of a GUI used to evaluate a circuit by estimating statistical parameters of captured images.

[0041] Figure 12 This is a structural diagram showing the GUI used to implement learning processing.

[0042] Figure 13 This is a schematic diagram illustrating an example of a semiconductor measurement system.

[0043] Figure 14 This is a schematic diagram showing the structure of a scanning electron microscope. Detailed Implementation

[0044] This invention relates to image processing techniques for processing image data. In particular, it relates to image processing techniques applicable to inspections using image data. One example of an object being inspected includes a semiconductor circuit.

[0045] Hereinafter, the image processing method, shape inspection method, image processing system, and shape inspection system, which are embodiments of the present invention, will be described.

[0046] The image processing method and system calculate the captured image statistics based on the design data and process information, as corresponding to the changes in the captured images. These captured image statistics represent the probability distribution of the values ​​that each pixel of the captured image can take.

[0047] The image processing system incorporates a CNN model, which calculates a pixel-level probability distribution representing variations in the captured image based on design data and process information. Here, CNN stands for Convolutional Neural Network.

[0048] The image processing system uses a calculated pixel-level probability distribution to evaluate the impact of process information on the circuit or its captured image. Furthermore, the shape inspection system uses the calculated pixel-level probability distribution to create a template image that can be used for pattern matching, thereby performing pattern matching with high precision. Further, this embodiment also includes determining the parameters (model data) included in the mathematical model used in machine learning, such as a CNN.

[0049] In addition to semiconductor circuits, the inspection scope can also be applied to various items such as automotive parts (pistons, etc.), trays, containers such as bottles, and LCD panels. Furthermore, the shape specifications include the size and length of the sample (item).

[0050] The image processing method described below relates to an image processing method for directly estimating changes in captured images of a circuit manufactured under the conditions of the design data and process information using design data, process information, and learned model data as reference data for the circuit, as well as an image inspection system using the same.

[0051] Furthermore, as a specific example, an example of a method is shown below, in which machine learning is used to learn the correspondence between design data images (visualized as design data), process information, and photographic images of the circuit. Using the learned model data, changes in photographic images of the corresponding circuits are directly inferred based on any design data image and any process information. Additionally, the changes in photographic images of the circuit are processed as a probability distribution of pixel values ​​that each pixel of the image can take, using prescribed statistics (mean, variance, etc.). Thus, pixel values ​​and their fluctuations, circuit deformation, and changes in the image quality of the photographed images can be captured.

[0052] The following description, using accompanying drawings, illustrates an apparatus or measurement and inspection system that functions as follows: receiving design data, process information, and learned model data of any circuit as input; directly estimating the changes in the captured image of the circuit corresponding to the combination of design data and process information as a statistical measure of pixel values; and outputting the estimated statistical measure. More specifically, an apparatus and system comprising a length measuring scanning electron microscope (CD-SEM) as one type of measuring device will be described.

[0053] In the following description, a charged particle beam device is exemplified as an apparatus for forming images of a circuit. This specification describes an example using a scanning electron microscope (SEM) as one type of charged particle beam device, but it is not limited to this; for example, a focused ion beam (FIB) device that forms an image by scanning an ion beam on a sample can also be used as a charged particle beam device. However, in order to measure further refined patterns with high precision, extremely high magnification is required, and therefore it is generally desirable to use an SEM that has superior resolution compared to an FIB device.

[0054] Example

[0055] Figure 13 This is a schematic structural diagram illustrating an example of a semiconductor measurement system, showing a measurement / inspection system with multiple measuring or inspection devices connected to a network. Here, the measurement / inspection system is included within an image processing system or a shape inspection system.

[0056] The system shown in this figure includes: a scanning electron microscope 1301 (CD-SEM) for measuring the length of patterns such as semiconductor wafers and photomasks; a defect inspection device 1302 for acquiring images by irradiating a sample with an electron beam and extracting defects based on comparisons of the images with pre-registered reference images; a condition setting device 1303; a simulator 1304; and a storage medium 1305 (storage unit). Furthermore, they are connected via a network.

[0057] The condition setting device 1303 has the function of setting the measurement position, measurement conditions, etc., based on the design data of the semiconductor device. The simulator 1304 has the function of simulating the completion of the pattern based on the design data of the semiconductor device and the manufacturing conditions of the semiconductor manufacturing apparatus. Furthermore, the storage medium 1305 stores and registers the layout data of the semiconductor device, the design data of the manufacturing conditions, etc. In addition, the learned model data can also be stored in the storage medium 1305.

[0058] Design data may be represented in formats such as GDS or OASIS (registered trademark), and stored in a given format. Furthermore, the type of design data is not limited as long as the software displaying the design data can display its format and process it as graphic data.

[0059] Furthermore, the storage medium 1305 can also be built into the control device, condition setting device 1303, or simulator 1304 of the measuring or inspection device. In addition, the length measuring scanning electron microscope 1301 and the defect inspection device 1302 are equipped with their own control devices to perform the necessary control for each device, but the functions of the simulator and the setting functions of the measurement conditions, etc., can also be combined in these control devices.

[0060] In SEM, the electron beam emitted from the electron source is focused by a multi-stage lens, and the focused electron beam is scanned on the sample in one or two dimensions by a scanning deflector.

[0061] Secondary electrons (SE) or backscattered electrons (BSE) emitted from the sample by scanning with an electron beam are detected by a detector and stored synchronously in a storage medium such as a frame memory, in conjunction with the scanning deflector. The image signal stored in the frame memory is accumulated by a computing unit integrated within the control device. Furthermore, the scanning achieved by the scanning deflector can be performed for any size, position, and orientation.

[0062] The controls described above are performed by the control devices of each SEM, and the images and signals obtained as a result of electron beam scanning are sent to the condition setting device 1303 via a communication line network.

[0063] Furthermore, in this example, the control device for controlling SEM and the condition setting device 1303 are described as separate devices, but this is not a limitation. For example, the control and measurement processing of the device may be performed by the condition setting device 1303 together, or the control and measurement processing of SEM may be performed by each control device together.

[0064] Furthermore, the condition setting device 1303 or the control device stores a program for performing measurement processing, and the measurement or calculation is performed according to the program.

[0065] Furthermore, the condition setting device 1303 functions as a manufacturing process unit, capable of creating a program (manufacturing method) to control the operation of the SEM based on semiconductor design data. Specifically, it sets desired measurement points, autofocus points, automarkers, address points, and other locations required for SEM processing on design data, pattern outline data, or simulated design data. Then, based on these settings, a program is created for automatically controlling the SEM's sample stage, deflector, etc. Additionally, for template creation (described later), a program is built-in or stored that extracts information about the area to be used as the template from the design data and enables a template-making processor or general-purpose processor to create the template based on this extracted information. Furthermore, this program can also be distributed via a network.

[0066] Figure 14 This is a schematic diagram showing the structure of a scanning electron microscope.

[0067] The scanning electron microscope shown in this figure includes an electron source 1401, an extraction electrode 1402, a condenser lens 1404 which serves as a focusing lens, a scanning deflector 1405, an objective lens 1406, a sample stage 1408, a conversion electrode 1412, a detector 1413, and a control device 1414.

[0068] An electron beam 1403, drawn from an electron source 1401 by an extraction electrode 1402 and accelerated by an accelerating electrode (not shown), is focused by a condenser lens 1404. Furthermore, it is scanned one-dimensionally or two-dimensionally on a sample 1409 by a scanning deflector 1405. The electron beam 1403 is slowed down by a negative voltage applied to an electrode disposed on a sample stage 1408 and focused onto the sample 1409 by the lens action of an objective lens 1406.

[0069] When the electron beam 1403 irradiates the sample 1409, electrons 1410, such as secondary electrons and backscattered electrons, are released from the irradiated area. The released electrons 1410 are accelerated towards the electron source by the negative voltage applied to the sample, and collide with the conversion electrode 1412 to generate secondary electrons 1411. The secondary electrons 1411 released from the conversion electrode 1412 are captured by the detector 1413, and the output I of the detector 1413 varies according to the amount of secondary electrons captured. The brightness of a display device (not shown) changes according to this output I. For example, in the case of forming a two-dimensional image, an image of the scanned area is formed by synchronizing the deflection signal to the scanning deflector 1405 with the output I of the detector 1413. Furthermore, the scanning electron microscope illustrated in this figure includes a deflector (not shown) for moving the scanning area of ​​the electron beam.

[0070] Furthermore, the example in this figure illustrates an example of one-end conversion and detection of electrons released from the sample by a conversion electrode, but it is of course not limited to such a structure. For example, it can also be configured as a structure such as arranging an electron multiplier tube and the detection surface of the detector on the orbit of the accelerated electron.

[0071] The control device 1414 has the function of controlling the various structures of the scanning electron microscope and forming an image based on the detected electrons, and the function of determining the pattern width of the pattern formed on the sample based on the intensity distribution of the detected electrons, which is called the line profile.

[0072] Next, we will explain an example of using machine learning to estimate the changes in the captured images of the circuit as a statistical measure of pixel values, to learn the parameters (model data) of the model that can estimate the statistical measure, or to perform evaluation processing or pattern matching processing using the process information of the statistical measure.

[0073] The statistical estimation processing or model data learning processing can also be performed by a computing device built into the control device 1414 or a computing device with image processing capabilities. Alternatively, processing can be performed via a network by an external computing device (e.g., the condition setting device 1303). Furthermore, the processing load sharing between the computing device built into the control device 1414 or the computing device with image processing capabilities and the external computing device can be appropriately configured, and is not limited to the examples described above.

[0074] Figure 1A This is an example of a photograph obtained based on design data and process information.

[0075] In this figure, the photographed image 104 of the circuit is obtained based on the design data image 101 and the given process information 102.

[0076] Design data image 101 is a form of reference data representing the wiring or configuration of a circuit.

[0077] Figure 1B This is another example of a photograph obtained based on design data and process information.

[0078] In this figure, the photographed image 105 of the circuit is obtained based on the design data image 101 and the given process information 103.

[0079] These figures show that even when using the same design data, the images taken under different process information are different.

[0080] In this embodiment, a design data image is used, which visualizes the design data described by CAD data, etc. As an example, a binary image can be used, showing the color differentiation between the wiring portions of the circuit and the areas outside of them. In the case of semiconductor circuits, there are also multi-layer circuits with two or more wiring layers. For example, if the wiring is a single layer, it can be used as a binary image of the wiring and the areas outside of it; if the wiring is a two-layer layer, it can be used as a ternary image of the wiring portions of the lower and upper layers and the areas outside of them. Furthermore, the design data image is an example of reference data and is not limited thereto.

[0081] Process information 102 and 103 refers to one or more parameters used in various processes from circuit manufacturing to imaging. In this embodiment, the process information is treated as actual numerical values. Specific examples of processes include etching, photolithography, and SEM-based imaging. Specific examples of parameters, in the case of photolithography, include exposure (dose) and focal length.

[0082] The circuit images 104 and 105 are images of circuits manufactured using process information 102 and 103 respectively, based on the design data shown in design data image 101. In this embodiment, the images are processed as grayscale images captured by SEM. Therefore, the images themselves have arbitrary height and width, and the number of channels in the image is set to 1.

[0083] Depending on the manufacturing process parameters, the circuit may undergo a degree of deformation that is permissible without causing electrical problems, resulting in a circuit shape that differs from the design data. Furthermore, the way the circuit is represented in photographic images varies depending on the parameters of the SEM imaging process. Therefore, although photographic images 104 and 105 correspond to the same design data image 101, due to differences in process information, they do not represent the same amount of circuit deformation, and the image quality also differs. Specific examples of image quality include noise and contrast variations.

[0084] Furthermore, even if the design data and process information are the same, the resulting photographic images of the circuit will not be exactly the same. This is because, even if the parameters of the manufacturing process or the photographing process are set, there are still process variations that will cause fluctuations in the results.

[0085] In this embodiment, the reference data is set as the design data image, the process information is set as the actual values ​​representing its parameter values, and the circuit image is set as an image captured by SEM, but there are no restrictions on them.

[0086] Next, we will explain how to use changes in captured images as statistical measures of pixel values ​​for estimation.

[0087] Figure 2 This is a structural diagram illustrating the image processing system of this embodiment.

[0088] As shown in this figure, the image processing system includes an input receiving unit 201, an estimation unit 202, and an output unit 203. Furthermore, the image processing system appropriately includes a storage unit.

[0089] The input receiving unit 201 receives the reference data 204, process information 205, and the learned model data 206 as input. Then, the estimation unit 202 transforms the input received by the input receiving unit 201 into a statistical quantity of the changes in the captured image of the circuit. The output unit 203 outputs this statistical quantity as the captured image statistical quantity 207.

[0090] Reference data 204 describes the shape or configuration of the wiring of the circuit, and in this embodiment it is processed as design data or design data visualized thereon.

[0091] The estimation unit 202 transforms the input received by the input receiving unit 201 into statistics representing changes in the captured image of the corresponding circuit. To perform this transformation, the estimation unit 202 has a mathematical model that sets parameters using model data 206 and estimates the captured image statistics based on design data images and process information.

[0092] Specifically, a Convolutional Neural Network (CNN) is used. In a CNN, the encoder consists of two or more convolutional layers and pooling layers, and the decoder consists of two or more deconvolutional layers. In this case, the model data consists of the weights (transformation parameters) of the filters in each layer of the CNN. Furthermore, the mathematical model used to estimate the statistical properties of the captured image can also be a model other than a CNN model; it is not limited to this.

[0093] The input receiving unit 201 reads in the reference data 204, process information 205, and model data 206 in the given format.

[0094] The output unit 203 outputs the calculation results from the estimation unit 202 in a given format.

[0095] Furthermore, the input receiving unit 201, estimation unit 202, and output unit 203 shown in this figure are part of the constituent elements of the system shown in this embodiment, and can also be distributed among multiple computers connected by a network. In addition, data including input reference data 204, process information 205, and learned model data 206 can be input by the user from an external source, but can also be stored in a given storage device.

[0096] The correspondence between the design data images and the captured images is described.

[0097] Specifically, using Figures 8A to 8D Examples of shape deviations in wiring in design data images and inspection object images are illustrated.

[0098] Figure 8A This is an example diagram showing a design data image.

[0099] In this figure, the design data image 801 has wiring 811 consisting of blank pixels (squares). The design data image 801 is obtained based on design data, and thus shows ideally orthogonal wiring 811.

[0100] Figures 8B to 8D It is shown that... Figure 8A The image is an example of the captured image corresponding to the design data image 801.

[0101] exist Figure 8B The image shown is a captured image 802 corresponding to the design data image 801.

[0102] exist Figure 8C The image shown is a captured image 803 corresponding to the design data image 801.

[0103] exist Figure 8D The image shown is a captured image 804 corresponding to the design data image 801.

[0104] Figure 8B Image 802, Figure 8C The captured image 803 and Figure 8DThe captured image 804 is affected by at least one of the manufacturing conditions and the shooting conditions. Therefore, the shape of the wiring 811 differs in each of the captured images 802, 803, and 804. In other words, the difference in the shape of the wiring 811 can occur depending on both the manufacturing batch and the shooting batch. Therefore, when a pixel in the design data image takes an arbitrary brightness value, the same pixel in the captured image can take multiple brightness values.

[0105] For example, if images 802, 803, and 804 are grayscale images, then each pixel can take brightness values ​​from 0 to 255 integers. In this case, the brightness value distribution represents the frequency relative to brightness values ​​from 0 to 255. As examples of statistical measures, if the brightness value distribution is normally distributed, the mean and standard deviation can be considered; if it is a Poisson distribution, arrival rate, etc., can be considered.

[0106] In summary, it is possible to define the probability density distribution of pixel values ​​such as brightness values ​​based on design data under certain manufacturing or shooting conditions.

[0107] Figure 10A This is a diagram showing an example of a design data image.

[0108] In this figure, the pixel of interest 1001 and its surrounding area 1002 are shown in the design data image 1000a.

[0109] Figure 10B This is a diagram showing an example of a captured image.

[0110] In this figure, pixel 1003 is shown in the captured image 1000b.

[0111] Figure 10A Focus on pixel 1001 and Figure 10B Pixel 1003 is located at the same coordinates when its position is aligned for the image of the comparison circuit (sample). The statistical values ​​that pixel 1003 can take are estimated based on the pixel values ​​of the focus pixel 1001 and the surrounding region 1002. This is because computations in the convolutional layers of the CNN include calculations involving the surrounding pixels. Furthermore, the size of the surrounding region 1002 is determined by the CNN's filter size, stride size, etc.

[0112] Figure 3A And 3B is a structural diagram showing the flow of data processed in the image processing system of this embodiment.

[0113] In these figures, the input receiving unit 201 receives the input of design data image 101, process information 102 or 103, and model data 301. The estimation unit 202 transforms the input into statistical quantities of changes in the captured images of the specified corresponding circuit. The output unit 203 outputs the calculated captured image statistical quantities 302 or 305.

[0114] If comparison Figure 3A and Figure 3B Even if the design data image 101 and the model data 301 are the same, if... Figure 3A The process information 102 has been changed to Figure 3B If the process information is 103, then the output will also become the same as... Figure 3A Statistical count of 302 different images captured Figure 3B The captured image statistics 305. The average image 306 and standard deviation 307, as output formats, differ from the average image 303 and standard deviation image 304. Thus, information can be obtained regarding the changes in the average circuit image caused by differences in process information, differences in image quality, and the location and extent of large fluctuations.

[0115] Figure 9 This is a graph illustrating an example of how the statistics of captured images are represented.

[0116] In this figure, the captured image statistics are represented as a probability density function 901, which is the probability distribution of pixel values ​​in each pixel. For example, in... Figure 3A When the captured image statistics 302 are expressed as a probability density function 901, the average and standard deviation values ​​of the probability density function 901 are obtained. Similarly, if the average and standard deviation values ​​for each pixel are calculated, the average image 303 and the standard deviation image 304 are obtained.

[0117] The probability density function 901 is represented by the probability density function of the frequency of occurrence of each pixel value that can be taken in an image captured by a certain circuit. Specifically, if the captured image is a grayscale image, the distribution can be defined as the frequency of occurrence of 256 pixel values. Alternatively, as a statistical measure, it can also be in units other than pixels.

[0118] For example, if we assume that the probability density function 901 is a Gaussian distribution, then the probability density function 901 can be uniquely defined by its mean and standard deviation (or variance).

[0119] The average image 303 and the standard deviation image 304 are examples of the output formats of the captured image statistics 302. If the captured image statistics are set to a Gaussian distribution per pixel, they can be estimated and output as the average image and standard deviation image after transforming their mean and standard deviation values ​​into the image.

[0120] The average image 303 is the image after the Gaussian distribution of each pixel is averaged and transformed into a grayscale image. If the captured image statistics 302 are assumed to be Gaussian distribution, then the average value of its distribution is consistent with the most frequent value. Therefore, the obtained average image 303 is the captured image with the most average circuit shape using the design data image 101 and under the conditions of process information 102.

[0121] The standard deviation image 304 is an image obtained by transforming the standard deviation of the Gaussian distribution of each pixel into a grayscale image. By image processing while maintaining the relative relationship of the standard deviations between pixels, it is possible to visualize image areas with large circuit deformations and significant image quality variations. For example, in semiconductor circuits, deformation often occurs at the edges of wiring lines, resulting in large fluctuations (standard deviations). On the other hand, deformation is less in areas outside the edges of wiring lines and in the space outside the wiring lines, resulting in smaller fluctuations. In this embodiment, the standard deviation serves to absorb process variations during manufacturing and imaging under certain design data and process information.

[0122] As mentioned above, the shape of the manufactured circuit and the image quality of its captured images depend on the process information.

[0123] Through such Figure 3A As shown in 3B, with the design data and the learned model data, the impact of changes to the input process information on the circuit and the captured images can be known without actual manufacturing and photography.

[0124] Figure 4 This is a flowchart illustrating an example of the learning process used to generate inferences about the statistical properties of captured images.

[0125] The learning process is handled by the Machine Learning Department.

[0126] In the learning process shown in this figure, the user inputs model data (S401), design data images, and process information (S402). Then, the machine learning unit infers and outputs the captured image statistics based on these inputs (S403). Here, the inputs provided by the user can also be made by someone other than the user; for example, the data can be automatically filtered from the data in the given storage unit and read in by the machine learning unit.

[0127] Then, determine whether the learning termination condition is met (learning necessity determination process S404).

[0128] If the termination condition is not met, the captured images are input as training data (S405). Then, the captured images (training data) and the inferred image information (captured image statistics) are compared (S406), and the model data is updated based on the comparison result (S407). As an example of the comparison method, there is a method that transforms the inferred image information (captured image statistics) into "inferred captured images" for comparison. In other words, the inferred captured images can be generated based on the captured image statistics.

[0129] On the other hand, if the termination condition is met in S404, the model data is saved (S408), and the learning process ends.

[0130] Additionally, in advance, the storage medium 1305 ( Figure 13 If the learned model data is stored, the input of S401 can be omitted.

[0131] Furthermore, S401 and S402 are collectively referred to as the "input process". Additionally, S403 is also called the "estimated process". Further, from the perspective of performing and... Figure 2 From the perspective of the processing corresponding to the output unit 203, S403 can also be called the "output process".

[0132] The following details the processing steps.

[0133] The model data input in S401, updated in S407, and saved in S408 consists of the weights of the filters in the convolutional or deconvolutional layers used in S403. In other words, it contains the structural information of each layer of the CNN encoder and decoder used in S403, and their transformation parameters (weights). These transformation parameters are determined by minimizing the loss function calculated using the captured image statistics estimated in S403 and the captured image input in S405 during the comparison process in S406. The model data in S401, after learning processing, can estimate the corresponding captured image based on the design data image and process information. Specific examples of loss functions include mean squared error and cross-entropy error.

[0134] The reference data input in S402 is a design data image in this embodiment.

[0135] Examples of S404's determination of whether learning is necessary include whether the number of learning repetitions exceeds a certain limit and whether the loss function used for learning converges.

[0136] The model data stored in S408 is saved by outputting the weights of each layer of the CNN to a file in a given format.

[0137] Next, the relationship between the design data images and the photographed images of the circuit used in the learning process will be explained.

[0138] In S406, the estimated captured image statistics (estimated captured image) are compared with the captured image. At this point, for accurate comparison, the design data needs to be positioned correctly against the captured image. Therefore, the training dataset (training dataset) needs to be a pair of position-aligned design data images and captured images. Generally, it is preferable that the training dataset contains a large number of images. Furthermore, it is preferable that the shape of the circuit used for training is similar to the shape of the circuit used for evaluation.

[0139] Furthermore, in order to learn circuit variations starting from the design data, the design data received in S401 and the captured image received in S405 need to be aligned. The positions on the images are aligned to ensure the circuit pattern is consistent, using the design data image for learning and the captured image of the manufactured circuit. An example of a position alignment method is to determine and position the wiring outlines of the design data image and the captured image so that the center of gravity of the shape enclosed by the outlines is consistent.

[0140] The process information used in the learning process, or the process information used in the inference process based on the statistical data of the captured images obtained from the learned model data, can be either only the parameters that need to be considered, or all parameters involved in the manufacturing and capturing processes can be used. However, if the process information increases, the computational load in the CNN increases, so from the perspective of processing speed, it is preferable to use only the minimum required parameters.

[0141] As an example of the comparison processing in S406, there is a difference calculation between the image sampled based on statistical measures and the captured image.

[0142] In summary, the machine learning department determines the necessity of learning from the model data. If the necessity of learning is determined to be necessary in the necessity-determination process, it receives input including benchmark data for learning, process information, and captured images. It then compares the captured image statistics with the data from the captured images in the learning dataset and updates the model data based on the comparison result. Conversely, if the necessity of learning is determined to be unnecessary in the necessity-determination process, the storage department saves the parameters used by the estimation department to calculate the captured image statistics as model data.

[0143] Next, use Figure 6A as well as Figure 6B and Figure 7A as well as Figure 7B This section provides examples of the input formats for design data images and process information in S402.

[0144] Figure 6A An example of transforming a design data image into feature quantities is illustrated.

[0145] This figure is an example of a design data image 601 and a feature quantity 602 calculated from it by two or more convolutional layers of a neural network model.

[0146] Design data image 601 is a binary image generated by visualizing design data such as CAD data. Here, the squares divided by the grid represent the individual pixels that make up the image.

[0147] Feature quantity 602 is calculated on design data image 601 using the convolutional layer (encoder layer) of the CNN in the image statistics estimation unit (estimation unit), and is represented as a matrix. Feature quantity 602 includes design information such as which part of the wiring unit and others each pixel in the design data image belongs to, and design information related to the shape and configuration of the wiring, such as the vicinity of the edges and corners of the wiring. Feature quantity 602 can be represented as a three-dimensional matrix with height, width, and channels. At this time, the height, width, and channels of feature quantity 602 calculated from design data image 601 depend on the number of convolutional layers in the CNN, its filter size, stride size, or padding size, etc.

[0148] Figure 6B An example of a combination of characteristic quantities and process information is shown.

[0149] As shown in this figure Figure 6A The feature quantity 602 is represented as a three-dimensional matrix combined with process information 603, 604, and 605.

[0150] Process information 603, 604, and 605 displays the actual values ​​representing manufacturing and shooting conditions as a three-dimensional matrix. This three-dimensional matrix is ​​provided as a matrix with the same height and width as feature quantity 602 and a channel size of 1. Specifically, an example is a three-dimensional matrix prepared with all element values ​​of 1, a height and width equal to feature quantity 602, and a channel size of 1, and the actual values ​​representing manufacturing and shooting conditions multiplied by it to obtain the three-dimensional matrix.

[0151] When the input to the CNN used in the image statistics estimation unit is set, the design data image 601 is transformed into feature quantity 602 through the convolutional layer (encoder layer) of the CNN. The feature quantity 602 is combined with the process information 603, 604, and 605 in channel order, and the combined information is input into the deconvolutional layer (decoder layer) of the CNN. Here, the case with two pieces of process information is described, but the standard process information can be one or more pieces, and there is no restriction on it.

[0152] Figure 7A This is a diagram illustrating an example of the input format in this embodiment.

[0153] This figure schematically illustrates examples of design data image 701, process information 702, and process information 703.

[0154] Design data image 701 is an image that visualizes design data such as CAD. For example, a binary image showing the coloring of wiring and space sections in a circuit can be used. In the case of semiconductor circuits, wiring sometimes consists of two or more layers. For example, if the wiring is single-layered, a binary image of the wiring and space sections can be used; if the wiring is two-layered, a ternary image of the lower layer wiring and the upper layer wiring and space sections can be used. Furthermore, the design data image is an example of a reference image and is not limited thereto.

[0155] Process information 702 and process information 703 provide actual values ​​representing manufacturing conditions and shooting conditions as an image of the same size as the design data image. Specifically, an example is a matrix obtained by multiplying the actual values ​​representing manufacturing conditions and shooting conditions with a matrix in which all elements have a value of 1 and the image size is the same as the design data.

[0156] Figure 7B This is a diagram illustrating an example of the combination form in this embodiment.

[0157] This figure schematically illustrates examples of design data image 701, process information 702, and process information 703.

[0158] An example of a method for assuming the input of a CNN to the image statistics estimation unit is to combine the design data image 701, process information 702, and process information 703 in the order of the image channels. Here, the case where there are two pieces of process information is described, but there is no restriction on the number of pieces of process information used, which can be one or more.

[0159] In addition, regarding Figures 6A to 7B The method of combining the process information shown is not restricted.

[0160] In addition, examples can be given of the impact of evaluation process information on circuits or their captured images.

[0161] For example, by changing only one of the parameters in the process information and calculating the statistics of the captured images, it becomes possible to observe the deformation that occurs during actual manufacturing and imaging based on the average image, and to observe the conceivable range of deformation at various parts of the circuit based on the standard deviation image. Therefore, with pre-learned model data, the impact on circuit deformation or image quality can be evaluated without actual manufacturing or imaging. If the change in the average image is small and the standard deviation value in the standard deviation image is small, it can be said that the parameter has a small impact on the shape deformation and the degree of its fluctuation in the circuit.

[0162] In this embodiment, the case where two process information items are set and only one of them is changed is described, but this is not a limitation. The number of parameters in the process information can be one or more. Furthermore, it is possible to execute the process by changing only one parameter in the process information or by changing multiple parameters in the process information.

[0163] Next, as Figure 2 Another embodiment of the estimation unit 202 will be described in the case of creating a template image for pattern matching.

[0164] Figure 5 This is a structural diagram showing the flow of data processed in a shape inspection system, illustrating an example of pattern matching processing using captured image statistics.

[0165] The shape inspection system shown in this figure includes an input receiving unit 501 for inputting captured image statistics 207, an input receiving unit 505 for inputting captured images 504, a template image creation unit 502, a pattern matching processing unit 503, and an output unit 506. Furthermore, the data flow shown in this figure is an example of a shape inspection method.

[0166] Image 504 is a captured image (the actual captured image) of the object set as a pattern match.

[0167] The number of captured images is 207. Figure 2 The input receiving unit 201 shown receives process information during the manufacturing and capturing of the circuit of the captured image 504, design data images of the circuit of the captured image 504, and model data generated by learning processing, and the data is calculated by the estimation unit 202 and output by the output unit 203.

[0168] The pattern matching process shown in this figure is performed as follows.

[0169] The input receiving unit 501 receives the captured image statistics 207, and the template image creation unit 502 transforms the captured image statistics 207 into a template image and passes it to the pattern matching processing unit 503. On the other hand, the input receiving unit 505 receives the captured image 504 and passes it to the pattern matching processing unit 503.

[0170] In the pattern matching processing unit 503, pattern matching processing is performed using the captured image 504 and the template image. Then, the output unit 506 outputs the matching result 507.

[0171] The pattern matching processing unit 503 performs a process of comparing the template image and the captured image 504 and aligning their positions.

[0172] As a specific example of the method, while offsetting the relative positions of the template image and the captured image 504, the standardized cross-correlation is calculated as a similarity score, and the relative position with the highest similarity score is output. The matching result 507 can be in the form of, for example, two-dimensional coordinate values ​​representing the amount of image movement, or it can be an image where the template image and the captured image 504 overlap at the position with the highest similarity.

[0173] The input captured image statistics 207 are obtained by using the design data image corresponding to the captured image 504 as the matching object, and process information. Figure 2 The estimation unit 202 estimates this. At this time, the model data provided to the estimation unit 202 is expected to be data prepared in advance through learning processing for pattern matching processing.

[0174] Examples of template images produced by the template image production unit 502 include an average image that visualizes the average value of the captured image statistics 207, and a sampled image obtained by sampling the values ​​of each pixel from the captured image statistics 207.

[0175] The images of the circuits used in the learning process prior to pattern matching can be images taken from previously manufactured wafers or images taken from the wafers of the matching targets.

[0176] Figure 11 This is a structural diagram illustrating the GUI used to estimate statistical parameters of captured images and to evaluate the circuitry. Here, GUI is short for Graphical User Interface.

[0177] The GUI (1100) shown in this figure includes a design data image setting unit 1101, a model data setting unit 1102, a process information setting unit 1103, an evaluation result display unit 1104, and an image display operation unit 1107.

[0178] The design data image setting unit 1101 is an area for setting design data images that are needed to estimate the statistical values ​​of captured images.

[0179] The model data setting unit 1102 is the area for setting up the model data that has been learned and is required to estimate the statistical properties of the captured images.

[0180] The process information setting unit 1103 is an area for setting process information related to the estimation of statistical values ​​of captured images. For example, as a method for setting process information, one can include inputting the parameters required for each process, such as photolithography and etching, separately.

[0181] In the design data image setting unit 1101, model data setting unit 1102, and process information setting unit 1103, data is read in by specifying a storage area saved in a given format.

[0182] The evaluation result display unit 1104 is an area that displays information related to the statistical measures of the captured images, which are estimated based on the data set by the design data image setting unit 1101, the model data setting unit 1102, and the process information setting unit 1103. Examples of the information displayed include an average image 1105 and a standard deviation image 1106 created based on the captured image statistics.

[0183] The image display operation unit 1107 is an area for performing operations related to the information displayed by the evaluation result display unit 1104. Examples of operations include switching the displayed image to another image, and zooming in or out of the image.

[0184] Figure 12 This is a structural diagram showing the GUI used to implement learning processing.

[0185] The GUI (1200) shown in this figure displays a learning dataset setting unit 1201, a model data setting unit 1202, a learning condition setting unit 1203, and a learning result display unit 1204.

[0186] The learning dataset setting unit 1201 is the area for setting up the learning dataset, which includes design data images, process information, and captured images used in the learning process. Here, data is read in by specifying a storage area saved in a given format.

[0187] The model data setting unit 1202 is an area for setting parameters related to the model data that is input, updated, and saved during the learning process. Here, the learning condition setting unit 1203, which reads model data by specifying a storage area saved in a given format, is an area for setting parameters related to the learning process. For example, as a determination of whether learning is necessary, S404, either the number of learning iterations or the value of the loss function used as a basis for ending the learning process can be specified.

[0188] The learning results display unit 1204 is an area that displays the learning results during or after the learning process. It can display a graph 1205 showing the time variation of the loss function, or an image 1206 that visualizes the captured image statistics estimated by the model during or after the learning process.

[0189] GUIs (1100) and (1200) can be separate or combined into a single GUI related to learning processing and evaluation. Furthermore, the areas shown in GUI (1100) or GUI (1200) for setting, displaying, or manipulating are examples; the GUI does not necessarily need to contain all of them, and only a portion may be implemented. Moreover, the devices performing these processes, like programs, can be either a single device or different devices.

[0190] Regarding Figure 2 , Figure 3A as well as Figure 3B The statistical data of the captured images are used for estimation. Figure 4 Learning processing and Figure 5 The pattern matching process can be performed by separate programs or by individual programs. Furthermore, the devices that perform these processes, like the programs, can be a single device or different devices.

[0191] Furthermore, the present invention is not limited to the embodiments described above, but includes various modifications. For example, the embodiments described above have been explained in detail for ease of understanding of the present invention, and are not limited to all the structures described.

[0192] According to this embodiment, based on the correspondence between reference images such as design data of the specimen, process information, and captured images, the deformation range of the specimen's shape corresponding to the process information can be estimated using statistical quantities based on the design data image. The estimated statistical quantities can then be used for pattern matching of captured images of the specimen.

[0193] Furthermore, this embodiment can also be applied to objects other than semiconductor circuits as the evaluation object. Additionally, it can use input data other than images (based on radar shape measurement).

[0194] The effects of this invention are summarized and explained below.

[0195] According to the present invention, based on reference data such as the design data of the specimen, process information which is set as parameters in the manufacturing process or shooting process of the specimen, and the correspondence between the specimen's photographed images, it is possible to estimate the deformation or physical properties of the specimen and the changes in the image quality of the specimen's photographed images based on any reference data of the specimen and its process information.

[0196] For example, a mathematical model can be constructed by learning the correspondence between some or all of the circuit design data, circuit manufacturing process, or process information used in the photographing process, obtained before evaluation such as measurement and inspection, and the photographed image. Based on any design data image and any process information, the deformation range of the circuit under certain conditions can be directly estimated. Therefore, if a pattern matching template image is created based on the estimation result, high-precision pattern matching can be achieved, taking into account the differences in deformation range caused by differences in process information.

[0197] Furthermore, by using design data, process information, and captured images to learn the correspondences, it is possible to incorporate parameters from multiple manufacturing or imaging processes (photolithography, etching, imaging, etc.) into the process information, thereby inferring the dependencies between these parameters as changes in the shape of the circuit reflected in the captured image or changes in the image quality. Conventional process simulations are time-consuming, thus this invention offers a speed advantage.

[0198] Furthermore, according to the present invention, a computer program for predicting deformations of circuits generated in accordance with process information or changes in the image quality of captured images thereof, and a semiconductor inspection apparatus using the computer program, are provided.

[0199] Symbol Explanation

[0200] 101: Design data image; 102, 103: Process information; 104, 105, 504: Captured images; 202: Estimation unit; 204: Baseline data; 205: Process information; 206, 301: Model data; 207: Captured image statistics; 303: Average image; 304: Standard deviation image; 502: Template image creation unit; 503: Pattern matching processing unit; 901: Probability density function; 1100, 1200: GUI.

Claims

1. An image processing method, using a system having an input receiving unit, an estimation unit, and an output unit, acquires data of the estimated image used when comparing an estimated image obtained based on reference data of a specimen under inspection with an actual image of the specimen, wherein, The image processing method includes: The input process involves the input receiving unit receiving the reference data, process information, and learned model data. The process information includes at least one of the manufacturing conditions of the sample or the shooting conditions of the captured image. In the estimation process, the estimation unit uses the reference data, the process information, and the model data as a probability density function representing the probability distribution of each pixel value of the captured image data to calculate the captured image statistics; and In the output process, the output unit outputs the statistical data of the captured image. The estimated captured image can be generated based on the statistical parameters of the captured image. The process information was combined with the feature values ​​of the captured image.

2. The image processing method according to claim 1, wherein, The system also includes a machine learning unit and a storage unit. The image processing method further includes: a learning necessity determination step, wherein the machine learning unit determines the necessity of learning from the model data. If the necessity of learning is determined to be required in the learning necessity determination process, The system receives an input of a learning dataset, including the baseline data used for learning, the process information, and the captured images. The statistical measures of the captured images are compared with the data of the captured images in the learning dataset. The model data is updated based on the results of the comparison. If, in the process of determining whether learning is necessary, the necessity of learning is determined to be unnecessary, The storage unit saves the parameters used by the estimation unit when calculating the statistics of the captured image as the model data.

3. The image processing method according to claim 1, wherein, The image processing method further includes a step of using the captured image statistics to evaluate the impact of the process information on the sample.

4. The image processing method according to claim 1, wherein, The statistical measures of the captured images include the average image and the standard deviation image.

5. The image processing method according to claim 1, wherein, The sample is a semiconductor circuit.

6. A shape inspection method, comprising using the captured image statistics obtained by the image processing method of claim 1 to inspect the shape of the sample, wherein, The system also includes a template image creation unit and a pattern matching processing unit. The input receiving unit receives the data of the captured image as input. The template image creation unit creates template images based on the statistical data of the captured images. The pattern matching processing unit performs pattern matching between the template image and the captured image. The output unit outputs the pattern matching result.

7. A shape inspection method, comprising using the captured image statistics obtained by the image processing method of claim 2 to inspect the shape of the sample, wherein, The system also includes a template image creation unit and a pattern matching processing unit. The input receiving unit receives the data of the captured image as input. The template image creation unit creates template images based on the statistical data of the captured images. The pattern matching processing unit performs pattern matching between the template image and the captured image. The output unit outputs the pattern matching result.

8. An image processing system, wherein data of the estimated image is acquired when comparing an estimated image obtained based on reference data of an inspected specimen with an actual image of the specimen, wherein, The image processing system includes: The input receiving unit receives the reference data, process information, and learned model data. The process information includes at least one of the manufacturing conditions of the sample or the shooting conditions of the captured image. The estimation unit uses the reference data, the process information, and the model data as a probability density function representing the probability distribution of each pixel value of the captured image data to calculate the captured image statistics; and The output unit outputs the statistical data of the captured images. The estimated captured image can be generated based on the statistical parameters of the captured image. The process information was combined with the feature values ​​of the captured image.

9. The image processing system according to claim 8, wherein, The image processing system also includes a machine learning unit and a storage unit. The machine learning unit determines the necessity of learning from the model data. When the machine learning unit determines that the learning is necessary, The system receives an input of a learning dataset, including the baseline data used for learning, the process information, and the captured images. The statistical measures of the captured images are compared with the data of the captured images in the learning dataset. The model data is updated based on the results of the comparison. If the machine learning unit determines that the learning is unnecessary, The storage unit saves the parameters used by the estimation unit when calculating the statistics of the captured image as the model data.

10. The image processing system according to claim 8, wherein, The statistical measures of the captured images are used to evaluate the impact of the process information on the sample.

11. The image processing system according to claim 8, wherein, The statistical measures of the captured images include the average image and the standard deviation image.

12. The image processing system according to claim 8, wherein, The sample is a semiconductor circuit.

13. A shape inspection system, comprising the image processing system of claim 8, further comprising a template image creation unit and a pattern matching processing unit, wherein the captured image statistics are used to inspect the shape of the sample, wherein... The input receiving unit receives the data of the captured image as input. The template image creation unit creates template images based on the statistical data of the captured images. The pattern matching processing unit performs pattern matching between the template image and the captured image. The output unit outputs the pattern matching result.

14. A shape inspection system, comprising the image processing system of claim 9, further comprising a template image creation unit and a pattern matching processing unit, wherein the captured image statistics are used to inspect the shape of the sample, wherein... The input receiving unit receives the data of the captured image as input. The template image creation unit creates template images based on the statistical data of the captured images. The pattern matching processing unit performs pattern matching between the template image and the captured image. The output unit outputs the pattern matching result.

Images