Evaluation method, evaluation device, and computer program

Abstract

Provided are an evaluation method, an evaluation device, and a computer program.　The present invention executes, by means of a computer, processing of: acquiring a surface image of a material to be evaluated; extracting a first feature quantity from the acquired surface image; reading, from a storage unit, a plurality of second feature quantities that have been extracted from surface images of a plurality of materials having structural uniformity and stored in the storage unit in advance; calculating a distance in a feature space between the first feature quantity and the plurality of second feature quantities; calculating an evaluation value for quantitatively evaluating the structural uniformity of the material to be evaluated on the basis of the calculated distance; and outputting the calculated evaluation value.

Description

Evaluation method, evaluation device, and computer program

[0001] The present invention relates to an evaluation method, an evaluation device, and a computer program.

[0002] Conventionally, visual inspection of substrates has been performed in various processing steps of substrate processing (see, for example, Patent Document 1). In visual inspection, for example, an operator visually checks a surface image obtained by capturing an image of the surface of the substrate to be inspected, thereby determining the quality of the process.

[0003] Japanese Patent Application Laid-Open No. 2018-197696

[0004] The present disclosure provides an evaluation method, an evaluation device, and a computer program that enable quantitative evaluation regardless of the skill of the worker.

[0005] The evaluation method disclosed herein includes a computer-implemented process of acquiring a surface image of a material to be evaluated, extracting a first feature from the acquired surface image, reading from a memory unit a plurality of second feature values ​​extracted from surface images of a plurality of materials having structural uniformity and stored in advance, calculating a distance in feature space between the first feature value and the plurality of second feature values, calculating an evaluation value that quantitatively evaluates the structural uniformity of the material to be evaluated based on the calculated distance, and outputting the calculated evaluation value.

[0006] According to the present disclosure, quantitative evaluation can be performed regardless of the skill of the worker.

[0007] FIG. 1 is an explanatory diagram illustrating the configuration of an evaluation system according to an embodiment. FIG. 2 is a schematic diagram illustrating the structure of a substrate to be evaluated. FIG. 3 is a schematic diagram illustrating an example structure of a substrate with reduced structural uniformity. FIG. 4 is an explanatory diagram illustrating the configuration of a main part of a learning model. FIG. 5 is a block diagram illustrating the internal configuration of an evaluation device. FIG. 6 is a flowchart illustrating an extraction procedure for extracting feature amounts from a non-defective substrate. FIG. 7 is a flowchart illustrating an evaluation procedure for evaluating a substrate to be evaluated. FIG. 8 is a schematic diagram illustrating an example display of evaluation values ​​and evaluation images. FIG. 9 is a schematic diagram illustrating another example display of evaluation images. FIG. 10 is a conceptual diagram illustrating an example of a table storing parameters included in a process recipe and evaluation values ​​in association with each other. FIG. 11 is a graph illustrating the correlation between parameters included in a process recipe and evaluation values. FIG. 12 is a flowchart illustrating an optimization procedure for a process recipe.

[0008] An embodiment will be described below with reference to the drawings. In the description, the same elements or elements having the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0009] 1 is an explanatory diagram illustrating the configuration of an evaluation system according to an embodiment of the present invention. The evaluation system according to the embodiment includes a substrate processing apparatus 100, an observation apparatus 200, and an evaluation apparatus 300.

[0010] The substrate processing apparatus 100 is an apparatus for performing required processing on a substrate (wafer). For example, the substrate processing apparatus 100 is a semiconductor manufacturing apparatus such as an exposure apparatus, an etching apparatus, a film formation apparatus, an ion implantation apparatus, an ashing apparatus, or a sputtering apparatus. Alternatively, the substrate processing apparatus 100 may be a display manufacturing apparatus for manufacturing flat display panels (FDPs) such as liquid crystal display panels and organic electroluminescence (EL) panels.

[0011] When a substrate is processed in the substrate processing apparatus 100, various set values ​​are set, such as the substrate temperature, the pressure and gas flow rate in the chamber, and the voltage applied from the high-frequency power supply. The set values ​​are given, for example, by a process recipe. The substrate processing apparatus 100 is also provided with various sensors and devices for measuring the substrate temperature, the pressure and gas flow rate in the chamber, the voltages applied to the upper electrode and the lower electrode, the plasma emission intensity, and the like, and various measurement values ​​are taken during the process. In addition to the above-mentioned measurement values, the substrate processing apparatus 100 also collects appropriate time-series data, such as images of the substrate before and after the process and process logs, as needed.

[0012] The observation device 200 is a device that observes the surface of a substrate processed by the substrate processing device 100. The observation device 200 is, for example, a scanning electron microscope (SEM). A scanning electron microscope irradiates an electron beam onto the surface of a sample and detects electrons (particularly secondary electrons) scattered by the surface of the sample. The scanning electron microscope measures the amount of secondary electrons while scanning the electron beam, thereby generating an image that reflects the surface structure of the sample. The observation device 200 uses such a scanning electron microscope to observe the surface of the substrate and generate an image that reflects the surface structure of the substrate (hereinafter referred to as a surface image). The observation device 200 outputs the generated surface image to the evaluation device 300.

[0013] In the embodiment, the observation device 200 is a scanning electron microscope. Alternatively, the observation device 200 may be a transmission electron microscope (TEM), a scanning acoustic microscope (SAM), or another observation device. Furthermore, the observation device 200 may be an observation device that is mounted on the substrate processing apparatus 100, observes the surface of the substrate during processing, and outputs the obtained surface image to the evaluation device 300.

[0014] The evaluation device 300 is a computer that quantitatively evaluates the structural uniformity of a substrate based on the surface image input from the observation device 200. In the embodiment, structural uniformity refers to the uniformity of the structure formed on the surface of the substrate. For example, when a substrate having a hole pattern formed thereon is evaluated, the evaluation device 300 may evaluate the uniformity of the hole arrangement or the hole shape as the structural uniformity. Although the evaluation method will be described in detail later, the evaluation device 300 according to the present embodiment calculates an evaluation value that quantitatively evaluates the structural uniformity by comparing feature amounts extracted from a surface image of a substrate that is deemed to be a non-defective with feature amounts extracted from a surface image of the substrate being evaluated, and outputs the calculated evaluation value.

[0015] In this embodiment, the evaluation object is described as a substrate processed by the substrate processing apparatus 100, but the evaluation object is not limited to a substrate processed by the substrate processing apparatus 100 and may be any material.

[0016] Furthermore, in this embodiment, the substrate processing apparatus 100, the observation apparatus 200, and the evaluation apparatus 300 are described as separate apparatuses, but the evaluation apparatus 300 may be provided inside the substrate processing apparatus 100 or inside the observation apparatus 200.

[0017] Below, we will explain a substrate on which a hole pattern is formed as an example of a substrate to be evaluated. FIG. 2 is a schematic diagram showing the structure of the substrate to be evaluated. FIG. 2A shows a side cross-section of a substrate on which a hole pattern is formed. Dry etching technology, for example, is used to form the hole pattern. In dry etching, multiple reactive gases are introduced into a reaction chamber, and the reactive gases are excited, dissociated, and ionized by plasma. The various particle species (radicals and ions) generated during this process are then irradiated onto the substrate. Because the ions react by entering the substrate surface perpendicularly, highly anisotropic processing is possible. By protecting a portion of the film to be processed with an appropriate mask material, a hole pattern that reflects the mask shape can be formed.

[0018] 2B is a schematic diagram of a surface image obtained by the observation device 200. When a substrate on which a hole pattern is formed is observed using a scanning electron microscope (observation device 200), the amount of electrons detected by the detector of the scanning electron microscope differs between regions where holes are formed and regions where holes are not formed, which appears as a shading in the image. For example, because of the property that more secondary electrons are scattered from areas with steep angles and fewer from flat areas (edge ​​effect), if a shading image is generated so that the areas with a greater amount of detected secondary electrons are darker, an image is obtained in which the shading values ​​in the regions corresponding to the holes are low and the shading values ​​in the other regions are high (i.e., the image is darker in the regions corresponding to the holes than in the surrounding area), as shown in FIG.

[0019] 2B shows an example of a surface image when the hole pattern is uniformly formed. When the hole pattern is uniformly formed, the grayscale value is low only in the areas corresponding to the holes, and the grayscale value of other areas is not low. Furthermore, when the holes have a shape close to a perfect circle, the areas with low grayscale values ​​will have a shape close to a perfect circle that reflects the shape of the holes.

[0020] In contrast, various shape anomalies can occur with actual etching techniques. For example, fluctuations in the number of photons during exposure, nonuniformity in resist materials, and reduced contrast in the exposure equipment can stochastically cause pattern defects on the substrate surface. Furthermore, the high radical flux from the substrate surface to the sidewalls above the mask facilitates deposition of films. Furthermore, the sidewalls are less susceptible to ion irradiation, resulting in a protruding shape known as necking. When necking reduces the hole opening diameter, resputtered material adheres to the opposing sidewall, making the hole more susceptible to clogging, a condition known as hole blockage. The hole's middle is prone to enlargement, known as bowing. The hole sidewalls can also develop vertical stripe-like shape anomalies known as striations. Furthermore, tilting, a phenomenon in which a vertically formed hole becomes oblique, or twisting or bending, a hole's bottom, can occur. Furthermore, the hole diameter can gradually shrink across the substrate, forming a tapered shape.

[0021] The occurrence of such shape anomalies reduces the structural uniformity of the substrate. Figure 3 is a schematic diagram showing an example of the structure of a substrate with reduced structural uniformity. Figure 3A shows an example of a substrate with a minute step (pattern defect) on a portion of the substrate surface. That is, the step occurs in the portion indicated by the dashed line in Figure 3A. Such a step is formed stochastically due to factors such as fluctuations in the number of photons during exposure, non-uniformity in the resist material, and reduced contrast in the exposure device. When a substrate with a step formed on its surface is observed with a scanning electron microscope (observation device 200), a surface image such as that shown in Figure 3B is obtained. Because a hole pattern is formed on the substrate, the grayscale value of the region corresponding to the hole is low, as in the example of Figure 2. In addition, the grayscale value of the region where the step is formed may also be low. For simplicity's sake, Figure 3B shows the region where the step occurs as a circle, and the grayscale value within that region is constant. However, in reality, the shape of the region corresponding to the pattern defect is complex, reflecting the shape of the pattern defect, and the grayscale value within that region varies widely.

[0022] 3 shows an example in which a step is formed on the substrate, but not limited to the formation of a step, if a shape abnormality such as necking, clogging, bowing, striation, tilting, twisting, bending, or taper occurs on the substrate, a shading reflecting the shape abnormality will appear in the surface image (SEM image). Such shape abnormalities may cause the shape of each hole to deviate from a perfect circle, and may also reduce the uniformity of the pattern.

[0023] In conventional visual inspections, workers visually check the surface images of the substrate to determine the quality of the process. In other words, the quality of the process is determined subjectively by the worker, and no quantitative evaluation is performed. In addition, because workers visually check the surface images of the substrate, there is a possibility that minute shape abnormalities may be overlooked.

[0024] In order to solve this problem, in this embodiment, feature amounts are extracted from a surface image obtained of a non-defective product (a substrate having structural uniformity), and the structural uniformity of the substrate is quantitatively evaluated by comparing the feature amounts with feature amounts extracted from a surface image of the substrate to be inspected.

[0025] In this embodiment, a trained learning model is used to extract features from a surface image. FIG. 4 is an explanatory diagram illustrating the essential configuration of the learning model MD. The learning model MD shown in FIG. 4 is a learning model using ResNet (Residual Neural Network) and includes multiple intermediate layers. In the example of FIG. 4, a configuration including four feature extraction layers FE1 to FE4 is shown as the intermediate layers. Of the four feature extraction layers FE1 to FE4 included in the learning model MD, outputs are extracted from the second feature extraction layer L2 and the third feature extraction layer L3. The vertical and horizontal dimensions of the outputs are aligned in average pooling layers AP1 and AP2, and then the outputs are combined in the channel direction in the combining layer CL, thereby extracting features (feature vectors) from a single surface image. In the example of FIG. 4, features are generated as many times as the number of patch images obtained by dividing the surface image (= the number of vertical divisions of the image × the number of horizontal divisions). By extracting features using this method, features that retain positional information within the image can be obtained.

[0026] While Fig. 4 shows a learning model MD using ResNet, the model structure of the learning model MD is not limited to that shown in Fig. 4, and any model structure including a general convolutional neural network (CNN) structure can be adopted. For example, the learning model MD may have a model structure used in Deep Nearest Neighbor Anomaly Detection (DN2), Sub-Image Anomaly Detection with Deep Pyramid Correspondences (SPADE), a Patch Distribution Modeling Framework for Anomaly Detection and Localization (PaDiM), Towards Total Recall in Industrial Anomaly (PatchCore), etc.

[0027] In this embodiment, of the four feature extraction layers FE1 to FE4 provided in the learning model MD, the outputs from the second feature extraction layer L2 and the third feature extraction layer L3 are taken out and features are extracted, so the output of the learning model MD itself does not necessarily have to be features, but may be some kind of probability value calculated based on the features, etc.

[0028] The following describes the configuration of an evaluation device 300 that evaluates a surface image using the learning model MD. Fig. 5 is a block diagram showing the internal configuration of the evaluation device 300. The evaluation device 300 is a dedicated or general-purpose computer, and includes, for example, a control unit 301, a storage unit 302, a communication unit 303, an operation unit 304, and a display unit 305.

[0029] The control unit 301 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The ROM included in the control unit 301 stores a control program that controls the operation of each hardware unit included in the evaluation device 300. The CPU in the control unit 301 reads and executes the control program stored in the ROM and a computer program (described below) stored in the storage unit 302, and controls the operation of each hardware unit, thereby causing the entire device to function as the evaluation device of the present disclosure. The RAM included in the control unit 301 temporarily stores data used during execution of calculations.

[0030] In the embodiment, the control unit 301 is configured to include a CPU, a ROM, and a RAM, but the configuration of the control unit 301 is not limited to the above. The control unit 301 may be, for example, one or more control circuits or arithmetic circuits including a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), a quantum processor, volatile or non-volatile memory, etc. The control unit 301 may also have functions such as a clock that outputs date and time information, a timer that measures the elapsed time from when a measurement start instruction is given until when a measurement end instruction is given, and a counter that counts numbers.

[0031] The storage unit 302 includes a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an electronically erasable programmable read-only memory (EEPROM), etc. The storage unit 302 stores various computer programs executed by the control unit 301 and various data used by the control unit 301.

[0032] The computer program (program product) stored in the storage unit 302 includes an evaluation program PG that causes a computer to execute a process of deriving an evaluation value that quantitatively evaluates the uniformity of processing accuracy based on a surface image of the substrate to be evaluated and outputting the derived evaluation value. The evaluation program PG may be a single computer program or a program group consisting of multiple computer programs. The evaluation program PG may be executed by multiple computers in cooperation. The evaluation program PG may also partially use an existing library.

[0033] A computer program including the evaluation program PG is provided by a non-transitory recording medium RM on which the computer program is readably recorded. The recording medium RM is a portable memory such as a CD-ROM, USB memory, a Secure Digital (SD) card, a microSD card, or a CompactFlash (registered trademark). The control unit 301 reads various computer programs from the recording medium RM using a reading device (not shown) and stores the read various computer programs in the memory unit 302. The computer program including the evaluation program PG may be provided via communication. In this case, the control unit 301 acquires the computer program including the evaluation program PG by communication via the communication unit 303 and stores the acquired computer program in the memory unit 302.

[0034] The above-described learning model MD is stored in the storage unit 302. In this embodiment, it is assumed that the learned learning model MD is stored in the storage unit 302. The learning model MD can be learned using a known method, and the description thereof will be omitted in this embodiment.

[0035] In this embodiment, the evaluation device 300 is configured to include a learning model MD. Alternatively, the learning model MD may be stored in an external device. In this case, the control unit 301 of the evaluation device 300 may access the external device via a communication network, transmit the surface image acquired from the observation device 200 to the external device, and acquire the evaluation value obtained as a result of calculation by the external device via the communication network.

[0036] Furthermore, the storage unit 302 includes a feature memory FM that stores feature amounts extracted from surface images of substrates that are deemed to be non-defective. The feature amounts of non-defective substrates are collected in a learning phase before the start of actual operation (substrate evaluation) and are stored in advance in the feature memory FM. The feature memory FM does not need to be hardware, but may be a storage area formed within the storage unit 302. The feature memory FM may be provided inside the evaluation device 300 or outside the evaluation device 300.

[0037] The communication unit 303 includes a communication interface for transmitting and receiving various data to and from an external device. A communication interface conforming to a communication standard such as a local area network (LAN) can be used as the communication interface of the communication unit 303. The external device may be the substrate processing apparatus 100, the observation apparatus 200, or a user terminal (not shown). When data to be transmitted is input from the control unit 301, the communication unit 303 transmits the data to the destination external device, and when data transmitted from the external device is received, the communication unit 303 outputs the received data to the control unit 301.

[0038] The operation unit 304 includes operation devices such as a touch panel, a keyboard, and switches, and receives various operations and settings from a user, etc. The control unit 301 performs appropriate control based on various pieces of operation information provided by the operation unit 304, and stores setting information in the storage unit 302 as necessary.

[0039] The display unit 305 includes a display device such as a liquid crystal monitor or an organic EL (Electro-Luminescence) monitor, and displays information to be notified to the user or the like in response to an instruction from the control unit 301 .

[0040] The evaluation device 300 in this embodiment may be a single computer, or may be a computer system configured with multiple computers and peripheral devices, etc. Furthermore, the evaluation device 300 may be a virtual machine whose entity is virtualized, or may be a cloud.

[0041] The operation of the evaluation device 300 will be described below. The evaluation device 300 according to this embodiment extracts features from non-defective products in a learning phase before the start of actual operation. It is assumed that a trained learning model MD is stored in the storage unit 302 of the evaluation device 300.

[0042] 6 is a flowchart showing the extraction procedure for extracting feature quantities from a non-defective substrate. The control unit 301 of the evaluation device 300 acquires a surface image of a substrate that is deemed to be a non-defective product via the communication unit 303 (step S101). Here, an operator checks the surface images of the substrate and acquires multiple surface images that are deemed to be non-defective (hereinafter also referred to as non-defective product images). The operator may determine whether a substrate is non-defective by focusing on the uniformity of the pattern, or by focusing on the shape (roundness) of each hole. Furthermore, the operator may determine whether a substrate is non-defective by focusing on the surface condition following cleaning of the substrate, or by focusing on the degree of surface deterioration following substrate processing.

[0043] The control unit 301 performs preprocessing on the acquired surface images. Specifically, the control unit 301 enlarges or reduces each of the acquired surface images so that the imaging magnifications of the multiple surface images are the same (step S102). Furthermore, the control unit 301 masks unnecessary portions of the surface images (step S103). The control unit 301 may execute the processes of steps S102 and S103 as necessary. For example, if surface images with the same imaging magnification are obtained in the observation device 200, the process of step S102 may be omitted. Furthermore, if the surface image includes only the structure to be evaluated (e.g., a hole pattern) and does not include other unnecessary structures, the process of step S103 may be omitted.

[0044] The control unit 301 inputs the preprocessed surface image into the learning model MD and extracts features (step S104). For example, in the process of executing calculations using the learning model MD, the control unit 301 extracts outputs from the intermediate layers (feature extraction layers FE2 and FE3), aligns the dimensions of the extracted outputs, and combines them in the channel direction to extract features. The features are represented by feature vectors (or feature tensors).

[0045] The control unit 301 stores the feature amounts extracted in step S104 in the feature amount memory FM (step S105). The control unit 301 may selectively store, from the feature amounts extracted in step S104, feature amounts that represent non-defective product images in the feature amount memory FM. The control unit 301 executes the processes of steps S102 to S105 for each of the multiple surface images acquired in step S101, and stores the extracted feature amounts in the feature amount memory FM.

[0046] By the above procedure, a plurality of feature amounts extracted from the surface image of the substrate that is determined to be a non-defective product are stored in the feature amount memory FM.

[0047] In the actual operation phase after the learning phase is completed, the evaluation device 300 performs a quantitative evaluation of the substrate to be evaluated.

[0048] 7 is a flowchart showing the evaluation procedure for evaluating a substrate to be evaluated. The control unit 301 of the evaluation apparatus 300 acquires a surface image of the substrate to be evaluated via the communication unit 303 (step S121). The substrate to be evaluated is a substrate (e.g., a substrate on which a hole pattern is formed) processed in the substrate processing apparatus 100. The surface image is an image obtained by observing the substrate processed in the substrate processing apparatus 100 with the observation apparatus 200.

[0049] The control unit 301 performs preprocessing on the acquired surface images. Specifically, the control unit 301 enlarges or reduces each of the acquired surface images so that the imaging magnification of the multiple surface images is the same (step S122). Furthermore, the control unit 301 masks unnecessary portions in the surface images (step S123). These processes are the same as the processes in steps S102 and S103 performed on the non-defective product images.

[0050] The control unit 301 inputs the preprocessed surface image into the learning model MD and extracts features (step S124). For example, in the process of performing calculations using the learning model MD, the control unit 301 extracts outputs from the intermediate layers (feature extraction layers FE2 and FE3), aligns the dimensions of the extracted outputs, and combines them in the channel direction to extract features. By using the learning model MD, the control unit 301 can extract features equal to the number of patch images obtained by dividing the surface image.

[0051] The control unit 301 reads out the features (features of the non-defective image) stored in the feature memory FM and calculates the distance between the feature extracted in step S124 and the feature read out from the feature memory FM (step S125). For example, the control unit 301 maps the feature extracted in step S124 and the feature read out from the feature memory FM into a feature space and calculates the distance between the features in the mapped feature space. A single surface image is divided into a plurality of patch images, each of which has a feature (feature holding position information) obtained. The control unit 301 calculates the distance between each of the feature extracted from the patch image and the feature read out from the feature memory FM. The distance calculated in step S125 may be any measure representing the similarity / dissimilarity between the features, such as Euclidean distance or Mahalanobis distance.

[0052] The control unit 301 removes the mask applied to the surface image and resizes it to the original image size (step S126), i.e., the control unit 301 executes the reverse procedure of steps S122 and S123.

[0053] The control unit 301 calculates an evaluation value that quantitatively evaluates the structural uniformity of the evaluation target substrate based on the distance calculated in step S125 (step S127). In step S125, the control unit 301 calculates the distance between the feature amount of each patch image and the feature amount of the non-defective image. The control unit 301 calculates, for example, the average value, variance, minimum value, maximum value, etc. of the calculated distance as the evaluation value (evaluation value for one surface image). Alternatively, the control unit 301 may use the distance calculated for each patch image as the evaluation value (evaluation value that holds position information). Furthermore, the control unit 301 may calculate the inverse of the average value, variance, minimum value, and maximum value of the calculated distance as the evaluation value, or may calculate the evaluation value using a function that uses these values ​​as variables.

[0054] Based on the distance calculated in step S125, the control unit 301 generates an evaluation image indicating an evaluation of the structural uniformity of the substrate being evaluated (step S128). The control unit 301 generates the evaluation image according to the length of the distance calculated for each patch image. A relatively short calculated distance indicates a high similarity to the non-defective image and high structural uniformity. Conversely, a relatively long calculated distance indicates a low similarity to the non-defective image and low structural uniformity. Therefore, the control unit 301 generates, as the evaluation image, a grayscale image or a heat map obtained by setting the grayscale and color of the image according to the length of the distance calculated for each patch image.

[0055] If, during learning using non-defective images, non-defective images with high pattern uniformity are selected and feature amounts are extracted from the selected non-defective images to construct the feature memory FM, an evaluation image that reflects the pattern uniformity is obtained in step S128. On the other hand, if, during learning using non-defective images, non-defective images with holes that are nearly circular are selected and feature amounts are extracted from the selected non-defective images to construct the feature memory FM, an evaluation image that reflects the circularity of the hole shapes is obtained in step S128.

[0056] The reliability may be increased by comparing the evaluation value obtained using this method with the evaluation value of the roundness derived using another method such as contour extraction, and selecting the features stored in the feature memory FM based on the comparison result.

[0057] Furthermore, an evaluation value and evaluation image relating to pattern uniformity and an evaluation value and evaluation image relating to circularity may be presented to the operator, and the operator may select the evaluation value and evaluation image to be fed back to the process recipe.

[0058] Furthermore, if, during learning using non-defective images, the surface condition of the substrate after cleaning is observed to select non-defective images with good surface conditions, and feature amounts are extracted from the selected non-defective images to construct the feature memory FM, then an evaluation image that reflects the surface condition of the substrate is obtained in step S128. Also, if, during learning using non-defective images, the surface condition of the substrate after processing is observed to select non-defective images with little surface degradation, and feature amounts are extracted from the selected non-defective images to construct the feature memory FM, then an evaluation image that reflects the degree of degradation of the substrate surface is obtained in step S128.

[0059] The control unit 301 outputs the evaluation value calculated in step S127 and the evaluation image generated in step S128 (step S129). The control unit 301 outputs the evaluation value and the evaluation image by displaying the calculated evaluation value and the generated evaluation image on the display unit 305. Alternatively, the control unit 301 may notify the user's terminal device of the calculated evaluation value and the generated evaluation image via the communication unit 303.

[0060] FIG. 8 is a schematic diagram showing an example of display of evaluation values ​​and evaluation images. FIG. 8 shows an example of an evaluation image in which the maximum value of the distance calculated within one surface image is used as the evaluation value, and the longer the calculated distance within the surface image, the darker the shaded area. FIG. 8A shows an example of an evaluation image with a small evaluation value and little unevenness due to shading, indicating high structural uniformity. On the other hand, FIG. 8B shows an example of an evaluation image with a large evaluation value and much unevenness due to shading, indicating low structural uniformity. By referring to the evaluation values ​​and evaluation images displayed on the display unit 305, an operator can determine the quality of the process performed when a substrate is processed in the substrate processing apparatus 100. The evaluation image is not limited to a shaded image, but may be a heat map in which colors are used according to the evaluation value, or a contour map using contour lines.

[0061] The control unit 301 may superimpose the evaluation image on the surface image. Specifically, the control unit 301 may display the evaluation image, such as a shaded image, transparently and superimpose it on the surface image of the substrate to be inspected. The control unit 301 may also display an evaluation value within the image. The evaluation value may be an evaluation value for each set area, or may be an average of the evaluation values ​​for each area.

[0062] Furthermore, the control unit 301 may generate an evaluation image for each process recipe and display the evaluation images in descending order of evaluation on the display unit 305. Fig. 9 is a schematic diagram showing another display example of the evaluation images. Fig. 9 shows an example in which the evaluation images are displayed in a list from left to right in descending order of evaluation.

[0063] As described above, in the first embodiment, the structural uniformity of a substrate can be quantitatively evaluated by comparing the feature amounts extracted from the surface image of the substrate to be inspected with the feature amounts extracted from the surface image of a non-defective product (a substrate having structural uniformity).

[0064] In this embodiment, the user's task is to select uniform data (surface images), and it is expected that the quality of annotation will be improved compared to regression analysis or classification processing.

[0065] Furthermore, in this embodiment, since the evaluation value is calculated by comparing feature amounts, it is also applicable to evaluation targets for which no quantified value exists.

[0066] In the second embodiment, a configuration is described in which a correlation between a parameter included in a recipe used when a substrate is processed and an evaluation value calculated based on a surface image is calculated, and the recipe is optimized based on the calculated correlation. The overall configuration of the evaluation system and the internal configuration of the evaluation device 300 are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0067] The evaluation device 300 according to the second embodiment acquires multiple types of recipes (process recipes) used when processing substrates in the substrate processing device 100, associates the parameters included in each process recipe with evaluation values ​​calculated from surface images, and stores them in the memory unit 302.

[0068] 10 is a conceptual diagram showing an example of a table storing parameters included in a process recipe in association with evaluation values. The example in FIG. 10 shows the relationship between the gas flow rates when substrates are processed using variously changed flow rates of specific types of gases (Gas A and Gas B) and the evaluation values ​​obtained from surface images observed in each process. That is, FIG. 10 shows the relationship between the parameter (flow rate of Gas A) when substrates are processed according to multiple recipes with variously changed flow rates of Gas A and the evaluation values ​​obtained from surface images obtained in each process, and the relationship between the parameter (flow rate of Gas B) when substrates are processed according to multiple recipes with variously changed flow rates of Gas B and the evaluation values ​​obtained from surface images obtained in each process. The gas flow rates may be set values ​​in the process recipe or may be actual values ​​measured by a flow sensor provided in the substrate processing apparatus 100.

[0069] 10 shows an example of a table storing the relationship between the flow rate of specific types of gases (Gas A and Gas B) and the evaluation value. Alternatively, the parameters registered in the table may be any parameters set in the process recipe, such as the total flow rate of all gases, the substrate temperature, and the voltage and current applied when generating plasma in the processing chamber.

[0070] The control unit 301 of the evaluation device 300 derives the correlation between the parameters included in the process recipe and the evaluation value. FIG. 11 is a graph showing the correlation between the parameters included in the process recipe and the evaluation value. FIG. 11A shows the correlation between the flow rate of a specific type of gas (Gas A) and the evaluation value, and FIG. 11B shows the correlation between the flow rate of another type of gas (Gas B) and the evaluation value. The horizontal axis of each graph represents the gas flow rate, and the vertical axis represents the evaluation value. These graphs show that changing the flow rate of Gas A causes a significant change in the evaluation value, and that the flow rate of Gas A has a significant impact on the structural uniformity of the substrate. Existing methods such as linear regression, ridge regression, regression trees, random forests, and neural networks are used to derive the correlation.

[0071] The evaluation apparatus 300 can optimize the process recipe for processing the substrate based on the correlation between the parameters included in the process recipe and the evaluation value. For example, when the correlation shown in Fig. 11 is obtained, the evaluation value fluctuates significantly when the flow rate of Gas A is changed, so the set value of the flow rate of Gas A in the process recipe can be adjusted to improve the structural uniformity of the substrate.

[0072] 12 is a flowchart showing the process recipe optimization procedure. The control unit 301 of the evaluation device 300 acquires the process recipe used to process the substrate and a surface image obtained by observing the substrate (step S201).

[0073] The control unit 301 calculates an evaluation value from the observed image (step S202). The procedure for calculating the evaluation value is the same as in the first embodiment, and the evaluation value that quantitatively evaluates the structural uniformity of the substrate is calculated by comparing the feature values ​​extracted from the surface image of the substrate to be evaluated with the feature values ​​extracted from the image of a non-defective substrate.

[0074] The control unit 301 calculates the correlation between the evaluation value calculated in step S202 and the parameters included in the process recipe (step S203), and identifies parameters that have a high correlation with the evaluation value (step S204).

[0075] The control unit 301 adjusts the identified parameters to optimize the process recipe so as to improve the structural uniformity of the substrate (step S205). The control unit 301 optimizes the process recipe by rewriting the values ​​of the parameters in the process recipe. An operator may perform substrate processing in the substrate processing apparatus 100 according to the optimized process recipe, and then re-evaluate the process using the evaluation apparatus 300 to determine whether the desired results have been obtained.

[0076] 12 has described a procedure for acquiring a plurality of process recipes, calculating a correlation between a parameter included in each acquired process recipe and an evaluation value calculated for a substrate processed in accordance with each process recipe, and optimizing the process recipe based on the calculated correlation. Alternatively, the control unit 301 may execute the processes from step S204 onward based on a correlation (a correlation between a parameter included in the process recipe and an evaluation value for the substrate) acquired in advance.

[0077] As described above, in the second embodiment, the correlation between the evaluation value calculated from the surface image and the parameters included in the process parameters is calculated, and the process recipe can be optimized so as to improve the structural uniformity of the substrate.

[0078] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0079] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0080] REFERENCE SIGNS LIST 100 Substrate processing apparatus 200 Observation apparatus 300 Evaluation apparatus 301 Control section 302 Storage section 303 Communication section 304 Operation section 305 Display section PG Evaluation program MD Learning model FM Feature memory RM Recording medium

Claims

1. An evaluation method comprising the steps of: acquiring a surface image of a material to be evaluated; extracting a first feature from the acquired surface image; reading from a storage unit a plurality of second feature values ​​extracted from surface images of a plurality of materials having structural uniformity and stored in advance in the storage unit; calculating a distance in feature space between the first feature value and the plurality of second feature values ​​based on the calculated distance; calculating an evaluation value that quantitatively evaluates the structural uniformity of the material to be evaluated; and outputting the calculated evaluation value.

2. The evaluation method according to claim 1, wherein the computer executes a process of: generating an evaluation image of the structural uniformity based on the distance; and displaying the generated evaluation image together with the evaluation value.

3. The evaluation method according to claim 1, wherein the computer executes the following process: acquiring a plurality of recipes for processing the material to be evaluated; calculating a correlation between a parameter included in each acquired recipe and an evaluation value calculated for the material processed according to each recipe; and optimizing the recipe for processing the material based on the calculated correlation.

4. The evaluation method according to claim 1, wherein the computer executes a process of optimizing a recipe for processing the material based on a correlation between parameters included in multiple recipes for processing the material to be evaluated and an evaluation value calculated for the material processed according to each recipe.

5. The evaluation method according to claim 3 or 4, further comprising the steps of: generating an evaluation image for each recipe; and displaying the evaluation images in descending order of evaluation value, the evaluation image being executed by the computer.

6. The evaluation method according to claim 1, wherein the computer executes a process of extracting the first feature amount and the second feature amount using a trained learning model having a feature amount extraction layer.

7. The evaluation method according to claim 1, wherein the structural uniformity includes at least one of the uniformity of a pattern structure formed on the surface of the material, the circularity of a plurality of holes formed in the material, the uniformity of the surface condition of the material after cleaning, and the degree of surface deterioration after processing of the material.

8. The evaluation method according to claim 1, wherein the material to be evaluated is a substrate processed by a substrate processing apparatus.

9. An evaluation device comprising: a processing unit; and a memory unit, wherein the memory unit stores a plurality of feature amounts extracted from surface images of a plurality of materials having structural uniformity, and the processing unit acquires a surface image of a material to be evaluated, extracts feature amounts from the acquired surface image, reads out the plurality of feature amounts from the memory unit, calculates a distance in feature space between the extracted feature amounts and the plurality of feature amounts read out from the memory unit based on the calculated distance, calculates an evaluation value that quantitatively evaluates the structural uniformity of the material to be evaluated, and outputs the calculated evaluation value.

10. A computer program for causing a computer to execute the following processes: acquiring a surface image of a material to be evaluated; extracting a first feature from the acquired surface image; reading from a memory unit a plurality of second feature values ​​extracted from surface images of a plurality of materials having structural uniformity and stored in advance in the memory unit; calculating a distance in feature space between the first feature value and the plurality of second feature values ​​based on the calculated distance; calculating an evaluation value that quantitatively evaluates the structural uniformity of the material to be evaluated; and outputting the calculated evaluation value.

Images