Error factor estimation device, error factor estimation method, and computer-readable medium

Abstract

This error factor estimation device 100 is a device for estimating the error factor of errors that occur, and comprises: a feature-quantity-group-generating unit A2a that processes data including inspection results collected from an inspection device and generates a plurality of feature quantities; a model-generating unit 4 that generates a model A5a for learning the relationship between the plurality of feature quantities generated by the feature-quantity-group-generating unit A2a and errors; a contribution-degree-calculating unit 11 that calculates a contribution degree indicating the degree of contribution to the output of the model A5a for at least one of the plurality of feature quantities used for the model A5a learning; and an error factor acquisition unit 15 that acquires error factors labeled with feature quantities selected on the basis of the usefulness calculated from the contribution degree calculated by the contribution-degree-calculating unit 11.

Description

[Technical field]

[0001] The present disclosure relates to an error cause estimation device, an error cause estimation method, and a computer-readable medium for estimating an error cause of an error that has occurred. [Background technology]

[0002] Semiconductor inspection equipment performs inspection and measurement operations for each inspection point on the surface of a semiconductor wafer according to setting parameters called a recipe. In general, recipe adjustments are performed by engineers manually optimizing each item according to the attributes of the object to be inspected and the characteristics of the equipment. Therefore, for example, the use of an insufficiently adjusted recipe may result in an error in the inspection operation. On the other hand, unlike such recipe-induced errors, the inspection result may also be an error due to deterioration or malfunction of hardware. When an error occurs, engineers correct the recipe for recipe-induced errors, and replace parts that have deteriorated over time or perform maintenance on parts that have malfunctioned for hardware-induced errors. As such, since the countermeasures to be taken differ depending on the error cause, it is very important to estimate the error cause.

[0003] A classification method using machine learning or the like is used to estimate the error cause (see, for example, Patent Document 1). Patent Document 1 discloses a technique for increasing the amount of failure data by generating learning data about failure data with a common circuit or a common process, as a measure to deal with cases where there is not a sufficient amount of failure data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-199338 A Summary of the Invention [Problem to be solved by the invention]

[0005] Data drift occurs when data trends change continuously or discontinuously due to various causes such as recipe changes, equipment part updates, and changes in inspection targets. When data drift occurs, the formulation of error factor estimation obtained by learning past inspection results no longer applies to new inspection results. Therefore, a classification model that has learned the relationship between past inspection results and error factors has difficulty classifying current inspection results that have undergone data drift by error factors.

[0006] The present disclosure aims to provide a technique capable of estimating the cause of an error that occurs even when data drift occurs, in which the inspection results change continuously or discontinuously. [Means for solving the problem]

[0007] In order to solve the above problem, the error factor estimation device disclosed herein is an error factor estimation device that estimates an error factor of an erroneous inspection result, and includes a computer system having one or more processors and one or more memories, and the computer system executes: a first feature amount generation process that processes data including the inspection results collected from an inspection device to generate a plurality of feature amounts; a model generation process that generates a first model that learns the relationship between the plurality of feature amounts generated by the first feature amount generation process and an error; a contribution degree calculation process that calculates a contribution degree indicating the degree of contribution to an output of the first model for at least one of the plurality of feature amounts used in learning the first model; and an error factor acquisition process that acquires an error factor labeled with a feature amount or a combination of feature amounts selected based on the contribution degree calculated by the contribution degree calculation process or a usefulness degree calculated from the contribution degree. Effect of the Invention

[0008] According to the present disclosure, even when the inspection results change continuously or discontinuously, it is possible to estimate the cause of an error that occurs.

[0009] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an overall configuration of an error factor estimating device according to a first embodiment. [Diagram 2] FIG. 2 is a hardware block diagram of a computer system of the error cause estimating device. [Diagram 3] FIG. 13 is a diagram showing the data structures of feature amount groups A and B. [Figure 4] This is a diagram in which the test results are plotted for each test ID, and a diagram in which the feature values ​​are plotted for each test ID. [Diagram 5] FIG. 13 is a diagram showing a selection screen for selecting a feature amount defined in a feature amount list. [Figure 6] 11 is a diagram for explaining a method of learning a detection rule for an error record. FIG. [Figure 7] 4 is a block diagram showing details of an error cause estimation unit; FIG. [Figure 8] FIG. 13 is a diagram illustrating a method for calculating the usefulness of a feature amount. [Figure 9] 13 is a screen showing an analysis result displayed on an output device. [Figure 10] 11 is a flowchart showing an error cause estimation method. [Figure 11] FIG. 11 is a block diagram showing details of an error factor estimation unit according to the second embodiment. [Figure 12] 10 is a flowchart showing an error factor estimating method according to a second embodiment. [Figure 13] FIG. 11 is a diagram illustrating a data structure of an error dictionary according to the second embodiment. [Figure 14] FIG. 11 is a block diagram showing details of a model generating unit according to a third embodiment. [Figure 15] FIG. 13 is a diagram showing an estimation result of an error probability by an error probability estimation unit of the third embodiment. [Figure 16] 13 is a flowchart showing an example of use of the error factor estimating device according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the embodiments described below, "semiconductor inspection equipment" includes equipment that measures the dimensions of patterns formed on the surface of a semiconductor wafer, equipment that inspects patterns formed on the surface of a semiconductor wafer for defects, equipment that inspects bare wafers on which no patterns are formed for defects, and composite equipment that combines these equipment.

[0012] In the embodiments described below, "inspection" means measurement or inspection, and "inspection operation" means measurement operation or inspection operation. In the embodiments described below, "inspection target" means a wafer to be measured or inspected, or an area on the wafer to be measured or inspected. In the embodiments described below, "error" includes not only measurement problems and equipment failures, but also warning messages and other signs of errors.

[0013] <Example 1> An error factor estimation device 100 according to a first embodiment will be described with reference to FIG. 1. The error factor estimation device 100 according to the first embodiment estimates an error factor of an inspection result (hereinafter, referred to as error data as appropriate) that has resulted in an error in a semiconductor inspection device 10. The semiconductor inspection device 10 performs an inspection operation for each inspection point on the surface of a semiconductor wafer according to setting parameters called a recipe. The error factor estimation device 100 may be on-premise operated in a facility managed by a user of the semiconductor inspection device 10, or may be cloud operated outside the facility managed by a user of the semiconductor inspection device 10. The error factor estimation device 100 may also be incorporated in the semiconductor inspection device 10. The error factor estimation device 100 includes a feature group A generation unit 2a, a feature group B generation unit 2b, a feature list storage unit 3 that stores feature lists A3a and B3b, a model generation unit 4, a model A5a, a model B5b, an error factor estimation unit 6, a feature-error factor list 8, and a feature-weight list 9. The error factor estimation device 100 of the first embodiment has two feature amount group generation units (2a, 2b), two feature amount lists (A3a, B3b), and two models (A5a, B5b). The error factor estimation device 100 may have three or more feature amount groups, three or more feature amount lists, and three or more models.

[0014] (Data to be analyzed 1) The analysis target data 1 is data collected from the semiconductor inspection equipment 10. The analysis target data 1 input to the error cause estimation device 100 stores the inspection results of the semiconductor inspection equipment 10 including error data for which the cause of the error is to be analyzed. The inspection results are stored in the analysis target data 1 in association with an inspection ID, equipment data, a recipe, and the presence or absence of an error. The analysis target data 1 may be stored in an internal storage of the semiconductor inspection equipment 10, or may be stored in an external storage communicatively connected to the semiconductor inspection equipment 10.

[0015] The inspection ID is a number that is given to an inspection object each time it is inspected by the semiconductor inspection equipment 10, and is a number for identifying the inspection result.

[0016] The equipment data includes equipment-specific parameters, individual difference correction data, and observation condition parameters. The equipment-specific parameters are correction parameters used to operate the semiconductor inspection equipment 10 according to the specified specifications. The individual difference correction data are parameters used to correct individual differences between the semiconductor inspection equipment 10. The observation condition parameters are parameters that specify the observation conditions of a SEM (Scanning Electron Microscope), such as the acceleration voltage of the electron optical system.

[0017] The recipe includes a wafer map, a pattern matching image, alignment parameters, addressing parameters, and length measurement parameters. The wafer map is a coordinate map (e.g., coordinates of a pattern) on a semiconductor wafer. The pattern matching image is a searched image used to detect measurement coordinates. The alignment parameters are parameters used to correct a deviation between a coordinate system on a semiconductor wafer and a coordinate system inside the semiconductor inspection device 10, for example. The addressing parameters are information that identifies a characteristic pattern that exists within an inspection target area among patterns formed on a semiconductor wafer, for example. The length measurement parameters are parameters that describe conditions for measuring a length, and are parameters that specify, for example, which portion of a pattern is to be measured.

[0018] The inspection results include measurement results, image data, and an operation log. The measurement results are information regarding the length of a pattern on a semiconductor wafer. The image data is an observed image of the semiconductor wafer. The operation log is data describing the internal state of the semiconductor inspection equipment 10 in each operation process of alignment, addressing, and measurement, and includes, for example, the operating voltage of each component, the coordinates of the observation field, and the like. Data drift occurs in which the trend of the inspection results of the semiconductor inspection equipment 10 changes continuously or discontinuously due to changes in the internal environment of the semiconductor inspection equipment 10, such as changes in recipes and updates to equipment parts, and changes in the external environment of the semiconductor inspection equipment 10, such as changes in the inspection target.

[0019] The presence or absence of an error is a parameter indicating whether the inspection result is error data indicating an error or normal data indicating normality. This parameter may indicate the process in which the error occurred among the operation processes of error alignment, addressing, and length measurement.

[0020] (Hardware Configuration of Error Cause Estimation Device 100) The error factor estimation device 100 includes a computer system 200 having one or more processors and one or more memories. The computer system 200 functions as the feature group A generation unit 2a, the feature group B generation unit 2b, the feature list storage unit 3, the model generation unit 4, the model A 5a, the model B 5b, the error factor estimation unit 6, the feature-error factor list 8, and the feature-weight list 9 shown in FIG. 1. The computer system 200 executes each process of the flowchart in FIG. 10, which will be described later. FIG. 2 is a diagram showing a hardware configuration of the computer system 200. The hardware configuration of the computer system 200 will be described with reference to FIG. 2.

[0021] The computer system 200 includes a processor 201, a communication interface 202 (hereinafter, the interface is abbreviated as I / F), a memory 203, a storage 204, a RAID controller 205, and a bus 206 that communicatively connects the above-mentioned modules. The processor 201 executes program instructions that cause each process in the flowchart of FIG. 10 to be executed. The processor 201 is, for example, a central processing unit (CPU), a digital signal processor (DSP), or an application specific integrated circuit (ASIC). The processor 201 deploys the program instructions stored in the storage 204 in an executable manner in a working area of ​​the memory 203. The memory 203 stores the program instructions executed by the processor 201, data processed by the processor 201, and the like. The memory 203 is, for example, a flash memory, a random access memory (RAM), a read only memory (ROM), and the like. The storage 204 stores an OS, a boot program, and a Web application. The storage 204 also stores the above-mentioned feature lists A3a and B3b, feature groups A and B, which will be described later, a model A5a and a model B5b, a feature-error cause list 8, and a feature-weight list 9. The storage 204 is a hard disk drive (HDD), a solid state drive (SSD), or the like.

[0022] The communication I / F 202 is communicatively connected to a storage that stores the above-mentioned analysis target data 1, and receives the analysis target data 1 from this storage. The communication I / F 202 also outputs the analysis result 900 (see FIG. 9) to an output device 7 locally or on a network. The RAID controller 205 logically operates the multiple storages 204 as if they were a single device. The RAID controller 205 writes various data to the multiple storages 204 and reads various data from the multiple storages 204.

[0023] (Feature group generation part) The feature group A generation unit 2a processes the analysis target data 1 to generate one or more feature amounts. The one or more feature amounts generated by the feature group A generation unit 2a are referred to as feature group A. The feature amounts generated by the feature group A generation unit 2a are defined in a feature list A3a. Moreover, the feature group B generation unit 2b processes the analysis target data 1 to generate one or more feature amounts. The one or more feature amounts generated by the feature group B generation unit 2b are referred to as feature group B. The feature amounts generated by the feature group B generation unit 2b are defined in a feature list B3b.

[0024] The data structures of the above-mentioned feature groups A and B will be described with reference to Fig. 3. Every time the semiconductor inspection equipment 10 inspects an inspection target, an inspection ID is assigned, and a recipe and inspection results (X1,1, X1,2, ...) are recorded for this inspection ID. The feature group A generation unit 2a processes the analysis target data 1 to generate feature amounts A1 and A2, etc. defined in the feature list A3a. Moreover, the feature group B generation unit 2b processes the analysis target data 1 to generate feature amounts B1 and B2, etc. defined in the feature list B3b.

[0025] (Examples of features) Next, a specific example of the feature amount will be described. The feature amount is, for example, an index related to the variation of the test results within the same device, and is the difference between the test result and the median or average of the test results within the same device for a certain test item. Another feature amount is, for example, an index relating to the variability of the test results at the same measurement point, such as the difference between the test result and the median or average of the test results at the same measurement point for a certain test item. Another feature amount is, for example, an index related to the variation of the inspection results for the same recipe, which is the difference between the inspection result and the median or average of the inspection results for the same recipe for a certain inspection item. Another feature amount is, for example, an index relating to the variation of the inspection results within the same wafer, which is the difference between the inspection result and the median or average of the inspection results within the same wafer for a certain inspection item. Another feature is, for example, an index of variation in the inspection results at measurement points using the same reference image for pattern matching, which is the difference between the inspection result and the median or average of the inspection results at measurement points using the same reference image for pattern matching for a certain inspection item. Another characteristic amount may be, for example, an error rate for a specific device or specific coordinates.

[0026] (Test results and features) With reference to FIG. 4, a comparison between the test results for a certain test item and the feature generated by processing the test results will be described. In FIG. 4, a circle indicates a normal record, and a cross indicates an error record. The diagram on the left side of FIG. 4 is diagram 401 in which the test results of test item X1 are plotted for each test ID. The diagram on the right side of FIG. 4 is diagram 402 in which the feature A1 is plotted for each test ID. In diagram 401 on the left side of FIG. 4, normal records and error records of the raw data (test results) of test item X1 are mixed in the same range, and it is difficult to determine a threshold value to distinguish between error records and normal records. In contrast, in diagram 402 on the right side of FIG. 4, by generating a feature value that is an index regarding the variation of the test results as described above, it is possible to determine a threshold value to distinguish between error records and normal records. If the feature value and the error factor have a close relationship, it is possible to determine a threshold value to distinguish between error records caused by error factors that have a close relationship with the feature value by plotting the feature value for each test ID as in diagram 402 on the right side of FIG. 4.

[0027] (Feature List Storage Unit 3) The feature list storage unit 3 stores a feature list A3a and a feature list B3b. The feature list A3a defines one or more feature amounts to be generated by the feature group A generation unit 2a. In other words, the feature group A generation unit 2a generates one or more feature amounts defined in the feature list A3a. Also, the feature list B3b defines one or more feature amounts to be generated by the feature group B generation unit 2b. In other words, the feature group B generation unit 2b generates one or more feature amounts defined in the feature list B3b.

[0028] The features defined in the feature lists A3a and B3b can be arbitrarily selected by the user. FIG. 5 is a diagram showing a selection screen 500 for selecting a feature. The user can select a feature for each of the feature lists A3a and B3b. The user selects an arbitrary feature from the feature list 501 of the selection screen 500 and adds it to the feature list field 502. The feature displayed in the feature list field 502 is the feature defined in the feature list A3a. The user can also select and delete the feature added to the feature list field 502. The computer system 200 executes a selection process for selecting a plurality of feature values ​​generated by the feature group A generating unit 2a and the feature group B generating unit 2b according to an instruction from the user. The user also sets a weight 503 for each feature value in the feature list field 502. The weight 503 set for each feature value is stored in the feature-weight list 9 for each feature value.

[0029] A user can select a combination of feature quantities suitable for estimating an error cause via a selection screen 500. This selection screen 500 may be displayed on a display unit of the output device 7, or may be displayed on a display unit connected to the error cause estimation device 100. For example, the selection screen 500 is a screen provided by a Web application executed by the error cause estimation device 100, and the Web browser of the output device 7 displays the selection screen 500 provided by the Web application. That is, the Web application executed by the error cause estimation device 100 executes a display control process so that the selection screen 500 is displayed on the display unit of the output device 7.

[0030] For example, when a hardware-induced error is to be detected as an error factor, a feature that is a difference between the inspection result and the median or average of the inspection results in the same device is defined in the feature list A3a. When a recipe-induced error is to be detected as an error factor, a feature that is a difference between the inspection result and the median or average of the inspection results in the same recipe is defined in the feature list B3b. That is, the user defines one or more feature amounts related to the hardware-induced error in the feature list A3a, and defines one or more feature amounts related to the recipe-induced error in the feature list B3b. Note that the feature amounts defined in the feature lists A3a and B3b are arbitrary, so a feature amount related to the recipe-induced error may be defined in the feature list A3a, and a feature amount related to the hardware-induced error may be defined in the feature list B3b. A feature amount common to both the feature lists A3a and B3b may be defined.

[0031] (Features - Error Factor List 8) The feature amount-error cause list 8 stores feature amounts labeled with error causes. In the feature amount-error cause list 8, for example, a feature amount that is a difference between the inspection result and the median or average of the inspection results in the same device is labeled with a hardware-induced error. In addition, in the feature amount-error cause list 8, for example, a feature amount that is a difference between the inspection result and the median or average of the inspection results in the same recipe is labeled with a recipe-induced error. Note that the error cause may be not only a hardware-induced error or a recipe-induced error, but also a detailed error cause such as an inappropriate recipe parameter or a defective part of the device.

[0032] (Feature-weight list 9) The feature amount-weight list 9 stores the feature amount and the weight set for the feature amount in association with each other. The weight set for the feature amount is the weight set in the feature amount list field 502 of the selection screen 500. The weight stored in the feature amount-weight list 9 is set according to the degree of relevance to the error cause. This weight is a value used when calculating the usefulness, which will be described later. The default value of the weight can be a value adjusted on another site.

[0033] (Model Generation Part 4) The model generation unit 4 generates models A5a and B5b that learn the relationship between a plurality of features and errors. A model trained with the features of the feature group A generated by the feature group A generation unit 2a is called model A5a, and a model trained with the features of the feature group B generated by the feature group B generation unit 2b is called model B5b. The models A5a and B5b are constructed using a decision tree-based algorithm such as Random Forest or Gradient Boosting Tree, or a machine learning algorithm such as Neural Network. An image of a learning method when a model is constructed with an algorithm based on a decision tree is shown in FIG. 6. This model is a model that learns a classification method for classifying error records and normal records using each feature of the input feature group. FIG. 6 shows an example in which a classification method for classifying error records and normal records is learned using feature A1 and feature A2.

[0034] (Error factor estimation unit 6) The error factor estimation unit 6 calculates the usefulness of each feature for the error prediction results of models A5a and B5b, and estimates the error factor based on this usefulness. The error factor estimation unit 6 estimates the error factor of the error data based on the feature-error factor list 8 and the feature-weight list 9. As shown in FIG. 7, the error factor estimation unit 6 includes a contribution degree calculation unit 11, an extraction unit 13, a usefulness calculation unit 14, and an error factor acquisition unit 15.

[0035] (Contribution calculation unit 11) The contribution degree calculation unit 11 calculates the contribution degree indicating the degree to which each feature of the feature group A used in the training of the model A5a contributed to the error prediction result which is the output of the model A5a. Also, the contribution degree calculation unit 11 calculates the contribution degree indicating the degree to which each feature of the feature group B used in the training of the model B5b contributed to the error prediction result which is the output of the model B5b. For example, when the model is constructed by an algorithm based on a decision tree, the contribution degree is a variable importance (Feature Importance) calculated based on the number of times each feature appears in a branch in the model, the improvement value of the objective function, etc. Also, the contribution degree calculation unit 11 may calculate the contribution degree using a sensitivity analysis for the model such as SHAP (SHapley Additive exPlanations) or a feature selection algorithm. In this way, the contribution calculation unit 11 calculates the contribution of each feature of feature group A used in training model A5a (hereinafter, contribution 12a of feature group A), and calculates the contribution of each feature of feature group B used in training model B5b (hereinafter, contribution 12b of feature group B).

[0036] (Extraction part 13) The extraction unit 13 extracts one or more feature amounts based on the contribution degree calculated by the contribution degree calculation unit 11. The extraction unit 13 may extract, for example, the top N feature amounts with high contribution degrees (N is a predetermined number), or may extract feature amounts having a contribution degree equal to or greater than a predetermined threshold. The combination of feature amounts extracted by the extraction unit 13 is independent of whether the feature amounts belong to feature amount groups A and B, and for example, all of the top N feature amounts may belong to feature amount group A.

[0037] (Usefulness calculation unit 14) The usefulness calculation unit 14 calculates the usefulness of each feature extracted by the extraction unit 13 based on the contribution of the feature and the weight of the feature. This usefulness is used to estimate the cause of the error. As shown in Fig. 8, the usefulness is calculated by multiplying the contribution of the feature φ and the weight of the feature w. Note that the usefulness e may be calculated based on the contribution of the feature φ and the weight of the feature w, and the calculation method is not limited to the multiplication of the contribution of the feature φ and the weight of the feature w.

[0038] (Error cause acquisition unit 15) The error factor acquisition unit 15 selects one or more feature quantities based on the usefulness calculated by the usefulness calculation unit 14, and acquires the error factor labeled with the selected feature quantity. For example, the error factor acquisition unit 15 refers to the feature quantity-error factor list 8 to acquire the error factor labeled with the feature quantity with the highest usefulness. The error factor acquisition unit 15 may acquire the error factors labeled with the top M feature quantities (M is a predetermined number) with the highest usefulness. Then, the error factor acquisition unit 15 transmits the analysis result 900 to the output device 7. As shown in FIG. 9, the analysis result 900 includes the acquired error factor 901, the top M feature quantities with the highest usefulness 902, the contribution degree 903 of the feature quantities, and a diagram 904 in which the feature quantity (the feature quantity with the highest usefulness) is plotted for each test ID.

[0039] (Output device 7) The output device 7 is a display device, and receives and displays the analysis result 900 transmitted by the error factor acquisition unit 15. Specifically, as shown in FIG. 9, the output device 7 displays an error factor 901, top M feature quantities with high usefulness 902, contributions 903 of the feature quantities, and a diagram 904 in which the feature quantities (the feature quantities with the highest usefulness) are plotted for each test ID so that the user can recognize them. In addition, when the error factor acquisition unit 15 acquires error factors labeled with the top M feature quantities with high usefulness, the output device 7 may display the error factors as candidates for the error factor in order of usefulness. The output device 7 may be a device locally connected to the error factor estimation device 100, or may be a device connected via a network. The contributions 903 may be usefulness.

[0040] (Method of estimating error factors) Next, the error factor estimation method executed by the error factor estimation device 100 will be described in detail with reference to Fig. 10. Each step of the flowchart shown in Fig. 10 is executed by a computer system 200 functioning as the feature amount group A generation unit 2a, the feature amount group B generation unit 2b, the model generation unit 4, and the error factor estimation unit 6. Note that program instructions for executing this error factor estimation method are stored in a non-transitory computer-readable medium, for example, the storage 204.

[0041] The computer system 200 (feature group A generating unit 2a, feature group B generating unit 2b) generates a feature group A including the features defined in the feature list A3a, and a feature group B including the features defined in the feature list B3b (S101 [first feature generation process and second feature generation process]). Next, the computer system 200 (model generating unit 4) generates a model A5a trained with the features of the feature group A, and a model B5b trained with the features of the feature group B (S102 [model generation process]). Then, the computer system 200 (contribution degree calculating unit 11) calculates the contribution degree of each feature of the feature group A and the contribution degree of each feature of the feature group B (S103 [contribution degree calculating process]).

[0042] Next, the computer system 200 (extraction unit 13) extracts one or more feature amounts based on the contribution degree calculated in S103 (S104 [extraction process]). Next, the computer system 200 (usefulness calculation unit 14) calculates the usefulness of each feature amount extracted by the extraction unit 13 (S105 [usefulness calculation process]). The usefulness is calculated based on the contribution degree of the feature amount and the weight of the feature amount. Then, the computer system 200 (error cause acquisition unit 15) selects one or more feature amounts based on the usefulness, refers to the feature amount-error cause list 8, and acquires the error cause labeled with the selected feature amount (S106 [error cause acquisition process]). The computer system 200 transmits the analysis result 900 to the output device 7. As a result, the output device 7 displays, so that the user can recognize, error factors 901, the top M most useful features 902, contributions 903 of these features, and a diagram 904 in which the features (the most useful feature) are plotted for each test ID.

[0043] (Effects of Example 1) A general classification model that prepares a large amount of error data labeled with error factors and learns the relationship between this error data and error factors cannot handle data drift, in which the trend of error occurrence changes continuously or discontinuously. Therefore, in the first embodiment, the feature-error factor list 8 is referenced to acquire error factors labeled with feature amounts selected based on the usefulness. As a result, even if data drift occurs in which the trend of error data changes, by labeling the feature amount that responds to errors with the error factor, it becomes possible to estimate the error factor as long as the feature amount does not change. Furthermore, in the first embodiment, by labeling the feature quantities with error factors, the number of steps required for labeling can be significantly reduced compared to a general method of labeling error data with error factors.

[0044] In addition, in the first embodiment, by preparing a feature-error cause list 8 that stores feature values ​​labeled with error causes, it is possible to easily obtain the error cause from the feature values ​​selected based on the usefulness.

[0045] Furthermore, in the first embodiment, the usefulness of the feature is calculated based on the contribution of each feature and the weight of the feature set according to the degree of relevance to the error factor. This allows the weight set according to the degree of relevance to the error factor to be taken into consideration in order to identify the error factor, making it possible to obtain error factors that are highly relevant to the error factor, thereby improving the accuracy of estimating the error factor.

[0046] In addition, in the first embodiment, by calculating the usefulness of the features extracted by the extraction unit 13, the computational load associated with calculating the usefulness can be reduced compared to the case where the usefulness is calculated for all features.

[0047] If features that commonly react to multiple error factors are mixed, features that are useful for identifying error factors such as hardware-induced errors and recipe-induced errors may not be used in model learning. Therefore, in the first embodiment, feature groups that are generated according to the phenomenon to be captured, such as hardware-induced errors and recipe-induced errors, are divided so that features that are useful for identifying error factors are used in model learning. As a result, it is possible to obtain error factors labeled with these features, improving the accuracy of estimating error factors.

[0048] By displaying a selection screen 500 for selecting features to be generated by the feature group A generation unit 2a and the feature group B generation unit 2b, an engineer or the like can select a feature that is considered to be related to the error cause from the list of features. As a result, feature values ​​that are considered to be unrelated to the error cause can be eliminated in advance, improving the accuracy of estimating the error cause.

[0049] Furthermore, in the first embodiment, the user can grasp the error cause of the error data by checking the screen displayed by the output device 7. In addition, the user can confirm that the extracted feature amount is correlated with the error by checking the feature amount and trend that contributed to the estimation of the error cause, and can confirm the validity of the estimated error cause. This allows the user to take measures with a sense of satisfaction, such as correcting the recipe if the estimated error is a recipe-related error, or performing maintenance on the device if the estimated error is a hardware-related error.

[0050] In addition, models A5a and B5b of Example 1 can easily acquire the features that contribute to outputting the error measurement result by learning the threshold for classifying error records and normal records using multiple features.

[0051] In addition, in the first embodiment, by using an index related to the variability of the inspection results as a feature amount, even if data drift of the inspection results occurs, it is possible to estimate the error factor as long as the index related to the variability is not affected by the data drift.

[0052] <Example 2> An error factor estimation device 100 according to the second embodiment will be described with reference to Fig. 11 to Fig. 13. As shown in Fig. 11, the error factor estimation device 100 according to the first embodiment includes a feature quantity-error factor list 8 and an error factor acquisition unit 15 that acquires an error factor by referring to the feature quantity-error factor list 8. On the other hand, the error factor estimation device 100 according to the second embodiment includes an error dictionary 22 and an error factor acquisition unit 21 that acquires an error factor by referring to the error dictionary 22.

[0053] Next, an error factor estimating method by the error factor estimating device 100 according to the second embodiment will be described with reference to Fig. 12. Since steps S121 to S125 in Fig. 12 are similar to steps S101 to S105 in Fig. 10 according to the first embodiment, the description thereof will be omitted.

[0054] The error cause acquisition unit 21 searches the error dictionary 22 for a combination of features that matches or is highly similar to the combination of features selected based on the usefulness calculated by the usefulness calculation unit 14, and acquires the error cause labeled with that combination (S126).

[0055] Here, the data structure of the error dictionary 22 will be described with reference to FIG. 13. A combination of feature quantities labeled with an error cause is recorded in each row of the error dictionary 22. In FIG. 13, the value of a feature quantity related to an error cause is indicated as 1, and a feature quantity not related is indicated as 0. Note that the feature quantity related to an error cause may be defined as a value within a range of 0 to 1 according to the importance. In this case, it is sufficient to search for a combination of feature quantities having an importance that is highly similar to the value of the usefulness of the feature quantity from within the error dictionary 22. For example, collaborative filtering can be used as a search method. The error cause acquisition unit 21 acquires the error cause labeled with the combination of feature quantities thus searched. In addition, the error causes acquired here may be the top K ones with the highest similarity.

[0056] (Effects of Example 2) In the second embodiment, by referring to an error dictionary that stores combinations of feature quantities labeled with error causes, information that can be used to identify the error cause is increased. This makes it possible to estimate more detailed error causes, such as inappropriate recipe parameters for recipe-related errors and defective parts for hardware-related errors.

[0057] <Example 3> An error factor estimating device 100 according to the third embodiment will be described with reference to Fig. 14 and Fig. 15. As shown in Fig. 14, the model generating unit 4 of the error factor estimating device 100 according to the third embodiment includes an error probability estimating unit 31 and an error probability learning unit 32, unlike those in the first and second embodiments.

[0058] The error probability estimation unit 31 estimates the probability of an error for normal records that are not recorded as errors in the analysis target data 1. A method for estimating the error probability of a normal record will be described with reference to FIG. 14. As shown in FIG. 4, the error probability of an error record is 1.0. The error probability of a normal record is estimated based on its positional relationship with the error record in the feature space. This error probability can be estimated from a model that predicts whether an error label has been assigned, such as Positive and Unlabeled Learning.

[0059] The error probability learning unit 32 generates a model that learns the error probability estimated by the error probability estimation unit 31. The estimation model that estimates this error probability is constructed using a decision tree-based algorithm such as Random Forest or Gradient Boosting Tree, or a machine learning algorithm such as Neural Network.

[0060] (Effects of Example 3) For example, in the case of measurement errors in a CD-SEM (Critical Dimension-Scanning Electron Microscope), errors may or may not occur even in data with similar characteristics due to minute differences in the operation of the device for each measurement. In order to improve the detection accuracy for such accidentally occurring error records, new detection rules are learned to separate the accidentally occurring error records as error records by increasing the feature amount used for learning. Therefore, in the third embodiment, a model is used to learn the error probability of each record, making it unnecessary to model to identify the boundary between the accidentally occurring error records and normal records. This suppresses the learning of feature amounts that are less related to the error factor, so that overlearning of the model can be suppressed. As a result, the generalization performance of the model and the extraction accuracy of feature amounts that contribute to the estimation of the error factor are improved, and the error factor can be estimated with higher accuracy.

[0061] <Example 4> 16 is a flowchart showing an example of how a user uses the error factor estimating device 100. In the fourth embodiment, an example of how a user uses the error factor estimating device 100 will be described with reference to FIG.

[0062] As a preparation step before using the error factor estimation device 100, analysis target data 1 of error factors is extracted from a database in which inspection results of one or more semiconductor inspection devices 10 are accumulated. Methods for extracting the analysis target data 1 include designating the product name, recipe name, and their measurement period. Then, the extracted analysis target data 1 is input to the error factor estimation device 100, and the analysis result 900 by the error factor estimation device 100 is displayed on the output device 7.

[0063] The user checks the analysis result 900 (error factors, feature amounts that contributed to the estimation of the error factors, and trend of the feature amounts) displayed on the output device 7 (S161). Then, the user judges whether the error factors displayed on the output device 7 are appropriate (S162). If the user judges that the displayed error factors are appropriate (S162: Yes), the user modifies the recipe or performs maintenance on the device to eliminate the error factors based on the displayed analysis result 900 (S163).

[0064] When it is determined that the displayed error factor is not valid (S162: No), the user rejects the analysis result 900 (S164). Then, the user adjusts the weight of the feature amount related to the rejected analysis result 900 so that the correct error factor is estimated (S165). That is, the computer system 200 executes an adjustment process to adjust the weight of the feature amount related to the rejected analysis result 900 to be relatively low. This weight may be adjusted automatically using an existing optimization algorithm such as Bayesian optimization or a metaheuristic algorithm, or may be adjusted manually on the selection screen of FIG. 5. In addition, when an error dictionary is used as in the second embodiment, the combination of feature amounts stored in the error dictionary is compared with the combination of feature amounts with high usefulness calculated by the usefulness calculation unit 14, and the weight of the matching feature amount is adjusted to be high and the weight of the non-matching feature amount is adjusted to be low. This is because it can be determined that the feature amount that does not match the error dictionary is not important for estimating the error factor, and the feature amount that matches the error dictionary is important for estimating the error factor. The weight adjustment may be performed each time an analysis result 900 is rejected, or the rejected analysis results 900 may be accumulated and the weight adjustment may be performed all at once at any timing.

[0065] (Effects of Example 4) In this way, by adjusting the weights of the feature amounts related to the analysis results 900 rejected by the user, it is possible to improve the accuracy of estimating the error factors according to the product and recipe used.

[0066] <Modification> The present disclosure is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present disclosure, and it is not necessary to include all of the configurations described. In addition, a part of an embodiment can be replaced with a configuration of another embodiment. In addition, a configuration of another embodiment can be added to a configuration of an embodiment. In addition, a part of the configuration of each embodiment can be added to, deleted from, or replaced with a part of the configuration of another embodiment.

[0067] For example, in the above-mentioned first to fourth embodiments, the error cause of the semiconductor inspection equipment 10 is estimated, but it is also possible to estimate the error cause of an error occurring in a device other than the semiconductor inspection equipment 10.

[0068] Furthermore, the error factor estimation device 100 of the above-described embodiments 1 to 4 has two feature groups A and B and two models A5a and B5b, but the error factor estimation device 100 may be a device having one feature group and one model trained with the features of that feature group.

[0069] In addition, in the above-described first to fourth embodiments, error factors labeled with features selected based on usefulness are acquired, but error factors labeled with features selected based on contribution may be acquired.

[0070] In the above-described first to fourth embodiments, the usefulness of each feature amount extracted by the extraction unit 13 is calculated, but the usefulness calculation unit 14 may calculate the usefulness of all feature amounts. In this case, the error cause acquisition unit 15 refers to the feature amount-error cause list 8 and acquires the error cause based on the calculated usefulness. [Explanation of symbols]

[0071] 1: data to be analyzed, 2a: feature group A generation unit, 2b: feature group B generation unit, 3: feature list storage unit, 3a: feature list A, 3b: feature list B, 4: model generation unit, 5a: model A, 5b: model B, 6: error factor estimation unit, 7: output device, 8: feature-error factor list, 9: feature-weight list, 10: semiconductor inspection device, 11: contribution degree calculation unit, 12a: contribution degree of feature group A, 12b: contribution degree of feature group B, 13: extraction unit, 14: usefulness calculation unit, 15: error factor acquisition unit, 21: error factor acquisition unit, 22: error dictionary, 31: error probability estimation unit, 32: error probability learning unit, 100: error factor estimation device

Claims

1. An error cause estimation device that estimates an error cause of an erroneous test result, comprising: A computer system having one or more processors and one or more memories, The computer system includes: a first feature amount generation process for processing data including the inspection results collected from an inspection device to generate a plurality of feature amounts; a model generation process for generating a first model that learns a relationship between the plurality of feature quantities generated by the first feature quantity generation process and an error; a contribution degree calculation process for calculating a contribution degree indicating a degree of contribution of at least one of the plurality of feature amounts used in training the first model to an output of the first model; an extraction process of extracting one or more feature amounts having a high degree of contribution from the plurality of feature amounts; a usefulness calculation process for calculating a usefulness of the one or more feature amounts extracted by the extraction process; and an error factor acquisition process for acquiring an error factor labeled with a feature value or a combination of feature values ​​selected based on the usefulness calculated by the usefulness calculation process.

2. The computer system includes: an error cause list that stores the feature quantities to which the error causes are labeled, 2. The error factor estimation device according to claim 1, wherein in the error factor acquisition process, the error factor list is referenced to acquire an error factor labeled with a feature quantity selected based on the contribution degree or the usefulness degree.

3. The computer system includes: a dictionary in which the error causes are labeled to combinations of the feature amounts; 2. The error factor estimation device according to claim 1, wherein in the error factor acquisition process, the dictionary is referenced to acquire error factors labeled with combinations that match or are similar to a combination of feature quantities selected based on the contribution degree or the usefulness degree.

4. The computer system includes: a weight list in which the plurality of feature amounts and weights set for the plurality of feature amounts are stored in association with each other; 2. The error factor estimating device according to claim 1, wherein in the usefulness calculation process, the usefulness is calculated based on the contribution degree for the feature amount and the weight stored in association with the feature amount.

5. An error cause estimation device for estimating an error cause of an erroneous test result, comprising: A computer system having one or more processors and one or more memories, The computer system includes: a first feature amount generation process for processing data including the inspection results collected from an inspection device to generate a plurality of feature amounts; a model generation process for generating a first model that learns a relationship between the plurality of feature quantities generated by the first feature quantity generation process and an error; a contribution degree calculation process for calculating a contribution degree indicating a degree of contribution of at least one of the plurality of feature amounts used in training the first model to an output of the first model; an error cause acquisition process for acquiring an error cause labeled with a feature value or a combination of feature values ​​selected based on the contribution degree calculated by the contribution degree calculation process or a usefulness degree calculated from the contribution degree; an error factor estimation device that executes an adjustment process to adjust, when the error factor acquired by the error factor acquisition process is rejected by a user, a weight of a feature quantity to which the rejected error factor is labeled lower.

6. An error cause estimation device for estimating an error cause of an erroneous test result, comprising: A computer system having one or more processors and one or more memories, The computer system includes: a first feature amount generation process for processing data including the inspection results collected from an inspection device to generate a plurality of feature amounts; a model generation process for generating a first model that learns a relationship between the plurality of feature quantities generated by the first feature quantity generation process and an error; a contribution degree calculation process for calculating a contribution degree indicating a degree of contribution of at least one of the plurality of feature amounts used in training the first model to an output of the first model; an error cause acquisition process for acquiring an error cause labeled with a feature value or a combination of feature values ​​selected based on the contribution degree calculated by the contribution degree calculation process or a usefulness degree calculated from the contribution degree; executing a second feature amount generation process for processing data including the inspection results collected from the inspection device to generate a plurality of feature amounts different from the plurality of feature amounts generated by the first feature amount generation process; generating a second model that learns a relationship between the plurality of feature quantities generated by the second feature quantity generation process and an error, In the contribution degree calculation process, the contribution degree is calculated for at least one of the plurality of feature amounts used in training the second model; an error factor estimation device, characterized in that, in the error factor acquisition process, an error factor labeled with a feature amount or a combination of feature amounts selected based on the contribution degree or the usefulness calculated by the contribution degree calculation process is acquired.

7. The computer system includes:

2. The error factor estimating device according to claim 1, further comprising a selection process for selecting the plurality of feature quantities generated by the first feature quantity generating process from among a plurality of feature quantities.

8. The computer system includes: the error factor estimation device according to claim 1, further comprising a display control process for causing a display unit to display the error factors acquired by the error factor acquisition process, a list of the feature amounts selected based on the contribution degree or the usefulness, or a trend of the feature amounts.

9. 2. The error factor estimating device according to claim 1, wherein the model generation process generates a model that learns a classification method for classifying error records and normal records using the plurality of feature quantities generated by the first feature quantity generation process.

10. 2. The error factor estimation device according to claim 1, wherein the model generation process generates a model that learns an error probability of each record estimated based on a positional relationship between an error record and a normal record in a feature space of the plurality of features.

11. An error cause estimation device for estimating an error cause of an erroneous test result, comprising: A computer system having one or more processors and one or more memories, The computer system includes: a first feature amount generation process for processing data including the inspection results collected from an inspection device to generate a plurality of feature amounts; a model generation process for generating a first model that learns a relationship between the plurality of feature quantities generated by the first feature quantity generation process and an error; a contribution degree calculation process for calculating a contribution degree indicating a degree of contribution of at least one of the plurality of feature amounts used in training the first model to an output of the first model; an error factor acquisition process for acquiring an error factor labeled with a feature value or a combination of feature values ​​selected based on the contribution degree calculated by the contribution degree calculation process or the usefulness degree calculated from the contribution degree; The feature amount is An index of variation in test results within the same device, An index of variation in test results at the same measurement point, Indicators of variability in test results for the same recipe, An index relating to the variation of inspection results within the same wafer, and An index of the variability of the inspection results at the measurement points using the same reference image for pattern matching; The error factor estimation device is characterized in that the error factor estimation device is at least one of the above.

12. An error cause estimation method for estimating an error cause of an erroneous test result, comprising: Processing data including the inspection results collected from the inspection device to generate a plurality of feature quantities; generating a first model that learns a relationship between the generated plurality of feature quantities and an error; calculating a contribution degree indicating a degree of contribution of at least one of the plurality of feature amounts used in training the first model to an output of the first model; an extraction process of extracting one or more feature amounts having a high degree of contribution from the plurality of feature amounts; a usefulness calculation process for calculating a usefulness of the one or more feature amounts extracted by the extraction process; acquiring an error factor labeled with a feature value or a combination of feature values ​​selected based on the usefulness calculated by the usefulness calculation process.

13. providing an error cause list storing the feature quantities labeled with the error causes; 13. The error factor estimating method according to claim 12, wherein acquiring the error factor includes referring to the error factor list and acquiring an error factor labeled with a feature quantity selected based on the contribution degree or the usefulness degree.

14. providing a dictionary in which the combination of the feature quantities is labeled with the error factors; 13. The error factor estimating method according to claim 12, wherein acquiring the error factor includes referring to the dictionary and acquiring an error factor labeled with a combination that matches or is similar to a combination of feature quantities selected based on the contribution degree or the usefulness degree.

15. A non-transitory computer-readable medium storing program instructions for executing an error cause estimation method for estimating an error cause of an erroneous test result, comprising: The error factor estimation method includes: Processing data including the inspection results collected from the inspection device to generate a plurality of feature quantities; generating a first model that learns a relationship between the generated plurality of feature quantities and an error; calculating a contribution degree indicating a degree of contribution of at least one of the plurality of feature amounts used in training the first model to an output of the first model; an extraction process of extracting one or more feature amounts having a high degree of contribution from the plurality of feature amounts; a usefulness calculation process for calculating a usefulness of the one or more feature amounts extracted by the extraction process; acquiring an error factor labeled with a feature or a combination of feature factors selected based on the usefulness calculated by the usefulness calculation process.

16. The error factor estimation method includes: providing an error cause list in which the feature quantities are labeled with the error causes; 16. The computer-readable medium of claim 15, wherein acquiring the error factor includes referring to the error factor list to acquire an error factor labeled with a feature quantity selected based on the contribution degree or the usefulness degree.

17. The error factor estimation method includes: providing a dictionary in which the combination of the feature quantities is labeled with the error factors; 16. The computer-readable medium of claim 15, wherein obtaining the error factors includes referring to the dictionary to obtain error factors labeled with combinations that match or are similar to a combination of features selected based on the contribution degree or the usefulness degree.

Images