Error cause presumption device and presumption method

Abstract

The error cause presumption device includes: a feature quantity generation unit that generates a feature quantity suitable for a machine learning model using data transmitted from the outside; a model database that has at least one or more error prediction models that use the feature quantity as input data for determination of whether an error has occurred; a model evaluation unit that compares a prediction result of the error prediction model with a true error result actually measured to evaluate the performance of the error prediction model; a model selection unit that selects an error prediction model whose evaluation value calculated by the model evaluation unit is a predetermined value or more from the model database; and an error prediction model generation unit that generates a new error prediction model for a measured error in a case where the error prediction model selected by the model selection unit does not meet the predetermined value. Thus, even without prior annotation of error causes, the causes of various errors that have occurred can be presumed.

Description

Technical Field

[0001] This invention relates to an apparatus and method for estimating the cause of an error. Background Technology

[0002] Semiconductor measurement and inspection equipment performs inspection and measurement actions on each inspection point on the surface of a semiconductor wafer that is determined to be abnormal, according to preset parameters known as a scheme. The scheme parameters are generally adjusted manually by engineers to optimize each item based on the properties of the object being measured / inspected or the characteristics of the equipment. Therefore, errors may occur during inspection and measurement actions, for example, when using an insufficiently adjusted scheme or when the characteristics of the equipment change over time. Such errors, as errors caused by the scheme itself, are called scheme errors.

[0003] When such errors occur, service engineers typically analyze internal data from semiconductor measurement and inspection devices to pinpoint the cause. However, with the miniaturization and diversification of semiconductors, the number of solutions and solution specifications increases, as does the complexity of solution generation. Therefore, determining the cause of errors takes time, contributing to reduced system uptime.

[0004] Patent Document 1 discloses the following: By identifying faults in a measuring tool used to determine the desired size of an ultra-small electronic mechanism, the user can quickly focus on the most problematic solution, use error logs typical of any measuring tool to determine the root cause, and automate the process, etc.

[0005] In Patent Document 2, as a technique for estimating the cause of a defect when a defect occurs on the machined surface of a workpiece, a defect cause estimation device is disclosed as follows: using a machine learning device, the inspection result of the inspection device on the machined surface of the workpiece is observed as a state variable, label data representing the cause of the defect in the machined surface is obtained, and the state variable is associated with the label data for learning.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent No. 4398441

[0009] Patent Document 2: Japanese Patent No. 6530779 Summary of the Invention

[0010] The problem that the invention aims to solve

[0011] In the method described in Patent Document 1, the root cause can be automatically determined using a typical error log. However, Patent Document 1 does not specify what kind of error is involved in the normalized quantity of errors related to the scheme used by the measuring tool.

[0012] The applicability of the processing defect cause estimation device described in Patent Document 2 is limited to situations where state variables can be correlated with tag data for learning. In other words, annotation is required.

[0013] When estimating the causes of errors in semiconductor inspection equipment, the mechanisms by which these errors occur depend on the product and manufacturing process. Furthermore, errors vary in many ways. Therefore, it is difficult to anticipate and compile all possible causes for future learning.

[0014] The purpose of this invention is to presume the cause of a problem for a variety of errors, even without prior annotation of the cause.

[0015] Methods for solving problems

[0016] The error cause estimation apparatus of the present invention comprises: a feature quantity generation unit that generates feature quantities suitable for a machine learning model using data sent from an external source; a model database having at least one error prediction model that uses the feature quantities as input data to determine whether an error has occurred; a model evaluation unit that evaluates the performance of the error prediction model by comparing the prediction results of the error prediction model with the actual measured error results; a model selection unit that selects an error prediction model from the model database whose evaluation value calculated by the model evaluation unit is above a predetermined value; and an error prediction model generation unit that generates a new error prediction model for the measured error if no suitable error prediction model is selected by the model selection unit.

[0017] The error cause estimation method of the present invention includes: a feature quantity generation step, which uses data sent from an external source to generate feature quantities suitable for a machine learning model; a model evaluation step, which compares the prediction results of error prediction models stored in a model database with the actual measured error results, and uses the feature quantities as input data to evaluate the performance of the error prediction model used in determining whether an error has occurred; a model selection step, which selects an error prediction model from the model database whose evaluation value calculated in the model evaluation step is above a predetermined value; and an error prediction model generation step, which generates a new error prediction model for the measured error if no error prediction model selected by the model selection step is suitable.

[0018] Invention Effects

[0019] According to the present invention, even without prior annotation of the cause of the error, it is possible to presume the cause of the error for a variety of errors that occur. Attached Figure Description

[0020] Figure 1 This is a block diagram illustrating an information processing system that includes the error cause estimation device of Embodiment 1.

[0021] Figure 2 It means Figure 1 The structural diagram of the error cause estimation device.

[0022] Figure 3 This is a flowchart illustrating the steps involved in generating a new error prediction model in the first error prediction model generation unit of Embodiment 1.

[0023] Figure 4 This is a table representing an example of the data structure for the input data in Example 1.

[0024] Figure 5 It means Figure 2 The diagram illustrates the processing and display examples in the model analysis section.

[0025] Figure 6 This is a schematic diagram illustrating the processing and display examples in the model evaluation unit and model analysis unit of Embodiment 1.

[0026] Figure 7 This is a graph illustrating an example of the relationship between the value of a characteristic quantity in Example 1 and its contribution to the occurrence of the error.

[0027] Figure 8 This is a flowchart illustrating the second error prediction model generation step in Example 1.

[0028] Figure 9 This is a structural diagram showing the error cause estimation device having the data classification unit of Embodiment 2.

[0029] Figure 10 This is a structural diagram showing the data classification section of Embodiment 2.

[0030] Figure 11 It is a graph that represents the state of erroneous data by using the relationship between the value of a feature and its contribution to the error.

[0031] Figure 12 This is a structural diagram of the error cause estimation device that performs error cause estimation only in Embodiment 3.

[0032] Figure 13 This is a structural diagram of an error cause estimation device having an error cause label acquisition unit and an error cause label database, as shown in Embodiment 3.

[0033] Figure 14 This is a diagram representing the candidate causes of user errors in Example 3. Detailed Implementation

[0034] In the embodiments described below, the "semiconductor inspection apparatus" includes an apparatus for measuring the size of a pattern formed on the surface of a semiconductor wafer, an apparatus for inspecting whether there are defects in the pattern formed on the surface of a semiconductor wafer, or an apparatus for inspecting whether there are defects in a bare wafer without a pattern, and also includes a composite apparatus composed of multiple of these apparatuses.

[0035] Additionally, in the embodiments described below, "inspection" is used to mean measurement or inspection, "inspection action" is used to mean measurement or inspection action, and "inspection object" refers to the wafer that is the object of measurement or inspection, or the area of ​​the wafer that is the object of measurement or inspection.

[0036] In addition, in this specification, the error cause estimation device and the error cause estimation method are synonymous.

[0037] Hereinafter, a preferred embodiment of the error cause estimation device and estimation method will be described.

[0038] Preferably, the estimation device includes a feature quantity generation unit, a model database, a model evaluation unit, a model selection unit, and an error prediction model generation unit. The estimation device also includes a data classification unit, which classifies the error data in the input data according to each error cause.

[0039] Preferably, in the estimation device, the error prediction model generation unit assigns a different label to each erroneous data after classification, generates an error prediction model together with the label, and sends the error prediction model to the model database.

[0040] Preferably, the estimation device further includes a model analysis unit that quantifies the contribution of feature quantities to the erroneous judgment results in the erroneous prediction model selected by the model selection unit.

[0041] Preferably, the estimation device has the following structure: it prompts the user with the feature quantity with a high contribution value calculated by the model analysis unit as a candidate error cause.

[0042] Preferably, the estimation device has the following structure: when multiple erroneous prediction models are selected by the model selection unit, the model evaluation unit calculates the model evaluation value, uses the model evaluation value to correct the contribution of each feature quantity calculated by the model analysis unit, and provides the user with a suggestion as a candidate cause of error for the feature quantity whose corrected contribution value is high calculated from the multiple erroneous prediction models respectively.

[0043] Preferably, the estimation device further includes another error prediction model generation unit, which generates an error prediction model in such a way that the corrected error cause is included in the analysis result of the model analysis unit when the user has modified the candidate error cause.

[0044] Preferably, the estimation device further comprises: an error cause label database, which stores the relationship between error causes and at least one of the feature quantity generated by the feature quantity generation unit and the combination of the feature quantity; and an error cause label acquisition unit, which uses the label relationship in the error cause label database to assign the corresponding error cause to the feature quantity corresponding to the contribution value after being numerically converted by the model analysis unit.

[0045] Preferably, in the estimation device, the error prediction model generation unit uses the action process in which the error occurred and the input data from the action processes preceding that action process to generate a new error prediction model.

[0046] Furthermore, regarding the relationship between the structure of the estimation device and the procedures of the estimation method, the feature quantity generation unit corresponds to the feature quantity generation procedure, the model evaluation unit corresponds to the model evaluation procedure, the model selection unit corresponds to the model selection procedure, and the error prediction model generation unit corresponds to the error prediction model generation procedure. Moreover, these procedures are not limited to being implemented in a single device; they can also be implemented using multiple devices that are distributed across the device.

[0047] Example 1

[0048] Figure 1 This is a diagram illustrating an example of an information processing system that includes the error cause estimation device of Embodiment 1.

[0049] In this diagram, the semiconductor inspection device 1 is connected to the database 2 and the error cause estimation device 3 via network 101. The error cause estimation device 3 is connected to the terminal 4 (GUI). The error cause estimation device 3 estimates the cause of errors in the inspection operations performed by the semiconductor inspection device 1.

[0050] The data sent from the semiconductor inspection device 1 may include, for example, device data, measurement scheme (hereinafter sometimes simply referred to as "scheme"), measurement results, and error results. Additionally, the scheme may include the number of measurement points, coordinate information of the measurement points (Evaluation Point: EP), shooting conditions when capturing images, and shooting sequence. Furthermore, the scheme may also include, along with the measurement points, the coordinates and shooting conditions of images obtained during the preparation stage for measuring the measurement points.

[0051] The device data includes inherent device parameters, device error correction data, and observation condition parameters. Inherent device parameters are correction parameters used to ensure the semiconductor inspection apparatus 1 operates according to predetermined specifications. Device error correction data are parameters used to correct errors between semiconductor inspection apparatuses. Observation condition parameters are, for example, parameters specifying the observation conditions of a scanning electron microscope (SEM), such as the accelerating voltage of the electron optics system.

[0052] The scheme includes wafer pattern, alignment parameters, addressing parameters, and length measurement parameters as scheme parameters. The wafer pattern is a coordinate diagram of the surface of a semiconductor wafer (e.g., the coordinates of a pattern). Alignment parameters are, for example, parameters used to correct the deviation between the coordinate system of the semiconductor wafer surface and the coordinate system inside the semiconductor inspection apparatus 1. Addressing parameters are, for example, information determining characteristic patterns within the inspection area of ​​the pattern formed on the surface of the semiconductor wafer. Length measurement parameters are parameters describing the conditions for measuring length, such as parameters specifying which part of the pattern's length to measure.

[0053] The measurement results include length measurement results, image data, and a work log. The length measurement results record the measurement of the length of the pattern on the surface of the semiconductor wafer. The image data are observed images of the semiconductor wafer. The work log records data on the internal state of the semiconductor inspection device 1 during each work process, including alignment, addressing, and length measurement. For example, it may include the operating voltage of each component and the coordinates of the viewing field.

[0054] Error results indicate which of the alignment, addressing, and length measurement processes the error occurred in, in the event of an error.

[0055] Data such as device information, plans, measurement results, and error results are accumulated in database 2 via network 101. The accumulated data is analyzed by error cause estimation device 3. The analysis results are displayed in a format that can be read by the user on terminal 4.

[0056] Figure 2 It means Figure 1 A detailed structural diagram of the device used to presume the cause of the error.

[0057] exist Figure 2In this device, the error cause estimation apparatus 3 includes a feature quantity generation unit 11 connected to an external database 2, an input data recording unit 5, a model database 12 (model DB), a model evaluation unit 13, a model selection unit 14, a model parsing unit 15, a first error prediction model generation unit 16, and a second error prediction model generation unit 17. The first error prediction model generation unit 16 is also simply referred to as the "error prediction model generation unit." The second error prediction model generation unit 17 is also referred to as "another error prediction model generation unit."

[0058] The feature generation unit 11 extracts features suitable for the machine learning model from the raw data such as device data, schemes, and measurement results sent by the database 2, and outputs the features to the input data recording unit 5. Here, feature extraction may also include data scaling, encoding of categorical variables, and generation of composite features that combine multiple data such as interaction features.

[0059] In model database 12, data from input data recording unit 5 is used as input to pre-record at least one error prediction model used in determining whether an error occurs at each checkpoint. This pre-recorded initial error prediction model can be reused from models generated in other semiconductor manufacturing plants or production lines, or it can be constructed based on the model generation steps described later for any error in database 2.

[0060] The model evaluation unit 13 evaluates the performance of the error prediction model in the model database 12 based on data such as scheme units, wafer units, and checkpoint units stored in the input data recording unit 5. Performance evaluation is obtained by comparing the error prediction results determined using the error prediction model with the actual error results stored in the input data recording unit 5. Performance evaluation values ​​can include accuracy, reproducibility, fit, F1 score, and AUC. Here, the F1 score is the harmonic mean of fit and reproducibility. AUC is short for Area Under the Curve.

[0061] The model selection unit 14 selects one or more models with high evaluation values ​​in the model evaluation unit 13 as models suitable for determining errors included in the input data recording unit 5. As a model selection method, a predetermined value is preset for the evaluation value used in the model evaluation unit 13, and the model with an evaluation value that is higher than or equal to that predetermined value is selected.

[0062] In the model selection unit 14, if there is no model with an evaluation value higher than the set predetermined value, it is considered that a new error has been input that is not suitable for the generated error prediction model, and a new error prediction model is generated by the first error prediction model generation unit 16.

[0063] The model analysis unit 15 analyzes the error prediction model selected by the model selection unit 14 to determine the extent to which each feature quantity in the input data recording unit 5 contributes to the error determination, and thereby extracts the feature quantity that is highly correlated with the error.

[0064] Figure 3 It means Figure 2 A diagram showing the steps for generating a new error prediction model in the first error prediction model generation unit 16.

[0065] In step S100, as described above, if it is determined that there is no model that has an evaluation value higher than the set predetermined value, the process proceeds to step S101.

[0066] In step S101, the learning data required for generating the error prediction model is selected (extracted). As a selection method, data is extracted from the database 2 or the input data recording unit 5, which contains the same or similar schemes as errors that cannot be detected in the error prediction model.

[0067] Next, in step S102, the weights of which feature quantities in the learning data are preferentially used are adjusted. As a method of adjustment, existing parameter search methods such as random search and Bayesian optimization can be effectively utilized.

[0068] In step S103, an error prediction model is generated based on the weights calculated in step S102 using the learning data as input. This error prediction model is the learning model used to determine whether an error has occurred within the learning data. This error prediction model can also be generated using any machine learning algorithm, such as decision trees or neural networks.

[0069] In step S104, the performance of the error prediction model generated in step S103 is evaluated. As an evaluation method, it is compared with the model evaluation unit 13 ( Figure 2 Similarly, metrics such as accuracy, reproducibility, fit, F1 score, and AUC can be used. Furthermore, it is preferable to be able to calculate these evaluation values ​​using methods such as cross-validation.

[0070] In step S105, it is determined whether the evaluation value calculated in step S104 is above a predetermined value. If it is below the predetermined value, the process returns to step S102 and repeats the same process. If it is above the predetermined value, the generation of a new error model is considered complete and saved to the model database 12. Figure 2 )middle.

[0071] Furthermore, as a method for selecting a similar scheme, one could select parameters representing registration information of patterns formed on the surface of a semiconductor wafer, or a method for determining magnification values ​​that are close to these parameters. When extracting data from database 2, feature quantities are generated in feature quantity generation unit 11 in a form suitable for the device learning model. Additionally, the period for extracting the data can be specified. Since the wafer manufacturing conditions and device status may have changed if the data is from the past, it is preferable to specify the data extraction period tracing back from the time the error occurred.

[0072] Furthermore, the learning data can include the work process in which the error occurred (the object of prediction) and the plans or measurement results from previous work processes. Semiconductor Inspection Apparatus 1 ( Figure 1 The measurement process includes continuous work steps such as alignment, addressing, and length measurement.

[0073] Figure 4 This is a table representing an example of a data structure containing input data containing values ​​of characteristic quantities measured from a semiconductor inspection device.

[0074] This figure shows the characteristic quantities (Z1, Z2, ..., Z) for each measurement index. m The value of ), the work process, and whether or not an error occurred (error result).

[0075] In this figure, an error occurs when the measurement index is 2 and the work step is 2. In this case, the features of work step 3 and beyond, which are not related to the error, can be excluded. However, the features of the preceding work step 1 may have affected the error and are therefore included in the training data.

[0076] In this way, when errors arise that cannot be handled by the learned model, new prediction models are generated as new errors, enabling the generation of error prediction models according to each error cause without prior annotation.

[0077] Next, the method for calculating the contribution of the model analysis unit 15 to error determination and the method for visualizing the calculation results to the user will be explained.

[0078] Figure 5 This is a schematic diagram illustrating the calculation of the contribution of the input features to the error prediction model to the error determination and the visualization method for the user.

[0079] In this figure, the input data of the input data recording unit 5 and the model database 12 ( Figure 2The error prediction model A(210) is input to the model analysis unit 15. Then, the calculation result 220 of the contribution is output, and a graph 230 showing the contribution of each feature quantity to the error is displayed. In other words, the contribution of the feature quantity is quantified. Here, the input data of the input data recording unit 5 is shown to have feature quantities (Z1, Z2, ..., Z...). m This is an example of a data structure that stores the values ​​of ) and the results of errors according to each measurement index assigned to the various work processes of alignment, addressing, and length measurement.

[0080] Regarding the contribution of these features to the error prediction results in the error prediction model, for example, when the error prediction model is constructed using a decision tree-based algorithm, it can be evaluated using variable importance calculated based on the number of times each feature appears in the branches within the model, the improvement value of the objective function, etc., and the SHAP value, which calculates the sensitivity of each feature value to the model output. Here, SHAP is a method used to determine the contribution of each variable (feature) to the model's prediction results; it is short for Shapley Additive exPlanations.

[0081] exist Figure 2 If the model selected in the model selection unit 14 is the error prediction model A, then, for example, the data of the row with measurement index 1 is input into the error prediction model A. Based on the difference between the output of the error prediction model A that includes feature Z1 and that excludes feature Z1, the contribution to the error determination is calculated. By repeatedly performing this calculation on all features and all measurement indices, the degree to which each feature affects the prediction model's outcome can be numerically quantified. The sum of the SHAP values ​​for each feature... The features can be displayed sequentially on terminal 4, starting with the features with the largest values. In other words, features with high contribution values ​​calculated by the model analysis unit 15 can be used as candidate error causes and prompted to the user via terminal 4.

[0082] In this way, by analyzing the sensitivity of the feature quantities in the input data recording unit 5 relative to the error prediction model, the user can be prompted with feature quantities that are highly correlated with the error.

[0083] Furthermore, by selecting a model with good performance from the data input by the model selection unit 14, even when there is a mixture of erroneous data with various characteristics within the data, it is possible to avoid extracting low-correlation features as noise and improve the accuracy of the extracted features.

[0084] In addition, if two or more models are selected by the model selection unit 14, the analysis results of these multiple models can be combined to indicate highly relevant features.

[0085] Next, the method for prompting error cause candidates when error prediction model A and error prediction model B are selected as two models in the model selection unit 14 will be explained.

[0086] Figure 6 This is a schematic diagram illustrating the processing and display examples in the model evaluation unit and model analysis unit of this embodiment.

[0087] In this figure, the input data of the input data recording unit 5 and the model database 12 ( Figure 2 The error prediction model A (210) is input to the model evaluation unit 13a and the model analysis unit 15a. Additionally, the input data from the input data recording unit 5 and the model database 12 ( Figure 2 The error prediction model B(211) is input to the model evaluation unit 13b and the model analysis unit 15b. Furthermore, a graph 231 can be used to display the contribution of the feature quantity to the error.

[0088] The functions of model evaluation units 13a and 13b and model analysis units 15a and 15b Figure 2 Same. Additionally, in Figure 6 For illustrative purposes, two model evaluation units 13a and 13b and two model analysis units 15a and 15b are shown, but in practice, the processing can also be performed sequentially or in parallel by one model evaluation unit 13a and 13b and one model analysis unit 15a and 15b, respectively.

[0089] In this figure, the sum of the SHAP values ​​for each feature quantity calculated by the model analysis units 15a and 15b is shown. The final contribution is calculated by correcting the model evaluation values ​​obtained from model evaluation units 13a and 13b. In its simplest terms, for example, when the model performance evaluation index in model evaluation sections 13a and 13b is the reproducibility rate, the model's reproducibility rate is compared with the SHAP value. Multiplying them together yields the final contribution. The final contribution calculated by each error prediction model is displayed in descending order of value. It can present the user with the features that are highly correlated with errors from multiple models in a sorted manner via Terminal 4.

[0090] Furthermore, examples are provided to illustrate the relationship between the values ​​of features and their contribution to the errors, focusing on the features that rank highly in the order of their contribution.

[0091] Figure 7 This is a graph illustrating an example of the relationship between the values ​​of the feature quantities and the contribution of the parameters to error generation in this embodiment. The horizontal axis represents the feature quantities, and the vertical axis represents the contribution of the parameters to error generation.

[0092] As shown in the figure, by displaying the determination of normal and dangerous areas based on the value of contribution to the error, and the actual location of the error on the screen of terminal 4, the user can evaluate the appropriateness of the extracted feature quantity.

[0093] exist Figure 2 If the selected error cause differs from the actual cause, the user can specify (correct) the correct error cause via terminal 4. Upon specifying the cause, a new error prediction model is generated by the second error prediction model generation unit 17.

[0094] Next, the model generation steps in the second error prediction model generation unit 17 will be explained.

[0095] Figure 8 This is a flowchart illustrating the second error prediction model generation step in this embodiment.

[0096] In this diagram, if the user modifies the cause parameter (step S800), the process proceeds to step S201.

[0097] In step S201, the learning data required for generating the error prediction model is selected (extracted). The selection method is the same as... Figure 3 The steps S101 are different. Figure 8 In step S201, for errors detected by the error prediction model, data containing the same or similar schemes are extracted.

[0098] The subsequent processing of S102 to S105 in this diagram is... Figure 3 same.

[0099] Next, in step S202, the error prediction model is analyzed, and the contribution of the features extracted in the learning data in step S201 to the error judgment is numericalized. This is related to... Figure 2 The same processing is applied to the model analysis section 15 in the model.

[0100] In step S203, if the high-contribution features in the predetermined order do not contain the error cause specified by the user, the same process is repeated from S102. If they do contain the error cause, the model is saved in the model database 12 as the generation of a new error prediction model is complete (step S300).

[0101] In this way, when machine learning cannot find the correct solution, the learning model can be improved by manually providing the correct solution.

[0102] Example 2

[0103] Figure 9 This is a structural diagram illustrating the error cause estimation device with a data classification unit in Embodiment 2.

[0104] The difference between this embodiment (the figure) and Embodiment 1 is that a data classification unit 18, which classifies the data sent from the input data recording unit 5, is provided in the error cause estimation device 3. Other structures are the same as those shown in the figure. Figure 2 same.

[0105] In Example 1, assuming that the errors generated in the same scheme are due to the same cause, data from the same or similar schemes are used to perform model evaluation and model generation.

[0106] In contrast, in this embodiment ( Figure 9 In the data classification section 18, the error data is classified according to each error cause, and model evaluation and model generation are performed according to each classified error data.

[0107] Next, the structure of the data classification section 18 will be explained.

[0108] Figure 10 It means Figure 9 A structural diagram detailing the data classification department.

[0109] exist Figure 10 In the data classification unit 18, there are an error prediction model generation unit 19, a model parsing unit 115, an error cause clustering unit 20, a data segmentation unit 21, and a segmented data recording unit 122.

[0110] Error prediction model generation unit 19 generates an error prediction model, which is used to determine whether an error exists in the input data sent from input data recording unit 5. In the process of error prediction model generation unit 19, it is possible to use... Figure 8 The same process as step S103.

[0111] The model analysis unit 115 uses, for example, SHAP values ​​to calculate the extent to which each feature contributes to the judgment result of the model generated by the error prediction model generation unit 19.

[0112] The error cause clustering unit 20 classifies the contribution of each feature quantity represented by the SHAP value calculated by the model parsing unit 115 to the error using unsupervised learning.

[0113] The data segmentation unit 21 divides the data into categorized error and normal data. The segmented data is stored in the segmented data record unit 122.

[0114] Next, the methodology for this classification will be explained.

[0115] Figure 11 This is a graph that represents the state of erroneous data by using the relationship between the values ​​of features and the contribution of parameters to error generation. The horizontal axis represents the features, and the vertical axis represents the contribution of parameters to error generation.

[0116] In this diagram, categories 1 and 2, corresponding to different error causes, are divided into their respective regions, and the data that generated the error (error data) is separated within each region. This is because the branching methods within the model used to determine the error differ depending on the cause, and the range of the SHAP value, which represents the contribution to the error determination, is expected to vary according to each feature associated with the error cause. Error data is then separated based on this SHAP value.

[0117] For error data separated according to each error cause, through Figure 9 The first error prediction model generation unit 16 and the second error prediction model generation unit 17 generate error prediction models, and the model parsing unit 15 parses the models. Therefore, even if there are multiple error causes in the input data of the input data recording unit 5 that cannot be addressed by the error prediction models in the model database 12, it is possible to avoid extracting low-correlation features as noise, thereby improving the accuracy of the extracted features. Here, when using the error data segmented by the data classification unit 18 to generate the error prediction model, normal data with the same or similar schemes as the error data of the target can be used together as training data.

[0118] Alternatively, each erroneous data point after classification can be assigned a different label, and this label, along with the generated error prediction model, can be saved to the model database 12. The labeling can be either automatically assigned different indices sequentially, or the error cause can be labeled by the user. This labeling of error causes can be performed on a per-segment basis, thus significantly reducing the number of processing steps compared to the labeling of a single error-generating unit in existing methods. In this case, Figure 9 The model evaluation unit 13 is not needed, and the function of the model selection unit 14 is replaced by the action of selecting the label of the erroneous data that is consistent with the label of the erroneous prediction model.

[0119] Example 3

[0120] In the error cause estimation device of Embodiment 1, it can also be configured to... Figure 2When the model database 12 stores sufficient changes to the error prediction model, it does not generate a new error prediction model, but only applies the inference of the cause of the error to the input error data.

[0121] The structure of the error cause estimation device in this case will be described as Example 3.

[0122] Figure 12 This is a structural diagram illustrating the error cause estimation device in this embodiment.

[0123] In this figure, as error data, the dataset of measurement points that produced errors, measurement results, and measurement points of the same wafer and the same scheme, is input from database 22.

[0124] In the feature generation unit 11, features suitable for the machine learning model are extracted from these data, and error data is output to the error data recording unit 23.

[0125] Subsequent handling and Figure 2 Similarly, from the model database 12, one or more error prediction models with evaluation values ​​higher than the error predictions of the model input error data recording unit 23 are selected as the analysis results of that model, and the features that contribute significantly to the error predictions are displayed to the user via the terminal 4. In this case, the model selection unit 14 is not required.

[0126] In this way, limiting the function of the error cause estimation device 3 to the estimation of the cause of the error also minimizes the required input data.

[0127] Figure 13 This is a structural diagram illustrating a modified example of an error cause estimation device.

[0128] In this figure, the error cause estimation device 3 includes an error cause label acquisition unit 24 and an error cause label database 25 (error cause label DB). Therefore, candidate error causes can be obtained based on the feature quantities extracted by the model analysis unit 15.

[0129] The error cause label database 25 stores the relationship between the error causes corresponding to each feature or a combination of features. In this case, the error causes of the features need to be labeled in advance, but compared with the labeling of a single unit that produces an error as an existing method, the number of processes required can be significantly reduced.

[0130] In the error cause label acquisition unit 24, the label relationships in the error cause label database 25 are used to assign corresponding error causes to the ranked feature quantities obtained by the model parsing unit 15.

[0131] The cause of the marked error is displayed to the user via terminal 4. At this time, the contribution of each feature quantity calculated by the model analysis unit 15 can also be converted into the confidence level of the corresponding error cause and displayed.

[0132] Next, the example shown will be explained.

[0133] Figure 14 This is a diagram illustrating an example of displaying candidate error reasons to the user in this embodiment.

[0134] The chart on the right side of the figure shows the contribution of each feature quantity calculated by the model analysis unit 15. The chart on the left side shows the confidence level of the error cause calculated based on the data shown in the chart on the right.

[0135] In this way, by analyzing the error prediction model to calculate the feature quantities that help in error judgment, and labeling the error causes corresponding to each feature quantity or the combination of the feature quantities, the system can suggest to the user the feature quantities or corresponding error cause candidates that are highly relevant to the generated error.

[0136] (other)

[0137] This invention is not limited to the embodiments described above, and includes various modifications. The embodiments described above are detailed examples for the purpose of readily understanding and illustrating the invention, and are not limited to possessing all the described structures. For example, Embodiment 3... Figure 13 The error reason label acquisition unit 24 and the error reason label database 25 shown can also be used in Embodiment 1. Figure 2 And Example 2 Figure 9 Combinations can show users the reasons for errors associated with feature quantities and combinations of feature quantities, respectively.

[0138] In the above embodiments, examples of presuming the cause of errors in a semiconductor inspection device were described. However, by generating parameters that define the operation of the device and a predictive model regarding whether an error will occur when these parameters are used, the present invention can also be applied to devices other than semiconductor inspection devices.

[0139] Furthermore, in the above embodiments, an example was shown of using SHAP values ​​to quantify the contribution of feature quantities to the error prediction model, but other evaluation values ​​such as feature importance can also be applied.

[0140] Explanation of reference numerals in the attached figures

[0141] 1: Semiconductor inspection device; 2, 22: Database; 3: Error cause estimation device; 4: Terminal; 5: Input data recording unit; 11: Feature quantity generation unit; 12: Model database; 13, 13a, 13b: Model evaluation unit; 14: Model selection unit; 15, 15a, 15b, 115: Model parsing unit; 16: First error prediction model generation unit; 17: Second error prediction model generation unit; 18: Data classification unit; 19: Error prediction model generation unit; 20: Error cause clustering unit; 21: Data segmentation unit; 23: Model input error data recording unit; 24: Error cause label acquisition unit; 25: Error cause label database; 101: Network.

Claims

1. An error cause estimation device, applied to a semiconductor inspection apparatus for measuring or inspecting semiconductor wafers, characterized in that, The error cause estimation device includes: The feature generation unit uses data sent from the semiconductor inspection device, including device data, measurement scheme, measurement results, and error results, to generate feature quantities suitable for a machine learning model. A model database having at least one error prediction model that uses the feature quantity as input data to determine whether an error has occurred during the measurement or inspection of a semiconductor wafer. The model evaluation unit evaluates the performance of the error prediction model by comparing its prediction results with the actual measured error results. The model selection unit selects from the model database the error prediction model whose evaluation value calculated by the model evaluation unit is above a predetermined value set in advance; The error prediction model generation unit generates a new error prediction model for the measured error when the error prediction model selected by the model selection unit does not match. as well as The model parsing unit quantifies the contribution of the feature quantities to the error judgment results in the error prediction model selected by the model selection unit. In the case where multiple incorrect prediction models are selected by the model selection unit... The model evaluation unit calculates the model evaluation value. The contribution of each feature quantity calculated by the model analysis unit is corrected using the model evaluation value. For the feature quantity with a high corrected contribution value calculated from the plurality of error prediction models, it is suggested to the user as a candidate error cause that causes measurement or inspection operation errors of the semiconductor inspection device.

2. The error cause estimation device according to claim 1, characterized in that, The error cause estimation device further includes a data classification unit that classifies the error data in the input data according to each error cause.

3. The error cause estimation device according to claim 2, characterized in that, The error prediction model generation unit assigns a different label to each of the classified error data, generates the error prediction model together with the label, and sends the error prediction model to the model database.

4. The error cause estimation device according to claim 1, characterized in that, For feature quantities whose contribution values ​​calculated by the model parsing unit are high, the user is prompted with an error reason candidate.

5. The error cause estimation device according to claim 4, characterized in that, The error cause estimation device also has another error prediction model generation unit, which generates an error prediction model in such a way that the user modifies the candidate error cause, and the modified error cause is included in the analysis result of the model analysis unit.

6. The error cause estimation device according to claim 1, characterized in that, The error cause estimation device also includes: An error cause label database stores the relationship between error causes and at least one of the feature quantities generated by the feature quantity generation unit and combinations of the feature quantities; as well as The error cause label acquisition unit uses the label relationships in the error cause label database to assign corresponding error causes to the feature quantities corresponding to the contribution values ​​after being numerically converted by the model parsing unit.

7. The error cause estimation device according to any one of claims 1 to 3, characterized in that, The error prediction model generation unit uses the work process in which the error occurred and the input data from previous work processes to generate a new error prediction model.

8. A method for estimating the cause of an error, applied to a semiconductor inspection apparatus for measuring or inspecting semiconductor wafers, characterized in that, The methods for estimating the cause of the error include: The feature generation process uses data sent from the semiconductor inspection device, including device data, measurement scheme, measurement results, and error results, to generate feature quantities suitable for machine learning models. The model evaluation process compares the prediction results of the error prediction model stored in the model database with the actual measured error results, and uses the feature quantity as input data to evaluate the performance of the error prediction model used in determining whether errors occur in the measurement or inspection of semiconductor wafers. The model selection process involves selecting, from the model database, the error prediction model whose evaluation value calculated in the model evaluation process is above a predetermined value; The error prediction model generation step, in the case that no error prediction model selected in the model selection step is suitable, generates a new error prediction model for the measured error; and The model analysis step quantifies the contribution of the feature quantities to the error prediction results in the error prediction model selected in the model selection step. In the case where multiple incorrect prediction models are selected through the model selection process, The model evaluation value is calculated in the model evaluation process. The contribution of each feature quantity calculated in the model analysis process is corrected using the model evaluation value. For the feature quantity with a high corrected contribution value calculated from the plurality of error prediction models, it is suggested to the user as a candidate error cause that may have caused a measurement or inspection error in the semiconductor inspection device.

Images