Analysis device and analysis method

Abstract

A predicted value calculation unit 131 inputs target data acquired by a device being analyzed to an index value prediction model, and acquires a plurality of predicted values obtained by predicting the values of a plurality of indices indicating the state of the device being analyzed. A relationship analysis unit 132, on the basis of the predicted values and correct answer label information indicating a correct answer label of training data used for constructing the index value prediction model, selects training data having correct answer values similar to the predicted values as comparison data. The relationship analysis unit 132 calculates the reliability of the prediction value on the basis of a first feature amount, which is an intermediate feature amount calculated when the target data is inputted to the index value prediction model, and a second feature amount, which is an intermediate feature amount calculated when the comparison data is inputted to the index value prediction model.

Description

Analytical device and analytical method

[0001] The present disclosure relates to an analytical device and an analytical method.

[0002] In semiconductor manufacturing processes, critical dimension-scanning electron microscopes (CD-SEMs), which are scanning electron microscopes for measuring the width of pattern elements, are typically used to measure the width of nanometer-order fine patterns formed on semiconductor wafers. As patterns formed on semiconductor wafers become increasingly finer, these scanning electron microscopes are required to perform dimensional measurements with higher measurement accuracy.

[0003] However, various factors can cause changes in the state of the scanning electron microscope, resulting in abnormalities such as distortion in the captured image, making it impossible to meet the required measurement accuracy. In such cases, it is necessary to analyze the captured image, identify the abnormality factor that caused the abnormality in the captured image, and adjust the parameters of the scanning electron microscope.

[0004] One method for identifying the cause of an abnormality is to use a machine learning model constructed based on training data that associates previously acquired captured images with the abnormality causes that cause an abnormality in the captured images.

[0005] However, if an image that has been disturbed by factors not included in the training data used to build the machine learning model is input to the machine learning model, there is a risk that the machine learning model will output an incorrect prediction result.In addition, because there are a wide variety of abnormality factors, it is difficult to build a machine learning model using training data that covers all abnormality factors.

[0006] In response to this, Non-Patent Document 1 discloses an image anomaly detection technology that detects abnormalities in a subject captured in an image, which is capable of detecting anomalies in a subject by using a machine learning model constructed without using abnormal images of abnormal subjects as training data. This technology detects the presence or absence of an abnormality in a subject based on the deviation of the feature values ​​of an image to be detected from the distribution of feature values ​​of normal images of normal subjects.

[0007] Rippel, Oliver, Patrick Mertens, and Dorit Merhof, “Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection” arXiv:2005.14140v2, 23 Oct 2020

[0008] The technology described in Non-Patent Document 1 is a general-purpose technology that targets changes in images that are visible to the human eye and simply detects the presence or absence of abnormalities. For this reason, it is difficult to apply the technology described in Non-Patent Document 1 to special cases such as the above-mentioned scanning electron microscope device, where minute changes in acquired data due to various factors are detected and the state of the analysis target device from which the data was acquired is adjusted.

[0009] An object of the present disclosure is to provide an analysis device and an analysis method that are capable of appropriately grasping a decrease in the accuracy of prediction results.

[0010] An analysis device according to one aspect of the present disclosure is an analysis device that analyzes the state of a device being analyzed, and includes: a memory unit that stores a machine learning model that predicts values ​​of a plurality of indicators that indicate the state of the device being analyzed using data acquired by the device being analyzed as input; correct answer label information that indicates, for each piece of training data, the values ​​of the plurality of indicators that are correct answer labels of the training data used to construct the machine learning model as correct answer values; a prediction unit that inputs target data that is the data to be analyzed into the machine learning model and obtains a plurality of predicted values ​​that predict the values ​​of the plurality of indicators; and an analysis unit that selects, as comparison data, the training data that has the correct answer value similar to the predicted value based on the predicted value and the correct answer label information, and calculates reliability of the plurality of predicted values ​​based on a first feature that is an intermediate feature calculated when the target data is input into the machine learning model and a second feature that is an intermediate feature calculated when the comparison data is input into the machine learning model.

[0011] According to the present invention, it is possible to appropriately grasp the decrease in accuracy of the prediction result.

[0012] FIG. 1 is a diagram illustrating an analysis device according to a first embodiment of the present disclosure; FIG. 2 is a diagram illustrating an example of correct answer label information; FIG. 3 is a diagram illustrating another example of training data information; FIG. 4 is a flowchart illustrating an example of a process of a predicted value calculation unit; FIG. 5 is a diagram illustrating an example of a prediction result of the predicted value calculation unit; FIG. 6 is a diagram illustrating an example of a display screen that is a flowchart illustrating an example of a process of a relationship analysis unit; FIG. 7 is a diagram illustrating an example of factor history information; and FIG. 8 is a diagram illustrating another example of a display screen.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] (First embodiment) Fig. 1 is a diagram illustrating an analysis device according to a first embodiment of the present disclosure. The analysis device 10 shown in Fig. 1 is a device for analyzing the state of an analysis target device, and includes a communication unit 11, a storage unit 12, and a calculation unit 13.

[0015] The communication unit 11 is connected to an external device so as to be able to communicate with the external device, and transmits and receives information to and from the external device. In this embodiment, the communication unit 11 is connected to an input / output device 30 via a network 20.

[0016] The input / output device 3 is a terminal device used by a user of the analysis device 10, and is implemented as an information processing device such as a PC (Personal Computer). The input / output device 3 transmits, for example, information input by the user to the analysis device 10, and also receives and displays information transmitted from the analysis device 10.

[0017] The storage unit 12 includes a RAM (Random Access Memory) and a ROM (Read Only Memory), and stores a program that defines the operation of the calculation unit 13 and various pieces of information that are used and generated by the program.

[0018] For example, the memory unit 12 stores an index value prediction model, which is a trained machine learning model for predicting the state of the device being analyzed, and training data information regarding the training data used to construct the index value prediction model.

[0019] Specifically, the index value prediction model is a model that inputs target data, which is data acquired by an apparatus under analysis, and outputs predicted values ​​that predict index values, which are the values ​​of multiple indicators that represent the state of the apparatus under analysis. In this embodiment, the apparatus under analysis is a scanning electron microscope, and the target data is image data acquired by the scanning electron microscope by capturing an image of a semiconductor wafer. However, the apparatus under analysis is not limited to a scanning electron microscope, and may be other inspection equipment or manufacturing equipment that produces a specified product. Furthermore, the target data is not limited to image data, and may be inspection data in other formats or data other than inspection data, such as internal data of a manufacturing equipment, as long as it reflects the state of the apparatus under analysis.

[0020] The output of the index value prediction model may be the predicted value itself that predicts the index value, but in this embodiment, it is the deviation of the index value from a reference value that indicates the reference state of the device being analyzed. Note that the predicted value calculation unit 131, which will be described later, may calculate the deviation of the predicted value, which is the output of the index value prediction model, from the reference value. The reference state is, for example, a state in which no disturbance occurs in the image data that is the target data. Furthermore, the index value prediction model may be based on a regression task that predicts the amount of deviation, which is the magnitude of the deviation in the index value, and the positive or negative direction of the deviation, or may be based on a classification task or multi-label task that predicts the probability that the index is deviated. In this embodiment, a model based on the regression task is assumed.

[0021] The training data for constructing an index value prediction model is data in which known data, which is data acquired with a scanning electron microscope device in a known state, is associated with a correct value, which is an index value that indicates the known state of the scanning electron microscope device, for each index as a correct label.

[0022] The training data information includes, for each training data, correct label information indicating the correct label of the training data (the correct value of each index). The training data information may also include, for each training data, feature amount information indicating intermediate feature amounts calculated by the index value prediction model when the training data is input to the index value prediction model.

[0023] 2 is a diagram showing an example of correct label information 200. The correct label information 200 shown in FIG. 2 has fields 201 to 202 for each record.

[0024] Field 201 stores path information indicating a storage location where training data is stored as information indicating training data. Note that field 201 may store identification information for identifying training data instead of or in addition to the path information, or may store the training data itself.

[0025] Field 202 stores the correct label of the training data. More specifically, a field 202 is provided for each index included in the correct label, and stores the index value (correct value) that is the value of the corresponding index. In FIG. 2, indices A to N are shown as indices. Examples of indices include AFC, which is a parameter (focal position of the primary electron irradiation beam) adjusted by the automatic focus control function of the scanning electron microscope, AAA, which is a parameter adjusted by automatic aperture alignment, and STIGMA, which is astigmatism.

[0026] 3 is a diagram showing an example of feature amount information 300. The feature amount information 300 shown in FIG. 3 has fields 301 to 302 for each record.

[0027] Field 301 stores path information indicating a storage location where training data is stored as information indicating training data. Note that field 301 may store identification information for identifying training data instead of or in addition to the path information, or may store the training data (known data) itself.

[0028] Field 302 stores intermediate features calculated by the index value prediction model when training data is input to the index value prediction model. Specifically, the intermediate features are vector quantities, and a field 302 is provided for each component of the intermediate features. In FIG. 3, features 1 to Z are shown as the components of the intermediate features. Z is an integer equal to or greater than 2.

[0029] Returning to the explanation of Fig. 1, the calculation unit 13 is a processor such as a CPU (Central Processing Unit), and realizes various functional units by reading and executing programs stored in the storage unit 12. In this embodiment, the calculation unit 13 realizes a predicted value calculation unit 131 and a relationship analysis unit 132 as functional units.

[0030] The predicted value calculation unit 131 is a prediction unit that reads out an index value prediction model stored in the memory unit 12, inputs target data into the index value prediction model, and obtains predicted values ​​that predict the index values ​​of each index.

[0031] The relationship analysis unit 132 is an analysis unit that calculates the reliability of the predicted value based on the predicted value acquired by the predicted value calculation unit 131 and the training data information stored in the storage unit 12 .

[0032] Specifically, the relationship analysis unit 132 selects, as comparison data, training data having a correct value similar to the predicted value based on the predicted value and correct label information included in the training data information. Then, the relationship analysis unit 132 calculates the reliability of the predicted value based on a first feature, which is an intermediate feature calculated by the index value prediction model when the target data is input to the index value prediction model, and a second feature, which is an intermediate feature calculated by the index value prediction model when the comparison data is input to the index value prediction model. The first feature and the second feature are intermediate features of the same type output from the same intermediate layer.

[0033] FIG. 4 is a flowchart illustrating an example of the process of the predicted value calculation unit 131.

[0034] The predicted value calculation unit 131 first acquires target data (step S101). The target data may be transmitted from the input / output device 3 or may be transmitted directly from the analysis target device. Alternatively, the predicted value calculation unit 131 may acquire the target data as multiple pieces of target data acquired by the analysis target device in the same state.

[0035] Next, the predicted value calculation unit 131 reads the index value prediction model from the storage unit 12 (step S102).

[0036] Then, the predicted value calculation unit 131 inputs the target data into the index value prediction model (step S103).

[0037] The predicted value calculation unit 131 obtains predicted values ​​for each index based on the output values ​​from the index value prediction model, outputs prediction results indicating the predicted values ​​(step S104), and ends the process. Note that if there are multiple target data, the predicted value calculation unit 131 obtains, for each index, a statistical value (e.g., an average value) of the output values ​​from the index value prediction model corresponding to each of the multiple analysis data.

[0038] Fig. 5 is a diagram showing an example of the prediction result of the prediction value calculation unit 131. The prediction result 400 shown in Fig. 5 has a field 401 for storing the index value for each index.

[0039] FIG. 6 is a diagram illustrating an example of the processing performed by the relationship analysis unit 132. As shown in FIG.

[0040] First, the relationship analysis unit 132 acquires the prediction result from the prediction value calculation unit 131 (step S201).

[0041] Next, the relationship analysis unit 132 acquires the correct value of each index corresponding to each training data from the correct label information stored in the storage unit 12 (step S202).

[0042] Then, based on the predicted values ​​of each index shown in the prediction result and the correct values ​​of each index corresponding to each training data, the relationship analysis unit 132 selects training data from the training data that has correct values ​​similar to the predicted values ​​as comparison data (step S203).

[0043] In this embodiment, the relationship analysis unit 132 selects, as comparison data, training data having a correct answer value whose difference from the predicted value falls within a predetermined range (e.g., a range of −10 to 10) for the index whose predicted value deviates most from the reference value. However, the method of selecting comparison data is not limited to this method. For example, the relationship analysis unit 132 may select, as comparison data, training data having a correct answer value whose difference from the predicted value falls within a predetermined range for all indexes, or may select, as comparison data, training data having a correct answer value whose difference from the predicted value falls within a predetermined range for any one index. Furthermore, the relationship analysis unit 132 may select comparison data based on a statistical value (e.g., a sum) of the differences between the predicted values ​​and the correct answer values ​​for all indexes or for a plurality of pre-specified indexes.

[0044] When the comparison data is selected, the relationship analysis unit 132 acquires a first feature, which is an intermediate feature calculated by the index value prediction model when the target data is input to the index value prediction model, and a second feature, which is an intermediate feature calculated by the index value prediction model when each comparison data is input to the index value prediction model (step S204). Note that, in this embodiment, the relationship analysis unit 132 acquires the second feature using the feature information 300, but it may also acquire the second feature by inputting the comparison data to the index value prediction model in step S204.

[0045] The relationship analysis unit 132 calculates the reliability of the prediction result of the prediction value calculation unit 131 based on the first feature, which is the intermediate feature of the target data, and the second feature, which is the intermediate feature of each comparison data (step S205).

[0046] Specifically, the relationship analysis unit 132 calculates the distance between the first feature and the second feature for each comparison data as a feature distance, and calculates the reliability based on the feature distance. In this embodiment, the feature distance is the Mahalanobis distance, but is not limited to this example and may be, for example, mean square error or cosine similarity. The reliability is the reciprocal of the statistical value (e.g., the mean value or the median value) of the feature distance corresponding to each comparison data.

[0047] The relationship analysis unit 132 calculates an untrained evaluation result based on the reliability, which evaluates the possibility of untrained factors, that is, data acquired by an analysis target device in which the same factors that caused the state of the analysis target device when the target data was acquired are occurring and not used as training data (step S206). In this embodiment, the untrained evaluation result indicates a high possibility of untrained factors when the reliability is equal to or greater than a threshold, and indicates a low possibility of untrained factors when the reliability is less than the threshold. The threshold may be set, for example, as the reciprocal of the feature distance that includes a predetermined percentage (e.g., 80%) of the comparison data. However, the untrained evaluation result is not limited to this two-level evaluation, and may be an evaluation of three or more levels.

[0048] Then, the relationship analysis unit 132 transmits a GUI (Graphical User Interface), which is a display screen showing the analysis results including the reliability and the untrained evaluation results, to the input / output device 3 (step S207), and ends the process. Note that the input / output device 3 receives and displays the GUI.

[0049] 7 is a diagram showing an example of a GUI 500. The GUI 500 shown in FIG. 7 includes a prediction result display area 501, a determination result display area 502, and a selection result display area 503.

[0050] The prediction result display area 501 is an area for displaying predicted values ​​of index values ​​for each index indicating the state of the analysis target device. Specifically, the prediction result display area 501 displays a device name 511 for identifying the analysis target device, a number of analysis target data 512 which is the number of target data, a prediction result 513 indicating the predicted value for each index, and a supplementary explanation 514 for the prediction result 513. In the example of FIG. 7 , the supplementary explanation 514 indicates the index whose predicted value deviates the most from the reference value.

[0051] The judgment result display area 502 is an area for displaying the untrained evaluation results. Specifically, the judgment result display area 502 displays result information 521 including the reliability and the untrained evaluation results, and a relationship area 522 showing the relationship between the target data and the training data (more specifically, the comparison data). The relationship area 522 shows the distribution of the training data and the target data in a space defined by axes of two features (FeatureX and FeatureY) obtained by compressing intermediate features into two dimensions.

[0052] In the example of FIG. 7 , the distribution of the training data and the distribution of the target data do not overlap in the relationship region 522. In other words, the feature space of the training data is sparse (an untrained space). In such a case, the accuracy of the prediction results decreases. Examples of methods for compressing intermediate features include principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), and uniform manifold approximation and projection (UMAP).

[0053] The selection result display area 503 is an area showing the training data selected as comparison data. The selection result display area 503 displays selection conditions 531, which are conditions for selecting comparison data, the number of selected comparison data 532, and distribution information 533, which shows the distribution of index values ​​(correct values) of the comparison data for a predetermined index. The predetermined index is, for example, the index with the largest deviation of the predicted value from the reference value.

[0054] As described above, according to this embodiment, the predicted value calculation unit 131 inputs target data acquired by an apparatus to be analyzed into an index value prediction model and obtains multiple predicted values ​​that predict the values ​​of multiple indicators that indicate the state of the apparatus to be analyzed. The relationship analysis unit 132 selects, as comparison data, training data having a correct value similar to the predicted value from the training data based on the predicted value and correct answer label information that indicates the correct answer label of the training data used to construct the index value prediction model. The relationship analysis unit 132 calculates the reliability of the predicted value based on a first feature that is an intermediate feature calculated when the target data is input into the index value prediction model and a second feature that is an intermediate feature calculated when the comparison data is input into the index value prediction model.

[0055] Therefore, the reliability of multiple predicted values ​​obtained by predicting the values ​​of multiple indicators indicating the state of the analysis target device is calculated based on the intermediate feature values ​​of the target data and the intermediate feature values ​​of the comparison data, which are training data having correct values ​​similar to the predicted values. This makes it possible to understand the decrease in accuracy of the prediction results due to the failure to use training data that reflects the state of the analysis target device at the time the target data was acquired, and to appropriately understand the decrease in accuracy of the prediction results.

[0056] In this embodiment, the relationship analysis unit 132 selects, as comparison data, training data having a correct value whose difference from the predicted value falls within a predetermined range for the index whose predicted value deviates the most from a predetermined reference value, thereby enabling more appropriate selection of comparison data.

[0057] In this embodiment, the relationship analysis unit 132 evaluates, based on the reliability, the possibility that data acquired from a target device where the same factors that caused the state of the target device when the target data was acquired are occurring are not used as training data. In this case, it becomes possible to more appropriately grasp the decline in accuracy of the prediction results.

[0058] In this embodiment, the predicted value calculation unit 131 inputs multiple pieces of target data into the index value prediction model and obtains, as predicted values, statistical values ​​of the values ​​predicted by the index value prediction model for each of the multiple pieces of target data. In this case, it is possible to obtain more accurate predicted values.

[0059] In this embodiment, the relationship analysis unit 132 acquires the second feature using the feature information stored in the storage unit 12. This eliminates the need to input the comparison data into the index value prediction model to calculate the second feature, thereby improving the processing speed. Furthermore, since there is no need to store the training data itself, it is also possible to reduce the storage capacity.

[0060] Second Embodiment In the first embodiment, it was possible to evaluate whether the cause of disturbance occurring in the analytical data was an untrained factor that had not been trained by the index value prediction model, but it was not possible to estimate the cause. In the second embodiment, untrained factors are estimated by preparing in advance factor history information, which is history information indicating factors determined by humans.

[0061] In this embodiment, the storage unit 12 of the analysis device 10 further stores factor history information. The factor history information indicates, for each analyzed data, which is target data that has been analyzed in the past (calculation of a predicted value and its reliability), an analyzed predicted value, which is a predicted value calculated from the analyzed data, and a disturbance factor, which is a factor that caused the state of the analysis target device when the analyzed data was acquired. The disturbance factor is identified, for example, by a user of the analysis device 10 after analyzing the target data and set in the factor history information.

[0062] 8 is a diagram showing an example of cause history information 600. The cause history information 600 shown in FIG. 8 has fields 601 to 603 for each record.

[0063] Field 601 stores path information indicating a storage location where analyzed data is stored as information indicating analyzed data. Note that field 601 may store identification information for identifying analyzed data instead of or in addition to the path information, or may store analyzed data itself.

[0064] Field 602 stores disturbance factors that cause the state of the analysis target device when the analyzed data is acquired. Field 603 stores analyzed predicted values, which are predicted values ​​acquired from the analyzed data. More specifically, field 603 is provided for each indicator, and stores predicted values ​​that predict the value of the corresponding indicator.

[0065] Furthermore, in this embodiment, the relationship analysis unit 132 of the analysis device 10 estimates the factors that cause the state of the device being analyzed when the target data is acquired, based on the target predicted value, which is a predicted value acquired by inputting the target data into the index value prediction model, and the factor history information.

[0066] Specifically, the relationship analysis unit 132 first selects, as similar data, analyzed data corresponding to analyzed predicted values ​​similar to the target predicted value. For example, similar to the selection of comparison data, the relationship analysis unit 132 selects analyzed data corresponding to analyzed predicted values ​​whose difference from the target predicted value falls within a predetermined range (e.g., 10) for the index with the largest deviation of the target predicted value from the reference value. However, the method of selecting comparison data is not limited to this method.

[0067] Next, the relationship analysis unit 132 calculates, for each disturbance factor corresponding to similar data, the reliability of the disturbance factor based on the first feature, which is an intermediate feature calculated by the index value prediction model when the target data is input to the index value prediction model, and the third feature, which is an intermediate feature calculated by the index value prediction model when the similar data is input to the index value prediction model. Specifically, the relationship analysis unit 132 calculates, for each disturbance factor, the reciprocal of the statistical value of the distance between the first feature and the third feature corresponding to each similar data, as the reliability.

[0068] Then, based on the reliability of each disturbance factor, the relationship analysis unit 132 estimates a target disturbance factor, which is a disturbance factor that causes the state of the analysis target device when the target data was acquired. For example, the relationship analysis unit 132 may estimate the disturbance factor with the highest reliability as the target disturbance factor, or may estimate a disturbance factor with a reliability equal to or greater than a threshold as the target disturbance factor. Furthermore, the relationship analysis unit 132 may determine that the target disturbance factor cannot be estimated if the reliability is less than the threshold.

[0069] According to this embodiment, it is possible to estimate the target disturbance factors that cause the state of the analysis target device when the target data is acquired, based on the predicted value corresponding to the target data and the factor history information.

[0070] The index value prediction model may be updated using analyzed data indicated in the factor history information as new training data. However, if the number of analyzed data corresponding to the same disturbance factor is small, updating the index value prediction model may result in a decrease in the accuracy of the index value prediction model. In such cases, the analysis device 10 of this embodiment is particularly useful.

[0071] Third Embodiment In this embodiment, a case will be described in which machine differences, which are differences in the states of a plurality of analysis target devices, are analyzed. The plurality of analysis target devices are, for example, devices of the same type arranged on the same production line.

[0072] In this embodiment, the predicted value calculation unit 131 obtains a predicted value of each index for each analysis target device in the same manner as in the first embodiment. The relationship analysis unit 132 generates an analysis result for each analysis target device in the same manner as in the first embodiment. At this time, the relationship analysis unit 132 calculates a comparison result by comparing the predicted values ​​for each analysis target device and adds the comparison result to the analysis result. For example, the relationship analysis unit 132 may calculate the difference in predicted values ​​for each index as the comparison result, or may calculate the difference in predicted values ​​for each index and use the index with the largest difference as the comparison result.

[0073] 9 is a diagram showing an example of a GUI displayed on the input / output device 3 of this embodiment. A GUI 700 shown in FIG. 9 includes a prediction result display area 701, a determination result display area 702, and a selection result display area 703.

[0074] The prediction result display area 701 displays device information 711 for each device under analysis, prediction results 712 showing predicted values ​​for each indicator of the device under analysis, and supplementary explanations 713 for the prediction results 712. The device information 711 includes, for example, the device name of the device under analysis and the number of target data items, which is the number of target data items acquired by the device under analysis. In the example of FIG. 9 , the supplementary explanations 714 indicate the indicator whose predicted value deviates most from the reference value.

[0075] The judgment result display area 702 is an area for displaying the untrained evaluation results. Specifically, the judgment result display area 702 displays a selection area 721 for selecting an analysis target device, result information 722 including the reliability and untrained evaluation results corresponding to the analysis target device selected in the selection area 721, and relationship information 723 indicating the relationship between the target data and the training data (more specifically, the comparison data).

[0076] The selection result display area 703 is an area showing the training data selected as comparison data. The selection result display area 703 displays selection conditions 731, which are conditions for selecting comparison data, the number of selected comparison data 732, and distribution information 733, which shows the distribution of index values ​​(correct values) of the comparison data for a predetermined index. Note that the predetermined index is, for example, the index with the largest deviation of the predicted value from the reference value.

[0077] According to this embodiment, it is possible to assist in determining whether the cause of the machine difference is due to a lack of training.

[0078] The above-described embodiments of the present disclosure are merely illustrative examples of the present disclosure, and are not intended to limit the scope of the present disclosure to these embodiments alone. Those skilled in the art may implement the present disclosure in various other forms without departing from the scope of the present disclosure.

[0079] 10: Analysis device 11: Communication unit 12: Storage unit 13: Calculation unit 131: Prediction value calculation unit 132: Relationship analysis unit

Claims

1. An analysis device that analyzes the state of a device under analysis, comprising: a memory unit that stores a machine learning model that predicts values ​​of multiple indicators that indicate the state of the device under analysis using data acquired by the device under analysis as input; and correct label information that indicates, for each piece of training data, the values ​​of the multiple indicators that are correct labels of the training data used to construct the machine learning model as correct values; a prediction unit that inputs target data, which is the data to be analyzed, into the machine learning model and obtains multiple predicted values ​​that predict the values ​​of the multiple indicators; and an analysis unit that selects, as comparison data, the training data that has the correct value similar to the predicted value based on the predicted value and the correct label information, and calculates the reliability of the multiple predicted values ​​based on a first feature that is an intermediate feature calculated when the target data is input into the machine learning model and a second feature that is an intermediate feature calculated when the comparison data is input into the machine learning model.

2. The analysis device described in claim 1, wherein the analysis unit selects as the comparison data the training data having the correct value whose difference from the predicted value falls within a predetermined range for the index in which the predicted value deviates the most from a predetermined standard value.

3. The analysis device described in claim 1, wherein the analysis unit evaluates, based on the reliability, the possibility that data acquired from the analysis target device in which the same factors that caused the state of the analysis target device when the target data was acquired are occurring is not used as the training data.

4. The analysis device described in claim 1, wherein the prediction unit inputs multiple pieces of target data into the machine learning model and obtains, as the predicted value, a statistical value of the value predicted by the machine learning model for each of the multiple pieces of target data.

5. The analysis device described in claim 1, wherein the memory unit further stores feature information indicating, for each piece of training data, intermediate features calculated when the training data is input into the machine learning model, and the analysis unit uses the feature information to obtain the second feature.

6. The analysis device of claim 1, wherein the memory unit further stores, for each analyzed data, which is the data that was input into the machine learning model to obtain the predicted value, the predicted value and historical information indicating the factors that caused the state of the analysis target device when the analyzed data was obtained, and the analysis unit estimates the factors that caused the state of the analysis target device when the target data was obtained based on the predicted value corresponding to the target data and the historical information.

7. The analysis device according to claim 1, wherein the prediction unit acquires the plurality of predicted values ​​for each of the plurality of target data acquired by each of the plurality of analysis target devices, and the analysis unit further calculates a comparison result by comparing the plurality of predicted values ​​corresponding to each of the plurality of analysis target devices with each other.

8. The analytical device according to claim 1, wherein the device to be analyzed is a scanning electron microscope device, and the target data is image data acquired by the scanning electron microscope device.

9. An analytical method using an analytical device that analyzes the state of a device under analysis, comprising: storing a machine learning model that predicts values ​​of multiple indicators that indicate the state of the device under analysis using data acquired by the device under analysis as input; and correct label information that indicates, for each piece of training data, the values ​​of the multiple indicators that are correct labels of the training data used to construct the machine learning model as correct values; inputting target data, which is the data to be analyzed, into the machine learning model to obtain multiple predicted values ​​that predict the values ​​of the multiple indicators; selecting, as comparison data, the training data that has the correct value similar to the predicted value based on the predicted value and the correct label information; and calculating the reliability of the multiple predicted values ​​based on a first feature that is an intermediate feature calculated when the target data is input into the machine learning model and a second feature that is an intermediate feature calculated when the comparison data is input into the machine learning model.

Images