Analytical apparatus and analytical methods

Abstract

The prediction calculation unit (131) inputs the object data obtained from the analysis object device into the index value prediction model and obtains a plurality of predicted values ​​for a plurality of indices representing the state of the analysis object device. The relationship analysis unit (132) selects training data with positive solution values ​​similar to the predicted values ​​as comparison data based on the predicted values ​​and positive solution label information representing the positive solution labels of the training data used to construct the index value prediction model. The relationship analysis unit (132) calculates the reliability of the predicted values ​​based on the intermediate feature quantity calculated when the object data is input into the index value prediction model, i.e., the first feature quantity, and the intermediate feature quantity calculated when the comparison data is input into the index value prediction model, i.e., the second feature quantity.

Description

[Technical Field]

[0001] This invention relates to an analytical device and an analytical method. [Previous Technology]

[0002] In semiconductor manufacturing processes, to measure the width of nanoscale micropatterns formed on semiconductor wafers, a length-measuring SEM (CD-SEM: Critical Dimension-Scanning Electron Microscope) is typically used as a scanning electron microscope device capable of magnifying and photographing the pattern for pattern width measurement. With the miniaturization of patterns formed on semiconductor wafers, higher precision dimensional measurement is required for this type of scanning electron microscope device.

[0003] However, due to various factors causing changes in the state of the scanning electron microscope (SEM), there may be abnormalities such as disturbances to the captured images, making it impossible to meet the required measurement accuracy. In such cases, it is necessary to analyze the captured images, identify the factors causing the abnormalities (i.e., the abnormal factors), and adjust the parameters of the SEM.

[0004] As a method for identifying specific anomalies, one example is the use of a machine learning model constructed based on training data, which corresponds a pre-acquired captured image to an anomaly that causes the captured image to be abnormal.

[0005] However, if images that have been disturbed due to factors not included in the training data used to construct the machine learning model are input into the machine learning model, the machine learning model may output incorrect predictions. In addition, since there are many different types of anomalous factors, it is difficult to use training data that includes all anomalous factors to construct a machine learning model.

[0006] In contrast, Non-Patent Document 1 discloses an image anomaly detection technique for detecting anomalies in a photographed subject. This technique utilizes a machine learning model constructed without using anomalous images of the photographed subject as training data to detect anomalies in the subject. In this technique, the presence or absence of anomalies in the subject is detected based on the deviation between the feature distribution of the image of the detected object and the feature distribution of a normal image of a normal subject. [Prior Art Documents] [Non-Patent Documents]

[0007] [Non-patent document 1] Rippel, Oliver, Patrick Mertens, and Dorit Merhof, “Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection” arXiv: 2005.14140v2, 23 Oct 2020 [Summary of the Invention]

[0008] [Problem to be Solved by the Invention] The technology described in Non-Patent Document 1 is a general-purpose technology that takes changes in images that are perceptible to the human eye as its object, and it merely detects the presence or absence of anomalies. Therefore, it is difficult to apply the technology described in Non-Patent Document 1 to special cases such as the scanning electron microscope device described above, where minute changes caused by various factors in the acquired data are detected and the state of the device for analyzing the acquired data is adjusted.

[0009] The object of the present invention is to provide an analysis apparatus and method capable of appropriately controlling the reduction in the accuracy of prediction results. [Means for solving the problem]

[0010] An analysis apparatus according to one aspect of the present invention analyzes the state of an analysis target device, comprising: a memory unit storing a machine learning model that uses data obtained from the analysis target device as input to predict the values ​​of a plurality of indicators representing the state of the analysis target device, and positive solution labels for training data used to construct the machine learning model, i.e., positive solution label information represented in each of the training data as positive solution values; a prediction unit that inputs the aforementioned data, i.e., object data, which is the analysis target, into the machine learning model to obtain a plurality of predicted values ​​that predict the values ​​of the plurality of indicators; and an analysis unit that, based on the predicted values ​​and the positive solution label information, selects the aforementioned training data having previously predicted positive solution values ​​similar to the predicted values ​​as comparison data, and calculates the reliability of the plurality of predicted values ​​based on an intermediate feature quantity (i.e., a first feature quantity) calculated when the aforementioned object data is input into the machine learning model, and an intermediate feature quantity (i.e., a second feature quantity) calculated when the aforementioned comparison data is input into the machine learning model. [Effects of the Invention]

[0011] According to the present invention, the reduction in the accuracy of the prediction results can be appropriately controlled.

Implementation Method

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0014] (First Embodiment) FIG1 is a diagram showing an analysis apparatus according to a first embodiment of the present invention. The analysis apparatus 10 shown in FIG1 is an apparatus for analyzing the state of an analysis target device, and includes a communication unit 11, a memory unit 12, and a calculation unit 13.

[0015] The communication unit 11 is communicatively connected to an external device and transmits and receives information with the external device. In this embodiment, the communication unit 11 is connected to the input / output device 30 via the network 20.

[0016] The input / output device 3 is a terminal device used by the user of the analysis device 10 and is implemented in an information processing device such as a PC (Personal Computer). For example, the input / output device 3 sends information input by the user to the analysis device 10, and also receives and displays information sent from the analysis device 10.

[0017] The memory unit 12 includes RAM (Random Access Memory) and ROM (Read Only Memory), and stores the program that specifies the operation of the calculation unit 13, as well as various information used and generated by the program.

[0018] For example, the memory unit 12 stores a learned machine learning model, i.e., an index value prediction model, for predicting the state of the target analysis device, and training data information related to the training data used to construct the index value prediction model.

[0019] Specifically, the index value prediction model takes the data obtained from the device being analyzed, i.e., the object data, as input and outputs a model that predicts the index values. These index values ​​display the individual values ​​of multiple indices representing the state of the device being analyzed. In this embodiment, the device being analyzed is a scanning electron microscope (SEM), and the object data is image data obtained by the SEM from photographing a semiconductor wafer. However, the device being analyzed is not limited to a SEM; it can be other inspection devices or manufacturing equipment for producing predetermined products. Furthermore, the object data is not limited to image data; it can be any data reflecting the state of the device being analyzed, including other forms of inspection data and data other than inspection data, such as internal data from a manufacturing apparatus.

[0020] The output of the index value prediction model can be the predicted value itself, which predicts the index value. However, in this embodiment, it is the deviation between the index value and the baseline value representing the baseline state of the analysis object device. Furthermore, the prediction value calculation unit 131, described later, can calculate the output of the index value prediction model, i.e., the deviation between the predicted value and the baseline value. For example, the baseline state is a state where the image data, which is the object data, is not disturbed. Also, the index value prediction model can be based on a regression task that predicts the magnitude of the index value deviation, i.e., the deviation amount and its positive or negative direction, or it can be based on a classification task or a multi-label task that predicts the probability of the index deviation. In this embodiment, it is assumed to be based on the aforementioned regression task.

[0021] The training data used to construct the index value prediction model are known data obtained from a scanning electron microscope device in a known state, which are used as positive solution labels. For each index, there is an index value representing the known state of the scanning electron microscope device, which is the positive solution value.

[0022] For each training data, the training data information includes positive solution label information representing the positive solution label (positive solution value of each index) of the training data. Furthermore, for each training data, the training data information may include feature quantity information representing the intermediate feature quantity calculated by the index value prediction model when the training data is input into the index value prediction model.

[0023] Figure 2 is an example of correct answer tag information. The correct answer tag information 200 shown in Figure 2 has fields 201 to 202 for each record.

[0024] Field 201 stores path information indicating the storage location of the training data as information representing the training data. In addition, field 201 may store identification information for identifying the training data, or the training data itself, as a substitute for path information or in addition to path information.

[0025] Field 202 stores the positive solution labels of the training data. More specifically, field 202 sets and stores the value of each index included in the positive solution labels, i.e., the index value (positive solution value). In Figure 2, indices A to N are labeled as indices. For example, indices are parameters adjusted by the automatic focus control function of the scanning electron microscope (the focal position of the primary electron beam), i.e., AFC; parameters adjusted by automatic aperture alignment, i.e., AAA; and astigmatism, i.e., STIGMA, etc.

[0026] Figure 3 is an example of feature information. The feature information 300 shown in Figure 3 has fields 301 to 302 for each record.

[0027] Field 301 stores path information indicating the storage location of the training data as information representing the training data. In addition, field 301 may store identification information for identifying the training data, or the training data (known data) itself, as a substitute for path information or in addition to path information.

[0028] Field 302 stores the intermediate feature quantities calculated by the indicator value prediction model when training data is input into the indicator value prediction model. Specifically, the intermediate feature quantities are vectors, and field 302 sets each component of the intermediate feature quantities. In Figure 3, the components of the intermediate feature quantities are represented by feature quantities 1 to Z. Furthermore, Z is an integer greater than or equal to 2.

[0029] Refer back to the explanation of Figure 1. The calculation unit 13 is a processor such as a CPU (Central Processing Unit), which reads the program stored in the memory unit 12 and executes the read program to implement various functional units. In this embodiment, the calculation unit 13 implements the prediction value calculation unit 131 and the relation analysis unit 132 as functional units.

[0030] The prediction value calculation unit 131 reads the indicator value prediction model stored in the memory unit 12 and inputs the object data into the indicator value prediction model to obtain the prediction value of each indicator.

[0031] The relationship analysis unit 132 is an analysis unit that calculates the reliability of the predicted value based on the predicted value obtained by the predicted value calculation unit 131 and the training data information stored in the memory unit 12.

[0032] Specifically, the relationship analysis unit 132 selects training data with positive solution values ​​similar to the predicted values ​​as comparison data based on the predicted values ​​and the positive solution label information contained in the training data information. Then, the relationship analysis unit 132 calculates the reliability of the predicted values ​​based on the intermediate feature quantity calculated by the index value prediction model when the object data is input into the index value prediction model, i.e., the first feature quantity, and the intermediate feature quantity calculated by the index value prediction model when the comparison data is input into the index value prediction model, i.e., the second feature quantity. Furthermore, the first feature quantity and the second feature quantity are the same type of intermediate feature quantity output from the same intermediate layer.

[0033] Figure 4 is a flowchart illustrating an example of the processing of the prediction value calculation unit 131.

[0034] First, the prediction calculation unit 131 acquires object data (step S101). The object data can be sent from the input / output device 3 or directly from the object analysis device. Furthermore, the prediction calculation unit 131 can acquire multiple object data sets acquired in the object analysis device in the same state.

[0035] Subsequently, the prediction calculation unit 131 reads the index value prediction model from the memory unit 12 (step S102).

[0036] Then, the prediction calculation unit 131 inputs the object data into the indicator value prediction model (step S103).

[0037] The prediction value calculation unit 131 obtains the predicted value of each indicator based on the output value from the indicator value prediction model, outputs the prediction result representing the predicted value (step S104), and ends the processing. In addition, when there are multiple objects of data, the prediction value calculation unit 131 obtains the statistical value (e.g., average value) of the output value from the indicator value prediction model corresponding to each of the multiple analysis data for each indicator as the predicted value.

[0038] Figure 5 is an example of the prediction result of the prediction calculation unit 131. The prediction result 400 shown in Figure 5 has a field 401 for storing the value of each indicator.

[0039] Figure 6 is a diagram showing an example of the processing of the relational analysis unit 132.

[0040] First, the relational analysis unit 132 obtains the prediction result of the prediction calculation unit 131 (step S201).

[0041] Subsequently, the relation analysis unit 132 obtains the positive value of each index corresponding to each training data from the positive label information stored in the memory unit 12 (step S202).

[0042] Then, the relation analysis unit 132 selects training data with positive values ​​similar to the predicted values ​​from the training data as comparison data based on the predicted values ​​of each indicator represented in the prediction results and the positive values ​​of each indicator corresponding to each training data (step S203).

[0043] In this embodiment, the relationship analysis unit 132 selects training data as comparison data from the indicators with the largest deviation between predicted and baseline values, specifically those with positive values ​​whose difference from the predicted value falls within a predetermined range (e.g., -10 to 10). However, the method for selecting comparison data is not limited to this method. For example, the relationship analysis unit 132 may select training data as comparison data from all indicators whose difference from the predicted value falls within a predetermined range, or it may select training data as comparison data from any single indicator whose difference from the predicted value falls within a predetermined range. Furthermore, the relationship analysis unit 132 may select comparison data based on statistical values ​​(e.g., total values) of the differences between predicted and positive values ​​from all indicators or from a pre-specified plurality of indicators.

[0044] After selecting the comparison data, the relationship analysis unit 132 obtains the intermediate feature quantity calculated by the indicator value prediction model when the object data is input into the indicator value prediction model, i.e., the first feature quantity, and the intermediate feature quantity calculated by the indicator value prediction model when each comparison data is input into the indicator value prediction model, i.e., the second feature quantity (step S204). In addition, in this embodiment, the relationship analysis unit 132 obtains the second feature quantity using the feature quantity information 300, but it can also be obtained by inputting the comparison data into the indicator value prediction model in step S204.

[0045] The relation analysis unit 132 calculates the reliability of the prediction result of the prediction calculation unit 131 based on the intermediate feature quantity of the object data, i.e., the first feature quantity, and the intermediate feature quantity of each comparison data, i.e., the second feature quantity (step S205).

[0046] Specifically, for each comparison data, the relationship analysis unit 132 calculates the distance between the first feature and the second feature as the feature distance, and calculates the reliability based on this feature distance. In this embodiment, the feature distance is the Mahalanobis distance, but it is not limited to this example and may be, for example, the mean square error or cosine similarity. Furthermore, the reliability is the reciprocal of the statistical value (e.g., the mean or median) of the feature distances of each comparison data.

[0047] The relationship analysis unit 132 calculates an untrained evaluation result based on reliability. This evaluation result assesses the probability that factors that cause the state of the analysis object device to be acquired and factors that were not used as training data by data acquired from analysis object devices that generate the same factors are not trained (step S206). In this embodiment, if the untrained evaluation result is above the reliability threshold, it indicates a high probability that the factor is not trained; if the reliability is below the threshold, the probability that the factor is not trained is low. For example, the threshold can be set to the reciprocal of the distance between the feature quantities of the comparison data that includes a predetermined ratio (e.g., 80%). However, the untrained evaluation result is not limited to these two-stage evaluations; it can also be an evaluation with three or more stages.

[0048] Then, the relational analysis unit 132 sends a GUI (Graphical User Interface) displaying the analysis results, including reliability and untrained evaluation results, to the input / output device 3 (step S207) and ends the process. Furthermore, the input / output device 3 receives and displays the GUI.

[0049] Figure 7 is a diagram illustrating an example of a GUI. The GUI 500 shown in Figure 7 includes a prediction result display area 501, a decision result display area 502, and a selection result display area 503.

[0050] The prediction result display area 501 is used to display the predicted values ​​of various indicators representing the state of the analyzed device. Specifically, the prediction result display area 501 displays the device name 511 for a specific analyzed device, the quantity of object data (i.e., the amount of analyzed object data) 512, the prediction result 513 representing the predicted value of each indicator, and supplementary explanations 514 for the prediction result 513. In the example of Figure 7, supplementary explanation 514 represents the indicator with the largest deviation between the predicted value and the benchmark value.

[0051] The judgment result display area 502 is used to display the untrained evaluation results. Specifically, the judgment result display area 502 displays result information 521 including reliability and untrained evaluation results, and a relational area 522 representing the relationship between the object data and the training data (more specifically, the comparison data). In the relational area 522, the distribution of the training data and the object data is represented on a space defined by two features (features X and Y) that compress the intermediate features into two dimensions.

[0052] Furthermore, in the example of Figure 7, the distribution of the training data and the distribution of the object data do not overlap in the relational region 522. That is, the feature space of the training data is sparse (the untrained space). In this case, the accuracy of the prediction results will decrease. Also, as a method for compressing intermediate features, examples include principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), or UMAP (Uniform Manifold Approximation and Projection).

[0053] The selection result display area 503 represents the area of ​​training data selected as comparison data. The selection result display area 503 displays the conditions used to select comparison data, namely selection condition 531, the number of comparison data selected 532, and distribution information 533 indicating the distribution of indicator values ​​(positive solution values) of the comparison data in the given indicator. In addition, the given indicator is, for example, the indicator with the largest deviation between the predicted value and the benchmark value.

[0054] As explained above, according to this embodiment, the prediction value calculation unit 131 inputs the object data obtained from the analysis object device into the index value prediction model and obtains a plurality of predicted values ​​for a plurality of indices representing the state of the analysis object device. The relationship analysis unit 132 selects training data with positive solution values ​​similar to the predicted values ​​from the training data as comparison data based on the predicted values ​​and positive solution label information representing the positive solution labels of the training data used to construct the index value prediction model. The relationship analysis unit 132 calculates the reliability of the predicted values ​​based on the intermediate feature quantity calculated when the object data is input into the index value prediction model (i.e., the first feature quantity) and the intermediate feature quantity calculated when the comparison data is input into the index value prediction model (i.e., the second feature quantity).

[0055] Therefore, the reliability of predicting a plurality of predicted values ​​for a plurality of indicators representing the state of the analyzed object device is calculated based on intermediate features of the object data and intermediate features of the training data, i.e., the comparison data, which have positive solution values ​​similar to the predicted values. Therefore, it becomes possible to grasp the reduction in the accuracy of the prediction results caused by the failure to use training data reflecting the state of the analyzed object device when the object data was acquired, and it becomes possible to appropriately grasp the reduction in the accuracy of the prediction results.

[0056] Furthermore, in this embodiment, the relationship analysis unit 132 selects training data with positive solution values ​​whose difference from the predicted value is within a predetermined range from the indicators with the largest deviation between the predicted value and the predetermined benchmark value as comparison data. In this case, the comparison data can be selected more appropriately.

[0057] Furthermore, in this embodiment, the relationship analysis unit 132 evaluates the possibility, based on reliability, that factors causing the state of the analysis target device and data obtained from analysis target devices that generate the same factors are not used as training data when acquiring target data. In this case, the reduction in the accuracy of the prediction results can be more appropriately grasped.

[0058] Furthermore, in this embodiment, the prediction value calculation unit 131 inputs a plurality of object data into the index value prediction model and obtains the statistical value of the prediction value of the index value prediction model for each of the plurality of object data as the prediction value. In this case, a more accurate prediction value can be obtained.

[0059] Furthermore, in this embodiment, the relational analysis unit 132 uses the feature information stored in the memory unit 12 to obtain the second feature. Therefore, since it is not necessary to input the comparison data into the index value prediction model and perform the calculation of the second feature, the processing speed can be improved. Also, since it is not necessary to store the training data itself, the memory capacity can be reduced.

[0060] (Second Embodiment) In the first embodiment, it is possible to evaluate whether the factors causing disturbances in the analytical data are untrained factors that have not been trained in the indicator value prediction model, but it is impossible to infer such factors. In the second embodiment, historical information representing factors of human judgment, i.e., factor history information, is prepared in advance to infer untrained factors.

[0061] In this embodiment, the memory unit 12 of the analysis device 10 further stores factor history information. For each object data that has been analyzed in the past (the calculation of the predicted value and its reliability), i.e., the analyzed data, the factor history information represents the predicted value calculated from the analyzed data, i.e., the analyzed predicted value, and the factors that caused the state of the analysis object device when the analyzed data was obtained, i.e., disturbance factors. For example, disturbance factors are specified and set in the factor history information by the user of the analysis device 10 or others after analyzing the object data.

[0062] Figure 8 is an example of factor historical information. The factor historical information 600 shown in Figure 8 has fields 601-603 for each record.

[0063] Field 601 stores path information indicating the storage location of the analyzed data as information indicating the analyzed data. In addition, field 601 may store identification information of the analyzed data, or the analyzed data itself, as a substitute for path information or in addition to path information.

[0064] Field 602 stores the disturbance factors that cause changes in the state of the analyzed device when the analyzed data is obtained. Field 603 stores the predicted values ​​obtained from the analyzed data, i.e., the predicted values ​​that have been analyzed. More specifically, field 603 is set for each indicator and stores the predicted values ​​that predict the value of the corresponding indicator.

[0065] Furthermore, in this embodiment, the relational analysis unit 132 of the analysis device 10 estimates the factors that cause the state of the analysis object device when the object data is obtained, based on the predicted value obtained by inputting the object data into the index value prediction model, namely the object prediction value and the historical information of the factors.

[0066] Specifically, the relationship analysis unit 132 first selects the analyzed data corresponding to the analyzed predicted values ​​that are similar to the object's predicted value as similar data. For example, similar to the selection of comparison data, the relationship analysis unit 132 selects the analyzed data corresponding to the analyzed predicted values ​​whose difference from the object's predicted value is within a predetermined range (e.g., 10) from the indicators with the largest deviation between the object's predicted value and the benchmark value. However, the method for selecting comparison data is not limited to this method.

[0067] Subsequently, the relationship analysis unit 132 calculates the reliability of each disturbance factor corresponding to similar data based on the intermediate feature quantity (i.e., the first feature quantity) calculated by the indicator value prediction model when the object data is input into the indicator value prediction model, and the intermediate feature quantity (i.e., the third feature quantity) calculated by the indicator value prediction model when similar data is input into the indicator value prediction model. Specifically, for each disturbance factor, the relationship analysis unit 132 calculates the reciprocal of the statistical value of the distance between the first feature quantity and the third feature quantity corresponding to each similar data as the reliability.

[0068] Then, the relationship analysis unit 132, based on the reliability of each disturbance factor, estimates the disturbance factor that causes the state of the analysis object device when acquiring object data, as the object disturbance factor. For example, the relationship analysis unit 132 can estimate the disturbance factor with the highest reliability as the object disturbance factor, or it can estimate the disturbance factor with a reliability above a threshold as the object disturbance factor. Furthermore, if the reliability is below the threshold, the relationship analysis unit 132 can determine that the object disturbance factor cannot be estimated.

[0069] According to this embodiment, based on the predicted value corresponding to the object data and the historical information of the factors, it is possible to estimate the object disturbance factors that cause the state of the analysis object device when the object data is obtained.

[0070] Furthermore, the analyzed data represented in the factor history information can be used as new training data to update the indicator value prediction model. However, when the amount of analyzed data corresponding to the same disturbance factor is small, the accuracy of the indicator value prediction model may decrease even if it is updated. In this case, the analysis device 10 of this embodiment is particularly useful.

[0071] (Embodiment 3) In this embodiment, the case of analyzing the differences in the states of a plurality of analytical objects, i.e., machine differences, will be explained. Furthermore, the plurality of analytical objects may be, for example, the same type of devices arranged on the same production line.

[0072] In this embodiment, the prediction value calculation unit 131 obtains the predicted value of each indicator for each analysis target device in the same manner as in the first embodiment. The relationship analysis unit 132 generates the analysis result for each analysis target device in the same manner as in the first embodiment. At this time, the relationship analysis unit 132 calculates a comparison result that compares the predicted values ​​of each analysis target device and adds the comparison result to the analysis result. For example, the relationship analysis unit 132 may use the difference between the predicted values ​​calculated for each indicator as the comparison result, or it may calculate the difference between the predicted values ​​for each indicator and use the indicator with the largest difference as the comparison result.

[0073] FIG9 is a diagram showing an example of the GUI of the input / output device 3 in this embodiment. The GUI 700 shown in FIG9 includes a prediction result display area 701, a decision result display area 702, and a selection result display area 703.

[0074] The prediction result display area 701 displays device information 711 for the analyzed device, prediction results 712 showing the predicted values ​​for each indicator of the analyzed device, and supplementary explanations 713 for the prediction results 712. Device information 711 includes, for example, the device name of the analyzed device and the quantity of object data obtained from the analyzed device. Supplementary explanation 714, in the example of Figure 9, shows the indicator with the largest deviation between the predicted value and the benchmark value.

[0075] The judgment result display area 702 is an area used to display the untrained evaluation results. Specifically, the judgment result display area 702 displays the selection area 721 for selecting the analysis object device, result information 722 containing the reliability and untrained evaluation results corresponding to the analysis object device selected in the selection area 721, and relational information 723 indicating the relationship between the object data and the training data (more specifically, the comparison data).

[0076] The selection result display area 703 represents the area of ​​training data selected as comparison data. The selection result display area 703 displays the conditions used to select comparison data, namely selection conditions 731, the number of comparison data selected 732, and distribution information 733 indicating the distribution of indicator values ​​(positive solution values) of the comparison data in the given indicator. In addition, the given indicator is, for example, the indicator with the largest deviation between the predicted value and the benchmark value.

[0077] According to this embodiment, it is possible to support the determination of whether the machine error is caused by untrained factors.

[0078] The embodiments of the present invention described above are merely examples for illustrating the present invention and are not intended to limit the scope of the present invention to these embodiments only. Those skilled in the art can implement the present invention in various other ways without departing from its scope. [Simplified Explanation of the Diagram]

[0012] [Figure 1] Figure 1 is a diagram showing an analysis apparatus according to a first embodiment of the present invention. [Figure 2] Figure 2 is a diagram showing an example of correct answer label information. [Figure 3] Figure 3 is a diagram showing another example of training data information. [Figure 4] Figure 4 is a flowchart illustrating an example of the processing of the prediction value calculation unit. [Figure 5] Figure 5 is a diagram showing an example of the prediction result of the prediction value calculation unit. [Figure 6] Figure 6 is a flowchart showing an example of the processing of the relationship analysis unit. [Figure 7] Figure 7 is a diagram showing an example of a display screen. [Figure 8] Figure 8 is a diagram showing an example of factor history information. [Figure 9] Figure 9 is a diagram showing another example of a display screen.

Claims

1. An analysis apparatus for analyzing the state of an analysis target device, comprising: a memory unit storing a machine learning model that uses data obtained from the analysis target device as input to predict the values ​​of a plurality of indicators representing the state of the analysis target device, and positive solution labels for training data used to construct the machine learning model, i.e., positive solution label information represented by the values ​​of the plurality of indicators as positive solution values ​​in each of the training data; a prediction unit that inputs the aforementioned data, i.e., object data, which is the analysis target, into the machine learning model to obtain a plurality of predicted values ​​that predict the values ​​of the plurality of indicators; and an analysis unit that, based on the predicted values ​​and the positive solution label information, selects the aforementioned training data having previously predicted positive solution values ​​similar to the predicted values ​​as comparison data, and calculates the reliability of the plurality of predicted values ​​based on an intermediate feature quantity, i.e., a first feature quantity, calculated when the aforementioned object data is input into the machine learning model, and an intermediate feature quantity, i.e., a second feature quantity, calculated when the aforementioned comparison data is input into the machine learning model.

2. The analytical apparatus as described in claim 1, wherein, The aforementioned analysis unit selects the aforementioned training data with the largest deviation between the aforementioned predicted value and the predetermined benchmark value from the aforementioned indicators as the aforementioned comparison data. The difference between the predicted value and the aforementioned training data is contained within the predetermined range.

3. The analytical apparatus as described in claim 1, wherein, Based on the aforementioned reliability, the aforementioned analysis unit evaluates the possibility that the factors that cause the state of the aforementioned analysis object device when acquiring the aforementioned object data, and the data acquired from the aforementioned analysis object device that has the same factors, are not used as the aforementioned training data.

4. The analytical apparatus as described in claim 1, wherein, The aforementioned prediction unit inputs a plurality of the aforementioned object data into the aforementioned machine learning model, and obtains the statistical values ​​of the values ​​predicted by the aforementioned machine learning model for the plurality of the aforementioned object data as the aforementioned prediction values.

5. The analytical apparatus as described in claim 1, wherein, The aforementioned memory unit then stores feature quantity information, which represents the intermediate feature quantity calculated when each of the aforementioned training data is input into the aforementioned machine learning model; the aforementioned analysis unit uses the aforementioned feature quantity information to obtain the aforementioned second feature quantity.

6. The analytical apparatus as described in claim 1, wherein, For each piece of data that has been input into the aforementioned machine learning model and whose aforementioned predicted value has been obtained, the aforementioned memory unit then stores the predicted value and historical information representing the factors that caused the state of the aforementioned analysis target device when the analyzed data was obtained; the aforementioned analysis unit, based on the aforementioned predicted value and the aforementioned historical information corresponding to the aforementioned target data, estimates the factors that caused the state of the aforementioned analysis target device when the aforementioned target data was obtained.

7. The analytical apparatus as described in claim 1, wherein, The aforementioned prediction unit obtains a plurality of predicted values ​​for a plurality of object data obtained from a plurality of the plurality of the aforementioned analysis object devices respectively; the aforementioned analysis unit then calculates a comparison result by comparing the plurality of predicted values ​​of the plurality of the plurality of the aforementioned analysis object devices with each of the aforementioned plurality of predicted values.

8. The analytical apparatus as described in claim 1, wherein, The aforementioned analysis device is a scanning electron microscope; the aforementioned object data is image data acquired by the aforementioned scanning electron microscope.

9. An analysis method for an analysis device that analyzes the state of the analysis object, wherein, The system stores a machine learning model that uses data obtained from the aforementioned analysis target device as input to predict the values ​​of a plurality of indicators representing the state of the aforementioned analysis target device, and positive solution labels for the training data used to construct the aforementioned machine learning model, i.e., positive solution label information for each of the aforementioned training data, where the values ​​of the aforementioned plurality of indicators are used as positive solution values; the aforementioned data that will become the analysis target, i.e., the object data, is input into the aforementioned machine learning model to obtain a plurality of predicted values ​​that predict the values ​​of the aforementioned plurality of indicators; based on the aforementioned predicted values ​​and the aforementioned positive solution label information, the aforementioned training data with previously predicted positive solution values ​​similar to the aforementioned predicted values ​​is selected as comparison data, and the reliability of the aforementioned plurality of predicted values ​​is calculated based on the intermediate feature quantity calculated when the aforementioned object data is input into the aforementioned machine learning model, i.e., the first feature quantity, and the intermediate feature quantity calculated when the aforementioned comparison data is input into the aforementioned machine learning model, i.e., the second feature quantity.

Images