Analysis apparatus for machine learning model

Abstract

To provide a technique for analysis depending on a kind of factor analysis for a machine learning model through the same apparatus and a display screen.SOLUTION: A machine learning model carries out class determination based on time-series data acquired during a manufacturing process of an object. An analysis apparatus of the machine learning model includes a quadrant classification unit for classifying, into a plurality of quadrants, a plurality of analysis data which contain objective variable corresponding to the class in the object and a prediction value as a class determination result of the machine learning model and information of the determined class, an analysis unit for receiving a kind of factor analysis of the machine learning model and analyzing the machine learning model in response to the kind of the received factor analysis and for carrying out analysis responding to the kind of the factor analysis by selecting two quadrants among the plurality of quadrants in response to the kind of the factor analysis to compare the analysis data classified into the selected quadrants with one another, and a display unit for displaying a result of analysis carried out by the analysis unit on a display apparatus.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an analysis device for a machine learning model. [Background technology]

[0002] Conventionally, machine learning models have been known that use time-series data acquired from a manufacturing process to determine whether a product is good or bad, or whether a device is normal or abnormal. To improve the determination accuracy of a machine learning model, it is common for the machine learning model to identify feature quantities used as determination factors. Non-Patent Document 1 discloses a technology that generates second time-series data from first time-series data determined to be abnormal by eliminating factors that characterize the abnormality determination, and identifies the abnormality factor by displaying the first time-series data and the second time-series data in an overlapping manner. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Fujitsu Limited and National Research Institutes, [online], FUJITSU homepage, [searched March 14, 2024], Internet<https: / / pr.fujitsu.com / jp / news / 2021 / 07 / 16.html> Summary of the Invention [Problem to be solved by the invention]

[0004] In order to improve the judgment accuracy of a machine learning model, it is necessary to consider factors that cause misjudgment in the machine learning model in addition to identifying factors that cause anomalies or defects. In Non-Patent Document 1, it is necessary to use different methods to identify factors that cause anomalies and to consider factors that cause misjudgment in the machine learning model. This can lead to the problem that analyses according to the type of factor analysis cannot be performed using the same device or display screen. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, there is provided an analysis device for a machine learning model, the machine learning model performing class determination based on time-series data acquired in a manufacturing process of an object, the analysis device including: a quadrant classification unit that classifies, into a plurality of quadrants, a plurality of analysis data sets, each of which includes a response variable corresponding to a class of the object and a predicted value that is the class determination result of the machine learning model and is information on the determined class; an analysis unit that receives a type of factor analysis of the machine learning model and performs an analysis of the machine learning model according to the received type of factor analysis, the analysis unit selecting two quadrants from the plurality of quadrants according to the type of factor analysis and comparing the analysis data sets classified into the selected quadrants; and a display unit that displays a result of the analysis performed by the analysis unit on a display device. (2) According to a second aspect of the present disclosure, there is provided an analysis device for a machine learning model, which performs class determination based on time-series data acquired in a manufacturing process of an object, and which includes: a two-dimensional plot acquisition unit that, among a plurality of analysis data including a response variable corresponding to a class of the object and a plurality of explanatory variables that are feature quantities in the time-series data, performs dimensionality reduction on the explanatory variables and acquires a two-dimensional plot by associating the explanatory variables with the response variable; an analysis unit that performs analysis of the machine learning model, wherein the analysis unit classifies and compares a first group of plots that is at least one or more selected plots from the two-dimensional plots with a second group of plots that is at least one or more selected plots other than the first group of plots; and a display unit that displays a result of the analysis performed by the analysis unit on a display device. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the configuration of an analysis device according to a first embodiment. [Figure 2]10 is a flowchart showing an analysis method using the analysis device. [Figure 3] FIG. 10 is a diagram for explaining a plurality of analysis data. [Figure 4] FIG. 10 is a diagram for explaining a setting image. [Figure 5] FIG. 4 is a diagram for explaining a quadrant classification unit. [Figure 6] 3 is a detailed flowchart of step S40 in FIG. 2. [Figure 7] FIG. [Figure 8] FIG. 10 is a diagram showing the configuration of an analysis device according to a second embodiment. [Figure 9] 10 is a flowchart showing an analysis method using the analysis device. [Figure 10] FIG. 10 is a diagram for explaining a plurality of analysis data. [Figure 11] FIG. 3 is a diagram for explaining a two-dimensional plot acquisition unit. [Figure 12] FIG. 10 is a diagram for explaining a setting image. [Figure 13] FIG. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: FIG. 1 is a diagram illustrating the configuration of an analysis device 200 according to a first embodiment. The analysis device 200 is an analysis device for a machine learning model 100. In this embodiment, the machine learning model 100 performs analysis by the analysis device 200, and performs class determination based on time-series data acquired during the manufacturing process of an object. More specifically, the machine learning model 100 performs class determination of a product as either good or defective based on time-series data acquired from a sensor or the like during the manufacturing process of an industrial product such as an ink cartridge or a circuit board. The analysis device 200 performs analysis according to the type of factor analysis based on the determination result of the machine learning model 100. The analysis device 200 may be configured as a single device or may be configured as two or more devices capable of data communication. In this embodiment, the analysis device 200 is an electronic computer such as a personal computer.

[0009] The analysis device 200 includes a processor 202 and a memory 205. The analysis device 200 also includes a display device 280 such as a monitor. The processor 202 and the memory 205 are connected via an internal bus 207 so as to be able to communicate bidirectionally. The memory 205 stores a computer program PG1.

[0010] By executing the computer program PG1, the processor 202 functions as a data acquisition unit 210, a setting image generation unit 220, a quadrant classification unit 230, an analysis unit 240, and a display unit 250. Note that some of the functions executed by the processor 202 may be realized by a hardware circuit. Here, in this disclosure, the term "processor" encompasses a CPU and a GPU.

[0011] The data acquisition unit 210 acquires multiple pieces of analysis data, including a response variable corresponding to a class of the object and a predicted value, which is a class determination result of the machine learning model 100 and is information on the determined class. The setting image generation unit 220 displays a setting image on the display device 280, which accepts input from the user for setting at least a portion of the analysis conditions. The quadrant classification unit 230 classifies the multiple pieces of analysis data acquired by the data acquisition unit 210 into multiple quadrants. The analysis unit 240 accepts the type of factor analysis of the machine learning model 100 and performs analysis of the machine learning model 100 according to the accepted type of factor analysis. The analysis unit 240 selects two quadrants from the multiple quadrants according to the type of factor analysis and compares the analysis data classified into the selected quadrants to perform an analysis according to the type of factor analysis. The display unit 250 displays the results of the analysis performed by the analysis unit on the display device 280. The data acquisition unit 210, the setting image generation unit 220, the quadrant classification unit 230, the analysis unit 240, and the display unit 250 will be described in detail later.

[0012] 2 is a flowchart showing an analysis method using analysis device 200. In step S10, data acquisition unit 210 acquires a plurality of analysis data including the determination results of machine learning model 100, and stores the data in memory 205 of analysis device 200 shown in FIG.

[0013] FIG. 3 is a diagram illustrating multiple pieces of analysis data. The multiple pieces of analysis data D1 shown in the upper part of FIG. 3 include a response variable corresponding to a class of the object and a predicted value, which is a class determination result of the machine learning model 100 and is information about the determined class. In this embodiment, the class determination result of the machine learning model 100 is either a first class or a second class. The first class is a class indicating a good product, and the second class is a class indicating a defective product. In other words, in this embodiment, the response variable and the predicted value both contain information about either a good product or a defective product. The response variable is information about a good or defective product obtained in the manufacturing process of the object. The predicted value is information as an output result obtained by the machine learning model 100 predicting whether the product is good or defective based on time-series data obtained in the manufacturing process of the object. In the multiple pieces of analysis data D1 shown in FIG. 3, a good product is denoted as "T" and a defective product is denoted as "F." In the following figures, a good product is denoted as "T" and a defective product is denoted as "F."

[0014] The plurality of analysis data D1 includes, in addition to the information on the objective variable and the predicted value, a variable that can uniquely identify the data. In this embodiment, the variable that can uniquely identify the data is a product ID.

[0015] Graph G1 shown at the bottom of FIG. 3 illustrates an example of time-series data. More specifically, graph G1 illustrates the time-series data of product ID 0001 among the multiple sets of analysis data D1. As described above, the machine learning model 100 analyzed by the analysis device 200 in this embodiment determines whether a product is good or bad based on time-series data acquired from a sensor during the manufacturing process of the target object. That is, each of product IDs 0001 to 0010 in the multiple sets of analysis data D1 has time-series data acquired from a sensor. In this disclosure, "time-series data" refers to time-series data representing a physical quantity detected by a sensor over a predetermined period. Examples of the physical quantity include pressure, temperature, current, and voltage. In this embodiment, the time-series data is, for example, data acquired by a temperature sensor during manufacturing process A when an unfinished product is heated for a time tx, as illustrated in graph G1 of FIG. 3.

[0016] A plurality of feature quantities Ft are extracted from the time-series data shown in graph G1. The extraction of the feature quantities Ft may be performed by the analysis device 200 or by another device. Each of the plurality of feature quantities Ft may include a maximum value Ft_max, a minimum value Ft_min, an integral value Ft_int, and an average value Ft_ave. Examples of each of the plurality of feature quantities Ft include the number of peaks, the amount of overshoot, the convergence time of the overshoot, and the absolute value of the area of ​​the difference from the target waveform data. In this embodiment, the plurality of feature quantities Ft correspond to explanatory variables, which will be described later.

[0017] In step S20 shown in FIG. 2, the setting image generating unit 220 causes the display device 280 to display a setting image for accepting input from the user for setting at least a part of the analysis conditions.

[0018] FIG. 4 is a diagram illustrating a setting image. The first setting image IM1 shown in FIG. 4 is a screen for the user to select analysis conditions for the machine learning model 100. The first setting image IM1 has a first condition setting image IM1a, a second condition setting image IM1b, and a third condition setting image IM1c. In this embodiment, the user selects the type of factor analysis as the analysis condition. The user can determine the type of factor analysis by selecting the third condition setting image IM1c from the first condition setting image IM1a.

[0019] In this embodiment, there are five types of factor analysis. Factor analysis 1 is an analysis of factors that cause the machine learning model 100 to erroneously determine a good product as a defective product. Factor analysis 2 is an analysis of factors that cause the machine learning model 100 to erroneously determine a defective product as a good product. Factor analysis 3 is an analysis of factors that cause the machine learning model 100 to fail to correctly determine a good product. Factor analysis 4 is an analysis of factors that cause the machine learning model 100 to fail to correctly determine a defective product. Factor analysis 5 is the identification of factors that cause defects. In the following explanation, the factors analyzed by factor analyses 1 and 2 will be collectively referred to as "factors that caused the model to make an incorrect determination," and the factors analyzed by factor analyses 3 and 4 will be collectively referred to as "factors that caused the model to fail to make a determination." Furthermore, the factors analyzed by factor analyses 1 to 4 will be collectively referred to as "factors that caused the model to make an incorrect determination."

[0020] First, the user selects which analysis they wish to perform in the first condition setting image IM1a: identifying factors causing misjudgment of the model or identifying factors causing defects. If identifying factors causing misjudgment of the model is selected, the four factor analyses 1 to 4 described above are selected. If identifying factors causing defects is selected, one factor analysis 5 described above is selected. In this embodiment, if the type of factor analysis is not determined in the first condition setting image IM1a, the user proceeds to select the second condition setting image IM1b. On the other hand, if identifying factors causing defects is selected and the type of factor analysis is determined, there is no need to select the second condition setting image IM1b or later. In FIG. 4, identifying factors causing misjudgment of the model is selected in the first condition setting image IM1a.

[0021] Next, in the second condition setting image IM1b, the user selects which of the factors causing the model to make an incorrect judgment they wish to analyze: factors causing the model to make an incorrect judgment or factors causing the model to be unable to distinguish. As described above, if a factor causing the model to make an incorrect judgment is selected, factor analysis 1 or 2 will be selected. If a factor causing the model to be unable to distinguish is selected, factor analysis 3 or 4 will be selected. In other words, regardless of which is selected, the type of factor analysis is not limited to one. Therefore, the user proceeds to set the third condition setting image IM1c. In Figure 4, the factor causing the model to make an incorrect judgment is selected in the second condition setting image IM1b.

[0022] Finally, the user selects either a good product or a defective product in the third condition setting image IM1c. For example, if a factor for which the model made an incorrect judgment is selected in the second condition setting image IM1b described above, and a good product is selected in the third condition setting image IM1c, the type of factor analysis is determined to be factor analysis 1. That is, the analysis device 200 analyzes the factors that caused the machine learning model 100 to mistakenly judge a good product as a defective product. On the other hand, if a defective product is selected in the third condition setting image IM1c, the type of factor analysis is determined to be factor analysis 2. That is, the analysis device 200 analyzes the factors that caused the machine learning model 100 to mistakenly judge a defective product as a good product. Similarly, if a factor for which the model could not be determined in the second condition setting image IM1b is selected, and a good product is selected in the third condition setting image IM1c, the type of factor analysis is determined to be factor analysis 3. On the other hand, if a defective product is selected in the third condition setting image IM1c, the type of factor analysis is determined to be factor analysis 4. In Fig. 4, a non-defective product is selected in the third condition setting image IM1c. As described above, the user can determine the type of factor analysis by selecting images from the first condition setting image IM1a to the third condition setting image IM1c in order.

[0023] 2, the quadrant classification unit 230 classifies the plurality of pieces of analysis data acquired by the data acquisition unit 210 into a plurality of quadrants. In this embodiment, the quadrant classification unit 230 classifies the plurality of pieces of analysis data into one of the following quadrants: a first quadrant in which the dependent variable and the predicted value are of the first class, a second quadrant in which the dependent variable is of the first class and the predicted value is of the second class, a third quadrant in which the dependent variable is of the second class and the predicted value is of the first class, and a fourth quadrant in which the dependent variable and the predicted value are of the second class.

[0024] FIG. 5 is a diagram for explaining the quadrant classification unit 230. Graph G2 in FIG. 5 shows a state in which the quadrant classification unit 230 classifies data into quadrants based on the plurality of pieces of analysis data D1 shown in FIG. 4. In this embodiment, the quadrant classification unit 230 classifies the plurality of pieces of analysis data D1 into any of the first to fourth quadrants. The first quadrant is a quadrant in which the objective variable and the predicted value are good products. The second quadrant is a quadrant in which the objective variable is good products and the predicted value is defective products. The third quadrant is a quadrant in which the objective variable is defective products and the predicted value is good products. The fourth quadrant is a quadrant in which the objective variable and the predicted value are defective products. As shown in graph G2, the quadrant classification unit 230 classifies the plurality of pieces of analysis data D1 into quadrants for each product ID.

[0025] 2, the analysis unit 240 analyzes the machine learning model 100 according to the type of factor analysis selected by the user. The analysis unit 240 selects any two quadrants from the first to fourth quadrants shown in FIG. 5 according to the type of factor analysis, and performs an analysis according to the type of factor analysis by comparing the analysis data classified into the selected quadrants.

[0026] 6 is a detailed flowchart of step S40 in FIG. 2. In step S410, the analysis unit 240 determines whether the type of factor analysis is a factor of erroneous determination. If the type of factor analysis is a factor of erroneous determination, the process proceeds to step S430. If the type of factor analysis is not a factor of erroneous determination, the process proceeds to step S420. Note that if the type of factor analysis is not a factor of erroneous determination, this is the case where the type of factor analysis is the identification of a defect factor, as described above. In other words, if the type of factor analysis is factor analysis 1 to 4, the process proceeds to step S430, and if the type of factor analysis is factor analysis 5, the process proceeds to step S420.

[0027] In step S420, the analysis unit 240 selects quadrant 1 and quadrant 4. That is, the analysis unit 240 selects quadrant 1 and quadrant 4 when the type of factor analysis is identification of defect factors. The time-series data of the product ID classified into quadrant 1 is correct data indicating that the actual product is a good product and that the machine learning model 100 has also determined that the product is a good product. The time-series data of the product ID classified into quadrant 4 is correct data indicating that the actual product is a defective product and that the machine learning model 100 has also determined that the product is a defective product. Therefore, by comparing the time-series data of quadrant 1 with the time-series data of quadrant 4, it is possible to identify the difference between a good product and a defective product.

[0028] In step S430, analysis unit 240 determines whether the type of factor analysis is a factor that caused the model to erroneously determine whether the product was good or bad, among the factors that caused the misjudgment. If the type of factor analysis is a factor that caused the model to erroneously determine whether the product was good or bad, the process proceeds to step S440. If the type of factor analysis is not a factor that caused the model to erroneously determine whether the product was good or bad, the process proceeds to step S470. Note that if the type of factor analysis is not a factor that caused the model to erroneously determine whether the product was good or bad, this is the case when the model was unable to distinguish between good and bad products, as described above. In other words, if the type of factor analysis is factor analysis 1 or 2, the process proceeds to step S440, and if the type of factor analysis is factor analysis 3 or 4, the process proceeds to step S470.

[0029] In step S440, analysis unit 240 determines whether the type of factor analysis is a factor that caused the model to erroneously determine a product as good, among the factors that caused the model to erroneously determine a product as good. If the type of factor analysis is a factor that caused the model to erroneously determine a product as good, the process proceeds to step S450. If the type of factor analysis is not a factor that caused the model to erroneously determine a product as good, the process proceeds to step S460. Note that if the type of factor analysis is not a factor that caused the model to erroneously determine a product as good, this is the case when the model erroneously determined a product as defective, as described above. In other words, if the type of factor analysis is factor analysis 1, the process proceeds to step S450, and if the type of factor analysis is factor analysis 2, the process proceeds to step S460.

[0030] In step S450, the analysis unit 240 selects quadrant 1 and quadrant 2. That is, when the type of factor analysis is a factor that caused the model to erroneously determine that the product was a good product, the analysis unit 240 selects quadrant 1 and quadrant 2. As described above, the time-series data of the product ID classified in quadrant 1 is correct data indicating that the actual product was a good product and that the machine learning model 100 also determined that the product was a good product. On the other hand, the time-series data of the product ID classified in quadrant 2 is incorrect data indicating that the machine learning model 100 determined that the product was a defective product, even though the actual product was a good product. Therefore, by comparing the time-series data of quadrant 1 and the time-series data of quadrant 2, it is possible to identify the factor that caused the model to erroneously determine that the product was a good product.

[0031] In step S460, the analysis unit 240 selects quadrant 3 and quadrant 4. That is, when the type of factor analysis is a factor that caused the model to erroneously determine a product as defective, the analysis unit 240 selects quadrant 3 and quadrant 4. The time-series data of the product ID classified in quadrant 3 is incorrect data that the machine learning model 100 determined to be a good product despite the fact that the product was actually defective. On the other hand, the time-series data of the product ID classified in quadrant 4 is correct data that the actual product was defective and that the machine learning model 100 also determined to be a defective product, as described above. Therefore, by comparing the time-series data of quadrant 3 and the time-series data of quadrant 4, it is possible to identify the factor that caused the model to erroneously determine a product as defective.

[0032] In step S470, the analysis unit 240 determines whether the type of factor analysis is a factor that prevented the model from discriminating a non-defective product among the factors that the model could not discriminate. If the type of factor analysis is a factor that prevented the model from discriminating a non-defective product, the process proceeds to step S480. If the type of factor analysis is not a factor that prevented the model from discriminating a non-defective product, the process proceeds to step S490. Note that if the type of factor analysis is not a factor that prevented the model from discriminating a non-defective product, this is the case when the factor analysis is a factor that prevented the model from discriminating a defective product, as described above. In other words, if the type of factor analysis is factor analysis 3, the process proceeds to step S480, and if the type of factor analysis is factor analysis 4, the process proceeds to step S490.

[0033] In step S480, the analysis unit 240 selects quadrant 1 and quadrant 3. That is, when the type of factor analysis is a factor that caused the model to be unable to distinguish between good and bad products, the analysis unit 240 selects quadrant 1 and quadrant 3. As described above, the time-series data of the product ID classified into quadrant 1 is correct data indicating that the actual product is a good product and that the machine learning model 100 has also determined that the product is a good product. On the other hand, the time-series data of the product ID classified into quadrant 3 is incorrect data indicating that the machine learning model 100 has determined that the actual product is a defective product. Therefore, by comparing the time-series data of quadrant 1 with the time-series data of quadrant 3, it is possible to identify the factor that caused the model to be unable to correctly distinguish between good and bad products.

[0034] In step S490, the analysis unit 240 selects quadrant 2 and quadrant 4. That is, when the type of factor analysis is a factor that caused the model to be unable to identify a defective product, the analysis unit 240 selects quadrant 2 and quadrant 4. As described above, the time-series data of the product ID classified in quadrant 2 is incorrect data that the machine learning model 100 has determined to be a defective product even though the actual product is a good product. On the other hand, the time-series data of the product ID classified in quadrant 4 is correct data that the actual product is a defective product and that the machine learning model 100 has also determined to be a defective product. Therefore, by comparing the time-series data of quadrant 2 with the time-series data of quadrant 4, it is possible to identify the factor that caused the model to be unable to correctly identify a defective product.

[0035] After the analysis unit 240 selects two quadrants in steps S420, 450, 460, 480, and 490, the process proceeds to step S495.

[0036] In step S495, the analysis unit 240 acquires representative data of the time-series data in each of the two selected quadrants. For example, if quadrant 1 and quadrant 2 are selected in step S450, the analysis unit 240 acquires two representative data: representative data in quadrant 1 and representative data in quadrant 2. For example, when acquiring representative data in quadrant 1 of graph G2 shown in FIG. 5, the analysis unit 240 averages the five time-series data items held by product IDs 0001, 0006, 0007, 0009, and 0010 to acquire new representative data. Similarly, when acquiring representative data in quadrant 2 of graph G2, the analysis unit 240 averages the two time-series data items held by product IDs 0003 and 0004 to acquire new representative data. When acquiring the representative data in quadrant 1, the analysis unit 240 may acquire the most average time series data from the five time series data held by each of the product IDs 0001, 0006, 0007, 0009, and 0010 as the representative data.

[0037] In step S50 shown in FIG. 2, the display unit 250 displays the results of the analysis performed by the analysis unit 240 on the display device 280.

[0038] FIG. 7 is a diagram showing an analysis result image. The analysis result image IM2 shown in FIG. 7 includes a first result image IM2a, a second result image IM2b, and a third result image IM2c. The first result image IM2a is an image showing the number of data items classified into each of the four quadrants of the graph G2 shown in FIG. 5. The second result image IM2b is an image showing the type of factor analysis used in the analysis. The second result image IM2b includes a first condition setting image IM1a, a second condition setting image IM1b, and a third condition setting image IM1c shown in FIG. 4. The third result image IM2c is an image showing the results of the analysis performed by the analysis unit 240. For example, in step S495 described above, if the analysis unit 240 acquires representative data items for quadrant 1 and quadrant 2, the display unit 250 displays the representative data items for quadrant 1 and quadrant 2 on the display device 280 in an overlapping manner. This allows the difference between the time-series data items for quadrant 1 and quadrant 2 to be visualized. In addition, the user can estimate the causes of misjudgments or defects in the machine learning model 100 from the differences in the time-series data.

[0039] In the first embodiment described above, analysis device 200 includes quadrant classification unit 230 that classifies a plurality of analysis data into quadrants, analysis unit 240 that performs analysis according to the type of factor analysis, and display unit 250 that displays the analysis results on a display screen. This allows analysis device 200 to perform analysis according to the type of factor analysis.

[0040] Furthermore, the analysis device 200 in the first embodiment automatically selects two quadrants to be used for analysis from the four quadrants described above according to the type of factor analysis selected by the user, allowing the user to easily and quickly perform analysis according to the type of factor analysis.

[0041] In the first embodiment, the types of factor analysis include identifying factors causing erroneous determination of the machine learning model 100 and identifying factors causing defects. In conventional analysis devices, identifying factors causing erroneous determination of the machine learning model and identifying factors causing defects requires different methods and cannot be performed using the same device or display screen. In contrast, in the present embodiment, identifying factors causing erroneous determination of the machine learning model 100 and identifying factors causing defects can be performed using the same device and display screen.

[0042] B. Second embodiment: FIG. 8 is a diagram showing the configuration of an analysis device 200b in the second embodiment. Similar to the first embodiment, the analysis device 200b is an analysis device for the machine learning model 100. The analysis device 200b includes a processor 202 and a memory 205. The processor 202 executes a computer program PG2 stored in the memory 205 to function as a data acquisition unit 210b, a two-dimensional plot acquisition unit 215, a setting image generation unit 220b, an analysis unit 240b, and a display unit 250b. The configuration of the analysis device 200b in the second embodiment is the same as that in the first embodiment unless otherwise specified. Furthermore, in FIG. 8 and subsequent figures, parts of the analysis device 200b corresponding to those of the analysis device 200 in the first embodiment are denoted by the same reference numerals as those of the analysis device 200.

[0043] The data acquisition unit 210b acquires multiple pieces of analysis data, including a response variable corresponding to a class of the object and multiple explanatory variables that are feature quantities in the time-series data. The two-dimensional plot acquisition unit 215 performs dimensional compression on the explanatory variables among the multiple pieces of analysis data acquired by the data acquisition unit 210b and acquires a two-dimensional plot by associating the explanatory variables with the response variable. The setting image generation unit 220b displays a setting image on the display device 280, which accepts input from the user for setting at least a portion of the analysis conditions. The analysis unit 240b classifies and compares a first group of plots, which are at least one or more selected plots, from the two-dimensional plots, with a second group of plots, which are at least one or more selected plots other than the first group of plots, to perform an analysis according to the type of factor analysis of the machine learning model 100. The display unit 250b displays the results of the analysis performed by the analysis unit on the display device 280. Details of the data acquisition unit 210b, the two-dimensional plot acquisition unit 215, the setting image generation unit 220b, the analysis unit 240b, and the display unit 250b will be described later.

[0044] 9 is a flowchart showing an analysis method using analysis device 200b. In step S60, data acquisition section 210b acquires a plurality of analysis data and stores them in memory 205 of analysis device 200b shown in FIG.

[0045] FIG. 10 is a diagram illustrating a plurality of analysis data. The plurality of analysis data D2 shown in FIG. 10 includes a dependent variable and a plurality of explanatory variables, which are feature quantities in time-series data. In the second embodiment, the dependent variable is, as in the first embodiment, information on whether a product is good or bad obtained in the manufacturing process of the target object. The explanatory variables are the plurality of feature quantities Ft described above. The plurality of analysis data D2 in the second embodiment has three feature quantities, namely, feature quantities Ft1 to Ft3. In the second embodiment, the feature quantity Ft1 is the maximum value Ft_max shown in FIG. 3, the feature quantity Ft2 is the minimum value Ft_min, and the feature quantity Ft3 is the number of peaks. The number of feature quantities Ft is not limited to three as described above, and may be any number.

[0046] 9, the two-dimensional plot acquisition unit 215 performs dimensional compression on the explanatory variables among the plurality of analysis data D2 acquired by the data acquisition unit 210b, and acquires a two-dimensional plot by associating the explanatory variables with the objective variables. The two-dimensional plot acquisition unit 215 associates information on the first class or the second class, which is the objective variable, with each plot in the two-dimensional plot.

[0047] FIG. 11 is a diagram illustrating the two-dimensional plot acquisition unit 215. The two-dimensional plot P1 shown in FIG. 11 is acquired by the two-dimensional plot acquisition unit 215 using the analysis data D2 shown in FIG. 10. The two-dimensional plot acquisition unit 215 compresses the feature quantities Ft1 to Ft3, which are explanatory variables, from among the multiple pieces of analysis data D2 into two dimensions. In the second embodiment, the three-dimensional data of the feature quantities Ft1 to Ft3 is compressed into two dimensions. Here, principal component analysis, for example, is used as a method for compressing the multiple feature quantities Ft into two dimensions. When the feature quantities Ft1 to Ft3 are compressed into two dimensions, a two-dimensional plot P1 is created having an X-axis that is the axis of the first principal component and a Y-axis that is the axis of the second principal component. The two-dimensional plot P1 has ten plots corresponding to the product IDs 0001 to 0010 shown in FIG. 10. For example, the plot located in the upper left of the two-dimensional plot P1 corresponds to product ID 0008. The method for compressing a plurality of feature quantities Ft into two dimensions is not limited to principal component analysis, and may be latent semantic analysis, linear discriminant analysis, independent component analysis, or the like.

[0048] The two-dimensional plot acquisition unit 215 associates information on whether a product is a good product or a defective product, which is the objective variable, with each plot in the two-dimensional plot. For example, product ID 0008 is a defective product, as shown in FIG. 10. Therefore, the two-dimensional plot acquisition unit 215 labels the plot of product ID 0008, which is located in the upper left corner of the two-dimensional plot P1 shown in FIG. 11, as a defective product. Note that in FIG. 11 and subsequent figures, plots labeled as defective products are hatched. This allows the user to visually distinguish between plots corresponding to good products and plots corresponding to defective products.

[0049] In step S80 shown in FIG. 9, the setting image generating unit 220b causes the display device 280 to display a setting image for accepting input from the user for setting at least a part of the analysis conditions.

[0050] Fig. 12 is a diagram for explaining a setting image. The setting image IM3 shown in Fig. 12 has three patterns of setting images IM3a to IM3c. The setting images IM3a to IM3c in the second embodiment show a two-dimensional plot P1 acquired by the two-dimensional plot acquisition unit 215. In the setting images IM3a to IM3c, the user can select any plot that constitutes the two-dimensional plot P1. The user selects a first plot group C1 and a second plot group C2 depending on the type of factor analysis.

[0051] In the setting image IM3a shown at the top of FIG. 12, three plots with product IDs 0001, 0007, and 0009 are selected for the first plot group C1a. Furthermore, three plots with product IDs 0002, 0005, and 0010 are selected for the second plot group C2a. All three plots in the first plot group C1a represent good products. All three plots in the second plot group C2a represent defective products. Here, all three plots in the first plot group C1a have large Y coordinate values. On the other hand, all three plots in the second plot group C2a have small Y coordinate values. In other words, the first plot group C1a, which represents good products, and the second plot group C2a, which represents defective products, are separated in terms of their positional relationship on the two-dimensional plot P1. This indicates that the machine learning model 100 can distinguish between the good products that make up the first plot group C1a and the defective products that make up the second plot group C2a by using the feature quantities Ft1 to Ft3 shown in Fig. 10. Therefore, the user can identify the cause of the defect by comparing the time series data of the product ID selected for the first plot group C1a with the time series data of the product ID selected for the second plot group C2a.

[0052] In the setting image IM3b shown in the center of FIG. 12, three plots with product IDs 0001, 0007, and 0009 are selected for the first plot group C1b. Two plots with product IDs 0002 and 0005 are selected for the second plot group C2b. All three plots in the first plot group C1b indicate good products. All two plots in the second plot group C2b indicate defective products. Unlike the setting image IM3a, the setting image IM3b selects plots in a predetermined positional relationship among the plots indicating defective products as the second plot group C2b. In the present disclosure, the term "predetermined positional relationship" refers to a distance between the plots being less than a threshold. More specifically, when there are two plots, this refers to a distance between the two plots being less than a threshold. When there are three or more plots, this refers to a maximum distance between the plots being less than a threshold. For example, in the setting image IM3b, among the three plots representing defective products with product IDs 0002, 0005, and 0010, the distance between the two plots representing product IDs 0002 and 0005 is small. In contrast, the distance between the plot representing product ID 0010 and product ID 0002 is large, and the distance between the plot representing product ID 0005 and product ID 0002 is also large. In other words, the two plots representing product IDs 0002 and 0005 are in a predetermined positional relationship. In contrast, the plot representing product ID 0010 is not in a predetermined positional relationship with either the plots representing product IDs 0002 and 0005. Therefore, the two plots representing product IDs 0002 and 0005 are selected as the second plot group C2. It can also be said that the plots representing product IDs 0002 and 0005 selected as the second plot group C2b and the plot representing product ID 0010 are separated in terms of their positional relationship on the two-dimensional plot P1. This indicates that the defective products with product IDs 0002 and 0005 may have different feature values ​​Ft that cause defects than the defective product with product ID 0010. In this disclosure, defective products caused by different feature values ​​may be referred to as "failure modes."From the above, the user can identify the cause of failure for each failure mode by comparing the time series data of the product ID selected in the first plot group C1b with the time series data of the product ID selected in the second plot group C2b.

[0053] In the setting image IM3c shown at the bottom of FIG. 12, two plots with product IDs 0003 and 0004 are selected for the first plot group C1c. Also, two plots with product IDs 0004 and 0008 are selected for the second plot group C2c. The two plots that make up the first plot group C1c both represent good products. The two plots that make up the second plot group C2c both represent defective products. Here, the two plots that make up the first plot group C1c both have small X-coordinate values ​​and large Y-coordinate values. Similarly, the two plots that make up the second plot group C2c both have small X-coordinate values ​​and large Y-coordinate values. In other words, the first plot group C1c, which represents good products, and the second plot group C2c, which represents defective products, are not separated in terms of their positional relationship on the two-dimensional plot P1. This suggests that the machine learning model 100 may not be able to distinguish between the good products that make up the first plot group C1c and the defective products that make up the second plot group C2c, even if it uses the feature values ​​Ft1 to Ft3 shown in Figure 10. Therefore, the user can analyze the cause of the misjudgment by the machine learning model 100 by comparing the time series data of the product IDs selected for the first plot group C1c with the time series data of the product IDs selected for the second plot group C2c.

[0054] 9, the analysis unit 240b compares the first plot group C1 and the second plot group C2 selected by the user to perform an analysis according to the type of factor analysis of the machine learning model 100. The analysis unit 240b acquires representative data of the time-series data for each of the first plot group C1 and the second plot group C2. Note that the representative data is acquired by averaging the time-series data of the selected IDs, as in the first embodiment.

[0055] In step S95 shown in FIG. 9, the display unit 250b displays on the display device 280 the time-series data for each of the first plot group C1 and the second plot group C2 classified by the analysis unit 240b.

[0056] FIG. 13 is a diagram showing an analysis result image. The analysis result image IM4 shown in FIG. 13 includes a first result image IM4a and a second result image IM4b. The first result image IM4a is an image showing the first plot group C1 and the second plot group C2 selected by the user in the setting image IM3 shown in FIG. 12. The second result image IM4b is an image showing the results of the analysis performed by the analysis unit 240b. The display unit 250b displays representative data for the first plot group C1 and the second plot group C2 selected by the user in an overlapping manner on the display device 280. This allows the differences in the time-series data between the first plot group C1 and the second plot group C2 to be visualized. Furthermore, the user can estimate the causes of misjudgment or defects in the machine learning model 100 from the differences in the time-series data.

[0057] In the second embodiment described above, analysis device 200b includes two-dimensional plot acquisition unit 215 that acquires two-dimensional plots by dimensionally compressing a plurality of explanatory variables contained in a plurality of analysis data, analysis unit 240b that performs an analysis according to the type of factor analysis using the two-dimensional plot, and display unit 250b that displays the analysis results on a display screen. This allows analysis device 200b to perform an analysis according to the type of factor analysis.

[0058] In the second embodiment, the analysis unit 240b performs an analysis to identify the defect factors caused by different feature values ​​Ft when plots for which the objective variable is non-defective products are selected as the first plot group C1 and when a plurality of adjacent plots for which the objective variable is defective products and which are separated from the first plot group C1 are selected as the second plot group C2. This enables the analysis device 200b to identify the defect factors for each defective product mode.

[0059] C. Other Embodiments: (C1) In the first embodiment described above, the analysis devices 200 and 200b include the data acquisition units 210 and 210b and the setting image generation units 220 and 220b. In contrast, the analysis devices 200 and 200b do not necessarily have to include the data acquisition units 210 and 210b and the setting image generation units 220 and 220b.

[0060] D. Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0061] (1) According to a first aspect of the present disclosure, there is provided an analysis device for a machine learning model, the machine learning model performing class determination based on time-series data acquired in a manufacturing process of an object, the analysis device including: a quadrant classification unit that classifies, into a plurality of quadrants, a plurality of analysis data sets, each of which includes a response variable corresponding to a class of the object and a predicted value that is the class determination result of the machine learning model and is information on the determined class; an analysis unit that receives a type of factor analysis of the machine learning model and performs an analysis of the machine learning model according to the received type of factor analysis, the analysis unit selecting two quadrants from the plurality of quadrants according to the type of factor analysis and comparing the analysis data sets classified into the selected quadrants; and a display unit that displays the results of the analysis performed by the analysis unit on a display device. According to this aspect, the analysis device can perform analysis according to the type of factor analysis.

[0062] (2) In the above embodiment, the quadrant classification unit may classify the plurality of analysis data into any of a first quadrant in which the objective variable and the predicted value are a first class, a second quadrant in which the objective variable is the first class and the predicted value is a second class, a third quadrant in which the objective variable is the second class and the predicted value is the first class, and a fourth quadrant in which the objective variable and the predicted value are the second class. According to this aspect, the quadrant classification unit can classify the analysis data into any of the first to fourth quadrants.

[0063] (3) In the above aspect, the analysis unit may select any two quadrants from the four quadrants, ie, the first quadrant to the fourth quadrant, depending on the type of the received factor analysis. According to this aspect, the analysis unit can select any two of the first to fourth quadrants.

[0064] (4) In the above embodiment, the first class is a class indicating a non-defective product, and the second class is a class indicating a defective product, When the type of the factor analysis is identification of a defect factor, the analysis section may select the first quadrant and the fourth quadrant to perform the analysis. According to this aspect, the analysis section can identify the cause of the defect by selecting the first quadrant and the fourth quadrant.

[0065] (5) In the above embodiment, the first class is a class indicating a good product, and the second class is a class indicating a defective product, and when the type of factor analysis is to analyze factors that caused the machine learning model to erroneously determine whether the product was good or defective, among factors that caused the machine learning model to erroneously determine whether the product was good or defective, the analysis unit may select two quadrants in which the objective variable is the same and the predicted value is different and perform the analysis. According to this aspect, the analysis unit can identify the factors that caused the machine learning model to erroneously determine whether a product is good or bad by selecting two quadrants that have the same objective variable but different predicted values.

[0066] (6) In the above embodiment, when the type of factor analysis is to analyze factors that caused the machine learning model to erroneously determine that the product was good, among factors that caused the machine learning model to erroneously determine that the product was good, the analysis unit may select the first quadrant and the second quadrant to perform the analysis. According to this aspect, the analysis unit can identify the cause of the machine learning model's incorrect determination of a non-defective product by selecting the first quadrant and the second quadrant.

[0067] (7) In the above embodiment, when the type of factor analysis is to analyze factors that caused the machine learning model to incorrectly determine that the product was defective, among factors that caused the machine learning model to incorrectly determine that the product was defective, the analysis unit may select the third quadrant and the fourth quadrant to perform the analysis. According to this aspect, the analysis unit can identify the cause of the machine learning model's incorrect determination of a defective product by selecting the third and fourth quadrants.

[0068] (8) In the above embodiment, the first class is a class indicating a good product, and the second class is a class indicating a defective product, and when the type of factor analysis is to analyze factors that caused the machine learning model to be unable to distinguish between the good product and the defective product, among factors that caused the machine learning model to make a misjudgment, the analysis unit may select two quadrants in which the predicted values ​​are the same and the objective variables are different and perform the analysis. According to this embodiment, the analysis unit can identify the factors that caused the machine learning model to be unable to distinguish between good and defective products by selecting two quadrants with the same predicted value but different target variables.

[0069] (9) In the above embodiment, when the type of factor analysis is to analyze factors that caused the machine learning model to misjudge the product and therefore could not determine whether the product was good or bad, the analysis unit may select the first quadrant and the third quadrant to perform the analysis. According to this aspect, the analysis unit can identify the cause of the machine learning model being unable to distinguish between non-defective products by selecting the first quadrant and the third quadrant.

[0070] (10) In the above embodiment, when the type of factor analysis is to analyze factors that caused the machine learning model to fail to identify the defective product among the factors that caused the machine learning model to misjudge the defective product, the analysis unit may select the second quadrant and the fourth quadrant to perform the analysis. According to this aspect, the analysis unit can identify the factors that caused the machine learning model to be unable to identify defective products by selecting the second and fourth quadrants.

[0071] (11) In the above aspect, the display unit may display the time-series data in each of the two quadrants selected by the analysis unit on the display device. According to this aspect, the display unit can display the time-series data in the selected two quadrants on the display device.

[0072] (12) In the above aspect, the display unit may display the time-series data in each of the two quadrants selected by the analysis unit in an overlapping manner on the display device. According to this aspect, the display unit can display the time-series data in the two selected quadrants in an overlapping manner on the display device.

[0073] (13) A second aspect of the present disclosure provides an analysis device for a machine learning model, which performs class determination based on time-series data acquired in a manufacturing process of an object, and includes: a two-dimensional plot acquisition unit that performs dimensionality reduction on explanatory variables among a plurality of analysis data sets, the explanatory variables including a response variable corresponding to a class of the object and a plurality of explanatory variables that are feature quantities in the time-series data, to acquire a two-dimensional plot associated with the response variable; an analysis unit that performs analysis of the machine learning model, the analysis unit classifying and comparing a first group of plots, which is at least one or more selected plots from the two-dimensional plots, with a second group of plots, which is at least one or more selected plots other than the first group of plots; and a display unit that displays a result of the analysis performed by the analysis unit on a display device. According to this aspect, the analysis device can perform analysis according to the type of factor analysis.

[0074] (14) In the above aspect, the two-dimensional plot acquisition unit may associate information on the first class or the second class, which is the response variable, with each plot in the two-dimensional plot. According to this aspect, the two-dimensional plot acquisition unit can assign a label of the response variable to each of the plots.

[0075] (15) In the above embodiment, the first class may be a class indicating good products and the second class may be a class indicating defective products, and the analysis unit may perform an analysis in which the type of factor analysis is identification of defective factors when plots for which the objective variable is the good products are selected as the first group of plots and plots for which the objective variable is the defective products and which are separated from the first group of plots are selected as the second group of plots. According to this embodiment, the analysis unit can identify the cause of defects by classifying plots for which the objective variable is a good product into a first plot group and plots for which the objective variable is a defective product and which are separated from the first plot group into a second plot group.

[0076] (16) In the above embodiment, when plots for which the objective variable is the non-defective product are selected as the first group of plots, and a plurality of plots for which the objective variable is the defective product and which are separated from the first group of plots and which are in a predetermined positional relationship are selected as the second group of plots, the analysis unit may perform an analysis in which the type of the factor analysis is identification of the defective factors that are caused by different feature quantities, among the identification of the defective factors. According to this aspect, the analysis unit can visualize the determination factors for each failure mode.

[0077] (17) In the above embodiment, the first class may be a class indicating good products and the second class may be a class indicating defective products, and the analysis unit may perform an analysis in which the type of factor analysis is a misjudgment factor of the machine learning model when plots in which the objective variable is the good products are selected as the first group of plots and plots in which the objective variable is the defective products and which are not separated from the first group of plots are selected as the second group of plots. According to this embodiment, the analysis unit can identify the cause of misjudgment in the machine learning model by classifying plots for which the objective variable is a good product into a first plot group and plots for which the objective variable is a defective product and which have not been separated from the first plot group into a second plot group.

[0078] (18) In the above aspect, the display unit may display the time series data for each of the first plot group and the second plot group classified by the analysis unit on the display device. According to this aspect, the display unit can display the time series data in the first plot group and the second plot group on the display device.

[0079] (19) In the above aspect, the display unit may display the time series data for the first plot group and the second plot group classified by the analysis unit in an overlapping manner on the display device. According to this aspect, the display unit can display the time series data in the first plot group and the second plot group in an overlapping manner on the display device.

[0080] The present disclosure can be realized in various forms, and in addition to the forms described above, it can also be realized in the form of a non-transitory recording medium on which a computer program is recorded. [Explanation of symbols]

[0081] 100...machine learning model, 200, 200b...analysis device, 202...processor, 205...memory, 207...internal bus, 210, 210b...data acquisition unit, 215...2D plot acquisition unit, 220, 220b...setting image generation unit, 230...quadrant classification unit, 240, 240b...analysis unit, 250, 250b...display unit, 280...display device, C1, C1a, C1b, C1c...first plot group, C2, C2a, C2b, C2c...second plot group, Ft...feature amount, IM1...first setting image, IM2...analysis result image, IM3...setting image, IM4...analysis result image

Claims

1. An analysis device for a machine learning model, The machine learning model performs class determination based on time-series data acquired in the manufacturing process of the object, The analysis device a quadrant classification unit that classifies a plurality of pieces of analysis data, each of which includes a response variable corresponding to a class of the object and a predicted value that is the class determination result of the machine learning model and is information on the determined class, into a plurality of quadrants; an analysis unit that receives a type of factor analysis of the machine learning model and analyzes the machine learning model according to the received type of factor analysis, selecting two quadrants from the plurality of quadrants according to the type of factor analysis, and comparing the analysis data classified into the selected quadrants to perform the analysis according to the type of factor analysis; and a display unit that displays a result of the analysis performed by the analysis unit on a display device.

2. The analysis device according to claim 1 , the quadrant classification unit classifies the plurality of analysis data into one of a first quadrant in which the dependent variable and the predicted value are of a first class, a second quadrant in which the dependent variable is of the first class and the predicted value is of a second class, a third quadrant in which the dependent variable is of the second class and the predicted value is of the first class, and a fourth quadrant in which the dependent variable and the predicted value are of the second class.

3. The machine learning model analysis device according to claim 2, The analysis unit selects any two quadrants from the four quadrants, i.e., the first quadrant to the fourth quadrant, depending on the type of the received factor analysis.

4. The analysis device according to claim 3, the first class is a class indicating a non-defective product, and the second class is a class indicating a defective product; The analysis device is configured such that, when the type of the factor analysis is identification of a defect factor, the analysis unit selects the first quadrant and the fourth quadrant and performs the analysis.

5. The analysis device according to claim 3, the first class is a class indicating a non-defective product, and the second class is a class indicating a defective product; When the type of factor analysis is a factor of a machine learning model's incorrect judgment, the analysis unit selects two quadrants in which the objective variable is the same and the predicted value is different and performs the analysis.

6. The analysis device according to claim 5, The analysis unit selects the first quadrant and the second quadrant and performs the analysis when the type of factor analysis is to analyze factors that caused the machine learning model to erroneously determine that the product was a good product, among factors that caused the machine learning model to erroneously determine that the product was a good product.

7. The analysis device according to claim 5, The analysis unit selects the third quadrant and the fourth quadrant and performs the analysis when the type of factor analysis is to analyze factors that caused the machine learning model to erroneously determine that the product was defective, among factors that caused the machine learning model to erroneously determine that the product was defective.

8. The analysis device according to claim 3, the first class is a class indicating a non-defective product, and the second class is a class indicating a defective product; When the type of factor analysis is to analyze factors that caused the machine learning model to be unable to distinguish between the good product and the defective product, among factors that caused the machine learning model to make a misjudgment, the analysis unit selects two quadrants in which the predicted values ​​are the same and the objective variables are different and performs the analysis.

9. The analysis device according to claim 8, The analysis unit selects the first quadrant and the third quadrant and performs the analysis when the type of factor analysis is to analyze factors that caused the machine learning model to fail to determine whether the product was a good product, among factors that caused the machine learning model to misjudge the product.

10. The analysis device according to claim 8, The analysis unit selects the second quadrant and the fourth quadrant and performs the analysis when the type of factor analysis is to analyze factors that caused the machine learning model to fail to identify the defective product among the factors that caused the machine learning model to misjudge the product.

11. The analysis device according to claim 3, The display unit displays the time series data in each of the two quadrants selected by the analysis unit on the display device.

12. The analysis device according to claim 11, The display unit displays the time-series data in each of the two quadrants selected by the analysis unit in an overlapping manner on the display device.

13. An analysis device for a machine learning model, The machine learning model performs class determination based on time-series data acquired in the manufacturing process of the object, The analysis device a two-dimensional plot acquisition unit that performs dimensional compression on explanatory variables among a plurality of analysis data including an objective variable corresponding to a class of the object and a plurality of explanatory variables that are feature quantities in the time-series data, and acquires a two-dimensional plot in association with the objective variable; an analysis unit that analyzes the machine learning model, and that classifies and compares a first group of plots, which is at least one or more selected plots from the two-dimensional plots, and a second group of plots, which is at least one or more selected plots other than the first group of plots, to perform an analysis according to a type of factor analysis of the machine learning model; and a display unit that displays a result of the analysis performed by the analysis unit on a display device.

14. The analysis device according to claim 13, The two-dimensional plot acquisition unit associates information on the first class or the second class, which is the objective variable, with each plot in the two-dimensional plot.

15. The analysis device according to claim 14, the first class is a class indicating a non-defective product, and the second class is a class indicating a defective product; The analysis unit performs an analysis in which the type of factor analysis is identification of defective factors when plots in which the objective variable is the non-defective product are selected as the first group of plots and plots in which the objective variable is the defective product and which are separated from the first group of plots are selected as the second group of plots.

16. The analysis device according to claim 15, the analysis unit performs an analysis in which, when plots for which the objective variable is the non-defective products are selected as the first group of plots, and a plurality of plots for which the objective variable is the defective products and which are separated from the first group of plots and which are in a predetermined positional relationship are selected as the second group of plots, the type of the factor analysis is identification of the defective factors that are caused by different feature quantities, among the identification of the defective factors.

17. The analysis device according to claim 14, the first class is a class indicating a non-defective product, and the second class is a class indicating a defective product; The analysis unit performs an analysis in which the type of factor analysis is a misjudgment factor of the machine learning model when plots for which the objective variable is the good product are selected as the first group of plots and plots for which the objective variable is the defective product and which are not separated from the first group of plots are selected as the second group of plots.

18. The analysis device according to claim 13, The display unit displays the time series data for each of the first plot group and the second plot group classified by the analysis unit on the display device.

19. 19. The analysis device according to claim 18, The display unit displays the time series data for the first plot group and the second plot group classified by the analysis unit in an overlapping manner on the display device.

Images