Machine learning device, electronic device, machine learning program, and simulation device

Abstract

The machine learning device has room for further consideration regarding detection of a state abnormality of the mechanical system.SOLUTION: The machine learning device includes a model holding unit and a calculation unit. The model storing unit is configured to store a machine learning model. Calculating a first calculation result by inputting input data to a machine learning model and performing inference, calculating a second calculation result by inputting output data of the first calculation result to the machine learning model and performing inference, and calculating an intermediate layer error based on first intermediate data included in an intermediate layer of the first calculation result and second intermediate data of the second calculation result; SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The invention disclosed in this specification relates to a machine learning device, an electronic device, a machine learning program, and a simulation device. [Background technology]

[0002] In the past, in the field of industrial factory equipment maintenance, AI (artificial intelligence) has been increasingly applied to condition-based maintenance of mechanical systems.

[0003] As an example of the related prior art, Patent Document 1 can be mentioned. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 035279

[0005] [overview] The machine learning device disclosed in Patent Document 1 leaves room for further consideration in terms of detecting abnormal conditions in mechanical systems.

[0006] The machine learning device disclosed in this specification includes a model holding unit and a calculation unit. The model holding unit is configured to hold a machine learning model including an input layer, an output layer, and at least one intermediate layer disposed between the input layer and the output layer. The calculation unit is configured to input input data to the machine learning model and perform inference to calculate a first calculation result, input output data included in the output layer of the first calculation result to the machine learning model and perform inference to calculate a second calculation result, and calculate an intermediate layer error based on first intermediate data included in the intermediate layer of the first calculation result and second intermediate data included in the intermediate layer of the second calculation result.

[0007] The electronic device disclosed in this specification includes the machine learning device having the above-described configuration.

[0008] The machine learning program disclosed in this specification is intended to function as the machine learning device having the above configuration.

[0009] The simulation device disclosed in this specification calculates output data and intermediate layer errors using the machine learning device configured as described above.

[0010] The present specification discloses an anomaly degree calculation method using a machine learning device, the machine learning device comprising: a model holding unit configured to hold a machine learning model including an input layer, an output layer, and at least one hidden layer disposed between the input layer and the output layer; and a calculation unit configured to input predetermined input data to the machine learning model, perform inference to calculate a calculation result, and calculate a hidden layer error based on a plurality of calculation results. The anomaly degree calculation method comprises the steps of: inputting first input data as input data to the machine learning model, performing inference, and calculating a first calculation result as a calculation result; inputting output data included in the output layer of the first calculation result to the machine learning model, performing inference, and calculating a second calculation result as a calculation result; and calculating a hidden layer error based on first intermediate data included in the hidden layer of the first calculation result and second intermediate data included in the hidden layer of the second calculation result. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing the configuration of a computer 100. As shown in FIG. [Figure 2] FIG. 2 is a block diagram showing the configuration of the machine learning device 6 according to the first embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram showing the configuration of the machine learning model 80. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of an aspect in which an input / output error da1 and a hidden layer error da3 are generated using input data 71 and a machine learning model 80. [Figure 5] FIG. 5 is a graph showing an example of the input data 71. [Figure 6] FIG. 6 is a graph showing an enlarged portion of FIG. 5 from time t1 to time t2. [Figure 7] FIG. 7 is a graph showing an enlarged portion of FIG. 5 from time t4 to time t5. [Figure 8] FIG. 8 is a graph showing the calculation results of the input / output error da1 and the hidden layer error da3 when the input data 71 is input to the machine learning model 80. [Figure 9] FIG. 9 is a flowchart showing a method for calculating an abnormality degree using the machine learning device 6. [Figure 10] FIG. 10 is a flowchart showing a detailed configuration of the first calculation step. [Figure 11] FIG. 11 is a flowchart showing a detailed configuration of the second calculation step.

[0012] [Detailed explanation] <About Computer 100> First, a computer 100 that functions as a machine learning device 6 according to the present disclosure will be described. Next, the machine learning device 6 according to the first embodiment of the present disclosure will be described in detail.

[0013] 1 is a diagram showing the configuration of a computer 100. The computer 100 functions as a machine learning device 6, which will be described later. The computer 100 is, for example, a PC (Personal Computer).

[0014] The computer 100 includes a CPU (Central Processing Unit) 100A, a memory 100B, an auxiliary storage device 100C, an operation input unit 100D, and a display unit 100E.

[0015] The CPU 100A has a control device and an arithmetic device (neither of which are shown). The control device interprets program instructions and controls each part of the computer 100. The arithmetic device performs arithmetic processing.

[0016] The memory 100B is a semiconductor storage device that temporarily stores programs or data. The information stored in the memory 100B is erased when the computer 100 is powered off.

[0017] The auxiliary storage device 100C is configured with a hard disk drive (HDD) or a solid state drive (SSD), and stores programs or data. The programs stored in the auxiliary storage device 100C are loaded into the memory 100B. The CPU 100A executes the programs loaded into the memory 100B.

[0018] Here, the auxiliary storage device 100C stores a simulation program P. The simulation program P is a program for causing the computer 100 to function as a machine learning device 6, which will be described later. The machine learning device 6 will be described in detail later.

[0019] The operation input unit 100D is configured with a keyboard, a mouse, or the like, and is a device that provides operation input to the computer 100. Information input from the operation input unit 100D is sent to the memory 100B.

[0020] The display unit 100E is configured by, for example, a liquid crystal display, and converts the information acquired from the memory 100B into an image and outputs it.

[0021] <Regarding the machine learning device 6 according to the embodiment> Next, a description will be given of the machine learning device 6 according to an embodiment of the present disclosure. The machine learning device 6 is configured with an MCU (Micro Controller Unit). The machine learning device 6 is incorporated into a predetermined mechanical system (such as a motor device) and controls this mechanical system. In addition to controlling the mechanical system, the machine learning device 6 can also perform machine learning using various data of this mechanical system as input data.

[0022] 2 is a block diagram showing the configuration of a machine learning device 6 according to the first embodiment of the present disclosure. As shown in FIG. 2, the machine learning device 6 includes a data storage unit 7, a model holding unit 8, a calculation unit 9, and an anomaly detection unit 10.

[0023] The data storage unit 7 stores input data 71 and initial value data 72. The input data 71 is, for example, time-series data output from a mechanical system or the like. This time-series data may be subjected to preprocessing such as normalization or FFT as necessary. The initial value data 72 is set to an initial value determined by the computer 100 as described above.

[0024] The model storage unit 8 stores a machine learning model 80. The machine learning model 80 is a machine learning model that is capable of learning and inference according to input data. Details of the machine learning model 80 will be described later.

[0025] The calculation unit 9 calculates a first calculation result 30 using input data 71 and a machine learning model 80. The first calculation result 30 includes first output data do1, an input / output error da1, and a first hidden layer vector ha, which will be described later. The calculation unit 9 also calculates a second calculation result 31 using the first output data do1 and the machine learning model 80. The calculation unit 9 will be described in detail as follows.

[0026] The calculation unit 9 includes a learning calculation unit 91, an inference calculation unit 92, and an anomaly degree calculation unit 93. The learning calculation unit 91 performs unsupervised learning using the machine learning model 80, input data 71, and initial value data 72.

[0027] The inference calculation unit 92 performs inference using the machine learning model 80, the input data 71, and the initial value data 72. The inference can be performed during the above-mentioned learning by the learning calculation unit 91 and after the learning is completed.

[0028] The anomaly degree calculation unit 93 calculates the input / output error da1 and the hidden layer error da3 using the input data 71 and the machine learning model 80. The anomaly degree calculation unit 93 transmits the calculated input / output error da1 and hidden layer error da3 to the anomaly detection unit 10 as a calculation result AS.

[0029] The input / output error da1 is the error between the value included in the input layer 50A (=input data 71) and the value included in the output layer 50C (=first output data do1) when input data 71 is input to the machine learning model 80 and inference is performed, and is calculated using a loss function described later. The hidden layer error da3 is the error between the first intermediate data 20 (see Figure 4 described later) and the second intermediate data 21 (see Figure 4 described later), and is calculated using a loss function described later.

[0030] The first intermediate data 20 is a value contained in the hidden layer 50B when the input data 71 is input to the machine learning model 80 and inference is performed. Here, the first hidden layer vector ha is treated as the first intermediate data 20. The first hidden layer vector ha is a feature vector of the hidden layer 50B when the input data 71 is input to the machine learning model 80 and inference is performed.

[0031] The second intermediate data 21 is a value contained in the hidden layer 50B when the first output data do1 is input to the machine learning model 80 and inference is performed. Here, the second hidden layer vector hb is treated as the second intermediate data 21. The second hidden layer vector hb is a feature vector of the hidden layer 50B when the first output data do1 is input to the machine learning model 80.

[0032] In the case of time-series data in which input data 71 repeats at a predetermined cycle, if the trend of the data changes during repetition, there is a possibility that the input / output error da1 and the hidden layer error da3 will also change. In this case, the input / output error da1 and the hidden layer error da3 may each have a different trend, or they may have a similar trend. Details of the method for calculating the input / output error da1 and the hidden layer error da3 will be described later.

[0033] The anomaly detection unit 10 receives the calculation result AS from the anomaly degree calculation unit 93 and detects whether an anomaly has occurred in the input data 71 from the data included in the calculation result AS. Specifically, the anomaly detection unit 10 refers to the trends of the input / output error da1 and the hidden layer error da3, determines whether a change has occurred in the data trend over time, and if a change has occurred, determines that an anomaly has occurred. The anomaly detection unit 10 outputs the detection result to the display unit 100E. When the computer 100 is made to function as the machine learning device 6 as described above, the anomaly detection unit 10 outputs the detection result to the display unit 100E.

[0034] Next, the machine learning model 80 will be described in detail. FIG. 3 is a diagram showing the configuration of the machine learning model 80. The machine learning model 80 is an inference model that can learn using predetermined training data. As shown in FIG. 3, the machine learning model 80 includes a three-layer neural network 50.

[0035] The three-layer neural network 50 is an AI model having an input layer 50A, a hidden layer 50B, and an output layer 50C. The hidden layer 50B is also called an intermediate layer. Generally, in a three-layer neural network 50, for n-dimensional input data x∈Rk×n with a batch size k, an n'-dimensional inference result y∈Rk×n' is obtained as y=G(x·α+b)β. Here, α∈Rn×m is a weight connecting the input layer 50A and the hidden layer 50B. β∈Rm×n' is a weight connecting the hidden layer 50B and the output layer 50C. Furthermore, b∈Rm is a bias for the hidden layer 50B. G is an activation function for the hidden layer 50B. For example, a Sigmoid or ReLU function can be used as the activation function.

[0036] The three-layer neural network 50 employs an algorithm that allows sequential learning with any batch size. When the i-th training data {xi∈Rki×n, ti∈Rki×n'} of batch size ki is obtained, it is necessary to find βi that minimizes the error shown in the following equation (1).

number

[0037] The optimized weight βi is calculated by the following equation (2). P i =P i-1 -P i-1 H i T (I+H i P i-1 H i T ) -1 H i P i-1 β i =β i-1 +P i H i T (t i -H i β i-1 ) (2)

[0038] Here, P0 and β0 are obtained by the following equation (3). P0=(H0 T H0) -1 β0=P0H0 T t0(3)

[0039] The learning algorithm is as follows: (1) The weight α and bias b are initialized with random numbers. (2) Calculate H0 for x0, and calculate P0 and β0. (3) Batch size k i Each time the i-th training data of P is obtained, i and β i It should be noted that instead of using the formula for calculating β0 in equation (3), a value initialized by a random number may be used as β0.

[0040] The computational bottleneck in the above equation (2) is (I+H i Pi-1 H i T ) -1 However, (I+H i P i-1 H i T ) has a matrix size of k × k, so when k = 1, the inverse matrix operation can be replaced with the reciprocal operation. Therefore, by fixing the batch size to k = 1, the operation becomes easy even for a computing device such as a microcomputer.

[0041] Furthermore, the machine learning model 80 according to this embodiment performs learning using an autoencoder. The autoencoder uses input data as training data and learns to reconstruct the input data as an inference result. In other words, in the above example, it learns with t = x. Because the autoencoder does not require separate training data creation, it is a type of unsupervised learning algorithm. Furthermore, by making the number of nodes in the hidden layer smaller than the number of nodes in the input and output layers, the hidden layer matrix can be considered a dimensionally compressed form of the input data once the error between the input data and the inference result has converged. That is, the encoding result of input data x is H = G(x·γ + b), and the decoding result of H is obtained as y = H·ι.

[0042] FIG. 4 is a diagram showing a schematic diagram of an aspect in which an input / output error da1 and a hidden layer error da3 are generated using input data 71 and a machine learning model 80.

[0043] As shown in Fig. 4, input data 71 is input to a machine learning model 80 and calculated by a calculation unit 9, thereby obtaining a first calculation result 30. The first calculation result 30 includes first output data do1, an input / output error da1, and a first hidden layer vector ha. Specifically, this is as follows.

[0044] Input data 71 is input to machine learning model 80 and inference is performed by inference calculation unit 92, thereby obtaining first output data do1 as the inference result. At this time, input / output error da1 is calculated by anomaly degree calculation unit 93. At this time, first hidden layer vector ha is obtained as the feature vector of hidden layer 50B.

[0045] Furthermore, the first output data do1 is input to the machine learning model 80 and calculated by the calculation unit 9, thereby obtaining a second calculation result 31. The second calculation result 31 includes the second output data do2 and the second hidden layer vector hb. Specifically, this is as follows.

[0046] By inputting the first output data do1 to the machine learning model 80 and performing inference using the inference calculation unit 92, the second output data do2 is obtained as the inference result. At this time, the second hidden layer vector hb is obtained as the feature vector of the hidden layer 50B.

[0047] The hidden layer error da3 is calculated based on the first hidden layer vector ha and the second hidden layer vector hb by the abnormality degree calculation unit 93. The calculation method at this time will be described in detail later.

[0048] Fig. 5 is a graph showing an example of input data 71. In Fig. 5, the input data 71 is shown as a time series graph with the horizontal axis representing time and the vertical axis representing a predetermined output value. Fig. 6 is a graph showing an enlarged view of the portion from time t1 to time t2 in Fig. 5. Fig. 7 is a graph showing an enlarged view of the portion from time t4 to time t5 in Fig. 5.

[0049] In Fig. 5, the start point of the data is time t0 and the end point of the data is time t6. Also in Fig. 5, the period from time t0 to time t3 is called a normal period T1. Also in Fig. 5, the period from time t3 to time t6 is called an abnormal period T2.

[0050] Normal period T1 is a period during which no abnormality occurs in the input data 71. In other words, normal period T1 is a period during which it is estimated that no particular abnormality occurs in the output value of the mechanical system implementing the machine learning device 6. On the other hand, abnormal period T2 is a period during which an abnormality occurs in the input data 71. In other words, abnormal period T2 is a period during which it is estimated that some kind of abnormality occurs in the output value of the mechanical system implementing the machine learning device 6.

[0051] As shown in Figures 5 and 6, the input data 71 during the normal period T1 has a waveform similar to a sine wave that repeats with a constant cycle. On the other hand, as shown in Figures 5 and 7, the input data 71 during the abnormal period T2 has a waveform that repeats with a constant cycle, but an abnormal portion (hereinafter also referred to as abnormal portion Ap) occurs in the latter half of one cycle of the waveform.

[0052] However, as shown in Fig. 5, it is difficult for a user to determine whether an abnormality has occurred in the input data 71 at a glance. Enlarging the graph of the input data 71 as shown in Fig. 7 may help in making a judgment, but it becomes cumbersome.

[0053] Fig. 8 is a graph showing the calculation results of the input / output error da1 and the hidden layer error da3 when input data 71 is input to a machine learning model 80. In Fig. 8, the input / output error da1 and the hidden layer error da3 are shown as a time series graph with the horizontal axis representing time and the vertical axis representing a predetermined value.

[0054] The period from time t0 to time ta shown in Figure 8 is referred to as learning period T3. Learning period T3 is a period during which input data 71 is input to machine learning model 80 and learning is performed. The period from time ta to time t6 is referred to as inference period T4. Inference period T4 is a period during which input data 71 is input to machine learning model 80 and inference is performed.

[0055] During the learning period T3, the features of the input data 71 are incorporated. As a result, the data values ​​(=values ​​on the vertical axis) of both the input / output error da1 and the hidden layer error da3 change significantly. During the learning period T3, the input / output error da1 and the hidden layer error da3 are unstable and react sensitively to changes in the input values, making them unsuitable for detecting abnormal values ​​in the input data 71.

[0056] As shown in FIG. 8, during the inference period T4 and the normal period T1, both the input / output error da1 and the hidden layer error da3 remain stable and do not show any significant changes.

[0057] On the other hand, during the abnormal period T2, the input / output error da1 shows a significant change in the data trend compared to the normal period T1. Specifically, compared to the normal period T1, the data values ​​rise more steeply and with higher frequency. The same is true for the hidden layer error da3. From this, it can be inferred that there is some kind of change in the trend of the input data 71 between the normal period T1 and the abnormal period T2, and therefore that an abnormality has occurred.

[0058] When the abnormality detection unit 10 detects a change in the trend of the input / output error da1 or the hidden layer error da3, it outputs the result as a detection result. The output method can be, for example, a method of displaying a warning, alarm, or the like on the display unit 100E.

[0059] By checking and analyzing the detection results output by the anomaly detection unit 10, the user can determine whether the input data 71 contains abnormal values, and ultimately whether an abnormality has occurred in the mechanical system or the like in which the machine learning device 6 is implemented.

[0060] <About input / output error da1> Next, the input / output error da1 and the hidden layer error da3 will be described in detail. The input / output error da1 represents the error between the first output data do1 and the input data 71. The first output data do1 is data of the inference result obtained as a result of inputting the input data 71 into the machine learning model 80 and performing inference. The input / output error da1 is calculated by a loss function based on the input data 71 and the first output data do1. Specifically, it is as follows.

[0061] Each input value included in the input data 71 is defined as an input value x. Each output value (each value of the inference result) included in the first output data do1 is defined as an output value y. As a loss function for calculating the input / output error da1, for example, MAE (Mean Absolute Error) or MSE (Mean Squared Error) can be used. When the loss function is MAE, the loss function L is expressed as in the following equation (4).

number

number

[0062] <About the hidden layer error da3> The hidden layer error da3 is calculated based on the first hidden layer vector ha and the second hidden layer vector hb. Specifically, it is as follows.

[0063] The first hidden layer vector ha is a feature vector of the hidden layer 50B when inference is performed by inputting the input data 71 to the machine learning model 80. The first hidden layer vector ha is expressed by the following equation (6).

number

[0064] The second hidden layer vector hb is a feature vector of the hidden layer 50B when inference is performed by inputting the first output data do1 to the machine learning model 80. The second hidden layer vector hb is expressed by the following equation (10).

[0065]

number

[0066] The hidden layer error da3 is calculated by a loss function L that represents the error between the first hidden layer vector ha and the second hidden layer vector hb. When the loss function is MAE, the loss function L is expressed by the following equation (11).

number

number

[0067] By calculating the input / output error da1 as described above, the user can determine whether the input data 71 contains an abnormal value, and ultimately whether an abnormality has occurred in the mechanical system or the like in which the machine learning device 6 is implemented.

[0068] Furthermore, the anomaly degree calculation unit 93 generates a hidden layer error da3 in addition to the input / output error da1. Depending on the input data 71, even if an abnormal value is included, there may be cases where a large change in the trend as described above cannot be confirmed in the input / output error da1. Even in such cases, by referring to the calculated hidden layer error da3, the user can easily determine whether or not the input data 71 includes an abnormal value.

[0069] As described above, the first hidden layer vector ha is a feature vector that indicates the characteristics of the hidden layer 50B when inference is performed using the input data 71. On the other hand, the second hidden layer vector hb indicates the characteristics of the hidden layer 50B when further inference is performed using the inference result (=first output data do1) based on the input data 71. Therefore, even if the input data 71 contains an abnormal value, the second hidden layer vector hb is a feature vector that indicates a feature in which the abnormal value is diluted compared to the first hidden layer vector ha.

[0070] The hidden layer error da3 is calculated based on the features of these two hidden layers (first hidden layer vector ha, second hidden layer vector hb). In other words, it is equivalent to determining an abnormal value of the input data 71 using the information of hidden layer 50B, which is a condensed version of the features of the input data 71. For this reason, even if only a slight trend change can be confirmed in the input / output error da1, a significant change may occur in the hidden layer error da3. Therefore, calculating the hidden layer error da3 in addition to the input / output error da1 makes it easier to detect an abnormal value of the input data 71.

[0071] Next, an abnormality degree calculation method using the machine learning device 6 will be described.

[0072] 9 is a flowchart of a method for calculating an anomaly degree using a machine learning device 6. As shown in FIG. 9, first, a first calculation step is executed (step St1). In the first calculation step, input data 71 is input to a machine learning model 80 to perform inference and calculate a first calculation result 30. As described above, the first calculation result 30 includes the first output data do1, the input / output error da1, and the first hidden layer vector ha.

[0073] Next, a second calculation step is executed (step St2). In the second calculation step, the first output data do1 is input to the machine learning model 80 to perform inference and calculate the second calculation result 31. As described above, the second calculation result 31 includes the second output data do2 and the second hidden layer vector hb.

[0074] Next, a third calculation step is executed (step St3). In the third calculation step, a hidden layer error da3 is calculated based on the first calculation result 30 and the second calculation result 31. The first calculation step (step St1) to the third calculation step (step St3) will be described in detail below.

[0075] Fig. 10 is a flowchart showing a detailed configuration of the first calculation step. As shown in Fig. 10, first, a user prepares input data 71 (step St11). Step St11 includes selection and extraction of input data, predetermined pre-processing (e.g., statistical processing, FFT analysis, etc.), etc.

[0076] Next, the inference calculation unit 92 executes inference based on the input data 71 and the machine learning model 80, and generates first output data do1 as the inference result (step St12). Next, the anomaly degree calculation unit 93 calculates the input / output error da1. The anomaly degree calculation unit 93 also acquires the first hidden layer vector ha based on the hidden layer 50B (step St13). Therefore, the first output data do1, the input / output error da1, and the first hidden layer vector ha are generated as the first calculation result 30 after the first calculation step (see FIG. 4). Then, the process proceeds to the second calculation step (step St2).

[0077] 11 is a flowchart showing a detailed configuration of the second calculation step. As shown in FIG. 11, in the second calculation step, first, the inference calculation unit 92 acquires first output data do1 from the first calculation result 30 (step St21). The first output data do1 is data included in the output layer 50C of the machine learning model 80 after the first calculation step. The acquisition of the first output data do1 may be performed by the inference calculation unit 92, another calculation unit, or a user may select the first output data do1 from the first calculation result 30.

[0078] Next, the inference calculation unit 92 inputs the acquired first output data do1 into the machine learning model 80 and executes inference again, generating second output data do2 as the inference result (step StSt22). Next, the anomaly degree calculation unit 93 acquires the second hidden layer vector hb based on the hidden layer 50B (step St23). Therefore, the second output data do2 and the second hidden layer vector hb are generated as the second calculation result 31 after the second calculation step (see FIG. 4). Then, the process proceeds to the third calculation step (step St3).

[0079] In the third calculation step, the abnormality degree calculation unit 93 calculates the hidden layer error da3 based on the acquired first hidden layer vector ha and second hidden layer vector hb. The method for calculating the hidden layer error da3 is as described above.

[0080] <Modification> The present disclosure is not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present disclosure. For example, in the configuration according to the above-described embodiments, the anomaly detection unit 10 detects the degree of anomaly of the input data 71 based on the input / output error da1 and the hidden layer error da3. However, the present disclosure is not limited to this. For example, the input / output error da1 and the hidden layer error da3 may be output as a graph as shown in FIG. 8, and the user may visually check the graph to confirm the changes in the trend as described above, thereby determining the degree of anomaly of the input data 71.

[0081] <Additional Notes> The machine learning device (6) includes: a model holding unit (8) configured to hold a machine learning model (80) including an input layer (50A), an output layer (50C), and at least one intermediate layer (50B) arranged between the input layer (50A) and the output layer (50C); and a calculation unit (9) configured to input input data (71) to the machine learning model (80) to perform inference to calculate a first calculation result (30), input output data (do1) included in the output layer (50C) of the first calculation result (30) to the machine learning model (80) to perform inference to calculate a second calculation result (31), and calculate an intermediate layer error (da3) based on first intermediate data (20) included in the intermediate layer (50B) of the first calculation result (30) and second intermediate data (21) included in the intermediate layer (50B) of the second calculation result (31) (first configuration).

[0082] In the machine learning device (6) according to the first configuration, the first intermediate data (20) includes a first intermediate layer vector (ha) which is a feature vector of the intermediate layer (50B) resulting from inputting input data (71) into a machine learning model (80) and executing inference, and the second intermediate data (21) includes a second intermediate layer vector (hb) which is a feature vector of the intermediate layer (50B) resulting from inputting output data (do1) into the machine learning model (80), and the calculation unit (9) calculates an intermediate layer error (da3) using a loss function (L) based on the first intermediate layer vector (ha) and the second intermediate layer vector (hb) (second configuration).

[0083] In the machine learning device according to the first or second configuration, the calculation unit (9) calculates an input-output error (input-output error da1) using a loss function (L) based on input data (71) and output data (do1) (third configuration).

[0084] The electronic device (100A) includes a machine learning device (6) according to any one of the first to third configurations (fourth configuration).

[0085] The machine learning program (P) is for causing the machine learning device (6) according to any one of the first to third configurations to function (fifth configuration).

[0086] The simulation device (100) calculates output data (do1) and an intermediate layer error (da3) using a machine learning device (6) according to any one of the first to third configurations (sixth configuration).

[0087] The anomaly degree calculation method is a method for calculating an anomaly degree using a machine learning device (6) including: a model holding unit (8) configured to hold a machine learning model (80) including an input layer (50A), an output layer (50C), and at least one intermediate layer (50B) arranged between the input layer (50A) and the output layer (50C); and a calculation unit (9) configured to input predetermined input data (71) to the machine learning model (80), execute inference, and calculate a calculation result, and to calculate an intermediate layer error (da3) based on a plurality of calculation results, wherein the machine learning device (6) uses first input data (71) as the input data (71) to perform machine learning. The seventh configuration includes a step (St1) of inputting the output data (do1) included in the output layer (50C) of the first calculation result (30) into a machine learning model (80) to execute inference and calculate a first calculation result (30) as the calculation result, a step (St2) of inputting the output data (do1) included in the output layer (50C) of the first calculation result (30) into the machine learning model (80) to execute inference and calculate a second calculation result (31) as the calculation result, and a step (St3) of calculating an intermediate layer error (da3) based on the first intermediate data (20) included in the intermediate layer (50B) of the first calculation result (30) and the second intermediate data (21) included in the intermediate layer (50B) of the second calculation result (31) (seventh configuration). [Explanation of symbols]

[0088] 6 Machine Learning Devices 7 Data storage unit 8 Model holding section 9 Arithmetic section 10. Anomaly detection unit 20 First Interim Data 21 Second Interim Data 30 First calculation result 31 Second operation result 50 Three-layer neural network 50A input layer 50B Hidden layer (intermediate layer) 50C output layer 71 Input Data 72 Initial Value Data 80 Machine Learning Models 91 Learning calculation unit 92 Inference calculation section 93 Abnormality calculation unit 100 Computer (simulation device) 100A CPU (electronic equipment) 100B memory 100C auxiliary storage 100D Operation input section 100E Display section Ap Abnormal location P Simulation Program T1 normal period T2 Abnormal period T3 Study Period T4 inference period da1 input / output error da3 Hidden layer error (intermediate layer error) do1 First output data do2 Second output data ha 1st hidden layer vector (1st intermediate layer vector) hb Second hidden layer vector (second intermediate layer vector) x input value y output value α weight β weight

Claims

1. a model storage unit configured to store a machine learning model including an input layer, an output layer, and at least one intermediate layer disposed between the input layer and the output layer; a calculation unit configured to input the input data to the machine learning model and perform inference to calculate a first calculation result, input output data included in the output layer of the first calculation result to the machine learning model and perform inference to calculate a second calculation result, and calculate an intermediate layer error based on first intermediate data included in the intermediate layer of the first calculation result and second intermediate data included in the intermediate layer of the second calculation result; A machine learning device comprising:

2. the first intermediate data includes a first intermediate layer vector, which is a feature vector of the intermediate layer, obtained by inputting the input data into the machine learning model and performing inference; the second intermediate data includes a second intermediate layer vector that is a feature vector of the intermediate layer obtained by inputting the output data into the machine learning model; The machine learning device according to claim 1 , wherein the calculation unit calculates the intermediate layer error using a loss function based on the first intermediate layer vector and the second intermediate layer vector.

3. The machine learning device according to claim 1 , wherein the calculation unit calculates an input / output error using a loss function based on the input data and the output data.

4. An electronic device comprising the machine learning device according to claim 1 or 2.

5. A machine learning program for causing the machine learning device according to claim 1 or 2 to function.

6. A simulation device that calculates the output data and the intermediate layer error using the machine learning device according to claim 1 or 2.

7. a model storage unit configured to store a machine learning model including an input layer, an output layer, and at least one intermediate layer disposed between the input layer and the output layer; a calculation unit configured to input predetermined input data into the machine learning model, execute inference, and calculate a calculation result, and to calculate an intermediate layer error based on a plurality of the calculation results; An anomaly degree calculation method using a machine learning device comprising: inputting first input data as the input data into the machine learning model to perform inference, and calculating a first calculation result as the calculation result; inputting output data included in the output layer of the first operation result into the machine learning model to perform inference, and calculating a second operation result as the operation result; calculating the intermediate layer error based on first intermediate data included in the intermediate layer of the first operation result and second intermediate data included in the intermediate layer of the second operation result; The abnormality degree calculation method includes the steps of:

Images