Method, apparatus and system for predicting anomaly of monitoring target
A dual-model system for anomaly detection enhances precision by using a first model for normal state detection and a second model for abnormal state detection, integrating intermediate processing results to improve the accuracy and speed of anomaly identification.
Patent Information
- Application Number
- JP2023079637
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-22
AI Technical Summary
Existing anomaly detection systems face a trade-off between early detection and accuracy, with early detection often leading to false positives, compromising overall precision.
A dual-model approach using a first trained model for normal state detection and a second trained model for abnormal state detection, where intermediate processing results from the second model are input into the first model to enhance accuracy, allowing for both timely and precise anomaly identification.
The system enables rapid and accurate detection of anomalies by combining the outputs of both models, improving the precision of abnormality judgments through enhanced feature extraction and analysis.
Smart Images

Figure 2024163765000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to methods, devices and systems for predicting anomalies in monitored objects. [Background technology]
[0002] In the field of IoT, systems to detect anomalies have been developed. For example, a specific target can be monitored using a sensor, and if data from the sensor exceeds a threshold, a warning can be output to assist the on-site manager.
[0003] In recent years, advances in artificial intelligence have made it possible to make more complex judgments to detect anomalies. For example, by using machine learning to learn from large amounts of data, it has become possible to detect anomalies that cannot be distinguished simply by whether they are greater than or less than a threshold.
[0004] Patent Document 1 discloses a method and system for detecting signs of anomaly in monitored equipment. Specifically, this document discloses that a bottleneck feature BNF is input to an anomaly detection system FDS that models a normal operating state using a machine learning algorithm, and a sign of an anomaly is detected in the monitored equipment. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] International Publication No. 2019 / 107315 Summary of the Invention [Problem to be solved by the invention]
[0006] Various anomaly detection mechanisms using machine learning have been proposed. Conceptually, anomaly detection includes two types of detection. The first detection is to detect that an anomaly has occurred. The second detection is to detect that an anomaly will occur or may occur in the future.
[0007] It is better for a user to know before an abnormality occurs than to know after it has occurred, and the earlier the user knows, the more time there is to take measures to deal with the abnormality.
[0008] However, earlier detection compromises accuracy. For example, even if a possible anomaly is detected at an early stage, it may not turn out to be an anomaly after all. In other words, there is a trade-off between earlier detection and more accurate detection.
[0009] The present invention aims to improve the accuracy of detection in a method, device and system for detecting anomalies at an earlier stage. [Means for solving the problem]
[0010] In order to achieve the above object, in one aspect, the present disclosure includes the following invention. (Invention 1) A method for predicting an anomaly in a monitored object using a system, comprising: The system includes an anomaly detection means and an anomaly identification means, The anomaly detection means includes a first trained model trained with training data regarding a normal state, The anomaly identification means includes a second trained model trained with learning data related to an abnormal state, The method comprises: A step (10) in which the abnormality identification means receives data of the monitored object; A step (20) of the anomaly identification means inputting the data of the monitored object into the second trained model; A step (30) in which the anomaly identification means extracts an intermediate processing result of the second trained model; A step (40) in which the anomaly detection means inputs the intermediate processing result into the first trained model; A step (50) of the anomaly detection means extracting an output (first output) of the first trained model; A method comprising: (Invention 2) In the method of invention 1, when the first output in the step (50) indicates that there is a possibility of an abnormality with a certain probability or higher, the method further comprises: A step (60) in which the anomaly identification means uses the data of the monitoring target and the first output together as data to be input to the second trained model; A step (70) in which the anomaly identification means extracts an output (second output) of the second trained model; A method comprising: (Invention 3) In the method of invention 1, when the first output in the step (50) indicates that there is a possibility of an abnormality with a certain probability or higher, the method further comprises: A step (60) in which the anomaly identification means uses the monitored data as data to be input to the second trained model, and the first output is not used; A step (70) in which the anomaly identification means extracts an output (second output) of the second trained model; A method comprising: (Invention 4) The method according to any one of Inventions 1 to 3, The method, wherein the step (40) includes combining the intermediate processing results with the monitored data and using them together as input data to the first trained model. (Invention 5) 5. The method according to any one of Inventions 1 to 4, wherein the intermediate processing result is an output other than an output of a bottleneck intermediate layer of a neural network. (Invention 6) A method according to invention 5, wherein the step (30) includes performing dimensionality reduction on intermediate processing results of the second trained model to extract the intermediate processing results. (Invention 7) A device for predicting an abnormality in a monitored object, comprising: The device includes an abnormality detection means and an abnormality identification means, The anomaly detection means includes a first trained model trained with training data regarding a normal state, The anomaly identification means includes a second trained model trained with learning data related to an abnormal state, The anomaly identification means is configured to receive data of the monitored object; The anomaly identification means is configured to input the data of the monitored object to the second trained model; The anomaly identification means is configured to extract an intermediate processing result of the second trained model; The anomaly detection means is configured to input the intermediate processing result to the first trained model; The anomaly detection means is configured to extract an output (first output) of the first trained model. Device. Effect of the Invention
[0011] In one aspect, the present invention includes an anomaly detection unit and an anomaly identification unit, which allows an anomaly to be detected early and also increases the accuracy of the determination of the anomaly detection. [Brief description of the drawings]
[0012] [Figure 1] 1 shows an overview of the system of the present disclosure in one embodiment. [Diagram 2] 1 shows an overview of the system of the present disclosure in one embodiment. [Diagram 3] 1 shows an overview of an apparatus of the present disclosure in one embodiment. [Figure 4] 1 shows an overview of an anomaly detection means of the present disclosure in one embodiment. [Diagram 5] 1 shows an overview of an anomaly identification means of the present disclosure in one embodiment. [Figure 6] 1 shows an overview of an anomaly identification means of the present disclosure in one embodiment. [Figure 7] 1 illustrates a flow diagram of a method according to the present disclosure in one embodiment. [Figure 8] 1 illustrates a flow diagram of a method according to the present disclosure in one embodiment. [Figure 9] 1 illustrates a flow diagram of a method according to the present disclosure in one embodiment. [Figure 10] 1 illustrates a flow diagram of a method according to the present disclosure in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, specific embodiments for carrying out the present invention will be described. The following description is intended to facilitate understanding of the invention, and is not intended to limit the scope of the present invention.
[0014] 1.Definition
[0015] In this specification, the term "anomaly detection" refers to either or both of detecting that an anomaly has occurred and detecting a sign before an anomaly occurs. In a preferred embodiment, the term "anomaly detection" includes detecting a sign before an anomaly occurs. Here, the meaning of detecting includes, for example, outputting a probability, likelihood, or score regarding the possibility of an anomaly. In another example, the meaning of detecting includes outputting a probability, likelihood, or score regarding the possibility of normality.
[0016] As used herein, the term "anomaly identification" refers to inferring the cause of an anomaly, where inferring does not necessarily mean determining the cause, and for example, inferring may include outputting respective probabilities, likelihoods, or scores for multiple possible causes.
[0017] In this specification, the terms "anomaly detection means" and "anomaly identification means" are modules realized by a combination of software (or a program) and hardware. It should be noted that the hardware does not need to be integrated into one device. For example, to realize the "anomaly detection means" and the "anomaly identification means", they may be distributed across multiple devices, with software installed in each device, and each device may be configured to cooperate with the multiple devices.
[0018] In this specification, the term "trained model" refers to a combination of an algorithm (program) and trained parameters. As a result of executing the algorithm according to the trained parameters, a highly accurate output can be obtained. The trained model may be based on either supervised learning or unsupervised learning. The trained model includes, but is not limited to, one or more types selected from the following or a combination thereof: hidden Markov model, SVR (support vector regression), PLS (partial least squares), neural network method, random forest method, decision tree method, etc. The neural network includes, but is not limited to, one or more types selected from the following or a combination thereof: CNN, GAN, autoencoder (e.g., stacked autoencoder, variational autoencoder, convolutional autoencoder, sparse autoencoder, saturating autoencoder), DNN, LSTM, RNN, etc.
[0019] In this specification, the term "intermediate processing result" means a combination of values extracted from any point between the input and output in the trained model described above, or the result of additional processing performed on the combination of values.
[0020] For example, in the case of a neural network, there is an intermediate layer. The "intermediate processing result" of the neural network means an extracted combination of values of each node in any intermediate layer, or a result of performing additional processing on the extracted combination. An example of the additional processing may include, for example, extracting features using another trained model.
[0021] 2. Execution environment
[0022] 2-1. System Overview
[0023] FIG. 1 shows an example of an execution environment for implementing the system and method of the present disclosure in one embodiment. A server 1010 and multiple terminals are connected through a network. In FIG. 1, the server 1010 is represented as a single piece of hardware, but may be distributed across multiple pieces of physical hardware. The server 1010 may be a cloud server or an on-premise server. The edge terminal 1020 is represented as multiple terminals, but may theoretically be a single terminal. However, since the server 1010 is often used by multiple users in common, typically the system has multiple edge terminals 1020. The network may be wired, wireless, or a combination of both.
[0024] FIG. 2 is a modified example of FIG. 1, and the system further includes a repeater 1030 and a sensor 1040. The edge terminal 1020 is connected to the repeater 1030 via another network (second network). This repeater 1030 can transmit information from the sensor 1040 to the edge terminal 1020 through the second network. Then, based on the information from the sensor 1040, anomaly detection and anomaly identification can be performed. In FIG. 2, one repeater 1030 and one sensor 1040 are shown, but there may be a plurality of repeaters. For example, there may be a plurality of repeaters corresponding to one edge terminal 1020. Also, there may be a plurality of sensors corresponding to one repeater 1030.
[0025] Also, in FIG. 2, the sensor 1040 is connected to the repeater 1030, but may be directly connected to the edge terminal 1020 if necessary.
[0026] 2-2. Overview of information processing device The above-mentioned server 1010 and edge terminal 1020 are typically information processing devices (or computing devices). The above-mentioned repeater 1030 may also be an information processing device.
[0027] The information processing device may typically include a processor 1080, a memory 1050, a non-transitory storage medium 1060, and a communication module 1070, as shown in FIG.
[0028] Examples of information processing devices include, but are not limited to, the following: servers, personal computers, single-board computers (e.g., Raspberry Pi, etc.), tablet terminals, smartphones, smart watches, smart glasses, etc.
[0029] The program is stored in a non-transitory storage medium 1060 (e.g., HDD, SSD, etc.), loaded into a memory 1050 (e.g., RAM, etc.) as appropriate, and executed by a processor 1080 (e.g., an electronic processor such as a CPU, etc.). If necessary, the program can connect to a network through a communication module 1070 to transmit and receive information.
[0030] In one embodiment, the program may be installed as application software on one information processing device and executed by the information processing device.
[0031] In another embodiment, the number of information processing devices is not limited to one, and multiple information processing devices may be used as necessary. In this case, the functions of the program may be distributed among the multiple information processing devices.
[0032] 2-3. Means of anomaly detection and anomaly identification In one embodiment, the anomaly detection means and the anomaly identification means may be implemented by an information processing device in which a desired program is installed. The anomaly detection means and the anomaly identification means may be implemented by the server 1010 or any edge terminal 1020 shown in FIG. 1 and FIG. 2.
[0033] For example, when an anomaly detection means and an anomaly identification means are implemented in the edge terminal 1020, the server 1010 may manage the versions of programs and the like related to the means. The edge terminal 1020 may download at least a part of the programs and the like from the server 1010 as appropriate to perform updates.
[0034] The anomaly detection means and the anomaly identification means may be implemented in the same device, or in some cases, may be implemented in multiple devices. In the case where they are implemented in multiple devices, the anomaly detection means may be implemented in one device, and the anomaly identification means may be implemented in another device.
[0035] Alternatively, either the anomaly detection means or the anomaly identification means may be further modularized, with some of the modules of the anomaly detection means being implemented in one device and another part of the modules of the anomaly detection means being implemented in another device.
[0036] When the server 1010 is provided with an anomaly detection means and an anomaly identification means, they may be distributed and implemented in a plurality of servers, similar to the edge terminal 1020 described above.
[0037] Furthermore, when multiple servers are used, the first server and the second server may be connected by a different network, although this is not shown in Figures 1 and 2. The first server may be connected by the network shown in Figures 1 and 2 and have a function of accepting requests from the edge terminal 1020 (e.g., a Web server). The first server may make an inquiry to the second server based on the request from the edge terminal 1020 and transmit the inquiry result to the edge terminal 1020. In this case, the second server is equipped with an anomaly detection means and an anomaly identification means, and the second server can output the inquiry result based on these means.
[0038] 3.Anomaly detection methods A configuration of the anomaly detection means in one embodiment is shown in Fig. 4. The anomaly detection means includes at least a first trained model 2010. Preferably, the anomaly detection means includes an input unit 2020 and an output unit 2030. The input unit 2020 can receive data from the outside and input the data to the first trained model 2010. In some cases, the input unit 2020 can perform preprocessing on the received data from the outside and input the preprocessed data to the first trained model 2010. The output unit 2030 can receive an output from the first trained model 2010 and transmit it to the outside.
[0039] As described above regarding the term “trained model,” the type of the trained model is not particularly limited. However, the first trained model 2010 in the anomaly detection means is constructed by training data related to a normal state.
[0040] For example, when detecting an abnormality from the sound of a machine, engine, etc. (e.g., when detecting signs of an abnormality), the learning data used to construct the first trained model 2010 may be sound data when the machine, engine, etc. is operating normally (and when no malfunction or the like occurs immediately thereafter).
[0041] As a result, when the input data is close to previously learned data, the first trained model 2010 can output a judgment that the data is normal, or that there is a high probability that the data is normal, etc. When the input data is far from previously learned data, the first trained model 2010 can output a judgment that the data is abnormal, that there is a possibility that an abnormality will occur, or that there is a high probability of these, etc.
[0042] For the purpose of constructing a training model by learning training data related to a normal state, an autoencoder is a suitable trained model. The type of the autoencoder is not particularly limited. For example, the autoencoder may be one or more of a stacked autoencoder, a variational autoencoder, a convolutional autoencoder, a sparse autoencoder, a saturating autoencoder, and the like.
[0043] In addition, the data input to the first trained model 2010 may be data to be monitored and / or may include data regarding intermediate processing results of the second trained model 3010 (details described later).
[0044] 4. Means of identifying anomalies A configuration of the anomaly identification means in one embodiment is shown in Fig. 5. The anomaly identification means includes at least a second trained model 3010. Preferably, the anomaly identification means includes an input unit 3020 and an output unit 3030. The input unit 3020 can receive data from the outside and input the data to the second trained model 3010. In some cases, the input unit 3020 can perform preprocessing on the received data from the outside and input the preprocessed data to the second trained model 3010. The output unit 3030 can receive output from the second trained model 3010 and transmit it to the outside.
[0045] As described above regarding the term “trained model,” the type of the trained model is not particularly limited. However, the second trained model 3010 in the anomaly identification means is constructed by training learning data related to an abnormal state.
[0046] In a preferred embodiment, as shown in FIG. 6, the anomaly identification means may include a plurality of second trained models 3010.
[0047] For example, the anomaly identification means may include a plurality of second trained models 3010 specialized for each type of anomaly cause. When there are a plurality of possible anomaly causes, data specialized for each cause may be trained separately to construct a plurality of trained models. For example, when there are three possible types of elements A, B, and C that may cause an anomaly, a trained model 1 specialized for determining whether an anomaly is caused by element A, a trained model 2 specialized for determining whether an anomaly is caused by element B, and a trained model 3 specialized for determining whether an anomaly is caused by element C may be constructed. Then, a predetermined value may be input, and the probability of an anomaly caused by element A, the probability of an anomaly caused by element B, the probability of an anomaly caused by element C, and the like may be output.
[0048] In another preferred embodiment, the possibility of one cause may be determined using the outputs of multiple second trained models 3010. As described above, the type of trained model is not particularly limited, but different types of second trained models may be combined to comprehensively determine the output results of each (parallel combination). Alternatively, different types of second trained models may be combined to receive the output result of one trained model by another trained model, and output yet another output result (serial combination).
[0049] Suitable trained models for the purpose of constructing training data regarding abnormal conditions through training include neural networks, hidden Markov models, or combinations thereof.
[0050] In addition, the data input to the second trained model 3010 may be the data to be monitored and the output result of the first trained model 2010 described above.
[0051] The data output from the second trained model 3010 may be the probability that an anomaly (or a precursor thereof) is due to a specific cause, etc.
[0052] The output unit 3030 can transmit to the outside the data output from the second trained model 3010. In addition, when there are multiple second trained models 3010, the output unit 3030 can transmit the output from each trained model to the outside.
[0053] 5. Anomaly detection process flow A process flow in an embodiment using the above-mentioned anomaly detection means and anomaly identification means will be described below. An example is shown in Figure 7. The method for detecting anomalies includes at least the following steps. A step (S10) in which the abnormality identification means receives data of a monitoring target A step (S20) in which the anomaly identification means inputs the monitored data into a second trained model. A step (S30) in which the anomaly identification means extracts intermediate processing results of the second trained model. A step (S40) in which the anomaly detection means inputs the intermediate processing result into the first trained model. A step (S50) in which the anomaly detection means extracts an output (first output) of the first trained model.
[0054] Each step is described in detail below.
[0055] 5-1. Step of receiving data to be monitored (S10) The step of receiving the data to be monitored is performed by the abnormality identification means (specifically, the input unit of the abnormality identification means). The type of data to be monitored is not particularly limited, and may be data in any format.
[0056] Examples of data may include at least one selected from the following: image data (e.g., still image data, video data, etc.), audio data (e.g., human voice, material sound, and a mixture of both), and time series data (e.g., data recording the temporal changes in values obtained from a sensor, etc.).
[0057] The data to be monitored may be derived from, for example, data obtained from the sensor shown in Fig. 2. Moreover, the data to be monitored does not need to be raw data obtained from the sensor shown in Fig. 2, and may be data obtained through processing such as A / D conversion, appropriate filtering, and feature extraction.
[0058] In another embodiment, the monitored data does not necessarily have to be derived from data obtained from a sensor. For example, time series data on the volume of trade on the Internet does not originate from a specific sensor, but is data obtained by aggregating buy and sell orders from an electronic commerce site, and does not use a sensor. In this embodiment, such time series data obtained without using a sensor may also be included in the monitored data.
[0059] 5-2. Step of inputting monitored data into the second trained model (S20) After receiving the data to be monitored, the step of inputting the data to be monitored into the second trained model is executed by the anomaly identification means (specifically, the input unit of the anomaly identification means).
[0060] 5-3. Step of extracting intermediate processing results of the second trained model (S30) Thereafter, the anomaly identification means executes a step of extracting intermediate processing results of the second trained model. To this end, the anomaly identification means may comprise an intermediate processing result extraction unit.
[0061] Extraction of intermediate processing results from a trained model is possible by using techniques known in the art. For example, in the case of a neural network, values input to each node of a certain intermediate layer can be extracted. In a preferred embodiment, in the case of a neural network, intermediate processing results can be extracted from intermediate layers other than the bottleneck. More preferably, intermediate processing results can be extracted from intermediate layers close to the output layer (e.g., the layer one, two, or three layers before the output layer). If the intermediate layer is close to the output layer, there is a high probability that a result that reflects the characteristics of the model can be obtained.
[0062] In the case of a hidden Markov model (HMM), the intermediate processing results may be used directly, or a neural network may be prepared as a sub-model separate from the HMM, and the intermediate processing results of the neural network may be extracted. For example, if the data to be monitored is sound, the target sound is input to the HMM frame by frame. Then, a state number output is obtained from the HMM. The neural network as a sub-model is trained to infer the state number when one frame of the target sound is input to the HMM. The same content as that input to the HMM may then be input to the trained neural network, and the intermediate processing results may be output from the intermediate layer.
[0063] As shown in Figure 6, if there are multiple second trained models, intermediate processing results may be extracted from each model.
[0064] If the number of parameters in the extracted intermediate processing result is large, dimensionality reduction may be performed. For example, a process of reducing the number of parameters may be performed in advance on the intermediate processing result by performing principal component analysis on the intermediate processing result. Therefore, the intermediate processing result in this specification includes not only a direct intermediate processing result (e.g., a value obtained from an intermediate layer of a neural network, etc.) but also a result obtained by performing another process on the intermediate processing result (e.g., dimensionality reduction). Preferably, the intermediate processing result may be a direct intermediate processing result or an intermediate processing result obtained by dimensionality reduction.
[0065] The anomaly identification means (e.g., the intermediate processing result extraction unit) can transmit the extracted intermediate processing result to the anomaly detection means. The input unit of the anomaly detection means can receive the transmitted intermediate processing result.
[0066] 5-4. Step of inputting intermediate processing results into the first trained model (S40) The input unit of the anomaly detection means inputs the intermediate processing result to the first trained model. This intermediate processing result can be regarded as a feature generated by the second trained model. Note that it is not essential to input all of the intermediate processing results obtained in step (30), and in some cases, at least a part of the intermediate processing results obtained may be input.
[0067] 5-5. Step of extracting output (first output) of the first trained model (S50) Thereafter, the output unit of the anomaly detection means can extract an output (first output) from the first trained model. The output may be an output indicating a possibility that an anomaly has occurred, or the opposite (i.e., an output indicating a possibility that the model is normal).
[0068] 5-6. Further steps (Figure 8) After extracting the first output, further steps may be performed depending on the result of the first output, for example, the following steps may be performed based on predefined criteria: A step (S60) in which the anomaly identification means uses the data of the monitoring target and the first output together as data to be input to the second trained model. A step (S70) in which the anomaly identification means extracts an output (second output) of the second trained model.
[0069] Here, the predetermined standard is a standard indicating whether there is a possibility of an abnormality with a certain probability or higher. The specific standard value is not particularly limited, and may be adjusted according to the purpose of implementation. For example, if there is a purpose such as preventing a major accident even with a fairly low probability, the standard value may be set low. For example, a borderline is set when the probability of an abnormality is 30%, and when the probability is equal to or exceeds 30%, the above-mentioned steps (S60) and (S70) may be executed.
[0070] Alternatively, it may be possible to set the program so that it is not executed unless the probability is extremely high (for example, the reference value is 70%). Alternatively, it may be possible to determine whether to execute the program using a 50 / 50 probability (for example, the reference value is 50%) as the boundary value.
[0071] Furthermore, if the possibility of an abnormality is equal to or less than a certain probability based on the above criteria, the above-mentioned steps (S60) and (S70) do not need to be executed.
[0072] The output unit of the anomaly detection means transmits the first output to the anomaly identification means. Here, the anomaly identification means for generating the intermediate processing result and the anomaly identification means for receiving the first output may be the same or different. For example, the second trained model in the anomaly identification means for generating the intermediate processing result may be trained without using the training data corresponding to the first output as training data. On the other hand, the second trained model in the anomaly identification means for receiving the first output may be trained using the training data corresponding to the first output as training data.
[0073] The above-mentioned steps (S60) and (S70) will be described in detail below.
[0074] 5-7. Step of using the monitored data and the first output together as data to be input to the second trained model (S60) The input unit of the anomaly identification means can receive the first output described above. The input unit of the anomaly identification means can also receive the monitored data. As described above, when the anomaly identification means for generating the intermediate processing result and the anomaly identification means for receiving the first output are the same, the monitored data has already been received at the stage of generating the intermediate processing result. The input unit of the anomaly identification means can input the received data to the second trained model. When there are multiple trained models as shown in FIG. 6 (particularly when multiple trained models are not serially connected but exist in parallel), the monitored data and the first output can be used together as input data for each model.
[0075] 5-8. Step of extracting output (second output) of second trained model (S70) Next, the output unit of the anomaly identification means extracts an output (second output) from the second trained model. The second output indicates the possibility that the anomaly is due to a specific cause. For example, as shown in FIG. 6, when the anomaly identification means has a plurality of second trained models, the second output may include an output indicating the likelihood of the cause corresponding to each of the second trained models. In another example, the anomaly identification means may have one type of second trained model, and the output layer of the one type of second trained model may have a plurality of nodes, and each node may have a node for each type of failure, such as a probability of normality, a probability of failure caused by element A, and a probability of failure caused by element B.
[0076] The above-described series of processes not only detects anomalies (particularly, detects signs of anomalies), but also makes it possible to determine the cause of the anomalies. In the above-described system, both an anomaly detection means and an anomaly identification means are used, and a determination is made by combining these. The anomaly identification means improves the accuracy of the determination by using not only the data of the monitored object but also the first output from the anomaly detection means as an input value. On the other hand, the anomaly detection means improves the accuracy of the determination by using the intermediate processing results from the anomaly identification means as an input value, thereby making it possible to utilize feature quantities. In this way, the interaction between the anomaly detection means and the anomaly identification means makes it possible to improve the accuracy of the determination.
[0077] 5-9. Variation 1 A modified example of the flow of the above-mentioned series of processes is shown in Fig. 9. The method for performing anomaly detection includes at least the following steps. The abnormality detection means and the abnormality identification means receive data to be monitored (S110). A step (S120) in which the anomaly identification means inputs the monitored data into the second trained model. A step (S130) in which the anomaly identification means extracts intermediate processing results of the second trained model. A step (S140) in which the anomaly detection means combines the intermediate processing result with the data to be monitored and uses the combined data as data to be input to the first trained model. A step (S150) in which the anomaly detection means extracts an output (first output) of the first trained model.
[0078] The difference is that in step (S110), not only the anomaly identification means but also the anomaly detection means receives the data to be monitored. Note that the timing at which the anomaly detection means receives the data to be monitored is not limited to step (S110), and may be any timing prior to step (S140).
[0079] The second difference is that in step (S140), the first trained model uses not only the intermediate processing results but also the monitored data.
[0080] By doing this, the first trained model can take into account not only the features of the intermediate processing results but also the monitored data before the features are extracted when determining whether an abnormality has been detected, thereby further improving the accuracy of the determination.
[0081] Further, in addition to steps (S110) to (S150) as shown in Fig. 8, further steps may be executed. For example, the following steps may be executed (Fig. 10). A step (S160) in which the anomaly identification means uses the data of the monitoring target and the first output together as data to be input to the second trained model. A step (S170) in which the anomaly identification means extracts an output (second output) of the second trained model.
[0082] The criterion for determining whether or not to execute these further steps (S160) to (S170) may be a criterion indicating whether or not there is a possibility of an abnormality with a certain degree of probability or higher, as described above.
[0083] 5-10. Variation 2 In another modified example, when the first output indicates that the possibility of an anomaly is extremely high, the anomaly identification means may use the monitored data as data to be input to the second trained model without using the first output. For example, when the degree of possibility of an anomaly indicated by the first output is within a first range (e.g., the probability of an anomaly is 40% to 80%), as described above, the anomaly identification means may use the monitored data and the first output together as data to be input to the second trained model. However, when the degree of possibility of an anomaly indicated by the first output is within a second range that is larger than the first range (e.g., when the lower limit value of the second range is larger than the upper limit value of the first range, e.g., the probability of an anomaly is greater than 80% to 100%), the anomaly identification means may use the monitored data as data to be input to the second trained model without using the first output. In this way, the contents to be input to the second trained model may be switched depending on the degree of possibility of an anomaly indicated by the first output.
[0084] 6. How to build a trained model In the above, at least two types of trained models are used: a first trained model and a second trained model. The first trained model is a model trained with training data on normal states. The second trained model is a model trained with training data on abnormal states.
[0085] 6-1. Construction method 1 When performing the processing shown in Fig. 7 or Fig. 9, a trained model can be constructed by the following procedure. The following describes an example using time-series data of speech.
[0086] First, time series data of voice in a normal state is prepared, and the first trained model is trained using this as training data. Next, time series data of voice in an abnormal state is prepared. Then, this time series data of voice in an abnormal state is input to the first trained model, and output data from the first trained model is prepared. Next, the output data and the time series data of voice in an abnormal state are trained using this as training data in a second trained model.
[0087] Next, the first trained model that was trained earlier is improved, or a new first trained model is constructed. Specifically, the second trained model that was trained earlier is prepared. Then, time-series data of voice in a normal state is prepared. Next, the time-series data of voice in a normal state is input into the second trained model, and the intermediate processing results are prepared.
[0088] A new first trained model is constructed using the obtained intermediate processing result (intermediate processing result in a normal state) as learning data. Then, this new first trained model and the previously constructed second trained model can be used in the process flows shown in FIGS. 7 to 10.
[0089] Alternatively, the intermediate processing result obtained (the intermediate processing result in the normal state) is used as learning data (together with the time-series data of the voice in the normal state as learning data) to perform reinforcement learning on the first trained model previously constructed. The first trained model after this reinforcement learning and the second trained model previously constructed can be used in the process flows shown in FIGS. 7 to 10.
[0090] 6-2. Construction method 2 In the above construction method, the first trained model is constructed first. However, in another example, the second trained model may be constructed first. For example, time series data of voice in an abnormal state is prepared, and the second trained model is trained using this as training data. Next, time series data of voice in a normal state is prepared. Then, this time series data of voice in the normal state is input to the second trained model, and an intermediate processing result is extracted from the second trained model. Next, the intermediate processing result and the time series data of voice in the normal state are trained using the first trained model as training data.
[0091] Next, the previously trained second trained model is improved or a new second trained model is constructed. Specifically, the previously trained first trained model is prepared. Then, time-series data of the voice in an abnormal state is prepared. Next, the time-series data of the voice in an abnormal state is input to the previously trained first trained model, and the output data is prepared.
[0092] A new second trained model is constructed using the obtained output data (output data in an abnormal state) and the time-series data of the voice in an abnormal state as training data. Then, the first trained model and the newly constructed second trained model can be used in the process flows shown in FIGS. 7 to 10.
[0093] Using the obtained output data (output data when an abnormal state occurs) as learning data (together with the time-series data of the voice when an abnormal state occurs as learning data), reinforcement learning is performed on the second trained model previously constructed. The first trained model after this reinforcement learning and the second trained model previously constructed can be used in the process flows shown in Figs. 7 to 10.
[0094] Furthermore, in the case where the second trained model is constructed first, the above method includes a first construction and a second construction. Here, in the second construction, a new trained model may be constructed without reusing the second trained model constructed the first time. In this case, for example, in the steps shown in FIG. 8, the second trained model constructed the first time may be used in steps (S10) to (S30). Then, the second trained model constructed the second time may be used in steps (S60) to (S70). With this method, it is not necessary to replace the second trained model with a new one during construction. [Industrial Applicability]
[0095] The system and method of the above-described embodiment can be used in various industrial fields.
[0096] As an example, the system and method of the above-described embodiment can be applied to a system that monitors the failure of parts of each vehicle of a train. Specifically, a material sound sensor can be provided near the parts of each vehicle to monitor the sound from the parts while the vehicle is running. Then, the normal state and the failure state (more specifically, the state immediately before the failure occurs) of the parts may be learned as learning data into a trained model of the anomaly detection means and a trained model of the anomaly identification means, respectively. Then, these trained models may be implemented in, for example, each vehicle or a specific vehicle (such as the leading vehicle or the rear vehicle of the train) by an information processing device (edge terminal in FIG. 1 and FIG. 2). Data from each sensor may be transmitted to the information processing device by providing a repeater in each vehicle and using the repeater.
[0097] In another example, the system and method of the above-described embodiment can be applied when monitoring disasters, weather changes, and the like using a surveillance camera. Specifically, the weather conditions on the mountain are monitored by a camera, and an information processing device (edge terminal in FIG. 1 and FIG. 2) at an observation station can receive image data using a repeater or the like. Then, a normal weather state (a state where the weather does not deteriorate) and an abnormal state (more specifically, a state immediately before the weather deteriorates) may be used as learning data to train a trained model of the anomaly detection means and a trained model of the anomaly identification means, respectively. Then, these trained models may be implemented, for example, in an information processing device.
[0098] In yet another example, data other than data from a sensor may be used. For example, the system and method of the above-described embodiment may be applied when monitoring human health using blood test data. For example, blood may be collected from a population for a specific period of time, and the health status of the population may be tracked. Then, the population may be classified into a group that had no health problems during the specific period of time, a group that suffered from a disease, etc., and the like. Furthermore, the groups may be further classified according to the type of disease, etc. The data thus obtained may be trained as normal state data and abnormal state data into a trained model of the anomaly detection means and a trained model of the anomaly identification means, respectively. Then, these trained models may be implemented in an information processing device (edge terminal in FIG. 1 and FIG. 2) for analyzing blood data.
[0099] Above, specific embodiments of the invention have been described. The above embodiments are merely examples, and the present invention is not limited to the above embodiments. For example, a technical feature disclosed in one of the above embodiments can be applied to other embodiments. In addition, unless otherwise specified, for a particular method, some steps can be replaced with other steps, and additional steps can be added between two specific steps. The scope of the present invention is defined by the claims. [Explanation of symbols]
[0100] 1010 Server 1020 Edge Terminal 1030 Repeater 1040 Sensor 1050 Memory 1060 storage medium 1070 Communication Module 1080 Processor 2010 First trained model 2020 Input section 2030 Output section 3010 Second trained model 3020 Input section 3030 Output section
Claims
1. A method for predicting an anomaly in a monitored object using a system, comprising: The system includes an anomaly detection means and an anomaly identification means, The anomaly detection means includes a first trained model trained with training data regarding a normal state, The anomaly identification means includes a second trained model trained with learning data related to an abnormal state, The method comprises: A step (10) in which the abnormality identification means receives data of the monitored object; A step (20) in which the anomaly identification means inputs the data of the monitored object into the second trained model; A step (30) in which the anomaly identification means extracts an intermediate processing result of the second trained model; A step (40) in which the anomaly detection means inputs the intermediate processing result into the first trained model; A step (50) in which the anomaly detection means extracts an output (first output) of the first trained model; A method comprising:
2. 2. The method according to claim 1, further comprising the steps of: when the first output in the step (50) indicates that there is a possibility of an abnormality with a certain probability or higher, the method further comprises: A step (60) in which the anomaly identification means uses the data of the monitoring target and the first output together as data to be input to the second trained model; A step (70) in which the anomaly identification means extracts an output (second output) of the second trained model; A method comprising:
3. 2. The method according to claim 1, further comprising the steps of: when the first output in the step (50) indicates that there is a possibility of an abnormality with a certain probability or higher, the method further comprises: A step (60) in which the anomaly identification means uses the monitored data as data to be input to the second trained model, and the first output is not used; A step (70) in which the anomaly identification means extracts an output (second output) of the second trained model; A method comprising:
4. 2. The method of claim 1, The method, wherein step (40) includes combining the intermediate processing results with the monitored data and using them together as input data to the first trained model.
5. 2. The method of claim 1, wherein the intermediate processing result is an output other than an output of a bottleneck hidden layer of a neural network.
6. 6. The method of claim 5, wherein step (30) includes performing dimensionality reduction on intermediate processing results of the second trained model to extract the intermediate processing results.
7. A device for predicting an abnormality in a monitored object, comprising: The device includes an abnormality detection means and an abnormality identification means, The anomaly detection means includes a first trained model trained with training data regarding a normal state, The anomaly identification means includes a second trained model trained with learning data related to an abnormal state, The anomaly identification means is configured to receive data of the monitored object; The anomaly identification means is configured to input the data of the monitoring target to the second trained model; The anomaly identification means is configured to extract an intermediate processing result of the second trained model, The anomaly detection means is configured to input the intermediate processing result to the first trained model; The anomaly detection means is configured to extract an output (first output) of the first trained model. Device.
Citation Information
Patent Citations
Method and system for detecting symptom of abnormality in apparatus being monitored
WO2019107315A1
