Method and system for predicting anomaly of monitoring target
By using a combination of multiple normal and abnormal models, the system enhances anomaly detection accuracy and timing, addressing the trade-off in existing systems to identify anomalies earlier and more precisely.
Patent Information
- Application Number
- JP2023079641
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-05-12
AI Technical Summary
Existing anomaly detection systems face a trade-off between early detection and accuracy, with faster detection often leading to reduced precision, and there is a challenge in collecting diverse anomaly data for machine learning models.
The system employs multiple normal and abnormal models trained on different states, using a first determining means for normal models and a second determining means for abnormal models, integrating their outputs to enhance detection accuracy and identify anomalies earlier.
This approach allows for earlier and more accurate anomaly detection by leveraging a combination of normal and abnormal models, improving the precision of identifying potential issues before they occur.
Smart Images

Figure 2024163768000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a method and system for predicting anomalies in a monitored object. [Background technology]
[0002] In the field of IoT, systems for detecting anomalies have been developed. For example, a specific target can be monitored using a sensor, and if data from the sensor exceeds a threshold, a warning can be output to assist the on-site manager.
[0003] In recent years, advances in artificial intelligence have made it possible to make more complex judgments to detect anomalies. For example, by using machine learning to learn from large amounts of data, it has become possible to detect anomalies that cannot be distinguished simply by whether they are greater than or less than a threshold.
[0004] Patent Document 1 (International Publication No. 2019 / 107315) discloses a method and system for detecting signs of anomaly in monitored equipment. Specifically, this document discloses that a bottleneck feature BNF is input to an anomaly detection system FDS that models a normal operating state using a machine learning algorithm, and a sign of an anomaly is detected in the monitored equipment.
[0005] Furthermore, Patent Document 2 (JP 2020-140518 A) discloses an information processing device, a control method for an information processing device, and a program. Specifically, this document discloses determining data that is effective for learning from data to be labeled and setting it as local data, setting a partial region to be labeled from the data to be labeled based on the distribution of the local data, and displaying the set partial region on a display unit. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] International Publication No. 2019 / 107315 [Patent Document 2] JP 2020-140518 A Summary of the Invention [Problem to be solved by the invention]
[0007] Meanwhile, various anomaly detection mechanisms using machine learning have been proposed. Conceptually, anomaly detection includes two types of detection. The first detection is to detect that an anomaly has occurred. The second detection is to detect that an anomaly will occur or may occur in the future.
[0008] It is better for a user to know before an abnormality occurs than to know after it has occurred, and the earlier the user knows, the more time there is to take measures to deal with the abnormality.
[0009] However, earlier detection usually comes with a decrease in accuracy. For example, even if a possible anomaly is detected at an early stage, it may not turn out to be an anomaly after all. In other words, there is a trade-off between earlier detection and more accurate detection.
[0010] The present invention has been completed in view of the above problems, and has an object in one embodiment to provide a method for detecting anomalies at an earlier stage while increasing the accuracy of anomaly detection and identification. In another embodiment, the present invention has an object to provide a system for performing such a method. [Means for solving the problem]
[0011] In anomaly detection based on machine learning, since there are countless types of anomalies, it is difficult to collect the anomaly data required for model learning. Therefore, it is common to judge whether something is normal or abnormal using only a normal model. In order to detect anomalies with high accuracy using a normal model, it is necessary to observe abnormal data that is sufficiently far from normal data. In other words, depending on the type of anomaly and the observation environment, data that is sufficiently far from normal data is not always obtained, and there is a limit to the accuracy of judgment with only one normal model. Therefore, the inventors came up with the idea of preparing multiple normal models suitable for detecting each expected abnormal state in advance, and selecting an appropriate normal model according to the observation data to make a judgment.
[0012] As a result of further intensive research, the inventors have found that it is possible to improve the detection and accuracy of abnormalities and solve the above-mentioned problems by using a sub-normal model that is more detailed than a main normal model obtained by machine learning data on normal states and is suitable for detecting each abnormal state that is expected in advance, and a plurality of abnormal models further obtained by machine learning data on abnormal states in combination. The present invention has been completed based on the above findings, and is exemplified below.
[0013] [1] A method for predicting an anomaly in a monitored object using a system, comprising: The system includes a first determination means, a second determination means, and an integrated determination means; the first determination means includes a plurality of normal models trained with learning data relating to normal states, the plurality of normal models including a plurality of sub-normal models suitable for detecting different abnormal states; the second determination means includes a plurality of abnormality models trained with learning data relating to different abnormal states; The method comprises: a step (10) in which the first determination means and the second determination means each receive data to be monitored; A step (201) in which the first determination means outputs a score, probability, or likelihood for each of the plurality of normal models from the monitored data; a step (202) of the second determination means outputting a score, probability or likelihood for an abnormal state corresponding to each of the plurality of abnormality models from the monitored data; a step (30) of the integrated judgment means outputting a judgment result including information on a normal state or at least one abnormal state based on the output from the first judgment means and the output from the second judgment means; A method comprising: [2] The method of claim 1, wherein the first determination means further includes a main normal model suitable for comprehensive detection of abnormal conditions. [3] The method according to [1] or [2], wherein the plurality of sub-normal models are trained by extracting features suitable for detecting different abnormal states from training data regarding normal states. [4] The method according to any one of [1] to [3], wherein the first determination means includes at least one normal model having at least one type of environmental noise superimposed thereon. [5] The first determination means outputs a score for each of the plurality of normal models from the monitored data; The second determination means outputs a probability or likelihood of an abnormal state corresponding to each of the plurality of abnormality models from the data of the monitoring target; The method according to any one of [1] to [4], wherein the step (30) includes a step (301) of determining whether or not the state is normal based on the score output from the first determination means, and a step (302) of weighting the score with the probability or likelihood output from the second determination means if an abnormality is determined, to identify at least one abnormal state. [6] The second determination means further includes at least one normal model; The method described in [5], wherein the step (301) includes determining whether or not the state is normal based on both the score output from the first determination means and the probability or likelihood of a normal state output for the at least one normal model. [7] The method according to any one of [1] to [6], wherein the determination result includes a probability or likelihood of a normal state, or a plurality of abnormal states corresponding to the plurality of abnormal models. [8] A system for predicting an abnormality in a monitored object, comprising: The system includes a first determination means, a second determination means, and an integrated determination means; the first determination means includes a plurality of normal models trained with learning data relating to normal states, the plurality of normal models including a plurality of sub-normal models suitable for detecting different abnormal states; the second determination means includes a plurality of abnormality models trained with learning data relating to different abnormal states; the first determining means and the second determining means are each configured to receive data to be monitored; The first determination means is configured to output a score, or a probability or likelihood, for each of the plurality of normal models from the monitored data; the second determination means is configured to output a score, or a probability or likelihood, for an abnormal state corresponding to each of the plurality of anomaly models from the monitored data; The integrated determination means is configured to output a determination result including information on a normal state or at least one abnormal state based on the output from the first determination means and the output from the second determination means. system. [9] The system described in [8], wherein the first determination means further includes a main normal model suitable for comprehensive detection of abnormal conditions.
[10] The system described in [8] or [9], wherein the multiple sub-normal models are trained by extracting features suitable for detecting different abnormal states from training data regarding normal states.
[11] The system according to any one of [8] to
[10] , wherein the first determination means includes at least one normal model having at least one type of environmental noise superimposed thereon.
[12] the first determination means is configured to output a score for each of the plurality of normal models from the monitored data; the second determination means is configured to output a probability or likelihood of an abnormal state corresponding to each of the plurality of abnormality models from the data of the monitored object; The system described in any one of [8] to
[11] , wherein the integrated judgment means is configured to judge whether or not the state is normal based on the score output from the first judgment means, and if it judges an abnormality, to weight the score with the probability or likelihood output from the second judgment means and identify at least one abnormal state.
[13] The system described in
[12] , wherein the second determination means further includes at least one normal model, and the integrated determination means is configured to determine whether or not the state is normal based on both the output of the score from the first determination means and the probability or likelihood of a normal state output for the at least one normal model.
[14] The system according to any one of [8] to
[13] , wherein the determination result includes a probability or likelihood of a normal state or a plurality of abnormal states corresponding to the plurality of abnormality models. Effect of the Invention
[0014] According to an embodiment of the present invention, a method is provided that allows for earlier detection of anomalies while increasing the accuracy of anomaly detection and identification, and according to another embodiment of the present invention, a system for implementing such a method is provided. [Brief description of the drawings]
[0015] [Figure 1] 1 is a schematic diagram showing an overview of a system according to an embodiment of the present invention; [Diagram 2] FIG. 13 is a schematic diagram showing an overview of a system according to another embodiment of the present invention. [Diagram 3] 1 is a schematic diagram showing an overview of an information processing device according to an embodiment of the present invention; [Figure 4] FIG. 2 is a schematic diagram showing an overview of a first determination means in one embodiment of the present invention. [Diagram 5] FIG. 4 is a schematic diagram showing an overview of a first determination means in another embodiment of the present invention. [Figure 6] FIG. 4 is a schematic diagram showing an overview of a second determination means in one embodiment of the present invention. [Figure 7] FIG. 2 illustrates a flow diagram for a method according to one embodiment of the present invention. [Figure 8] FIG. 4 is a flow diagram illustrating a method according to another embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing an example of differences in learning data between a main normal model and a sub normal model in one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0016] Next, an embodiment of the present invention will be described in detail with reference to the drawings. It should be understood that the present invention is not limited to the following embodiment, and that appropriate changes and improvements in the design may be made based on the ordinary knowledge of a person skilled in the art without departing from the spirit of the present invention.
[0017] (1.Definition) In this specification, the term "anomaly detection" means either or both of detecting the occurrence of an anomaly and detecting a sign before an anomaly occurs. In a preferred embodiment, the term "anomaly detection" includes detecting a sign before an anomaly occurs. And, the term "anomaly detection means" means for performing anomaly detection.
[0018] In this specification, the term "anomaly identification" refers to estimating the cause of an anomaly. Here, estimating does not necessarily mean determining the cause. For example, estimating includes outputting respective probabilities or likelihoods for multiple possible causes. And, the term "anomaly identification means" refers to means for performing anomaly identification.
[0019] In this specification, the term "determination" is a general term for "anomaly detection" and "anomaly identification," and may refer to either or both of them.
[0020] In this specification, the term "trained model" refers to a combination of an algorithm (program) and trained parameters. By executing the algorithm according to the trained parameters, a highly accurate output can be obtained. The trained model includes, but is not limited to, one or more types selected from the following or a combination thereof: hidden Markov model, SVR (Support Vector Regression), PLS (Partial Least Squares), neural network method, random forest method, decision tree method, etc. The neural network includes, but is not limited to, one or more types selected from the following or a combination thereof: CNN (Convolutional Neural Network), GAN (Generative Adversarial Network), autoencoder, DNN (Deep Neural Network), LSTM (Long Short Term Memory), RNN (Recurrent Neural Network), etc.
[0021] In this specification, the term "intermediate processing result" means a combination of values extracted from any point between the input and output in the trained model described above, or the result of additional processing performed on the combination of values.
[0022] For example, in the case of a neural network, there is an intermediate layer. The "intermediate processing result" of the neural network means an extracted combination of values of each node in any intermediate layer, or a result of performing additional processing on the extracted combination. An example of the additional processing may include, for example, extracting features using another trained model.
[0023] (2. Execution Environment) [2-1. System Overview] FIG. 1 shows an example of an execution environment for implementing the system and method of the present disclosure in one embodiment. A server 1010 and multiple edge terminals 1020 are connected through a network. In FIG. 1, the server 1010 is represented as a single piece of hardware, but may be distributed across multiple pieces of physical hardware. The server 1010 may be a cloud server or an on-premise server. The edge terminal 1020 is represented as multiple terminals, but may theoretically be a single terminal. However, when using the server 1010, it is often the case that multiple users share the server, so typically, the system has multiple edge terminals 1020. The network may be wired, wireless, or a combination of both.
[0024] FIG. 2 is a modified example of FIG. 1, and the system further includes a repeater 1030 and a sensor 1040. The edge terminal 1020 is connected to the repeater 1030 via another network (second network). This repeater 1030 can transmit information from the sensor 1040 to the edge terminal 1020 through the second network. Then, based on the information from the sensor 1010, anomaly detection and anomaly identification can be performed. Although FIG. 2 shows one each of the repeater 1030 and the sensor 1040, there may be a plurality of these. For example, there may be a plurality of repeaters 1030 corresponding to one edge terminal. Furthermore, there may be a plurality of sensors 1040 corresponding to one repeater 1030.
[0025] In FIG. 2, the sensor 1040 is connected to the repeater 1030, but may be directly connected to the edge terminal 1020 as necessary.
[0026] [2-2. Overview of information processing device] The above-mentioned server 1010 and edge terminal 1020 are typically information processing devices (or computing devices). The above-mentioned repeater 1030 may also be an information processing device.
[0027] The information processing device may typically include a processor, a memory, a non-transitory storage medium, and a communication module, as shown in FIG.
[0028] Examples of information processing devices include, but are not limited to, the following: servers, personal computers, single-board computers (e.g., Raspberry Pi, etc.), tablet terminals, smartphones, smart watches, smart glasses, etc.
[0029] In the information processing device, the program is stored in a non-transitory storage medium (e.g., HDD, SSD, etc.), loaded into a memory (e.g., RAM, etc.) as appropriate, and executed by a processor (e.g., an electronic processor such as a CPU). If necessary, the program can connect to a network through a communication module to transmit and receive information.
[0030] In one embodiment, the program may be installed as application software on one information processing device and executed by the information processing device.
[0031] In another embodiment, the number of information processing devices is not limited to one, and multiple information processing devices may be used as necessary. In this case, the functions of the program may be distributed to the multiple information processing devices. Alternatively, the program may be installed in some information processing devices, and the other information processing devices may execute the program via a network.
[0032] [2-3. Arrangement of the first judgment means, the second judgment means, and the integrated judgment means] In one embodiment, the first determination means, the second determination means, and the integrated determination means may be implemented by an information processing device in which a desired program is installed. The first determination means, the second determination means, and the integrated determination means may be implemented in the server 1010 shown in Fig. 1 and Fig. 2, or in any edge terminal 1020.
[0033] For example, when the first determination means, the second determination means, and the integrated determination means are implemented in the edge terminal 1020, the server 1010 may manage the versions of the programs and the like related to these means. The edge terminal 1020 may update at least a part of the programs and the like by downloading them from the server 1010 as appropriate.
[0034] The first determination means, the second determination means, and the integrated determination means may be implemented in the same information processing device, or in some cases, may be implemented in multiple information processing devices. In the case where they are implemented in multiple information processing devices, the first determination means may be implemented in one information processing device, the second determination means may be implemented in another information processing device, and the integrated determination means may be implemented in yet another information processing device.
[0035] Alternatively, any of the first judgment means, the second judgment means, and the integrated judgment means may be further modularized, with some modules of the first judgment means, the second judgment means, or the integrated judgment means being implemented in one information processing device, and another part of the modules of the first judgment means, the second judgment means, or the integrated judgment means being implemented in another information processing device.
[0036] When the first determination means, the second determination means, and the integrated determination means are implemented in the server 1010, they may be distributed and implemented in a plurality of servers 1010, similar to the edge terminal 1020 described above.
[0037] Furthermore, when a plurality of servers 1010 are used, the first server 1010 and the second server 1010 may be connected to another network, not shown in Figs. 1 and 2. The first server 1010 may be connected to the network shown in Figs. 1 and 2 and have a function of accepting a request from the edge terminal 1020 (for example, a web server). The first server 1010 may make an inquiry to the second server 1010 based on the request from the edge terminal 1020 and transmit the inquiry result to the edge terminal 1020. In this case, the second server 1010 is implemented with a first determination means, a second determination means, and an integrated determination means, and the second server 1010 can output the inquiry result based on these means.
[0038] (3. First Determination Means) FIG. 4 shows the configuration of the first determination means in one embodiment. The first determination means includes a plurality of normal models that have been trained with learning data related to normal states. The plurality of normal models are all trained models. The plurality of normal models also include a plurality of sub-normal models (sub-normal models 2011 to 201N, where N is any integer equal to or greater than 2) that are suitable for detecting different abnormal states. The sub-normal models are normal models that are suitable for detecting or identifying individual abnormal states. The first determination means may further include a main normal model 2010 for comprehensively detecting abnormal states without being limited to individual abnormal states. The main normal model 2010 is a normal model for detecting some abnormal state (i.e., without limiting the type of abnormal state). The main normal model 2010 is effective for improving the accuracy of abnormality detection, for example, when a finite number of abnormal models cannot cover all abnormal states. On the other hand, the sub-normal models 2011 to 201N are each trained with different learning data so as to be suitable for identifying different abnormal states. One method for training the sub-normal model is to select a subset of the training data set that is suitable for identifying individual abnormal states. Note that, hereinafter, when simply referring to a "normal model," it can mean both the main normal model and the sub-normal model.
[0039] In a preferred embodiment, the sub-normal models 2011-201N are trained by extracting features suitable for detecting different abnormal states from training data related to normal states. That is, the sub-normal models 2011-201N are configured to be suitable for detecting different abnormal states. The features are not particularly limited as long as they can distinguish between different abnormal states, but in the case of a neural network, they may be bottleneck features.
[0040] Also, preferably, the first determination means includes an input unit 2020 and an output unit 2030. The input unit 2020 can receive data such as data of a monitoring target from the outside and input the data to a plurality of normal models. In some cases, the input unit 2020 can perform preprocessing on the received data from the outside and input the preprocessed data to a plurality of normal models. The output unit 2030 can receive outputs from a plurality of normal models and transmit them to the outside.
[0041] As described above, the term "trained model" is not particularly limited in terms of the type of trained model. However, the multiple normal models 2010 to 201N in the first determination means are constructed by training learning data related to normal states. Therefore, the first determination means mainly functions as an anomaly detection means.
[0042] Suitable trained models for the purpose of constructing training data regarding normal conditions through training include neural networks, hidden Markov models, or combinations of these.
[0043] The data output from the normal models 2010 to 201N is a score, probability, or likelihood for each normal model based on the data of the monitoring target. In this embodiment, a case will be described in which the first judgment means outputs a score for each normal model. The score is a measure of the degree to which the normal model is suitable for judging an abnormality. For example, in the case of an autoencoder that is widely used for anomaly detection, a neural network is trained so that the output data reproduces the input data, so that the score can be the value of an error function that indicates how far the output data is from the input data. Here, the mean square error is generally used as the error function. In general, the larger the score, the higher the possibility that an abnormality has occurred or will occur in the future.
[0044] Furthermore, in some preferred embodiments of the present invention, the sub-normal models 2011 to 201N may include at least one normal model with at least one kind of environmental noise superimposed thereon. That is, the normal model may be trained using training data obtained by superimposing data related to environmental noise on data related to a normal state.
[0045] The configuration of the first determination means in a preferred embodiment of the present invention is shown in Fig. 5. In this configuration, the input unit 2020 and the output unit 2030 are configured as described above, but the multiple sub-normal models 2011-201N are superimposed on environmental noises 1-M (M is an integer equal to or greater than 1), respectively, to obtain main normal models 2010-1-2010-M, sub-normal models 2011-1-201N-1, sub-normal models 2011-2-201N-2, etc.
[0046] Here, the reason why data on environmental noise is superimposed on data on the normal state is that various environmental noises may exist in the place where the monitoring target exists. In order to ensure the accuracy of judging an abnormality from the normal model, it is necessary to observe abnormal data that is sufficiently far from the normal state, but if environmental noise exists, it itself will also constitute a part of the abnormal data, and it may be difficult to evaluate how far the observed data actually differs from the normal state. Therefore, by superimposing environmental noise on normal data and learning the normal model, it becomes easier to evaluate how far the abnormal data differs from the normal state, and the accuracy of the judgment increases. The type and number of environmental noises can be set according to the specific environment in which the monitoring target exists. For example, in the case of voice data, voice data of noise present in the environment can be superimposed on voice data in a normal state. Note that when superimposing data, it is desirable to add at an appropriate ratio based on the ratio of observed data to environmental noise expected in the actual usage environment, that is, the so-called S / N ratio. In addition, when the expected S / N ratio has a certain range, the normal model may be learned by combining data on which environmental noise is superimposed at multiple S / N ratios.
[0047] (4. Second Judgment Means) The configuration of the second determination means in one embodiment is shown in Fig. 6. The second determination means includes a plurality of abnormal models 3011-301N (N is any integer equal to or greater than 2. Note that the value of N does not necessarily have to be the same in the first determination means and the second determination means) that have been trained with learning data relating to different abnormal states. The plurality of abnormal models 3011-301N are all trained models. The second determination means is configured to output a score, probability, or likelihood for an abnormal state corresponding to each of the plurality of abnormal models 3011-301N. In this embodiment, a case will be described in which the second determination means outputs a probability or likelihood for an abnormal state corresponding to each of the plurality of abnormal models 3011-301N.
[0048] The plurality of anomaly models 3011 to 301N are specialized for different causes of anomaly. That is, when there are a plurality of possible causes of anomaly, data specialized for each cause can be separately trained to construct a plurality of trained models. For example, when there are three types of elements A, B, and C that may cause anomaly, an anomaly model 3011 specialized for determining whether an anomaly is caused by element A, an anomaly model 3012 specialized for determining whether an anomaly is caused by element B, and an anomaly model 3013 specialized for determining whether an anomaly is caused by element C may be constructed. Then, a predetermined value can be input, and the probability or likelihood of an anomaly caused by element A, the probability or likelihood of an anomaly caused by element B, the probability or likelihood of an anomaly caused by element C, and the like can be output, respectively.
[0049] In another preferred embodiment, a plurality of anomaly models 3011-301N may be used to determine the probability or likelihood of one cause. That is, different types of anomaly models may be combined to comprehensively determine the output results of each (parallel combination). Alternatively, different types of anomaly models may be combined to receive the output result of one anomaly model from another anomaly model, which then outputs yet another output result (serial combination).
[0050] As described later, since the second judgment means is intended to improve judgment accuracy by performing integrated judgment in combination with the first judgment means, it is preferable that, corresponding to a certain abnormal state, there exist a normal model suitable for that abnormal state in the first judgment means and an abnormal model suitable for that abnormal state in the second judgment means. In one embodiment of the present invention, there exists a normal model in the first judgment means corresponding to all of the abnormal models in the second judgment means.
[0051] In a preferred embodiment of the present invention, the second determination means includes at least one normal model 3110. If the second determination means includes only abnormal models, when the monitored object is in a normal state, the probability or likelihood of any abnormal model may be low. In that case, it is not necessary to identify the abnormality, and it is sufficient to output that the monitored object is normal. Therefore, by including at least one normal model in the second determination means, it becomes easier to estimate whether the monitored object is normal or not depending on whether the probability or likelihood of the normal model is high or low. Note that the normal model 3110 in the second determination means is preferably the same as the main normal model 2010 in the first determination means, but may be different.
[0052] Also, preferably, the second determination means includes an input unit 3020 and an output unit 3030. The input unit 3020 can receive data such as data to be monitored from the outside and input the data to the multiple anomaly models 3011-301N. In some cases, the input unit 3020 can perform preprocessing on the data received from the outside and input the preprocessed data to the multiple anomaly models 3011-301N. The output unit 3030 can receive outputs from the multiple anomaly models 3011-301N and transmit them to the outside.
[0053] As described above about the term "trained model," the type of trained model is not particularly limited. However, the plurality of abnormality models 3011 to 301N in the second determination means are constructed by training learning data related to abnormal states. Therefore, the second determination means mainly functions as an abnormality identification means.
[0054] Suitable trained models for the purpose of constructing training data regarding abnormal conditions through training include neural networks, hidden Markov models, or combinations of these.
[0055] The data output from the multiple anomaly models 3011 to 301N is the probability or likelihood that an anomaly (or a sign of an anomaly) is a specific cause based on the data of the monitored object. The probability or likelihood is a measure of the possibility that an anomaly has occurred or will occur due to the cause, and in the case of a hidden Markov model, for example, these are probabilities or vectors that are determined and calculated by parameters of a trained model that shows a strong reaction according to the characteristics of the data of the monitored object by inputting the data of the monitored object to a trained model in a state in which parameters suitable for anomaly judgment are estimated by using the Baum-Welch algorithm or the like.
[0056] (5. Integrated judgment means) The integrated judgment means outputs a judgment result including information on a normal state or at least one abnormal state based on the output from the first judgment means and the output from the second judgment means. In this embodiment, a case will be described in which the integrated judgment means outputs a judgment result based on the score output from the first judgment means and the probability or likelihood output from the second judgment means. In this embodiment, the integrated judgment means outputs a judgment result including a normal state or a probability or likelihood of at least one abnormal state. Preferably, the judgment result may include the normal state and the probabilities or likelihoods of multiple abnormal states corresponding to all of the multiple abnormal models.
[0057] In a preferred embodiment of the present invention, the integrated judgment means is configured to weight the score output from the first judgment means with the probability or likelihood output from the second judgment means and output a judgment result. Although all the scores of the normal models may be used for outputting the judgment result, in some embodiments of the present invention, only the top few scores may be used.
[0058] By the above operation by the integrated judgment means, a normal model (which may be the main normal model or any of the sub-normal models) that is more suitable for the data to be monitored, i.e., a normal model that is farther away from the data to be monitored, can obtain a higher score, so that by weighting the normal model with a higher score, anomaly detection can be performed with higher accuracy, and by combining with different anomaly models, it is possible to output a result including not only the presence or absence of anomaly but also the type of anomaly. The output of the judgment result may include, for example, the probability or likelihood of normality, and the probability or likelihood of each anomaly.
[0059] The above operations by the integrated determination means will be described in detail later.
[0060] (6. Flow of outputting the judgment result) A flow of a process for outputting a judgment result using the above-mentioned first judgment means, second judgment means, and integrated judgment means according to an embodiment of the present invention will be described below. Fig. 7 shows a flow of a process according to an embodiment of the present invention. The method includes at least the following steps: A step (10) in which a first determination means and a second determination means each receive data to be monitored; A step (201) in which a first determination means outputs a score, probability, or likelihood for each of a plurality of normal models from the monitored data; A step (202) in which the second determination means outputs a score, or a probability or likelihood, for an abnormal state corresponding to each of a plurality of anomaly models from the monitored data; and A step (30) in which the integrated determination means outputs a determination result including information on a normal state or at least one abnormal state based on the output from the first determination means and the output from the second determination means.
[0061] In the following description, a case will be described in which the first determination means outputs a score for each of a plurality of normal models, and the second determination means outputs a probability or likelihood for an abnormal state corresponding to each of a plurality of abnormal models.
[0062] [6-1. Step (10) of receiving monitored data] The data to be monitored is input to both the first determination means and the second determination means. The type of data to be monitored is not particularly limited, and may be data in any format.
[0063] Examples of data may include at least one selected from the following: image data (e.g., still image data, video data, etc.), audio data (e.g., human voice, material sound, and a mixture of both), and time series data (e.g., data recording the temporal changes in values obtained from a sensor, etc.).
[0064] The data to be monitored may be derived from, for example, data obtained from the sensor 1040 shown in Fig. 2. Moreover, the data to be monitored does not need to be raw data obtained from the sensor 1040 shown in Fig. 2, and may be data obtained through processing such as A / D conversion, appropriate filtering, and feature extraction.
[0065] In another embodiment, the monitored data does not necessarily have to originate from data obtained from the sensor 1040. For example, time series data on the volume of commercial transactions on the Internet does not originate from a specific sensor 1040, but is data obtained by aggregating buy and sell orders from an electronic commerce site, and does not use the sensor 1040. In this embodiment, such time series data obtained without using the sensor 1040 may also be included in the monitored data.
[0066] [6-2. Step of outputting the score for the normal model (201)] The first judgment means calculates a score for each normal model from the input data of the monitoring target. The calculation of the score can be performed by the method described above. When the input data is normal, the scores for each normal model tend to be low overall, whereas when the input data corresponds to some abnormal state, the scores tend to be high overall, and in particular, the scores for the sub-normal models corresponding to the abnormal state tend to be higher. As a result, while a single normal model can only judge whether the data is normal or not, the sub-normal models can be used to estimate which abnormal state the input data is closer to. Furthermore, by integrating the output of the second judgment means, the judgment accuracy can be improved compared to using only a single judgment means.
[0067] [6-3. Step of outputting probability or likelihood for abnormal state (202)] The second determination means calculates the probability or likelihood of an abnormal state corresponding to each of the multiple anomaly models from the input data of the monitored object. The calculation of the probability or likelihood can be performed by the method described above. By calculating the probability or likelihood of an abnormal state corresponding to each of the multiple anomaly models, it is possible to estimate which abnormal state the input data is closest to. However, since the first determination means and the second determination means have different feature amounts extracted from the input data and different determination methods, the determination results do not necessarily become the same, and the determination accuracy can be improved by integrating the respective outputs as described below.
[0068] Incidentally, steps (201) and (202) can be performed serially, but are preferably performed in parallel.
[0069] [6-4. Step (30) for outputting the judgment result] The integrated judgment means outputs a judgment result including information on a normal state or at least one abnormal state based on the score output from the first judgment means and the probability or likelihood output from the second judgment means. The information on the at least one abnormal state may include the type of the abnormal state, the score of the abnormal state, or the probability or likelihood of the abnormal state. For example, if it is determined that the abnormal state is not a normal state, it can output a plurality of probabilities or likelihoods of abnormal states corresponding to a plurality of abnormal models. In a preferred embodiment of the present invention, the integrated judgment means includes a step (301) of judging whether or not the state is normal based on the score output from the first judgment means, and a step (302) of identifying at least one abnormal state by weighting the score with the probability or likelihood output from the second judgment means when it is determined that the state is abnormal. In a further preferred embodiment of the present invention, the second judgment means further includes at least one normal model, and the step (301) includes judging whether or not the state is normal based on both the score output from the first judgment means and the probability or likelihood of the normal state output for the at least one normal model.
[0070] An example of a method for calculating the judgment result is shown below.
[0071] A flowchart of a preferred embodiment of the method of the present invention is shown in Figure 8. In the normality / abnormality judgment step (301), the integrated judgment means first calculates a representative value sr from each score {s0, s1, s2, s3 ...} output by the first judgment means, multiplies this by the inverse 1 / y0 of the probability or likelihood y0 for a normal state output by the second judgment means, and compares the result with a predetermined threshold value. If the value is less than the threshold value, it is judged to be normal, and if it is equal to or greater than the threshold value, it is judged to be some kind of abnormality. Here, the representative value sr can be, for example, the average value of the scores {s0, s1, s2, s3 ...} or the maximum value s maxIt is preferable to use the above formula. If there is no main normal model, s0 can be excluded from the calculation. If y0 is 0, it may be determined that there is an abnormality directly. It may also be determined by directly comparing the representative value sr with a threshold value without multiplying it by 1 / y0. If it is determined that there is a normality, the normality is output as the determination result. In another embodiment of the present invention, the probability or likelihood of normality may be calculated and output.
[0072] If it is determined that there is an abnormality in the normality / abnormality determination step (301), the next step (302) is to calculate an integrated score {z1, z2, z3...} for each abnormality state by multiplying the scores {s1, s2, s3...} for each sub-normal model output by the first determination means by the probability or likelihood {y1, y2, y3...} for each abnormality state output by the second determination means (e.g. z1=s1×y1). Then, the abnormality state that has the maximum value among the integrated scores {z1, z2, z3...} for each abnormality state is output as the determination result (identification of the abnormality type). Alternatively, all integrated scores can be output to indicate the probability or likelihood of multiple abnormalities corresponding to all of the multiple abnormality models. Note that in {s1, s2, s3...} and {y1, y2, y3...}, the same numbers correspond to the same abnormality state. In this case, there is one abnormal model in the second judging means for one sub-normal model in the first judging means corresponding to each individual abnormal state.
[0073] Here, the purpose of multiplying the score for each sub-normal model by the probability or likelihood for each abnormal state to obtain an integrated score is to improve the accuracy of the determination by combining the output result of the first determination means and the output result of the second determination means. For example, even if the input data is actually the second abnormal state, the probability or likelihood y2 and y3 output by the second determination means are the same value, and the second determination means alone cannot correctly determine the state. If the score s2 output by the first determination means is greater than s3, a more accurate determination can be made by combining the scores.
[0074] Furthermore, in another embodiment of the present invention, the integrated judgment means may be configured to calculate a conversion score for each score {s0, s1, s2, s3 ...} output by the first judgment means prior to the normal / abnormal judgment step (301), in the following manner, and use the conversion score in the normal / abnormal judgment step (301) and the abnormality identification step (302).
[0075] First, a normalized score is calculated from each score {s0, s1, s2, s3...} output by the first judgment means using normalization parameters calculated in advance for each normal model. For example, for score s0 for the main normal model, a normalized score is calculated as sn0=(s0-M0) / S0 using normalization parameters (M0, S0) of the main normal model. Similarly, normalized scores sn1, sn2, sn3... are calculated for each sub-normal model score {s1, s2, s3...}.
[0076] The normalization parameters of each normal model use the average and standard deviation calculated from the scores for each learning sample used in learning. As a result, when the value of each score output by the first determination means is average for the corresponding normal model, the normalized score takes a value close to 0, making it possible to more appropriately compare the magnitude relationship of each score.
[0077] Next, each normalized score {sn0, sn1, sn2, sn3...} is input to a sigmoid function Sig(x)=1 / (1+exp(-x)) to obtain a converted score {s'0, s'1, s'2, s'3...}. This allows the converted score to take a value greater than 0 and less than 1, making it easy to handle probabilistically. Of the above scores, s0 and s'0 are related to the main normal model, and if there is no main normal model, it is not necessary to find the converted score s'0.
[0078] As described above, the present invention can improve the accuracy of detecting and identifying anomalies by integrating the outputs from the first and second determination means, making it possible to detect anomalies at an earlier stage.
[0079] (7. How to build a trained model) In the above, at least two of each of the two types of normal models and abnormal models are used. The normal model is a model trained with learning data related to a normal state. The abnormal model is a trained model trained with learning data related to an abnormal state. Note that the normal model and the abnormal model may be based on either supervised learning or unsupervised learning, and are not limited thereto.
[0080] A trained model can be constructed by the following procedure. The following describes an example using time-series voice data.
[0081] First, time-series data of voice in a normal state is prepared, and this is used as training data to train the main normal model. Next, feature amounts (e.g., band, window size) suitable for detecting different abnormal states are extracted from the training data, and each sub-normal model is trained on the extracted data.
[0082] FIG. 9 shows an example of the difference in learning data between the main normal model and the sub-normal model. When information on individual abnormal phenomena (abnormal state 1, abnormal state 2, etc.) related to the band f is available as prior knowledge, the main normal model 2010 performs model learning by extracting acoustic features from all bands without using the prior knowledge information. Next, acoustic features are extracted limited to frequency bands suitable for identifying individual abnormal states, and the sub-normal model 2011 and the like are trained. In this example, the abnormal state 1 has more low-frequency components than the normal state, and the abnormal state 2 has more high-frequency components than the normal state. Therefore, when data corresponding to the abnormal state 1 is observed, the data is farther away from the sub-normal model 2011, and the sub-normal model 2011 is more likely to detect the abnormal state 1 than the other sub-normal models. Then, by performing identification using an abnormal model corresponding to the abnormal state 1, the probability or likelihood of the abnormal state 1 can be obtained with higher accuracy.
[0083] In addition, when environmental noise is superimposed, the normal model can be trained by superimposing the environmental noise on the data corresponding to abnormal states 1 and 2. When there are multiple types of environmental noise, a set of normal models of the corresponding number should be prepared.
[0084] It should be noted that this example is based on the assumption that abnormal sound samples are obtained for each abnormal state to the extent that the frequency spectrum trend can be understood. In a specific implementation, it may be necessary to select or add other features.
[0085] For the anomaly models, time-series data of voice in a certain abnormal state is prepared, and each anomaly model is trained on the time-series data of voice in the abnormal state.
[0086] 8. Additional Embodiments The above describes a method of integrated judgment based on the score output from the first judgment means and the probability or likelihood output from the second judgment means, and a system for implementing the method. However, in another embodiment of the present invention, the first judgment means can be configured to output a probability or likelihood for each of a plurality of normal models, and the second judgment means can be configured to output a score for an abnormal state corresponding to each of a plurality of abnormal models, and the integrated judgment means can be configured to output a judgment result including the probability or likelihood of a normal state or at least one abnormal state, based on the output of the probability or likelihood from the first judgment means and the output of the score from the second judgment means.
[0087] That is, in the case of the above embodiment, the method of the present invention includes, following the above-mentioned step (10), a step (201A) of the first judgment means outputting a probability or likelihood for each of a plurality of normal models from the data of the monitored object, a step (202A) of the second judgment means outputting a score for an abnormal state corresponding to each of a plurality of abnormal models from the data of the monitored object, and a step (30A) of the integrated judgment means outputting a judgment result including the probability or likelihood of a normal state or at least one abnormal state based on the output of the probability or likelihood from the first judgment means and the output of the score from the second judgment means.
[0088] In the above embodiment, the step (30A) of outputting the judgment result includes a step (301A) of judging whether or not the state is normal based on the output of the probability or likelihood from the first judgment means, and a step (302A) of weighting the score from the second judgment means by the output of the probability or likelihood from the first judgment means to identify at least one abnormal state if the state is judged to be abnormal. In this embodiment, it is also preferable that the second judgment means includes at least one normal model, and in this case, the step (301A) includes judging whether or not the state is normal based on both the output of the probability or likelihood from the first judgment means and the score output for the at least one normal model.
[0089] Furthermore, in another embodiment of the present invention, the first judgment means may be configured to output a score for each of the normal models, the second judgment means may be configured to output a score for each of the abnormal states corresponding to each of the abnormal models, and the integrated judgment means may be configured to output a judgment result including a score for a normal state or at least one abnormal state based on the score output from the first judgment means and the score output from the second judgment means. In this case, for example, the scores {s1, s2, s3...} for each sub-normal model output by the first judgment means are multiplied by the inverses {1 / y1, 1 / y2, 1 / y3...} of the scores {y1, y2, y3...} for each abnormal state output by the second judgment means to calculate integrated scores {z1, z2, z3...} for each abnormal state (e.g., z1=s1×1 / y1). Then, the abnormal state having the maximum value among the integrated scores {z1, z2, z3...} for each abnormal state is output as the judgment result (identification of the abnormality type). Alternatively, the combined scores can all be output to show the scores for multiple anomaly conditions corresponding to all of the multiple anomaly models.
[0090] Furthermore, in another embodiment of the present invention, the first judgment means can be configured to output a probability or likelihood for each of a plurality of normal models, and the second judgment means can be configured to output a probability or likelihood for an abnormal state corresponding to each of a plurality of abnormal models, and the integrated judgment means can be configured to output a judgment result including the probability or likelihood of a normal state or at least one abnormal state based on the output of the probability or likelihood from the first judgment means and the output of the probability or likelihood from the second judgment means.
[0091] In the above additional embodiment, other than the outputs from the first judgment means and the second judgment means being different and the format of the output result of the integrated judgment means being different, other features of the method and system are common to the previously described embodiment, and therefore further explanation will be omitted.
[0092] 9. FIELD OF APPLICATION OF THE PRESENT ART The method and system of the present invention can be utilized in a variety of industrial sectors.
[0093] As an example, the system and method of the above-described embodiment can be applied to a system that monitors the failure of parts of each vehicle of a train. Specifically, a material sound sensor can be provided near the parts of each vehicle to monitor the sound from the parts while the vehicle is running. Then, the normal state and the failure state of the parts (more specifically, the state immediately before the failure occurs) can be used as learning data to learn the normal model of the first determination means and the abnormal model of the second determination means, respectively. When environmental noise is superimposed, the sound of a traffic light while the vehicle is running, the sound of thunder, etc. can be used as the environmental noise. Then, these learned models can be implemented in an information processing device (edge terminal 1020 in FIG. 1 and FIG. 2) in, for example, each vehicle or a specific vehicle (such as the leading vehicle or the rear vehicle of the train). Data from each sensor 1040 can be transmitted to the information processing device by providing a repeater 1030 in each vehicle and using the repeater 1030.
[0094] In another example, the system and method of the above-described embodiment can be applied when monitoring disasters, weather changes, and the like using a surveillance camera. Specifically, the weather conditions on the mountain are monitored by a camera, and an information processing device (edge terminal 1020 in FIG. 1 and FIG. 2) at an observation station can receive image data using a repeater or the like. Then, a normal state of the weather (a state where the weather does not deteriorate) and an abnormal state (more specifically, a state just before the weather deteriorates) may be used as learning data to learn the normal model of the first determination means and the abnormal model of the second determination means, respectively. When environmental noise is superimposed, the sound of a radio broadcast or a person's screaming can be used as the environmental noise. Then, these learned models may be implemented, for example, in an information processing device.
[0095] In yet another example, data other than data from a sensor may be used. For example, the system and method of the above-described embodiment may be applied when monitoring human health using blood test data. For example, blood may be collected from a population for a specific period of time, and the health status of the population may be tracked. Then, the population may be classified into a group that had no health problems during the specific period, a group that suffered from a disease, etc., and the like. Furthermore, the groups may be further classified according to the type of disease, etc. The normal model of the first determination means and the abnormal model of the second determination means may be trained. Then, these trained models may be implemented in an information processing device (edge terminal in FIG. 1 and FIG. 2) for analyzing blood data.
[0096] Above, specific embodiments of the invention have been described. The above embodiments are merely examples, and the present invention is not limited to the above embodiments. For example, a technical feature disclosed in one of the above embodiments can be applied to other embodiments. In addition, unless otherwise specified, for a particular method, some steps can be replaced with other steps, and additional steps can be added between two specific steps. The scope of the present invention is defined by the claims. [Explanation of symbols]
[0097] 1010 Server 1020 Edge Terminal 1030 Repeater 1040 Sensor 2010 Main normal model 2011~201N Sub-normal model 2020 Input section 2030 Output section 3011~301N Abnormal Model 3020 Input section 3030 Output section 3110 Normal model
Claims
1. A method for predicting an anomaly in a monitored object using a system, comprising: The system includes a first determination means, a second determination means, and an integrated determination means; the first determination means includes a plurality of normal models trained with learning data relating to normal states, the plurality of normal models including a plurality of sub-normal models suitable for detecting different abnormal states; the second determination means includes a plurality of abnormality models trained with learning data relating to different abnormal states; The method comprises: A step (10) in which the first determination means and the second determination means each receive data to be monitored; The first determination means outputs a score, probability, or likelihood for each of the plurality of normal models from the monitored data (201); The second determination means outputs a score, probability, or likelihood for an abnormal state corresponding to each of the plurality of anomaly models from the monitored data (202); a step (30) of the integrated judgment means outputting a judgment result including information on a normal state or at least one abnormal state based on the output from the first judgment means and the output from the second judgment means; A method comprising:
2. The method of claim 1 , wherein the first determining means further comprises a main normal model suitable for comprehensive detection of abnormal conditions.
3. The method according to claim 1 or 2, wherein the plurality of sub-normal models are trained by extracting features suitable for detecting different abnormal states from training data relating to normal states.
4. The method according to claim 1 or 2, wherein the first determining means includes at least one normal model convoluted with at least one type of environmental noise.
5. The first determination means outputs a score for each of the plurality of normal models from the monitored data; the second determination means outputs a probability or likelihood of an abnormal state corresponding to each of the plurality of abnormality models from the data of the monitoring target; 3. The method of claim 1 or 2, wherein the step (30) includes a step (301) of determining whether a normal state is present based on the score output from the first determination means, and a step (302) of weighting the score with the probability or likelihood output from the second determination means if an abnormality is determined, to identify at least one abnormal state.
6. The second determination means further includes at least one normal model; 6. The method of claim 5, wherein the step (301) includes determining whether or not a normal state is present based on both the score output from the first determination means and a probability or likelihood of a normal state output for the at least one normal model.
7. The method of claim 1 or 2, wherein the determination result comprises a probability or likelihood of a normal state or a plurality of abnormal states corresponding to the plurality of abnormal models.
8. A system for predicting an abnormality in a monitored object, comprising: The system includes a first determination means, a second determination means, and an integrated determination means; the first determination means includes a plurality of normal models trained with learning data relating to normal states, the plurality of normal models including a plurality of sub-normal models suitable for detecting different abnormal states; the second determination means includes a plurality of abnormality models trained with learning data relating to different abnormal states; The first determining means and the second determining means are each configured to receive data to be monitored; The first determination means is configured to output a score, probability, or likelihood for each of the plurality of normal models from the monitored data; the second determination means is configured to output a score, or a probability or likelihood, for an abnormal state corresponding to each of the plurality of anomaly models from the data of the monitoring target; The integrated determination means is configured to output a determination result including information on a normal state or at least one abnormal state based on the output from the first determination means and the output from the second determination means. system.
9. The system of claim 8 , wherein the first determining means further comprises a main normal model suitable for comprehensive detection of abnormal conditions.
10. The system according to claim 8 or 9, wherein the plurality of sub-normal models are trained by extracting feature amounts suitable for detecting different abnormal states from training data relating to normal states.
11. The system according to claim 8 or 9, wherein the first determination means includes at least one normal model convoluted with at least one type of environmental noise.
12. the first determination means is configured to output a score for each of the plurality of normal models from the monitored data; the second determination means is configured to output a probability or likelihood of an abnormal state corresponding to each of the plurality of abnormality models from the data of the monitored object; 10. The system according to claim 8 or 9, wherein the integrated determination means is configured to determine whether a state is normal or not based on the score output from the first determination means, and if an abnormality is determined, to weight the score with the probability or likelihood output from the second determination means to identify at least one abnormal state.
13. 13. The system of claim 12, wherein the second determination means further includes at least one normal model, and the integrated determination means is configured to determine whether or not a normal state is present based on both the score output from the first determination means and a probability or likelihood of a normal state output for the at least one normal model.
14. The system of claim 8 or 9, wherein the determination result includes a probability or likelihood of a normal state or a plurality of abnormal states corresponding to the plurality of abnormality models.
Citation Information
Patent Citations
Method and device for monitoring state of facility
JP2011070635A
Discrimination system, discrimination device, learning device, discrimination method, and program
JP2019159770A
Information processing method, information processor, and program
JP2019204321A
Detecting device, detecting method, and detecting program
JP2020095425A
Information processing device, control method of information processing device and program
JP2020140518A
Cited By
Target parameter estimation method based on bounded nonlinear function in impulse noise environment
CN120761988A
