Detection device

Abstract

This detection device has an extraction unit that extracts a feature value from time-series data acquired from a system, and a detection unit that detects deterioration in abnormality detection performance on occasions for detecting abnormalities from the system, by comparing evaluation items corresponding to feature values in a time series acquired as a result of the extraction unit performing the extraction multiple times, with comparison items corresponding to feature values in a time series acquired in advance.

Description

Detection device

[0001] The present invention relates to a detection device, a detection method, and a recording medium.

[0002] Techniques used to detect anomalies based on time-series data are known.

[0003] For example, Patent Document 1 describes a time series data processing device that converts a time series data set, which is a collection of multiple time series data, into a feature vector that indicates characteristics of the time series data set, thereby enabling search. Specifically, Patent Document 1 describes a time series data processing device that includes a data conversion unit, a storage unit, and a search unit. For example, the data conversion unit converts partial time series data sets (series segments), which are obtained by dividing a time series data set, which is a collection of multiple time series data, into feature vectors (features) that indicate characteristics of the partial time series data set. Furthermore, the storage unit stores multiple first partial time series data sets and multiple first feature vectors converted from the multiple first partial time series data sets by the data conversion unit, in association with each other. Then, the search unit selects at least one first feature vector from the multiple first feature vectors stored in the storage unit that is similar to the second feature vector converted from the input second partial time series data set by the data conversion unit, and outputs the first partial time series data set corresponding to the selected first feature vector.

[0004] International Publication No. 2020 / 049666

[0005] When the system state changes without the occurrence of an anomaly, such as when the system configuration is changed, an unprecedented state may occur continuously, resulting in a continuously increasing anomaly score. Since such a state may degrade anomaly detection performance, it is desirable to appropriately detect it. However, with the technology described in Patent Document 1, it has sometimes been difficult to detect the above-described degradation of anomaly detection performance.

[0006] Therefore, an object of the present invention is to provide a detection device, a detection method, and a recording medium that can solve the above-mentioned problems.

[0007] In order to achieve this object, a detection device according to one embodiment of the present disclosure has a configuration including: an extraction unit that extracts features from time-series data acquired from a system; and a detection unit that detects deterioration in anomaly detection performance when detecting an anomaly in the system by comparing an evaluation object corresponding to the time-series features acquired by multiple extractions by the extraction unit with a comparison object corresponding to the time-series features acquired in advance.

[0008] In addition, a detection method according to another aspect of the present disclosure is configured such that an information processing device extracts features from time series data acquired from a system, and compares an evaluation object corresponding to the time series features acquired by multiple extractions with a comparison object corresponding to the time series features acquired in advance, thereby detecting a deterioration in anomaly detection performance when detecting an anomaly in the system.

[0009] Another aspect of the present disclosure is a computer-readable recording medium having recorded thereon a program for causing an information processing device to extract features from time-series data acquired from a system, and compare an evaluation object corresponding to the time-series features acquired by multiple extractions with a comparison object corresponding to the time-series features acquired in advance, thereby detecting deterioration in anomaly detection performance when detecting an anomaly in the system.

[0010] According to the above-mentioned configurations, the above-mentioned problems can be solved.

[0011] FIG. 1 is a diagram for explaining an overview of a detection system according to a first embodiment of the present disclosure. FIG. 2 is a diagram showing an example of an anomaly score according to a change in system state. FIG. 3 is a block diagram showing an example configuration of a detection device. FIG. 4 is a diagram showing an example of a reference histogram. FIG. 5 is a diagram for explaining an example of feature extraction and conversion. FIG. 6 is a diagram showing an example of processing by a distribution calculation unit. FIG. 7 is a diagram for explaining an example of calculation of a deviation. FIG. 8 is a flowchart showing an example operation of a detection device. FIG. 9 is a diagram for explaining an example effect. FIG. 10 is a diagram for explaining an example use of a detection device. FIG. 11 is a diagram showing an example hardware configuration of a second detection device in the present disclosure. FIG. 12 is a block diagram showing an example configuration of a detection device.

[0012] [First Embodiment] A first embodiment of the present invention will be described with reference to FIGS. 1 to 10. FIG. 1 is a diagram illustrating an overview of a detection system 100. FIG. 2 is a diagram illustrating an example of an anomaly score according to a change in system state. FIG. 3 is a block diagram illustrating an example of the configuration of a detection device 200. FIG. 4 is a diagram illustrating an example of a reference histogram. FIG. 5 is a diagram illustrating an example of feature extraction and conversion. FIG. 6 is a diagram illustrating an example of processing by a distribution calculation unit 254. FIG. 7 is a diagram illustrating an example of deviation calculation. FIG. 8 is a flowchart illustrating an example of the operation of the detection device 200. FIG. 9 is a diagram illustrating an example of an effect. FIG. 10 is a diagram illustrating an example of use of the detection device 200. Note that in the present disclosure, the drawings may be associated with one or more embodiments.

[0013] In a first embodiment of the present disclosure, as shown in FIG. 1 , a detection system 100 is described that detects anomalies using features extracted from time-series data and detects degradation of anomaly detection performance when detecting system anomalies using time-series features acquired by multiple extractions. As described below, the detection system 100 detects anomalies in a system using features extracted from time-series data. For example, the detection system 100 detects anomalies by, for example, calculating an anomaly score by comparing the extracted features with pre-stored features. The detection system 100 also compares an evaluation target corresponding to the time-series features with a comparison target corresponding to pre-acquired time-series features. The detection system 100 then detects degradation of anomaly detection performance based on the comparison result.

[0014] FIG. 2 shows an example of how an anomaly score is calculated when the system state changes. As shown in FIG. 2, when the system state changes, the anomaly score may continually increase due to the continuous occurrence of unprecedented states. In such a state, the anomaly score continually increases, resulting in a deterioration in anomaly detection performance. Therefore, the detection system 100 described in this disclosure accurately detects deterioration in anomaly detection performance by using time-series feature quantities, etc.

[0015] Specifically, for example, the detection system 100 acquires a time-series binary code as a time-series feature. The detection system 100 also calculates an evaluation histogram by calculating the distribution of the time-series binary code. The detection system 100 then detects a deterioration in anomaly detection performance by comparing the calculated evaluation histogram with a reference histogram calculated in advance as a comparison target. In this manner, the detection system 100 detects a deterioration in anomaly detection performance by, for example, comparing the evaluation histogram, which is an evaluation target based on the time-series feature, with the reference histogram, which is a comparison target. As an example, the detection system 100 may detect a deterioration in anomaly detection performance by, for example, calculating the degree of deviation between the evaluation histogram and the reference histogram.

[0016] The detection system 100 described in the present disclosure can extract features from time-series data using a model trained in advance by machine learning. For example, the detection system 100 uses time-series data for training to train a model that extracts and outputs features that preserve local distance relationships for input time-series segments, which are time-series data corresponding to a certain time window. As an example, the detection system 100 may train the model using a method such as that described in Patent Literature 1. The detection system 100 inputs time-series segments to the trained model and extracts features obtained as the model output. The detection system 100 can then obtain binary codes by performing a predetermined conversion process on the features. The detection system 100 can also obtain time-series features or time-series binary codes by extracting features from multiple consecutive time windows.

[0017] FIG. 1 shows an overview of a detection system 100. Referring to FIG. 1, the detection system 100 includes a detection device 200, which is an information processing device that performs anomaly detection using a search technique. As shown in FIG. 1, the detection device 200 can acquire time-series data from various sensors and other external devices. Here, the time-series data may be, for example, data in which numerical data, such as observation data measured by one or more sensors at predetermined intervals, is arranged in order of measurement time, as shown in FIG. 1.

[0018] The detection device 200 performs system anomaly detection using binary codes converted from feature quantities extracted from time-series segments. Furthermore, the detection device 200 uses the time-series feature quantities to detect degradation of anomaly detection performance when detecting system anomalies. FIG. 3 shows an example configuration of the detection device 200. Referring to FIG. 3, the detection device 200 has, as its main components, an operation input unit 210, a screen display unit 220, a communication interface unit 230, a storage unit 240, and a calculation processing unit 250, for example.

[0019] 3 illustrates an example in which the functions of the detection device 200 are realized using one information processing device. However, at least some of the functions of the detection device 200 may be realized using multiple information processing devices, for example, on the cloud. Furthermore, the detection device 200 may not include some of the components illustrated above, such as not having the operation input unit 210 or the screen display unit 220, or may have a configuration other than those illustrated above.

[0020] The operation input unit 210 is made up of operation input devices such as a keyboard, a mouse, etc. The operation input unit 210 detects operations of the operator who operates the detection device 200 and outputs the operations to the arithmetic processing unit 250.

[0021] The screen display unit 220 is composed of a screen display device such as a liquid crystal display, an organic electroluminescence (EL) display, etc. The screen display unit 220 can display various information stored in the storage unit 240 on the screen in response to instructions from the arithmetic processing unit 250.

[0022] The communication interface unit 230 is composed of a data communication circuit, etc. The communication interface unit 230 performs data communication with an external device connected via a communication line.

[0023] The storage unit 240 is a storage device such as a hard disk or memory. The storage unit 240 stores processing information and a program 244 required for various processes in the arithmetic processing unit 250. The program 244 is read into the arithmetic processing unit 250 and executed to realize various processing units. The program 244 is read in advance from an external device or recording medium via a data input / output function such as the communication interface unit 230, and is stored in the storage unit 240. Main information stored in the storage unit 240 includes, for example, model information 241, feature amount information 242, and reference histogram information 243.

[0024] The model information 241 includes information about a model that extracts and outputs features that preserve local distance relationships for input time series segments. The model information 241 may include weight parameters and the like included in the trained model described above. For example, the model included in the model information 241 is trained in advance using training time series data (time series segments) inside or outside the detection device 200 and stored in the storage unit 240. The model information 241 can also be updated by the adaptation unit 257, which will be described later, or the like.

[0025] The model included in the model information 241 may be any model capable of handling time-series data. For example, the model may be one of a 1D-CNN (1-Dimensional-Convolutional Neural Network), a GRU (Gated Recurrent Unit), a LSTM (Long Short-Term Memory), a Transformer, etc. Furthermore, the loss function used as the model training criterion may be any function that preserves the local similarity between data in the input space. For example, the loss function may be one of triplet loss, pairwise loss, contrastive loss, etc. As an example, the model included in the model information 241 may be trained using a method such as that described in Patent Document 1. For example, Patent Document 1 discloses an example of model training using triplet loss or pairwise loss.

[0026] The feature amount information 242 includes feature amount information corresponding to feature amounts extracted from time-series segments previously acquired during a period when no abnormalities occur, such as the learning period shown in FIG. 2 . For example, the feature amount information 242 includes, as feature amount information, binary code obtained by converting feature amounts using an arbitrary method. The binary code included in the feature amount information 242 is previously acquired using an arbitrary method, such as acquisition from an external device via the communication interface unit 230 or conversion by the binary code conversion unit 253 of feature amounts extracted by the feature amount extraction unit 252, and is stored in the storage unit 240. In addition, the feature amount information 242 can be updated in accordance with processing by the adaptation unit 257, which will be described later, or the like.

[0027] The reference histogram information 243 includes a reference histogram, which is a histogram corresponding to the distribution of time-series binary codes previously acquired during a period that does not include model degradation or anomalies. In other words, the reference histogram information 243 includes a reference histogram, which is information to be compared when detecting degradation of anomaly detection performance. For example, the reference histogram included in the reference histogram information 243 is calculated in advance inside or outside the detection device 200 using time-series data (time-series segments) during a period that does not include model degradation or anomalies, and is stored in the storage unit 240. The reference histogram may be calculated in advance by the distribution calculation unit 254, which will be described later, or the like.

[0028] 4 shows an example of a reference histogram. For example, the reference histogram can be calculated by using a time-series binary code for which the reference histogram is to be calculated and tallying the number of 1s in each bit of the binary code (e.g., 256 bits). As an example, in FIG. 4, the x-axis corresponds to each bit, and the y-axis indicates the number of 1s in the corresponding bit. Note that the y-axis may indicate the percentage of 1s instead of the number of 1s.

[0029] The arithmetic processing unit 250 has an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 250 reads and executes a program 244 from the storage unit 240, thereby causing the above hardware and the program 244 to cooperate with each other to realize various processing units. Major processing units realized by the arithmetic processing unit 250 include, for example, a time-series data acquisition unit 251, a feature extraction unit 252, a binary code conversion unit 253, a distribution calculation unit 254, a deviation calculation unit 255, a deterioration detection unit 256, an adaptation unit 257, an anomaly detection unit 258, and an output unit 259.

[0030] In addition, the arithmetic processing unit 250 may have a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these, instead of the above-mentioned CPU.

[0031] The time-series data acquisition unit 251 acquires time-series data from various sensors included in the detection system 100 and other external devices. For example, the time-series data acquisition unit 251 may acquire, from one or more sensors included in the detection system 100, data in which numerical data such as observation data measured by each sensor at a predetermined period is arranged in order of measurement time, as time-series data.

[0032] The feature extraction unit 252 inputs the time series segments into a model stored as the model information 241, thereby extracting features corresponding to the time series segments. For example, as illustrated in FIG. 5 , the feature extraction unit 252 divides the time series data acquired by the time series data acquisition unit 251 into multiple time series segments using a time window of a certain period. Then, the feature extraction unit 252 inputs each divided time series segment into a trained model, thereby extracting features. Note that the feature extraction unit 252 may divide the time series data into the time series segments using any method. For example, the size of the time window may be set arbitrarily. Furthermore, the feature extraction unit 252 may divide the time series data into multiple time series segments so that they overlap for an arbitrary period, or may divide the time series data into multiple time series segments so that the time series segments do not overlap.

[0033] The feature extraction unit 252 can extract feature amounts when detecting an anomaly, etc. In addition to the above-described example, the feature extraction unit 252 may extract feature amounts during the learning period shown in FIG. 2 or the period for calculating the reference histogram described above.

[0034] The binary code conversion unit 253 converts the feature extracted by the feature extraction unit 252 into binary code, which is information corresponding to the feature. In the present disclosure, the method of conversion into binary code is not particularly limited. The binary code conversion unit 253 may convert the feature extracted by the feature extraction unit 252 into binary code using any method. For example, the binary code conversion unit 253 can convert the feature into a 256-bit binary code.

[0035] The distribution calculation unit 254 calculates the distribution of time-series binary codes. For example, the distribution calculation unit 254 calculates the distribution of time-series binary codes that are extracted and converted multiple times when performing anomaly detection and are to be detected for degradation of anomaly detection performance.

[0036] 6, the distribution calculation unit 254 counts the number of times that 1 is set for each bit using a time-series binary code that is the target of detection for anomaly detection performance degradation. In other words, the distribution calculation unit 254 calculates the distribution by counting the number of times that 1 is set for each bit using a time-series binary code that corresponds to a plurality of consecutive time windows. As a result, the distribution calculation unit 254 can calculate an evaluation histogram that is the target of evaluation to be compared with the reference histogram that is the target of comparison described above.

[0037] The number of binary codes used to calculate the reference histogram and the number of binary codes used to calculate the evaluation histogram may be the same or different. For example, by setting the y-axis of the reference histogram and the evaluation histogram to a ratio of 1's, it is possible to compare the reference histogram and the evaluation histogram without any problems even if the number of binary codes used to calculate the reference histogram and the number of binary codes used to calculate the evaluation histogram are different.

[0038] 7, the deviation calculation unit 255 compares the evaluation histogram, which is the evaluation target calculated by the distribution calculation unit 254, with a reference histogram included in the reference histogram information 243, which is stored in advance as a comparison target. For example, the deviation calculation unit 255 can perform the comparison by calculating the deviation between the evaluation histogram and the reference histogram.

[0039] As an example, the divergence calculation unit 255 calculates Kullback-Leibler Divergence or the like as the divergence. The divergence calculation unit 255 may calculate any index other than Kullback-Leibler Divergence that can quantify the similarity between histograms as the divergence. For example, the divergence calculation unit 255 may calculate a chi-square test, Jensen-Shannon Divergence, or the like as the divergence.

[0040] The degradation detection unit 256 detects degradation of the anomaly detection performance using the result of calculation by the deviation calculation unit 255. For example, the degradation detection unit 256 can detect degradation of the anomaly detection performance based on whether the deviation calculated by the deviation calculation unit 255 is equal to or greater than a predetermined threshold. As an example, the degradation detection unit 256 detects degradation of the anomaly detection performance when the deviation calculated by the deviation calculation unit 255 is equal to or greater than the predetermined threshold. On the other hand, the degradation detection unit 256 does not detect degradation of the anomaly detection performance when the deviation calculated by the deviation calculation unit 255 is less than the predetermined threshold. Note that the value of the threshold may be determined arbitrarily.

[0041] The adaptation unit 257 performs adaptive processing to mitigate degradation of the anomaly detection performance in accordance with the detection result by the degradation detection unit 256. For example, the adaptation unit 257 can perform adaptive processing when the degradation detection unit 256 detects degradation of the anomaly detection performance.

[0042] For example, as an adaptation process, the adaptation unit 257 performs a process of storing the binary code converted by the binary code conversion unit 253 in the storage unit 240 as feature information 242. For example, when detecting an anomaly, the binary code converted by the binary code conversion unit 253 is not stored in the storage unit 240 until the adaptation process by the adaptation unit 257 begins. On the other hand, when the degradation detection unit 256 detects degradation of the anomaly detection performance, the adaptation unit 257 stores the binary code converted by the binary code conversion unit 253 in the storage unit 240 as feature information 242. Note that the adaptation unit 257 can perform additional storage process until an arbitrary criterion is satisfied. As an example, after detecting degradation of the anomaly detection performance, the adaptation unit 257 may additionally store binary code until a predetermined number of additional storages have been performed.

[0043] Furthermore, after detecting a deterioration in anomaly detection performance, the adaptation unit 257 may update the model used to extract features by storing the time series segments in the storage unit 240. For example, after detecting a deterioration in anomaly detection performance, if it is determined that a sufficient number of time series segments to be used as learning data have been obtained, the adaptation unit 257 can perform model adaptation to update the model information 241. In other words, the adaptation unit 257 can perform model adaptation by updating the weight parameters of the model using the time series segments acquired after detecting a deterioration in anomaly detection performance as learning data. The adaptation unit 257 may update the weight parameters using a learning criterion based on a distance learning loss using a triplet loss, a pairwise loss, or the like, as described in Patent Document 1.

[0044] The adaptation unit 257 can perform at least one of the above-described additional storage process of binary code and the model adaptation process as the adaptation process. The adaptation unit 257 may also perform both the above-described additional storage process of binary code and the model adaptation process as the adaptation process.

[0045] The anomaly detection unit 258 detects an anomaly in the system based on the feature amounts extracted from the time-series segment to be detected. The anomaly detection unit 258 can detect an anomaly in the system by calculating an anomaly score based on the extracted feature amounts.

[0046] For example, the anomaly detection unit 258 acquires, from the binary code conversion unit 253, a binary code corresponding to a feature extracted from a time-series segment to be detected. Then, the anomaly detection unit 258 searches the feature information 242 for a binary code similar to the acquired binary code. In the present disclosure, the anomaly detection unit 258 may perform the search using any method. For example, the anomaly detection unit 258 calculates the distance between the acquired binary code and each binary code included in the feature information 242. Then, the anomaly detection unit 258 acquires, as a binary code similar to the acquired binary code, a binary code that satisfies any condition, such as a binary code with the smallest calculated distance among the binary codes included in the feature information 242.

[0047] The anomaly detection unit 258 also calculates an anomaly score according to the distance between the acquired binary code and the binary code searched for in the storage unit 240. The anomaly detection unit 258 then detects an anomaly based on the calculated anomaly score. For example, the anomaly detection unit 258 compares the calculated anomaly score with a predetermined anomaly threshold. The anomaly detection unit 258 then detects an anomaly when the calculated anomaly score exceeds the anomaly threshold. In this manner, the anomaly detection unit 258 calculates an anomaly score and detects an anomaly in the system based on the calculated anomaly score. The anomaly detection unit 258 may calculate the anomaly score using any method. For example, the anomaly detection unit 258 may calculate the anomaly score so that the longer the distance between the binary codes, the larger the anomaly score value. When calculating the anomaly score in this manner, a larger anomaly score indicates a higher likelihood of an anomaly occurring. The anomaly threshold to be compared with the anomaly score may be determined using any method.

[0048] The output unit 259 outputs the detection results of the deterioration detection unit 256 and the detection results of the abnormality detection unit 258. The output unit 259 may output the detection results to an external device via the communication interface unit 230, or may display the detection results on the screen display unit 220, etc.

[0049] The output unit 259 may be configured to output, in addition to the detection results by the deterioration detection unit 256 and the detection results by the anomaly detection unit 258, information used for deterioration detection such as a reference histogram, an evaluation histogram, and a deviation degree, and information used for anomaly detection such as an anomaly score. The output unit 259 may output information other than the above-mentioned examples.

[0050] The above is an example of the configuration of the detection device 200. Next, an example of the operation of the detection device 200 will be described with reference to FIG.

[0051] 8 shows an example of the operation of the detection device 200 when detecting a deterioration in anomaly detection performance. Referring to FIG. 8 , the feature extraction unit 252 extracts features from a time-series segment that is a target for anomaly detection. Furthermore, the binary code conversion unit 253 converts the features extracted by the feature extraction unit 252 into binary code, which is information corresponding to the features. In this way, the feature extraction unit 252 and the binary code conversion unit 253 extract features from the time-series segment that is a target for anomaly detection, and convert the extracted features into binary code (step S101).

[0052] The distribution calculation unit 254 calculates the distribution of the time-series binary code to calculate an evaluation histogram (step S102).

[0053] The deviation calculation unit 255 calculates the deviation between the evaluation histogram calculated by the distribution calculation unit 254 and the reference histogram included in the reference histogram information 243 stored in advance as a comparison target (step S103). For example, the deviation calculation unit 255 may calculate the Kullback-Leibler divergence or the like as the deviation between the evaluation histogram and the reference histogram.

[0054] The degradation detection unit 256 detects degradation of the anomaly detection performance using the result of the deviation calculation unit 255 (step S104). If the deviation calculated by the deviation calculation unit 255 is equal to or greater than a predetermined threshold (step S104, YES), the degradation detection unit 256 detects degradation of the anomaly detection performance (step S105). In response to this, the adaptation unit 257 performs adaptive processing to mitigate the degradation of the anomaly detection performance (step S106). The adaptation unit 257 may perform at least one of additional storage processing of binary code and model adaptation processing as the adaptive processing. On the other hand, if the deviation calculated by the deviation calculation unit 255 is less than the predetermined threshold (step S104, NO), the degradation detection unit 256 does not detect degradation of the anomaly detection performance. In this case, the adaptation unit 257 does not perform adaptive processing.

[0055] The above is an example of the operation of the detection device 200 when detecting deterioration in anomaly detection performance.

[0056] As described above, the detection device 200 includes a distribution calculation unit 254, a deviation calculation unit 255, and a deterioration detection unit 256. With this configuration, the deviation calculation unit 255 can calculate the deviation between the evaluation histogram calculated by the distribution calculation unit 254 and the reference histogram included in the reference histogram information 243 that is stored in advance as a comparison target. As a result, the deterioration detection unit 256 can detect deterioration in anomaly detection performance based on the deviation calculated by the deviation calculation unit 255. This makes it possible to accurately detect performance deterioration.

[0057] The detection device 200 also includes an adaptation unit 257. With this configuration, the adaptation unit 257 can perform adaptive processing in accordance with the detection result by the degradation detection unit 256. As a result, when degradation of anomaly detection performance is detected, it becomes possible to alleviate the degradation of anomaly detection performance and improve the anomaly detection performance.

[0058] FIG. 9 shows an example of the creation period of the reference histogram and the calculation results of the deviation. As shown in FIG. 9, the reference histogram can be calculated by the distribution calculation unit 254 or the like by using the time-series binary code converted by the binary code conversion unit 253 for a period that does not include model degradation or anomalies. Whether or not a period does not include model degradation or anomalies may be determined using any method. Also, referring to FIG. 9, it can be seen that the deviation calculated by the deviation calculation unit 255 does not increase in the case of short-term anomalies, but increases when anomaly detection performance deteriorates due to changes in the system state. Due to these characteristics, using the deviation can accurately detect deterioration of anomaly detection performance due to changes in the system state that are different from short-term anomalies.

[0059] The detection device 200 described in the present disclosure can be utilized to detect degradation of anomaly detection performance when detecting anomalies in an optical communication system such as the optical network 300 shown in FIG. 10 . For example, the time-series data acquisition unit 251 of the detection device 200 can acquire time-series data from an optical transponder included in the optical network 300, an optical performance monitor that measures the optical signal-to-noise ratio (OSNR), an amplifier, a switch, or any other sensor. As an example, the time-series data acquisition unit 251 can acquire at least some of information indicating received light intensity and transmitted light intensity, information corresponding to the state of the light itself, and other information indicating the performance of any optical communication, such as the optical signal-to-noise ratio, from the optical transponder. With this configuration, the detection device 200 can accurately detect degradation of anomaly detection performance in response to changes in the system configuration of the optical network 300.

[0060] In addition to the above-mentioned examples, the detection device 200 may also acquire time-series data from a plurality of sensors installed in a plant, a data center, a social infrastructure facility, etc. The detection device 200 may also acquire, as time-series data, biological data acquired by a sensor owned by a user in a healthcare-related system.

[0061] In the present disclosure, the detection device 200 calculates an evaluation histogram and compares the calculated evaluation histogram with a reference histogram to compare an evaluation target corresponding to time-series feature quantities with a comparison target corresponding to previously acquired time-series feature quantities. However, the detection device 200 may be configured to compare a time-series binary code with a comparison target time-series binary code without calculating an evaluation histogram or a reference histogram. For example, the detection device 200 may perform the above-described comparison by checking the degree to which each component of the time-series binary code and the comparison target time-series binary code match. Furthermore, the detection device 200 is not limited to comparing time-series binary codes, and may be configured to directly compare time-series feature quantities with comparison target time-series feature quantities. In this way, the evaluation target and comparison target may be time-series feature quantities, time-series binary codes, etc.

[0062] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a diagram illustrating an example of the hardware configuration of a detection device 400. Fig. 12 is a block diagram illustrating an example of the configuration of the detection device 400.

[0063] In a second embodiment of the present disclosure, a detection device 400 that is an information processing device that detects degradation in anomaly detection performance will be described. Fig. 11 shows an example of the hardware configuration of the detection device 400. Referring to Fig. 11, the detection device 400 has the following hardware configuration, as an example: a CPU (Central Processing Unit) 401 (arithmetic device); a ROM (Read Only Memory) 402 (storage device); a RAM (Random Access Memory) 403 (storage device); a group of programs 404 loaded into the RAM 403; a storage device 405 that stores the group of programs 404; a drive device 406 that reads and writes data from and to a storage medium 410 external to the information processing device; a communication interface 407 that connects to a communication network 411 external to the information processing device; an input / output interface 408 that inputs and outputs data; and a bus 409 that connects the various components.

[0064] 12 by the CPU 401 acquiring and executing the program group 404. The program group 404 is stored in advance in the storage device 405 or the ROM 402, for example, and is loaded into the RAM 403 or the like by the CPU 401 for execution as needed. The program group 404 may be supplied to the CPU 401 via the communication network 411, or may be stored in advance in the recording medium 410, and the drive device 406 may read out the program and supply it to the CPU 401.

[0065] 11 shows an example of the hardware configuration of the detection device 400. The hardware configuration of the detection device 400 is not limited to the above-described case. For example, the detection device 400 may be configured with only a part of the above-described configuration, such as excluding the drive device 406. Furthermore, the CPU 401 may be a GPU or the like exemplified in the first embodiment.

[0066] The extraction unit 421 extracts features from the time-series data acquired from the system.

[0067] The detection unit 422 detects a deterioration in the anomaly detection performance when detecting an anomaly in the system by comparing an evaluation object corresponding to the time series feature obtained by the extraction unit 421 through multiple extractions with a comparison object corresponding to the time series feature obtained in advance.

[0068] The above is an example of the configuration of the detection device 400.

[0069] As described above, the detection device 400 includes the extraction unit 421 and the detection unit 422. With this configuration, the detection unit 422 can detect a deterioration in anomaly detection performance when detecting an anomaly in the system by comparing an evaluation target corresponding to the time-series feature amounts acquired by multiple extractions by the extraction unit 421 with a comparison target corresponding to the time-series feature amounts acquired in advance. This makes it possible to accurately detect performance deterioration.

[0070] The above-described detection device 400 can be realized by incorporating a predetermined program into an information processing device such as the detection device 400. Specifically, a program according to another aspect of the present invention is a program for causing an information processing device such as the detection device 400 to implement processing of extracting features from time-series data acquired from a system, and comparing an evaluation target corresponding to the time-series feature values ​​acquired by multiple extractions with a comparison target corresponding to the time-series feature values ​​acquired in advance, thereby detecting deterioration in anomaly detection performance when detecting an anomaly in the system.

[0071] Furthermore, a detection method executed by an information processing device such as the detection device 400 described above is a method in which the information processing device extracts features from time-series data acquired from a system, and compares an evaluation object corresponding to the time-series features acquired by multiple extractions with a comparison object corresponding to the time-series features acquired in advance, thereby detecting deterioration in anomaly detection performance when detecting anomalies in the system.

[0072] Even if the invention is a program having the above-described configuration, or a computer-readable recording medium having the program recorded thereon, or a detection method, it can achieve the same functions and effects as the above-described detection device 400, and therefore can achieve the above-described objective of the present disclosure.

[0073] <Supplementary Notes> Part or all of the above-described embodiments can be described as follows: The detection device and other components of the present invention will be outlined below. However, the present invention is not limited to the following configuration.

[0074] (Supplementary Note 1) A detection device comprising: an extraction unit that extracts features from time-series data acquired from a system; and a detection unit that detects deterioration of anomaly detection performance when detecting anomalies in the system by comparing an evaluation object corresponding to the time-series features acquired by multiple extractions by the extraction unit with a comparison object corresponding to previously acquired time-series features. (Supplementary Note 2) The detection device according to Supplementary Note 1, wherein the extraction unit acquires binary codes obtained by converting the features extracted from the time-series data as the features, and the detection unit detects deterioration of anomaly detection performance by comparing an evaluation object corresponding to the time-series binary code converted from the features with a comparison object corresponding to previously acquired time-series binary code. (Supplementary Note 3) The detection device according to Supplementary Note 2, wherein the extraction unit calculates a distribution of the time-series binary code acquired by the extraction unit, and the detection unit detects deterioration of anomaly detection performance by comparing an evaluation object that is a calculation result by the distribution calculation unit with a comparison object that is a previously acquired binary code distribution calculation result. (Supplementary Note 4) The detection device according to any one of Supplements 1 to 3, including an adaptation unit that stores feature quantities extracted by the extraction unit in a storage device according to a detection result by the detection unit. (Supplementary Note 5) The detection device according to any one of Supplements 1 to 4, wherein the detection unit uses time-series feature quantities extracted in advance by the extraction unit in a period not including an anomaly as a comparison target according to the previously acquired time-series feature quantities. (Supplementary Note 6) The detection device according to any one of Supplements 1 to 5, wherein the extraction unit extracts feature quantities from time-series segments obtained by dividing time-series data, and the detection unit detects deterioration of anomaly detection performance by comparing an evaluation target according to the time-series feature quantities acquired from a plurality of consecutive time-series segments with a comparison target according to the previously acquired time-series feature quantities. (Supplementary Note 7) The detection device according to Supplementary Note 3, wherein the distribution calculation unit calculates an evaluation histogram by aggregating time-series binary codes as a distribution of the time-series binary codes, and the detection unit detects deterioration of anomaly detection performance by comparing the evaluation histogram, which is the calculation result by the distribution calculation unit, with a reference histogram calculated in advance.(Supplementary Note 8) The detection device according to Supplementary Note 7, wherein the detection unit calculates a deviation between the evaluation histogram and the reference histogram, and detects a deterioration in anomaly detection performance when the calculated deviation is equal to or greater than a predetermined threshold. (Supplementary Note 9) A detection method in which an information processing device extracts features from time-series data acquired from a system, and compares an evaluation object corresponding to the time-series features acquired by extracting them multiple times with a comparison object corresponding to the time-series features acquired in advance, thereby detecting a deterioration in anomaly detection performance when detecting an anomaly in the system. (Supplementary Note 10) A computer-readable recording medium having recorded thereon a program for causing an information processing device to implement a process of extracting features from time-series data acquired from the system, and comparing an evaluation object corresponding to the time-series features acquired by extracting them multiple times with a comparison object corresponding to the time-series features acquired in advance, thereby detecting a deterioration in anomaly detection performance when detecting an anomaly in the system.

[0075] Note that some or all of the configurations described in Supplementary Notes 2 to 8 that are dependent on the detection device described in Supplementary Note 1 may also be dependent in a similar dependent relationship on the detection method described in Supplementary Note 9 and the recording medium described in Supplementary Note 10. Furthermore, not limited to Supplementary Note 9 and Supplementary Note 10, some or all of the configurations described as Supplements may be similarly made dependent on various hardware, software, various recording means for recording software, or systems within the scope of the above-described embodiments.

[0076] Furthermore, the programs described in the above embodiments and appendices can be stored using various types of non-transitory computer-readable media and supplied to a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The programs may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can supply the programs to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0077] Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0078] REFERENCE SIGNS LIST 100 Detection system 200 Detection device 210 Operation input unit 220 Screen display unit 230 Communication interface unit 240 Storage unit 241 Model information 242 Feature information 243 Reference histogram information 244 Program 250 Arithmetic processing unit 251 Time series data acquisition unit 252 Feature extraction unit 253 Binary code conversion unit 254 Distribution calculation unit 255 Deviation calculation unit 256 Deterioration detection unit 257 Adaptation unit 258 Anomaly detection unit 259 Output unit 300 Optical network 400 Detection device 401 CPU 402 ROM 403 RAM 404 Program group 405 Storage device 406 Drive device 407 Communication interface 408 Input / output interface 409 Bus 410 Recording medium 411 Communication network 421 Extraction unit 422 Detection unit

Claims

1. A detection device having: an extraction unit that extracts features from time-series data acquired from a system; and a detection unit that detects deterioration in anomaly detection performance when detecting anomalies in the system by comparing an evaluation object corresponding to the time-series features acquired by the extraction unit through multiple extractions with a comparison object corresponding to the time-series features acquired in advance.

2. The detection device according to claim 1, wherein the extraction unit obtains binary codes obtained by converting the features extracted from the time-series data as the features, and the detection unit detects deterioration of anomaly detection performance by comparing an evaluation object corresponding to the time-series binary code obtained by converting the features with a comparison object corresponding to a time-series binary code obtained in advance.

3. A detection device as described in claim 2, further comprising a distribution calculation unit that calculates the distribution of the time-series binary code acquired by the extraction unit, and wherein the detection unit detects deterioration of anomaly detection performance by comparing an evaluation object, which is the calculation result by the distribution calculation unit, with a comparison object, which is the distribution calculation result of the binary code acquired in advance.

4. The detection device according to claim 1, further comprising an adaptation unit that stores the feature quantity extracted by the extraction unit in a storage device in accordance with the detection result by the detection unit.

5. The detection device according to claim 1, wherein the detection unit uses the time series feature values ​​extracted in advance by the extraction unit during a period not including an abnormality as a comparison target according to the time series feature values ​​acquired in advance.

6. The detection device according to claim 1, wherein the extraction unit extracts features from time series segments obtained by dividing time series data, and the detection unit detects deterioration of anomaly detection performance by comparing an evaluation object corresponding to time series features obtained from a plurality of consecutive time series segments with a comparison object corresponding to time series features obtained in advance.

7. The detection device according to claim 3, wherein the distribution calculation unit calculates an evaluation histogram by aggregating time-series binary codes as a distribution of the time-series binary codes, and the detection unit detects deterioration of anomaly detection performance by comparing the evaluation histogram, which is the calculation result by the distribution calculation unit, with a reference histogram calculated in advance.

8. The detection device according to claim 7, wherein the detection unit calculates the degree of deviation between the evaluation histogram and the reference histogram, and detects a deterioration in anomaly detection performance if the calculated degree of deviation is equal to or greater than a predetermined threshold value.

9. A detection method in which an information processing device extracts features from time-series data acquired from a system, and compares an evaluation object corresponding to the time-series features acquired by multiple extractions with a comparison object corresponding to the time-series features acquired in advance, thereby detecting deterioration in the anomaly detection performance of the system when detecting anomalies.

10. A computer-readable recording medium having recorded thereon a program for executing a process in an information processing device, which extracts features from time-series data acquired from a system, and compares an evaluation object corresponding to the time-series features acquired by multiple extractions with a comparison object corresponding to the time-series features acquired in advance, thereby detecting deterioration in the anomaly detection performance of the system when detecting anomalies.

Images