Learning device, learning method, and recording medium

Abstract

The generation unit (131) generates a model that calculates an abnormality score by learning data selected as unlearned data in the learning data. The selection unit (133) selects at least a part of data in the learning data for which the abnormality score calculated by the model generated by the generation unit (131) is equal to or higher than a threshold as unlearned data.

Description

TECHNICAL FIELD

[0001] The present application relates to a learning device, a learning method, and a recording medium. BACKGROUND

[0002] With the advent of the IoT era, a variety of devices are connected to the Internet in a variety of usage modes. For security measures for such IoT devices, research on a service session anomaly detection system for IoT devices, an intrusion detection system (IDS) has recently been popular.

[0003] In such an anomaly detection system, there is an anomaly detection system using a probability density estimator based on teacherless learning such as Variational Auto Encoder (VAE). The anomaly detection system using the probability density estimator generates high-dimensional data for learning called service feature amounts from actual communication, and learns the features of normal services using the feature amounts, whereby it is possible to estimate the occurrence probability of the normal communication pattern. In the following description, the probability density estimator will be simply referred to as a model.

[0004] After that, the anomaly detection system calculates the occurrence probability of each communication using the learned model, and detects communication with a small occurrence probability as an anomaly. Therefore, according to the anomaly detection system using the probability density estimator, it is possible to perform anomaly detection even if all malicious states are not known, and it is also possible to cope with unknown cyber attacks. In the anomaly detection system, sometimes an anomaly score is used in anomaly detection, and the smaller the occurrence probability is, the larger the anomaly score is.

[0005] Here, learning of the probability density estimator such as VAE is mostly not smoothly performed in a situation where the number of pieces of normal data of the learning target is biased. In particular, in service session data, a situation where the number of pieces is biased often occurs. For example, since HTTP communication is often used, data is concentrated in a large amount in a short time. On the other hand, it is difficult to collect data of NTP communication and the like which communicates only sporadically in a large amount. If learning based on the probability density estimator such as VAE is performed in such a situation, learning of NTP communication in which the number of pieces of data is small cannot be smoothly performed, the occurrence probability is estimated to be low, and sometimes it becomes a cause of false detection.

[0006] As a method of solving the problem due to the bias in the number of pieces of data, a method of performing learning of the probability density estimator in two stages is known (for example, refer to Patent Literature 1).

[0007] PRIOR ART DOCUMENTS

[0008] PATENT LITERATURE

[0009] Patent Literature 1: Japanese Patent Application Laid-Open No. 2019-101982 SUMMARY

[0010] PROBLEMS TO BE SOLVED BY THE INVENTION

[0011] However, in the related art, there is a problem that the processing time sometimes increases. For example, in the method described in Patent Literature 1, learning of the probability density estimator is performed in 2 stages, so the learning time is about twice as long as in the case of 1 stage.

[0012] MEANS FOR SOLVING THE PROBLEMS

[0013] To solve the above problems and achieve the object, a learning device is characterized by having: a generation section that learns data selected as unlearned data among learning data to generate a model that calculates an abnormality score; and a selection section that selects at least a part of data among the learning data for which the abnormality score calculated using the model generated by the generation section is equal to or higher than a threshold as the unlearned data.

[0014] EFFECTS OF THE INVENTION

[0015] According to the present application, learning can be performed with high accuracy in a short time even in a case where the number of pieces between normal data is deviated. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a diagram that explains the flow of learning processing.

[0017] Figure 2 is a diagram that shows a structure example of a learning device of the first embodiment.

[0018] Figure 3 is a diagram that explains selection of unlearned data.

[0019] Figure 4 is a flowchart that shows a processing flow of the learning device of the first embodiment.

[0020] Figure 5 is a diagram that shows a distribution of abnormality scores.

[0021] Figure 6 is a diagram that shows a distribution of abnormality scores.

[0022] Figure 7 is a diagram that shows a distribution of abnormality scores.

[0023] Figure 8 is a diagram that shows an ROC curve.

[0024] Figure 9 is a diagram that shows a structure example of an abnormality detection system.

[0025] Figure 10is a diagram showing an example of a computer that executes a learning program. DETAILED DESCRIPTION

[0026] Hereinafter, an embodiment of a learning device, a learning method, and a learning program according to the present application will be described in detail with reference to the drawings. In addition, the present application is not limited to the following described embodiment.

[0027] [Structure of the first embodiment]

[0028] First, the learning device according to the first embodiment will be described with reference to Figure 1 The flow of the learning process of the present embodiment will be described. Figure 1 is a diagram illustrating the flow of the learning process. As Figure 1 indicated, the learning device of the present embodiment repeatedly performs Step 1 and Step 2 until the end condition is satisfied. Thereby, the learning device generates a plurality of models. In addition, the generated models are added to the list.

[0029] Initially, all of the collected learning data is regarded as unlearned data. In Step 1, the learning device randomly samples a prescribed number of data from the unlearned data. Then, the learning device generates a model from the sampled data. For example, the model is a probability density estimator such as VAE.

[0030] Next, in Step 2, the learning device calculates the anomaly scores of the entire unlearned data using the generated model. Then, the learning device selects data having anomaly scores below a threshold value as learned data. On the other hand, the learning device selects data having anomaly scores above the threshold value as unlearned data. Here, if the end condition is not satisfied, the learning device returns to Step 1.

[0031] In STEP 1 at the second time and thereafter, data having anomaly scores above the threshold value in STEP 2 is regarded as unlearned data. In this way, in the present embodiment, sampling and evaluation (calculation of anomaly scores and selection of unlearned data) are repeatedly performed, and data of a dominant type is sequentially learned from among the unlearned data.

[0032] In addition, in the present embodiment, by performing sampling and reducing unlearned data, the data of the learning target is reduced, and thus the time required for learning can be shortened.

[0033] The structure of the learning device will be described. Figure 2 is a diagram showing an example of the structure of the learning device of the first embodiment. As Figure 2 indicated, the learning device 10 has an IF (interface) section 11, a storage section 12, and a control section 13.

[0034] The IF section 11 is an interface for input and output of data. For example, the IF section 11 is a NIC (Network Interface Card). In addition, the IF section 11 can also be connected to input devices such as a mouse and a keyboard, and output devices such as a display.

[0035] The storage section 12 is a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), and an optical disk. In addition, the storage section 12 can also be a semiconductor memory such as a RAM (Random Access Memory), a flash memory, and an NVSRAM (Non Volatile Static Random Access Memory) that can rewrite data. The storage section 12 stores an OS (Operating System) and various programs executed by the learning device 10.

[0036] The control section 13 controls the entire learning device 10. The control section 13 is, for example, an electronic circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit), an integrated circuit such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). In addition, the control section 13 has an internal memory for storing programs and control data that define various processing steps, and performs each processing using the internal memory. In addition, the control section 13 functions as various processing sections by working with various programs. For example, the control section 13 has a generation section 131, a calculation section 132, and a selection section 133.

[0037] The generation section 131 learns data selected as unlearned data in the learning data, and generates a model that calculates an abnormality score. The generation section 131 adds the generated model to a list. The generation section 131 can adopt a generation method of an existing VAE. In addition, the generation section 131 can generate a model based on data obtained by sampling a part of the unlearned data.

[0038] The calculation section 132 calculates the anomaly scores of the unlearned data based on the model generated by the generation section 131. The calculation section 132 can calculate the anomaly scores of all of the unlearned data, or can calculate the anomaly scores of a part of the unlearned data.

[0039] The selection section 133 selects at least a part of the data in which the anomaly scores calculated using the model generated by the generation section 131 are equal to or higher than the threshold value, as the unlearned data.

[0040] Using Figure 3 The selection of the unlearned data by the selection section 133 will be described. Figure 3 is a graph illustrating the selection of the unlearned data. Here, the model is a VAE, and is used to calculate the anomaly scores of the communication data in order to detect abnormal communication.

[0041] As described above, if there is a bias in the number of data, the number of cases of false detection increases. For example, in a case where a large number of HTTP communications and a small number of management FTP communications are simultaneously used as learning targets, a bias in the number of data occurs.

[0042] As illustrated in <First> in Figure 3 , it is assumed here that there are a large number of MQTT communication data, a medium number of DNS communication data, and a small number of video communication data. Figure 3 The graph of depicts the approximate value of the negative log likelihood of the probability density (-log p(x)), that is, the anomaly score, on the horizontal axis, and a histogram of the number of data on the vertical axis. Since the lower the density (frequency of occurrence) of the data point, the higher the value of the negative log likelihood of the probability density, it can be regarded as the anomaly score, that is, the degree of abnormality.

[0043] As illustrated in <First> in Figure 3 , the anomaly scores of the MQTT communications, which have a large number of data, become low, and the anomaly scores of the video stream communications, which have a small number of data, become high. Therefore, it is considered that the data of the video communications, which have a small number of data, become the cause of false detection.

[0044] Therefore, the selection section 133 selects the unlearned data from the data in which the anomaly scores are equal to or higher than the threshold value. Then, using a part or all of the selected unlearned data, a model in which false detection is suppressed is generated. In other words, the selection section 133 has a function of excluding data that does not need to be further learned.

[0045] The threshold value can also be determined based on a loss value obtained at the time of generation of the model. In this case, the selection section 133 selects at least a portion of data for which the abnormality score calculated using the model generated by the generation section 131 is equal to or higher than a threshold value calculated based on the loss value of each data obtained at the time of generation of the model, among the learning data, as the unlearning data. For example, the threshold value can also be calculated based on the mean and variance, such as mean + 0.3σ of the loss value.

[0046] As shown in Figure 3 The selection section 133 selects data of the DNS communication and data of the camera communication as the center based on the abnormality score calculated in the first time, as shown in the second time. In contrast, the selection section 133 hardly selects data of the MQTT communication having a large number of data.

[0047] Further, the learning device 10 can repeatedly perform the processing performed by the generation section 131, the calculation section 132, and the selection section 133 from the third time and thereafter. That is, each time data is selected as the unlearning data by the selection section 133, the generation section 131 learns the selected data and generates a model for calculating the abnormality score. Also, each time a model is generated by the generation section 131, the selection section 133 selects at least a portion of data for which the abnormality score calculated using the generated model is equal to or higher than the threshold value as the unlearning data.

[0048] In addition, the learning device 10 can also end the repetition at the time when the number of data for which the abnormality score is equal to or higher than the threshold value is smaller than a prescribed value. In other words, the selection section 133 selects at least a portion of data for which the abnormality score is equal to or higher than the threshold value as the unlearning data in a case where the number of data for which the abnormality score calculated using the model generated by the generation section 131 is equal to or higher than the threshold value among the learning data satisfies a prescribed condition.

[0049] For example, the learning device 10 can also repeatedly perform the processing until the number of data for which the abnormality score is equal to or higher than the threshold value is smaller than 1% of the number of the learning data collected initially. In addition, a model is generated each time the repetition is performed and is added to the list, and thus the learning device 10 can output a plurality of models.

[0050] The plurality of models generated by the learning device 10 are used for abnormality detection in a detection device or the like. As for the abnormality detection using the plurality of models, the method described in Patent Literature 1 can also be used. That is, the detection device can detect an abnormality based on a merged value or a minimum value of the abnormality scores calculated by the plurality of models.

[0051] [Processing of the first embodiment]

[0052] Figure 4is a flowchart showing a processing flow of the learning device of the first embodiment. First, the learning device 10 samples a part of the unlearned data (step S101). Next, the learning device 10 generates a model based on the sampled data (step S102).

[0053] Here, in a case where the end condition is satisfied (step S103, Yes), the learning device 10 ends the processing. On the other hand, in a case where the end condition is not satisfied (step S103, No), the learning device 10 calculates abnormality scores of the entire unlearned data using the generated model (step S104).

[0054] The learning device 10 selects data having an abnormality score of the threshold value or more as the unlearned data (step S105), and returns to step S101 to repeat the processing. Further, before step S105 is executed, selection of the unlearned data is temporarily initialized. That is, in step S105, the learning device 10 reselects the unlearned data with reference to the abnormality scores in a state where none of the unlearned data is selected.

[0055] [Effects of the First Embodiment]

[0056] As described thus far, the generating section 131 learns data selected as the unlearned data among the learning data, and generates a model that calculates an abnormality score. The selecting section 133 selects at least a part of data having an abnormality score calculated using the model generated by the generating section 131 of the threshold value or more as the unlearned data among the learning data. In this way, the learning device 10 can select data that is likely to be a cause of false detection after the model is generated, and generate the model again. As a result, according to the present embodiment, even in a case where the number of pieces among normal data is biased, learning can be performed with high accuracy in a short time.

[0057] Each time data is selected as the unlearned data by the selecting section 133, the generating section 131 learns the selected data, and generates a model that calculates an abnormality score. Each time the model is generated by the generating section 131, the selecting section 133 selects at least a part of data having an abnormality score calculated using the generated model of the threshold value or more as the unlearned data. In the present embodiment, a plurality of models are generated by repeating the processing in this way, and the accuracy of abnormality detection can be improved.

[0058] The selecting section 133 selects at least a part of data having an abnormality score calculated using the model generated by the generating section 131 of the threshold value or more, which is calculated based on the loss value of each data obtained at the time of generation of the model, as the unlearned data among the learning data. Thereby, a threshold value corresponding to the degree of deviation of the abnormality score can be set.

[0059] The selection section 133 selects at least a part of the data whose abnormality score calculated using the model generated by the generation section 131 is equal to or higher than the threshold as the unlearned data, in a case where the number of data whose abnormality score is equal to or higher than the threshold in the learning data satisfies a prescribed condition. In this way, by setting the end condition of the repeated processing in advance, it is possible to adjust the balance between the accuracy of the abnormality detection and the processing time required for the learning.

[0060] [Experimental Results]

[0061] Results of experiments performed using the present embodiment are shown. First, in the experiments, learning was performed using data mixed with the following communications.

[0062] MQTT communication: 1883 port 20951 pieces (large amount of data)

[0063] Camera communication: 1935 port 204 pieces (small amount of data)

[0064] In the experiments, a model was generated by learning, and the abnormality score of each data was calculated using the generated model. Figure 5 Figure 6 and Figure 7 is a graph showing the distribution of the abnormality scores.

[0065] First, in Figure 5 , the results of learning based on the conventional VAE (1-stage VAE) are shown. In Figure 5 , the time required for the learning was 268 seconds. In addition, in Figure 5 , the abnormality scores of the camera communication, which is the small amount of data, were calculated to be slightly high.

[0066] In Figure 6 , the results of learning based on the 2-stage VAE described in Patent Literature 1 are shown. In Figure 6 , the time required for the learning was 572 seconds. In addition, in Figure 6 , compared to Figure 5 , the abnormality scores of the camera communication, which is the small amount of data, were reduced.

[0067] Figure 7 shows the results of learning of the present embodiment. In Figure 7 , the time required for the learning was 192 seconds. In addition, as shown in Figure 7 , in the present embodiment, the abnormality scores of the camera communication were reduced to the same degree as in the case of the 2-stage VAE of Figure 6 , and the time required for the learning was further greatly shortened.

[0068] Figure 8 is a graph showing the ROC curve. As Figure 8 ​As shown, the present embodiment shows an ideal ROC curve compared to the 1-stage VAE and the 2-stage VAE. In addition, the detection accuracy of the present embodiment is 0.9949. In addition, the detection accuracy based on the 2-stage VAE is 0.9652. In addition, the detection accuracy based on the 1-stage VAE is 0.9216. Thus, according to the present embodiment, it is possible to improve the detection accuracy.

[0069] [Embodiment]

[0070] As Figure 9 shown, it is also possible to provide a server on a network to which IoT devices are connected with the same model generation function as the learning device 10 in the above-described embodiment and the anomaly detection function using the model generated by the learning device 10. Figure 9 is a diagram showing an example of the structure of an anomaly detection system.

[0071] In this case, the server collects service session information transmitted and received by IoT devices, and performs learning of the probability density of normal service sessions and detection of abnormal service sessions. The server, when learning the probability density of normal service sessions, applies the method of the embodiment, and can generate an anomaly detection model with high accuracy and at high speed even if there is a deviation in the number of session data.

[0072] [Structure of System, etc.]

[0073] In addition, each of the constituent elements of each of the devices shown is a functional conceptual element, and is not necessarily constituted in the physical aspect as shown. That is, the specific manner of the dispersion and integration of each of the devices is not limited to the manner shown, and all or a part of the functionality or the physicality can be dispersed or integrated in any unit according to various loads, usage conditions, and the like. Also, all or any part of each processing function performed by each of the devices can be realized by a CPU (Central Processing Unit) and a program executed by the CPU, or can be realized as hardware based on wiring logic. In addition, the program can be executed not only by the CPU but also by other processors such as a GPU.

[0074] In addition, all or a part of the processing described in the present embodiment as processing performed automatically can be manually performed, or all or a part of the processing described as processing performed manually can be automatically performed in a known manner. In addition, unless specifically described, the processing procedure, the control procedure, the specific name, the information including various data and parameters described in the above description or the drawings can be arbitrarily changed.

[0075] [Program]

[0076] As one embodiment, the learning device 10 can be installed by installing a learning program that performs the above-described learning processing as packaged software or online software to a desired computer. For example, by causing an information processing device to execute the above-described learning program, the information processing device can function as the learning device 10. The information processing device described here includes a desktop or notebook personal computer. In addition, among information processing devices, mobile body communication terminals such as smartphones, mobile phones, or PHS (Personal Handyphone System), and tablet terminals such as PDAs (Personal Digital Assistants) are also included in the category.

[0077] In addition, the learning device 10 can also be installed as a learning server device that uses a terminal device used by a user as a client and provides a service related to the above-described learning processing to the client. For example, the learning server device is installed as a server device that provides a learning service that inputs learning data and outputs information of a plurality of models that have been generated. In this case, the learning server device can be installed as a Web server or as a cloud that provides a service related to the above-described learning processing by outsourcing.

[0078] Figure 10 FIG. 1 is a diagram that shows an example of a computer that executes a learning program. The computer 1000 has, for example, a memory 1010, a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These parts are connected by a bus 1080.

[0079] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0080] The hard disk drive 1090 stores, for example, an OS 1091, an application 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the learning device 10 is installed as the program module 1093 in which a code that can be executed by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for performing the same processing as the functional configuration in the learning device 10 is stored in the hard disk drive 1090. Further, the hard disk drive 1090 can also be replaced by an SSD (Solid State Drive).

[0081] In addition, the setting data used in the processing of the above-described embodiments is stored in the memory 1010 and the hard disk drive 1090 as the program data 1094, for example. Further, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiments.

[0082] In addition, the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, and can be stored in a removable storage medium, for example, and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 can also be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), or the like). Further, the program module 1093 and the program data 1094 can also be read out by the CPU 1020 from the other computer via the network interface 1070.

[0083] Mark Description

[0084] 10 Learning device

[0085] 11 IF section

[0086] 12 Storage section

[0087] 13 Control section

[0088] 131 Generation section

[0089] 132 Calculation section

[0090] 133 Selection section

Claims

1. A learning device, characterized in that, have: The generation unit learns from the data selected as unlearned data in the learning data to generate a model for calculating anomaly scores, wherein the learning data is network communication data; as well as The selection unit selects at least a portion of the learning data whose anomaly scores calculated using the model generated by the generation unit are above a threshold as the unlearned data. The threshold is calculated based on the average and variance of the loss values ​​of each data point obtained during model generation. If the number of data in the learning data whose abnormal scores calculated by the model generated by the generation unit are above the threshold meets a predetermined condition, the selection unit selects at least a portion of the data whose abnormal scores are above the threshold as the unlearned data.

2. The learning device according to claim 1, characterized in that, Whenever data is selected as unlearned data by the selection unit, the generation unit learns from that selected data to generate a model for calculating outlier scores. Whenever a model is generated by the generation unit, the selection unit selects at least a portion of the data whose anomaly scores calculated using the generated model are above the threshold as the unlearned data.

3. A learning method, wherein the learning method is executed by a learning device, characterized in that, The learning methods include: The generation step involves learning from the data selected as unlearned data in the learning data to generate a model for calculating anomaly scores, wherein the learning data is network communication data; and In the selection step, at least a portion of the data in the learning data whose anomaly scores calculated using the model generated through the generation step are above a threshold are selected as the unlearned data. The threshold is calculated based on the average and variance of the loss values ​​of each data point obtained during model generation. In the selection step, if the number of data in the learning data whose anomaly scores calculated using the model generated in the generation step are above the threshold meets the specified conditions, at least a portion of the data whose anomaly scores are above the threshold is selected as the unlearned data.

4. A recording medium storing a learning program that causes a computer to perform the following steps: The generation step involves learning from the data selected as unlearned data in the learning data to generate a model for calculating anomaly scores, wherein the learning data is network communication data; and In the selection step, at least a portion of the data in the learning data whose anomaly scores calculated using the model generated through the generation step are above a threshold are selected as the unlearned data. The threshold is calculated based on the average and variance of the loss values ​​of each data point obtained during model generation. In the selection step, if the number of data in the learning data whose anomaly scores calculated using the model generated in the generation step are above the threshold meets the specified conditions, at least a portion of the data whose anomaly scores are above the threshold is selected as the unlearned data.

Images