Abnormality detection device, abnormality detection method, and abnormality detection program

Abstract

A creation unit (15b) creates a group for each of multiple prescribed communication feature amounts for alerts indicating that abnormal communication having a different pattern from normal communication has been detected. An identification unit (15c) identifies the cause of an alert notification for each created group of alerts.

Description

Anomaly detection device, anomaly detection method, and anomaly detection program

[0001] The present invention relates to an abnormality detection device, an abnormality detection method, and an abnormality detection program.

[0002] Conventionally, anomaly detection techniques for detecting known threats are known. For example, a method for grouping alerts for known threats by pattern matching is known (see Non-Patent Document 1). Another known method is for visualizing network intrusion monitoring by inputting the source IP address, destination IP address, time of occurrence, a positive integer ID representing the type of intrusion, and a five-level level representing the risk (see Non-Patent Document 2). Another known method is for reducing false positives by inputting the source IP address, time of occurrence, and signature and focusing on the number of security alerts generated by each signature in intrusion detection (see Non-Patent Document 3).

[0003] “Nozomi Networks Guardian”, [online], NOZOMI NETWORKS, [Retrieved May 27, 2022], Internet<URL:https: / / www.nozominetworks.com / products / guardian / > Ito, Takakura, and two others, "A visualization method for monitoring network intrusions," [online], Kyoto University, [Retrieved May 27, 2022], Internet<URL:http: / / itolab.is.ocha.ac.jp / ~itot / paper / ItotRDCPJ12.pdf> Iwasaki, Kakuta, and four others, “A method for reducing false positives by focusing on the volume of security alerts generated by each signature in intrusion detection,” [online], October 2018, Computer Security Symposium 2018, [Retrieved May 27, 2022], Internet <URL:https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=repository_uri&item_id=192157&file_id=1&file_no=1>

[0004] However, with conventional technologies, there is a risk that true alerts may be overlooked in an anomaly-based anomaly detection system. In other words, because conventional technologies depend on predefined attack conditions and attack results, they cannot be applied to anomaly-based anomaly detection systems that detect unknown threats by detecting anomalous communications that deviate from normal communication patterns.

[0005] Furthermore, even with an anomaly-based abnormality detection system, if there is a lack of training data or if there is a change in normal communication patterns, the number of false positives will increase, resulting in a large number of alerts that must be checked during monitoring, and there is a risk that true alerts will be overlooked.

[0006] The present invention has been made in view of the above, and aims to make it possible to avoid the risk of overlooking a true alert in an anomaly-based abnormality detection system.

[0007] In order to solve the above-mentioned problems and achieve the object, the anomaly detection device of the present invention is characterized by having a creation unit that creates groups for each predetermined communication feature for alerts that indicate that abnormal communication that differs from normal communication patterns has been detected, and an identification unit that identifies the cause of the notification of the alert for each group of created alerts.

[0008] According to the present invention, it is possible to avoid the risk of missing a true alert in an anomaly-based abnormality detection system.

[0009] FIG. 1 is a schematic diagram illustrating the general configuration of an anomaly detection device according to this embodiment. FIG. 2 is a diagram illustrating the processing of a creation unit and an identification unit. FIG. 3 is a diagram illustrating the processing of the identification unit. FIG. 4 is a diagram illustrating the processing of the identification unit. FIG. 5 is a diagram illustrating the processing of the identification unit. FIG. 6 is a diagram illustrating the processing of the identification unit. FIG. 7 is a diagram illustrating the processing of the identification unit. FIG. 8 is a diagram illustrating an example of a screen display of an anomaly detection processing result. FIG. 9 is a flowchart illustrating an anomaly detection processing procedure. FIG. 10 is a diagram illustrating an example of a computer that executes an anomaly detection program.

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0011] [Configuration of the Abnormality Detection Device] Fig. 1 is a schematic diagram illustrating the general configuration of the abnormality detection device of this embodiment. As illustrated in Fig. 1, the abnormality detection device 10 of this embodiment is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0012] The input unit 11 is realized by input devices such as a keyboard and a mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to input operations by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of the abnormality detection processing described below.

[0013] The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and an external device via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device or the like that manages various information related to the like.

[0014] The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance the processing program that operates the abnormality detection device 10, data used during execution of the processing program, and the like, or temporarily stores the data each time processing is performed. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

[0015] The control unit 15 is realized using a central processing unit (CPU), a network processor (NP), a field programmable gate array (FPGA), etc., and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a creation unit 15b, an identification unit 15c, and a determination unit 15d, as illustrated in FIG. 1, and executes an abnormality detection processing.

[0016] Each or some of these functional units may be implemented in different hardware. For example, the acquisition unit 15a may be implemented in hardware different from the other functional units. Furthermore, the control unit 15 may include other functional units. For example, the control unit 15 may be built into an anomaly-type abnormality detection system.

[0017] The acquisition unit 15a acquires an alert indicating that abnormal communication that differs from a normal communication pattern has been detected. Specifically, the acquisition unit 15a acquires an alert indicating abnormal communication that is output from the anomaly-type abnormality detection system via the input unit 11 or the communication control unit 13.

[0018] The acquisition unit 15a also acquires learning data used in training the anomaly-based anomaly detection system. For example, the acquisition unit 15a acquires the learning data used in training the detection model of the anomaly-based anomaly detection system via the input unit 11 or from a management device that manages the learning data via the communication control unit 13. The acquisition unit 15a may store the previously acquired alert and learning data in the storage unit 14. Alternatively, the acquisition unit 15a may immediately transfer the acquired alert and learning data to a functional unit described below without storing them in the storage unit 14.

[0019] The creation unit 15b creates groups for each predetermined communication feature for alerts indicating that abnormal communication different from normal communication patterns has been detected. Specifically, the creation unit 15b creates groups for which the predetermined communication feature is the same: the source IP address, the destination IP address, and the destination port number.

[0020] 2 is a diagram illustrating the processing of the creation unit and the identification unit. Fig. 2 illustrates examples of information items related to each communication, and for example, items 3 to 7 are 5-tuple information. The creation unit 15b groups together alerts that are targeted at communications that have the same source IP address, destination IP address, and destination port number included in the 5-tuple of information items (communication features) illustrated in Fig. 2.

[0021] 3 to 7 are diagrams for explaining the processing of the identifying unit 15c. The identifying unit 15c identifies the cause of the notification of the alert for each group of created alerts.

[0022] Specifically, the identifying unit 15c first identifies the cause of the alert notification by determining whether the communication of the alert group has been learned so that it can output whether it is normal or abnormal. That is, the identifying unit 15c determines whether the combination of the source IP address, destination IP address, and destination port number that is the same as that of the group exists in the learning data used to train the detection model of the anomaly-based anomaly detection system. Note that in this embodiment, "learned" means that learning has been performed using a normal pattern.

[0023] 3, if any of the source IP address, destination IP address, and destination port number differs from the learned data, it is determined that the information has not been learned. In this way, if the information has not been learned, the identifying unit 15c identifies that the cause of the alert notification is the occurrence of new communication with a new destination. In this case, the anomaly detection device 10 can recommend to the operator that they take action such as checking the details of the group's communications.

[0024] Next, when the identification unit 15c has already learned to be able to output whether the communications of the alert group are normal or abnormal, the identification unit 15c determines whether the features of the communications of this group are similar to the learned features, thereby identifying the cause of the alert notification. For example, the identification unit 15c focuses on the features of items 8 to 18 shown in FIG. 2, integrates these features, and performs clustering using a Gaussian Mixture Model (GMM). Then, it determines whether the clustered classes exist in the training data.

[0025] For example, Figure 4 illustrates the distribution of the total number of upstream bytes as a feature of each communication. If the distribution of the feature of the learning data and the distribution of the feature of the alert group differ, as illustrated in Figure 4, it is determined that the two feature amounts are not similar. In this way, if the feature of this group's communication is not similar to the feature of the learned learning data, the identifying unit 15c identifies that the cause of the alert notification is a change in the communication state, even if the communication destination remains unchanged. In this case, the anomaly detection device 10 recommends that the operator take action by checking the detailed content of the group's communication.

[0026] Next, the identification unit 15c identifies the cause of the alert notification by determining the degree of deviation of the value for detecting abnormal communication from a predetermined threshold value when the characteristics of the communication in the alert group are similar to the learned characteristics.

[0027] For example, Figure 5 illustrates a case where the threshold value for the anomaly determination is set to 40, and the anomaly determination values ​​for the group's communications are 100, 150, and 130. In this case, the identifying unit 15c determines that the degree of deviation is, for example, 1.2 times or more, indicating a large deviation. In this way, if the anomaly determination value for the group's communications for this alert deviates significantly from the threshold by a certain multiple or more, it is presumed that the alert was issued due to insufficient maturity of the anomaly detection model. In this case, the anomaly detection device 10 detects a lack of learning data and, for example, recommends to the operator that the communications of this group be added as learning data for the anomaly detection model.

[0028] On the other hand, Figure 6 illustrates a case in which the feature values ​​of the learning data (distribution of total number of uplink bytes) indicated by diagonal shading and the feature values ​​of the communication in the alert group are similar, and the anomaly determination values ​​for both data deviate slightly from the threshold by less than a certain multiple. In this way, if the anomaly determination value for the communication in this alert group deviates from the threshold by less than a certain multiple, it is presumed that an anomaly close to normal occurred, causing the alert to be notified. In this case, too, the anomaly detection device 10 will advise the operator that learning is insufficient, and will recommend, for example, that the operator add the communication in this alert group as learning data for the anomaly detection model.

[0029] In this way, the identifying unit 15c identifies the cause of the alert notification for the alert group of each group, as illustrated in FIG. 7. In FIG. 7, group (1) is an alert group for which the occurrence of communication with a new communication destination is identified as the cause of the alert notification. Group (2) is an alert group for which the change in the communication aspect is identified as the cause of the alert notification even if the communication destination remains unchanged. Group (3) is an alert group for which the cause of the alert notification is identified as insufficient maturity of the model because the anomaly determination value deviates significantly from a predetermined threshold. Group (4) is an alert group for which the cause of the alert notification is identified as insufficient learning because the anomaly determination value deviates slightly from a predetermined threshold.

[0030] The identifying unit 15c then outputs the cause of the alert notification identified for each alert group as a result of the anomaly detection process to the output unit 12. Here, FIG. 8 is a diagram showing an example of a screen display of the anomaly detection process results. As shown in FIG. 8, the identifying unit 15c outputs a similar cause name indicating the identified cause of the alert notification for each group identified by the source IP address, destination IP address, and destination port number. Furthermore, for the above groups (3) and (4), the identifying unit 15c outputs the number of learned learning data items.

[0031] In the example shown in FIG. 8 , "Occurrence of new communication" in cause classification (1) corresponds to the alert group in the above group (1). Also, "Change in communication status" in cause classification (2) corresponds to the alert group in the above group (2). Also, "Lack of model maturity" in cause classification (3) corresponds to the alert group in the above group (3). Also, "Abnormality close to normal data" in cause classification (4) corresponds to the alert group in the above group (4).

[0032] Returning to the explanation of FIG. 1 , the determination unit 15d determines whether or not there is periodicity in the alert occurrence times for each alert group. For example, the determination unit 15d tallies the alert occurrence times for each hour and identifies the number of alerts that have occurred. Then, the determination unit 15d determines whether or not there is periodicity in the alert groups of the group, and if there is periodicity, identifies the number of alerts per cycle.

[0033] The determination unit 15d adds the determined presence or absence of periodicity to the anomaly detection processing result and outputs the result, as illustrated in Fig. 8. In the example illustrated in Fig. 8, the number of alerts, the time of occurrence of the alert, and the presence or absence of periodicity are added for each group.

[0034] In this way, by presenting the cause of the alert notification for each alert group, the risk of overlooking true alerts that should be checked is reduced even when a large number of alerts are generated. For example, for alert groups of cause category (1) and cause category (2), it becomes possible for the operator to recognize that it is necessary to check the detailed content of the group's communications. Furthermore, for alert groups of cause category (3) and cause category (4), it is inferred that there is a lack of learning data, and the operator can recognize that an investigation is necessary.

[0035] [Abnormality Detection Processing] Next, the abnormality detection processing performed by the abnormality detection device 10 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the procedure of the abnormality detection processing. The flowchart in Fig. 9 starts, for example, when the user performs an operation input to instruct the start of the processing.

[0036] First, the creation unit 15 b creates groups for each predetermined communication feature for alerts indicating that abnormal communication different from normal communication patterns has been detected, for example, groups with the same source IP address, destination IP address, and destination port number as the predetermined communication feature (step S1).

[0037] Next, the identifying unit 15c identifies the cause of the alert notification for each created alert group. Specifically, the identifying unit 15c first determines whether the communication of the alert group has been learned so that it can output whether it is normal or abnormal (step S2). In other words, the identifying unit 15c determines whether the learned data contains a combination of a source IP address and a destination IP address that is the same as that of the group. If the communication of the group has not been learned (step S2, No), the identifying unit 15c identifies the occurrence of communication with a new destination as the cause of the alert notification (step S3) and proceeds to step S9.

[0038] On the other hand, if the communication of the alert group has been learned so that it can be output as to whether it is normal or abnormal (Yes in Step S2), the identifying unit 15c determines whether the feature of the communication of this alert group is similar to the learned feature (Step S4).If the feature of the communication of this alert group is not similar to the feature of the learned learning data (No in Step S4), the identifying unit 15c identifies that the cause of the alert notification is a change in the communication state (Step S5), and proceeds to Step S9.

[0039] On the other hand, for communications in the alert group, if the feature of the communications in the alert group is similar to the feature that has been learned so as to be able to output whether the communications are normal or abnormal (step S4, Yes), the identifying unit 15c determines the degree of deviation of the judgment value for determining that the communications are abnormal from a predetermined threshold (step S6). If the degree of deviation of the judgment value of the communications in this alert group from the threshold is greater than a predetermined value (step S6, Yes), the identifying unit 15c identifies that the cause of the alert notification is insufficient maturity of the anomaly detection model (step S7), and proceeds to step S9.

[0040] On the other hand, if the deviation of the judgment value of the communication of this alert group from the threshold value is smaller than a predetermined value (step S6, No), the identification unit 15c identifies that the cause of the alert notification is the occurrence of an abnormality that is close to normal (step S8), and proceeds to step S9.

[0041] In the process of step S9, the identifying unit 15c outputs the cause of the alert notification identified for each alert group to the output unit 12 as a result of the abnormality detection process.

[0042] The determination unit 15d also determines whether or not there is periodicity in the occurrence times of the alerts for each group of alerts, and adds the determined presence or absence of periodicity to the result of the abnormality detection process and outputs it. This completes the series of abnormality detection processes.

[0043] As described above, in the anomaly detection device 10 of this embodiment, the creation unit 15b creates groups for each predetermined communication feature for alerts indicating that abnormal communication that differs from normal communication patterns has been detected. In addition, the identification unit 15c identifies the cause of the alert notification for each created group of alerts.

[0044] Specifically, the creating unit 15b creates a group of packets having the same source IP address, destination IP address, and destination port number as the predetermined communication feature.

[0045] This makes it possible to determine whether or not a response is required to check the detailed content of the communication for each group of alerts. In this way, it is possible to narrow down the number of alerts that require a response that checks the detailed content of the communication, thereby shortening the time required for checking. Therefore, it is possible to avoid the risk of overlooking a true alert when it exists among a large number of alerts. In this way, it is possible to avoid the risk of overlooking a true alert in an anomaly-based abnormality detection system.

[0046] The identification unit 15c also identifies the cause of the alert by determining whether the communication in the alert group has been learned so that it can output whether the communication is normal or abnormal. For example, if the learning has not been completed, it can be identified that the reason is that communication with a new communication destination has occurred. Therefore, it is possible to recommend that the operator take action by checking the detailed content of the communication.

[0047] Furthermore, when the identification unit 15c has already learned to be able to output whether the communications in the alert group are normal or abnormal, it determines whether the features of the communications in this alert group are similar to the learned features, thereby identifying the cause of the alert notification. For example, when the features of the communications in this alert group are not similar to the features of the learned learning data, it can be assumed that the communication conditions have changed even if the communication destination remains unchanged. Therefore, it is possible to recommend that the operator take action by checking the detailed content of the communications.

[0048] Furthermore, when the feature of the communication of the alert group is similar to a feature that has already been learned so as to be able to output whether the communication is normal or abnormal, the identification unit 15c determines the degree of deviation of the value for determining abnormal communication from a predetermined threshold, thereby identifying the cause of the alert notification. For example, when the determination value of the communication of the alert group deviates significantly from the predetermined threshold, it is inferred that this is due to insufficient maturity of the anomaly detection model. Also, when the determination value of the communication of the alert group deviates only slightly from the predetermined threshold, it is inferred that this is due to an anomaly that is close to normal. In either case, it is possible to recommend to the operator that the communication of this alert group be added as learning data for the anomaly detection model.

[0049] Furthermore, the determination unit 15d determines whether or not there is periodicity in the occurrence time of the alerts for each group of alerts. This makes it possible to more reliably avoid overlooking true alerts by checking the alerts while focusing on periodicity.

[0050] [Program] A program written in a computer-executable language may be created to execute the processes executed by the anomaly detection device 10 according to the above embodiment. In one embodiment, the anomaly detection device 10 can be implemented by installing an anomaly detection program that executes the anomaly detection process as package software or online software on a desired computer. For example, by executing the anomaly detection program on an information processing device, the information processing device can function as the anomaly detection device 10. The information processing device referred to here includes desktop and notebook personal computers. Other examples of information processing devices include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants). The functions of the anomaly detection device 10 may also be implemented on a cloud server.

[0051] 10 is a diagram showing an example of a computer that executes an anomaly detection program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0052] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.

[0053] Here, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. The various pieces of information described in the above embodiments are stored in the hard disk drive 1031 or memory 1010, for example.

[0054] The abnormality detection program is stored in the hard disk drive 1031, for example, as a program module 1093 in which instructions to be executed by the computer 1000 are written. Specifically, the program module 1093 in which each process executed by the abnormality detection device 10 described in the above embodiment is written is stored in the hard disk drive 1031.

[0055] Furthermore, data used for information processing by the anomaly detection program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-described procedures.

[0056] The program module 1093 and program data 1094 related to the abnormality detection program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041. Alternatively, the program module 1093 and program data 1094 related to the abnormality detection program may be stored in another computer connected via a network such as a LAN or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

[0057] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0058] REFERENCE SIGNS LIST 10 Anomaly detection device 11 Input unit 12 Output unit 13 Communication control unit 14 Storage unit 15 Control unit 15a Acquisition unit 15b Creation unit 15c Identification unit 15d Discrimination unit

Claims

1. An anomaly detection device comprising: a creation unit that creates groups for each predetermined communication feature for alerts that indicate that abnormal communication that differs from normal communication patterns has been detected; and an identification unit that identifies the cause of the notification of the alert for each group of created alerts.

2. The anomaly detection device according to claim 1, wherein the creation unit creates the group having the same source IP address, destination IP address, and destination port number as the predetermined communication feature.

3. The anomaly detection device according to claim 1, characterized in that the identification unit identifies the cause by determining whether the communication of the group has been learned so that it can output whether the communication is normal or abnormal.

4. The anomaly detection device described in claim 3, characterized in that, when the communication of the group has already been learned, the identification unit identifies the cause by determining whether the characteristics of the communication of the group are similar to the learned characteristics.

5. The anomaly detection device described in claim 4, characterized in that the identification unit identifies the cause by determining the degree of deviation from a predetermined threshold value for detecting abnormal communication when the characteristics of the communication of the group are similar to learned characteristics.

6. The anomaly detection device according to claim 1, further comprising a determination unit that determines, for each of the groups, whether or not there is periodicity in the occurrence times of the alerts.

7. An anomaly detection method executed by an anomaly detection device, comprising: a creation step of creating groups for each predetermined communication feature for alerts indicating that abnormal communication that differs from normal communication patterns has been detected; and an identification step of identifying the cause of the notification of the alert for each group of created alerts.

8. An anomaly detection program that causes a computer to execute the following steps: a creation step of creating groups for each predetermined communication feature for alerts that indicate that abnormal communication that differs from normal communication patterns has been detected; and an identification step of identifying the cause of the notification of the alert for each of the created groups of alerts.

Images