Anomaly detection device, anomaly detection method, and anomaly detection program

Abstract

A creation unit (15b) creates a group for each of multiple prescribed communication feature amounts for alerts indicating that abnormal communication having a different pattern from normal communication has been detected. An identification unit (15c) identifies the cause of an alert notification for each created group of alerts.

Description

[Technical Field]

[0001] The present invention relates to an abnormality detection device, an abnormality detection method, and an abnormality detection program. [Background technology]

[0002] Conventionally, anomaly detection techniques for detecting known threats have been known. For example, a method for grouping alerts for known threats by pattern matching is known (see Non-Patent Document 1). Another known method is for visualizing network intrusion monitoring by inputting the source IP address, destination IP address, time of occurrence, a positive integer ID indicating the type of fraud, and a five-level level indicating the risk (see Non-Patent Document 2). Another known method is for reducing false positives by inputting the source IP address, time of occurrence, and signature and focusing on the number of security alerts generated by each signature in intrusion detection (see Non-Patent Document 3). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Nozomi Networks Guardian”, [online], NOZOMI NETWORKS, [Retrieved May 27, 2022], Internet<URL:https: / / www.nozominetworks.com / products / guardian / > [Non-patent document 2] Ito, Takakura, and two others, "A visualization method for monitoring network intrusions," [online], Kyoto University, [Retrieved May 27, 2022], Internet<URL:http: / / itolab.is.ocha.ac.jp / ~itot / paper / ItotRDCPJ12.pdf> [Non-patent document 3] Iwasaki, Kakuta, et al., “A method for reducing false positives by focusing on the volume of security alerts generated by each signature in intrusion detection,” [online], October 2018, Computer Security Symposium 2018, [Retrieved May 27, 2022], Internet <URL:https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=repository_uri&item_id=192157&file_id=1&file_no=1> Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technologies, there is a risk that true alerts may be overlooked in an anomaly-based anomaly detection system. In other words, because conventional technologies depend on predefined attack conditions and attack results, they cannot be applied to anomaly-based anomaly detection systems that detect unknown threats by detecting anomalous communications that deviate from normal communication patterns.

[0005] Furthermore, even with an anomaly-based abnormality detection system, if there is a lack of training data or if there is a change in normal communication patterns, the number of false positives will increase, resulting in a large number of alerts that must be checked during monitoring, and there is a risk that true alerts will be overlooked.

[0006] The present invention has been made in view of the above, and aims to make it possible to avoid the risk of overlooking a true alert in an anomaly-based abnormality detection system. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the anomaly detection device of the present invention is characterized by having a creation unit that creates groups for each predetermined communication feature for alerts that indicate that abnormal communication that differs from normal communication patterns has been detected, and an identification unit that identifies the cause of the notification of the alert for each group of created alerts. [Effects of the Invention]

[0008] According to the present invention, it is possible to avoid the risk of missing a true alert in an anomaly-based abnormality detection system. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating the general configuration of an abnormality detection device according to this embodiment. [Figure 2] FIG. 2 is a diagram for explaining the processing of the creating unit and the identifying unit. [Figure 3] FIG. 3 is a diagram for explaining the processing of the identification unit. [Figure 4] FIG. 4 is a diagram for explaining the processing of the identification unit. [Figure 5] FIG. 5 is a diagram for explaining the processing of the identification unit. [Figure 6] FIG. 6 is a diagram for explaining the processing of the identification unit. [Figure 7] FIG. 7 is a diagram for explaining the processing of the identification unit. [Figure 8] FIG. 8 is a diagram showing an example of a screen display of the abnormality detection processing result. [Figure 9] FIG. 9 is a flowchart showing the procedure of the abnormality detection process. [Figure 10] FIG. 10 is a diagram illustrating an example of a computer that executes an anomaly detection program. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0011] [Configuration of anomaly detection device] Fig. 1 is a schematic diagram illustrating the general configuration of an anomaly detection device according to this embodiment. As illustrated in Fig. 1, an anomaly detection device 10 according to this embodiment is implemented by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0012] The input unit 11 is realized using input devices such as a keyboard and a mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to input operations by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of the abnormality detection processing described below.

[0013] The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and an external device via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device or the like that manages various information related to the like.

[0014] The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance the processing program that operates the anomaly detection device 10, data used during execution of the processing program, and the like, or temporarily stores the data each time processing is performed. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

[0015] The control unit 15 is realized using a CPU (Central Processing Unit), an NP (Network Processor), an FPGA (Field Programmable Gate Array), etc., and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a creation unit 15b, an identification unit 15c, and a determination unit 15d, as exemplified in FIG. 1, and executes an abnormality detection processing.

[0016] Each or some of these functional units may be implemented in different hardware. For example, the acquisition unit 15a may be implemented in hardware different from the other functional units. Furthermore, the control unit 15 may include other functional units. For example, the control unit 15 may be built into an anomaly-type abnormality detection system.

[0017] The acquisition unit 15a acquires an alert indicating that abnormal communication different from a normal communication pattern has been detected. Specifically, the acquisition unit 15a acquires an alert indicating abnormal communication output from the anomaly-type abnormality detection system via the input unit 11 or the communication control unit 13.

[0018] Furthermore, the acquisition unit 15a acquires learning data used in training the anomaly-based anomaly detection system. For example, the acquisition unit 15a acquires the learning data used in training the detection model of the anomaly-based anomaly detection system via the input unit 11 or from a management device that manages the learning data, etc., via the communication control unit 13. The acquisition unit 15a may store the previously acquired alert and learning data in the storage unit 14. Alternatively, the acquisition unit 15a may immediately transfer the acquired alert and learning data to a functional unit described below without storing them in the storage unit 14.

[0019] The creating unit 15b creates groups for each predetermined communication feature for alerts indicating that abnormal communication different from normal communication patterns has been detected. Specifically, the creating unit 15b creates groups for which the predetermined communication feature is the same: the source IP address, the destination IP address, and the destination port number.

[0020] Here, Fig. 2 is a diagram for explaining the processing of the creating unit and the identifying unit. Fig. 2 illustrates information items related to each communication, and for example, item numbers 3 to 7 are 5-tuple information. The creating unit 15b groups together alerts that are targeted at communications that have the same source IP address, destination IP address, and destination port number included in the 5-tuple of information items (communication features) illustrated in Fig. 2.

[0021] 3 to 7 are diagrams for explaining the processing of the identifying unit 15c. The identifying unit 15c identifies the cause of the notification of the alert for each group of created alerts.

[0022] Specifically, the identification unit 15c first identifies the cause of the alert notification by determining whether the communication of the alert group has been learned so that it can output whether it is normal or abnormal. That is, the identification unit 15c determines whether the combination of the source IP address, destination IP address, and destination port number that is the same as that of the group exists in the learning data used to learn the detection model of the anomaly-based anomaly detection system. Note that in this embodiment, "learned" means that learning has been performed using a normal pattern.

[0023] For example, as shown in Fig. 3, if any of the source IP address, destination IP address, and destination port number differs from the learned data, it is determined that the information has not been learned. In this way, if the information has not been learned, the identifying unit 15c identifies that the cause of the alert notification is the occurrence of new communication with a new destination. In this case, the abnormality detection device 10 can recommend to the operator that they take action by checking the detailed content of the group's communications.

[0024] Next, when the identification unit 15c has already learned to be able to output whether the communication of the alert group is normal or abnormal, the identification unit 15c determines whether the feature of the communication of this group is similar to the learned feature, thereby identifying the cause of the alert notification. For example, the identification unit 15c focuses on the feature of items 8 to 18 shown in FIG. 2, integrates these feature, and performs clustering using a Gaussian Mixture Model (GMM). Then, it determines whether the clustered class exists in the training data.

[0025] For example, FIG. 4 illustrates the distribution of the total number of upload bytes as a feature of each communication. If the distribution of the feature of the learning data and the distribution of the feature of the alert group differ, as illustrated in FIG. 4, the two feature amounts are determined to be dissimilar. In this way, if the feature of this group's communication is dissimilar to the feature of the learned learning data, the identifying unit 15c identifies that the cause of the alert notification is a change in the communication state, even if the communication destination remains unchanged. In this case, the anomaly detection device 10 recommends that the operator take action by checking the detailed content of the group's communication.

[0026] Next, the identification unit 15c identifies the cause of the alert notification by determining the degree of deviation of the value for detecting abnormal communication from a predetermined threshold value when the characteristics of the communication in the alert group are similar to the learned characteristics.

[0027] For example, FIG. 5 illustrates a case where the threshold value for the anomaly determination is set to 40, and the anomaly determination values ​​for the group's communications are 100, 150, and 130. In this case, the identifying unit 15c determines that the deviation is large, for example, 1.2 times or more. In this way, if the anomaly determination value for the group's communications for this alert deviates significantly from the threshold by a certain multiple or more, it is presumed that the alert was issued due to insufficient maturity of the anomaly detection model. In this case, the anomaly detection device 10 determines that there is insufficient learning data, and recommends, for example, to the operator, to add the group's communications as learning data for the anomaly detection model.

[0028] On the other hand, Figure 6 illustrates a case where the feature values ​​of the learning data (distribution of total number of uplink bytes) indicated by diagonal shading and the feature values ​​of the communication of the alert group are similar, and the anomaly determination values ​​of both data deviate slightly from the threshold by less than a certain multiple. In this way, when the anomaly determination value of the communication of this alert group deviates slightly from the threshold by less than a certain multiple, it is presumed that an anomaly close to normal has occurred, causing the alert to be notified. In this case as well, the anomaly detection device 10 notifies the operator that learning is insufficient, and recommends that the operator, for example, add the communication of this alert group as learning data for the anomaly detection model.

[0029] In this way, the identifying unit 15c identifies the cause of the alert notification for the alert group of each group, as illustrated in FIG. 7. In FIG. 7, group (1) is an alert group for which the occurrence of communication with a new communication destination is identified as the cause of the alert notification. Group (2) is an alert group for which the change in the communication aspect is identified as the cause of the alert notification even though the communication destination remains unchanged. Group (3) is an alert group for which the cause of the alert notification is identified as insufficient maturity of the model, since the determination value for abnormality determination deviates significantly from a predetermined threshold. Group (4) is an alert group for which the cause of the alert notification is identified as insufficient learning, since the determination value for abnormality determination deviates slightly from a predetermined threshold.

[0030] Then, the identifying unit 15c outputs the cause of the alert notification identified for each alert group as a result of the anomaly detection process to the output unit 12. Here, FIG. 8 is a diagram showing an example of a screen display of the anomaly detection process result. As shown in FIG. 8, the identifying unit 15c outputs a similar cause name indicating the identified cause of the alert notification for each group identified by the source IP address, destination IP address, and destination port number. Furthermore, for the above groups (3) and (4), the identifying unit 15c outputs the number of learned learning data items.

[0031] In the example shown in Figure 8, "Occurrence of new communication" in cause classification (1) corresponds to the alert group in group (1) above. Also, "Change in communication status" in cause classification (2) corresponds to the alert group in group (2) above. Also, "Lack of model maturity" in cause classification (3) corresponds to the alert group in group (3) above. Also, "Abnormality close to normal data" in cause classification (4) corresponds to the alert group in group (4) above.

[0032] Returning to the explanation of FIG. 1, the determination unit 15d determines whether or not there is periodicity in the alert occurrence times for each alert group. For example, the determination unit 15d tallies the alert occurrence times for each hour and identifies the number of alerts that have occurred. Then, the determination unit 15d determines whether or not there is periodicity in the alert groups of the group, and if there is periodicity, identifies the number of alerts per cycle.

[0033] The determination unit 15d adds the determined presence or absence of periodicity to the anomaly detection processing result and outputs it, as illustrated in Fig. 8. In the example illustrated in Fig. 8, the number of alerts, the time of occurrence of the alert, and the presence or absence of periodicity are added for each group.

[0034] In this way, by presenting the cause of the alert notification for each alert group, the risk of overlooking true alerts that should be checked is reduced, even when a large number of alerts are generated. For example, for alert groups of cause category (1) and cause category (2), it becomes possible for the operator to recognize that it is necessary to check the detailed content of the group's communications. Furthermore, for alert groups of cause category (3) and cause category (4), it is inferred that there is a lack of learning data, and the operator can recognize that an investigation is necessary.

[0035] [Anomaly detection processing] Next, the abnormality detection process performed by the abnormality detection device 10 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the procedure of the abnormality detection process. The flowchart in Fig. 9 starts, for example, when the user performs an operation input to instruct the start of the process.

[0036] First, the creating unit 15b creates groups for each predetermined communication feature for alerts indicating that abnormal communication different from normal communication patterns has been detected. For example, the creating unit 15b creates groups for which the predetermined communication features are the same: source IP address, destination IP address, and destination port number (step S1).

[0037] Next, the identifying unit 15c identifies the cause of the alert notification for each group of created alerts. Specifically, the identifying unit 15c first determines whether the communication of the alert group has been learned so that it can output whether it is normal or abnormal (step S2). In other words, it determines whether the learned data contains a combination of a source IP address and a destination IP address that is the same as that of the group. If the communication of the group has not been learned (step S2, No), the identifying unit 15c identifies that the cause of the alert notification is the occurrence of communication with a new destination (step S3), and proceeds to step S9.

[0038] On the other hand, if the communication of the alert group has been learned so that it can be output as to whether it is normal or abnormal (step S2, Yes), the identifying unit 15c determines whether the feature of the communication of this alert group is similar to the learned feature (step S4).If the feature of the communication of this alert group is not similar to the learned feature of the learning data (step S4, No), the identifying unit 15c identifies that the cause of the alert notification is a change in the communication state (step S5), and proceeds to step S9.

[0039] On the other hand, for communications in the alert group, if the feature amount of the alert communication is similar to the feature amount that has been learned so as to be able to output whether the communication is normal or abnormal (step S4, Yes), the identifying unit 15c determines the degree of deviation of the judgment value for determining that the communication is abnormal from a predetermined threshold (step S6).If the degree of deviation of the judgment value of the communications in this alert group from the threshold is greater than a predetermined value (step S6, Yes), the identifying unit 15c identifies that the cause of the alert notification is insufficient maturity of the anomaly detection model (step S7), and proceeds to step S9.

[0040] On the other hand, if the deviation of the judgment value of the communication of this alert group from the threshold value is smaller than a predetermined value (step S6, No), the identification unit 15c identifies that the cause of the alert notification is the occurrence of an abnormality that is close to normal (step S8), and proceeds to step S9.

[0041] In the process of step S9, the identifying unit 15c outputs the cause of the alert notification identified for each alert group to the output unit 12 as a result of the abnormality detection process.

[0042] Furthermore, the determination unit 15d determines whether or not there is periodicity in the occurrence times of the alerts for each group of alerts, and adds the determined presence or absence of periodicity to the result of the abnormality detection processing and outputs it, thereby completing the series of abnormality detection processing.

[0043] [effect] As described above, in the anomaly detection device 10 of this embodiment, the creation unit 15b creates groups for each predetermined communication feature amount for alerts indicating that abnormal communication different from normal communication patterns has been detected. Furthermore, the identification unit 15c identifies the cause of the alert notification for each created group of alerts.

[0044] Specifically, the creating unit 15b creates a group of packets having the same source IP address, destination IP address, and destination port number as the predetermined communication feature.

[0045] This makes it possible to determine whether or not a response is required to check the detailed content of the communication for each group of alerts. In this way, it is possible to narrow down the number of alerts that require a response that checks the detailed content of the communication, thereby shortening the time required for checking. Therefore, it is possible to avoid the risk of overlooking a true alert when it exists among a large number of alerts. In this way, it is possible to avoid the risk of overlooking a true alert in an anomaly-based abnormality detection system.

[0046] The identification unit 15c also identifies the cause of the alert notification by determining whether the communication in the alert group has been learned so that it can output whether the communication is normal or abnormal. For example, if the learning has not been completed, it can be identified that the reason is that communication with a new communication destination has occurred. Therefore, it is possible to recommend that the operator take action by checking the detailed content of the communication.

[0047] Furthermore, when the identification unit 15c has learned to be able to output whether the communication of the alert group is normal or abnormal, the identification unit 15c determines whether the feature of the communication of this alert group is similar to the learned feature, thereby identifying the cause of the alert notification. For example, if the feature of the communication of this alert group is not similar to the feature of the learned learning data, it can be assumed that the communication status has changed even if the communication destination remains unchanged. Therefore, it is possible to recommend that the operator take action by checking the detailed content of the communication.

[0048] Furthermore, when the feature value of the communication in the alert group is similar to a feature value that has been learned so that the communication can be determined as normal or abnormal, the identification unit 15c determines the degree of deviation of the value for determining abnormal communication from a predetermined threshold, thereby identifying the cause of the alert notification. For example, when the determination value of the communication in the alert group deviates significantly from the predetermined threshold, it is inferred that this is due to the insufficient maturity of the anomaly detection model. When the determination value of the communication in the alert group deviates slightly from the predetermined threshold, it is inferred that this is due to an anomaly that is close to normal. In either case, it is possible to recommend to the operator that the communication in the alert group be added as learning data for the anomaly detection model.

[0049] Furthermore, the determination unit 15d determines whether or not there is periodicity in the occurrence times of the alerts for each group of alerts. This makes it possible to more reliably avoid overlooking true alerts by checking the alerts while paying attention to the periodicity.

[0050] [program] A program written in a computer-executable language may be created to execute the processing executed by the anomaly detection device 10 according to the above embodiment. In one embodiment, the anomaly detection device 10 can be implemented by installing an anomaly detection program that executes the anomaly detection processing described above as package software or online software on a desired computer. For example, by executing the anomaly detection program on an information processing device, the information processing device can function as the anomaly detection device 10. The information processing device referred to here includes desktop and notebook personal computers. Other examples of information processing devices include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants). The functions of the anomaly detection device 10 may also be implemented on a cloud server.

[0051] 10 is a diagram showing an example of a computer that executes an anomaly detection program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0052] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to, for example, a mouse 1051 and a keyboard 1052. The video adapter 1060 is connected to, for example, a display 1061.

[0053] Here, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiment is stored in the hard disk drive 1031 or memory 1010, for example.

[0054] The anomaly detection program is stored in the hard disk drive 1031, for example, as a program module 1093 in which instructions to be executed by the computer 1000 are written. Specifically, the program module 1093 in which each process executed by the anomaly detection device 10 described in the above embodiment is written is stored in the hard disk drive 1031.

[0055] Furthermore, data used for information processing by the anomaly detection program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-described procedures.

[0056] The program module 1093 and program data 1094 related to the abnormality detection program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041. Alternatively, the program module 1093 and program data 1094 related to the abnormality detection program may be stored in another computer connected via a network such as a LAN or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

[0057] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention. [Explanation of symbols]

[0058] 10. Anomaly detection device 11 Input section 12 Output section 13 Communication control section 14 Storage section 15 Control Unit 15a Acquisition part 15b Creation Department 15c Specific part 15d Discrimination part

Claims

1. a creating unit that creates groups for each predetermined communication feature amount for alerts indicating that abnormal communication different from normal communication patterns has been detected; an identification unit that identifies a cause of the notification of the alert for each of the groups of created alerts; and the identification unit identifies the cause by determining whether learning has been completed so that the communication of the group can be output as normal or abnormal; An anomaly detection device characterized in that, when the communication of the group has already been learned, the cause is identified by determining whether the characteristics of the communication of the group are similar to the learned characteristics.

2. 2. The anomaly detection device according to claim 1, wherein the creating unit creates the group in which the source IP address, destination IP address, and destination port number are the same as the predetermined communication feature.

3. The anomaly detection device according to claim 1, characterized in that the identification unit identifies the cause of the communication of the group by determining the degree of deviation of a value for detecting anomalous communication from a predetermined threshold value when the feature of the communication of the group is similar to a learned feature.

4. 2. The abnormality detection device according to claim 1, further comprising a determination unit that determines, for each group, whether or not there is periodicity in the occurrence times of the alerts.

5. An abnormality detection method executed by an abnormality detection device, a creation step of creating groups for each predetermined communication feature for alerts indicating that abnormal communication that differs from normal communication patterns has been detected; an identifying step of identifying a cause of the notification of the alert for each of the groups of created alerts; Including, the identifying step identifies the cause by determining whether the communication of the group has been learned so that whether the communication is normal or abnormal can be output; An anomaly detection method characterized in that, when the communication of the group has already been learned, the cause is identified by determining whether the characteristics of the communication of the group are similar to the learned characteristics.

6. a creation step of creating a group for each predetermined communication feature for an alert indicating that abnormal communication different from a normal communication pattern has been detected; an identifying step of identifying, for each of the groups of created alerts, a cause for which the alert was notified; on the computer, the identifying step identifies the cause by determining whether the communication of the group has been learned so that whether the communication is normal or abnormal can be output; An anomaly detection program characterized by identifying the cause by determining whether the characteristics of the group's communications are similar to the learned characteristics when the group's communications have already been learned.

Images