Traffic analysis device, traffic analysis program, and traffic analysis method

Abstract

To provide a traffic analysis device, a traffic analysis program, and a traffic analysis method that can analyze communication traffic and detect abnormal communication in real time and efficiently with a single device.SOLUTION: The present invention relates to a traffic analysis device. The traffic analysis device has first analysis means that includes one or more detection processing units that analyze data to be analyzed based on communication traffic data flowing on a network to be analyzed for different items with each other to detect the presence or absence of abnormal communication, and second analysis means for determining the presence or absence of abnormal communication by means of an AI analysis processing unit that performs AI analysis processing using a learning model for data to be analyzed for which presence of abnormal communication is detected by the one or more detection processing units.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This invention relates to a traffic analysis apparatus, a traffic analysis program, and a traffic analysis method, and can be applied, for example, to a system that collects and analyzes communication traffic flowing through a network and detects abnormal communication such as cyberattacks.

Background Art

[0002] Currently, IoT (Internet of Things) devices are being utilized in various environments of organizations such as offices and factories. Organizations such as companies manage the devices connected to the network, such as whether there are unmanaged devices connected to the internal network of the organization or whether unauthorized communication is being generated due to malware infection, etc. Conventionally, by capturing and analyzing the communication traffic flowing through the connection point (such as a core switch) between the core network and the Internet, a communication traffic analysis apparatus is installed to list up the connected devices and monitor (and in some cases, even block communication) whether the devices are engaging in unauthorized communication, thereby managing network devices. However, IoT devices often do not have an Internet connection during normal operation and are frequently used in closed network environments (referred to as edge networks) such as the production site at the end or within an office, and it may not be possible to cover the entire network by only monitoring the core network. Therefore, there is a need for a communication traffic analysis apparatus that installs monitoring points in the edge network where IoT devices are installed and captures and analyzes the traffic flowing through it.

[0003] One method for detecting malicious communications from IoT devices using communication traffic analysis equipment is to employ machine learning. One specific method for detecting malicious communications using machine learning involves unsupervised learning, which uses machine learning to learn the steady-state communication patterns of the network and detects communication patterns that deviate significantly from these patterns as abnormal communications. Unsupervised learning can detect a wide range of abnormal communications that may occur in edge networks, such as communications due to unknown malware infections or communications related to information leaks due to internal fraud. In the conventional unsupervised learning method for detecting malicious communications described above, the accuracy of malicious communication detection can be further improved by building machine learning models that have learned the steady-state communication patterns for each device (i.e., building a machine learning model for each device), thereby reflecting the unique steady-state communication patterns of each device in the learning process.

[0004] Here, since the number of communication traffic analysis devices installed in the edge network is equal to the number of monitoring points, it is desirable that they be inexpensive to introduce. Therefore, the computing resources that traffic analysis devices for edge networks can have are inevitably limited. Traditionally, it has been difficult for edge devices with limited computing resources to analyze computationally intensive tasks, such as machine learning models built using deep learning. However, in recent years, VPUs (Vision Processing Units) have been developed and put into practical use, enabling inference processing of machine learning models using deep learning even on edge devices. VPUs can load and execute machine learning models on hardware, and even when CPUs and memory are not abundant, they can execute inference processing of machine learning models built using deep learning at high speed.

[0005] Therefore, it is desirable for edge network communication traffic analysis devices to be equipped with VPUs, run machine learning models that have learned the steady-state communication patterns of each device on the VPU, and analyze communication traffic in real time to detect abnormal communications. However, since VPUs load machine learning models onto hardware, loading the models takes time. Therefore, if a machine learning model is prepared for each device and the analysis is performed by switching models according to the received traffic, the model switching time becomes a bottleneck, and there is a problem in that real-time analysis cannot be performed. Thus, there is a need for a method that can detect abnormal communications in real time even in a configuration in which model switching occurs.

[0006] In light of the above background, the technology described in Patent Document 1 has been proposed. The technology described in Patent Document 1 concerns a communication control device that collects communications from IoT devices, defines a steady communication range for each device, detects communications that deviate from that range as abnormal communications, and performs control such as communication blocking. Patent Document 1 also mentions a method for analyzing communications using shared information with other communication control devices when the volume of communications from IoT devices is large and cannot be processed by a single communication control device. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2019-153894 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, the technology described in Patent Document 1 is based on the premise that multiple communication control devices work together, and cannot be applied when operating as a single device.

[0009] Therefore, there is a need for a traffic analysis device, a traffic analysis program, and a traffic analysis method that can analyze communication traffic in real time and efficiently and detect abnormal communications using a standalone device. [Means for solving the problem]

[0010] The first traffic analysis device of the present invention includes a detection processing unit that analyzes data based on communication traffic data flowing on the network to be analyzed using different items to detect the presence or absence of abnormal communication. 2 A first analysis means comprising the above-mentioned means, and a second analysis means that determines the presence or absence of abnormal communication by an AI analysis processing unit that performs AI analysis processing using a learning model on the data to be analyzed which has been detected as having abnormal communication by one or more of the detection processing units. The system includes an analysis control means which performs a priority determination process to determine the priority of the data to be analyzed for which the second analysis means will perform the AI ​​analysis process, and which causes the second analysis means to perform the AI ​​analysis process on the data to be analyzed in the order determined by the result of the priority determination process, wherein the analysis control means determines the priority of the data to be analyzed according to the detection processing unit which has detected abnormal communication by the first analysis means. It is characterized by the following:

[0011] The second traffic analysis program of the present invention includes a detection processing unit that analyzes data based on communication traffic data flowing on the network under analysis using different items to detect the presence or absence of abnormal communication. 2 A first analysis means comprising the above-mentioned means, and a second analysis means that determines the presence or absence of abnormal communication by an AI analysis processing unit that performs AI analysis processing using a learning model on the data to be analyzed which has been detected as having abnormal communication by one or more of the detection processing units. The analysis control means performs a priority determination process to determine the priority of the data to be analyzed for which the second analysis means will perform the AI ​​analysis process, and functions as an analysis control means to cause the second analysis means to execute the AI ​​analysis process on the data to be analyzed in the order determined by the result of the priority determination process, and the analysis control means determines the priority of the data to be analyzed according to the detection processing unit which has detected abnormal communication by the first analysis means. It is characterized by the following:

[0012] The third aspect of the present invention relates to a traffic analysis method performed by a traffic analysis device, wherein the traffic analysis device comprises a first analysis means and a second analysis means Analytical control means and The first analysis means includes a detection processing unit that analyzes the data to be analyzed, which is based on communication traffic data flowing on the network to be analyzed, using different items to detect the presence or absence of abnormal communication. 2 The above-mentioned second analysis means determines whether or not abnormal communication exists by an AI analysis processing unit that performs AI analysis processing using a learning model on the data to be analyzed which has been detected as having abnormal communication by one or more of the detection processing units, The analysis control means performs a priority determination process to determine the priority of the data to be analyzed for which the second analysis means will perform the AI ​​analysis process, and instructs the second analysis means to perform the AI ​​analysis process on the data to be analyzed in the order determined by the result of the priority determination process, and the analysis control means determines the priority of the data to be analyzed according to the detection processing unit which has detected abnormal communication by the first analysis means. Characterized by doing so.

Advantages of the Invention

[0013] According to the present invention, it is possible to analyze communication traffic in real time and efficiently and detect abnormal communication with a single device.

Brief Description of the Drawings

[0014] [Figure 1] It is a block diagram showing the functional configuration of the traffic analysis device according to the first embodiment. [Figure 2] It is a block diagram showing the connection relationship of each device related to the first embodiment. [Figure 3] It is a block diagram showing an example of the hardware configuration of the traffic analysis device according to the first embodiment. [Figure 4] It is a diagram showing an example of the configuration of the priority information used in the AI analysis kicker unit according to the first embodiment. [Figure 5] It is a diagram showing an example of the configuration of the AI applied in the AI analysis unit according to the first embodiment. [Figure 6] It is a diagram showing an overview of the analysis process in the traffic analysis device according to the first embodiment. [Figure 7] It is a flowchart (Part 1) showing the operation of the traffic analysis device according to the first embodiment. [Figure 8] It is a flowchart (Part 2) showing the operation of the traffic analysis device according to the first embodiment. [Figure 9] It is a flowchart (Part 3) showing the operation of the traffic analysis device according to the first embodiment. [Figure 10] It is a flowchart (Part 4) showing the operation of the traffic analysis device according to the first embodiment. [Figure 11] It is a diagram showing a specific example of the prioritization process of the AI analysis target flow by the AI analysis kicker unit according to the first embodiment. [Figure 12] It is a block diagram showing the functional configuration of the traffic analysis device according to the second embodiment. [Figure 13] It is a block diagram showing the functional configuration of the traffic analysis device according to the third embodiment. [Figure 14] It is a block diagram showing the functional configuration of the traffic analysis device according to the fourth embodiment. [Figure 15] It is a block diagram showing the functional configuration of the traffic analysis device according to the fifth embodiment. [Figure 16] It is a diagram showing a configuration example of the abnormal reason condition list used in the abnormal determination cause estimation unit according to the fifth embodiment.

Embodiments for Carrying Out the Invention

[0015] (A) First Embodiment Hereinafter, the first embodiment of the traffic analysis device, traffic analysis program, and traffic analysis method according to the present invention will be described in detail with reference to the drawings.

[0016] (A-1) Configuration of the First Embodiment FIG. 2 is a block diagram showing the connection relationship of each device related to the first embodiment. The reference numerals in parentheses in FIG. 2 are the reference numerals used in the second to fifth embodiments described later.

[0017] Here, the traffic analysis device 10 is assumed to perform a process of analyzing traffic generated from monitoring target devices 30 (30-1, 30-2,...) as communication devices connected to a monitoring target network N1 (analysis target). In the configuration example of FIG. 2, it is assumed that a network switch 20 (layer 2 switch) and a gateway 40 (for example, a gateway corresponding to the function of a firewall) are arranged as network devices in the monitoring target network N1. Note that the communication devices connected to the monitoring target network N1 are not limited to IoT devices and may be various devices.

[0018] The monitored devices 30 include various communication terminals such as IoT devices and PCs (for example, PCs operated by users). The traffic analysis device 10 manages the type / attributes (hereinafter referred to as "device type") of each monitored device 30. In this embodiment, the device types managed by the traffic analysis device 10 will be described as including at least "IoT devices" and "PCs".

[0019] In the configuration example shown in Figure 2, each monitored device 30 is connected to a network switch 20 (Layer 2 switch), and a gateway 40 is connected between the network switch 20 and the internet N2. Therefore, in the example shown in Figure 2, each monitored device 30 is capable of communicating with the internet N2 via the network switch 20 and the gateway 40. Furthermore, the number of monitored devices 30 etc. connected to the network switch 20 is not limited and may increase or decrease during operation. The network switch 20 is assumed to be a switch that accommodates monitored devices 30 belonging to a so-called edge network. The specific functions, configurations, and communication content of each monitored device 30 are not limited.

[0020] In the example shown in Figure 2, the traffic analysis device 10 is assumed to be connected to the network switch 20. Here, it is assumed that the network switch 20 is configured to mirror packets (Ethernet frames) sent and received on the LAN port (Ethernet® port) connected to the gateway 40 to the port to which the traffic analysis device 10 is connected (so-called port mirroring connection). As a result, the traffic analysis device 10 can collect traffic (packets) flowing between the connected terminals (monitored devices 30) under the network switch 20 and the internet N2. In this embodiment, the network switch 20's port mirroring setting supplies the traffic analysis device 10 with data based on communication traffic flowing between the monitored devices 30 and the internet N2 (hereinafter also simply referred to as "communication traffic data"), but the specific configuration for supplying communication traffic data to the traffic analysis device 10 is not limited. Note that the configuration of the monitored network N1 shown in Figure 1 is just one example, and various configurations may be used as long as they can supply communication traffic data to the traffic analysis device 10.

[0021] The traffic analysis device 10 analyzes the communication of each monitored device 30 using communication traffic data or data based on communication traffic data (for example, flow information described later; hereinafter also referred to as "data to be analyzed"), and outputs the results. The traffic analysis device 10 can perform AI-based analysis processing (hereinafter referred to as "AI analysis processing") and non-AI analysis processing (hereinafter referred to as "non-AI analysis processing") on the data to be analyzed. In this embodiment, the traffic analysis device 10 is configured to receive the communication traffic data (packets) itself, but it may also be configured to receive data to be analyzed based on communication traffic data (for example, flow information for each packet).

[0022] Next, the internal configuration of the traffic analysis device 10 will be described.

[0023] Figure 1 is a block diagram showing the functional configuration of the traffic analysis device 10 according to the first embodiment.

[0024] As shown in Figure 1, the traffic analysis device 10 includes a traffic capture unit 11, a control unit 12, a learning unit 13, a non-AI analysis unit 14, an AI analysis kicker unit 15, a storage unit 16, an AI analysis unit 17, and a notification / visualization unit 18.

[0025] The traffic analysis device 10 may be composed entirely of hardware (e.g., a dedicated chip), or it may be composed entirely of software (a program). The traffic analysis device 10 may also be configured, for example, by installing a program (including the traffic analysis program of the embodiment) on a computer having a processor and memory.

[0026] Figure 3 shows an example of the hardware configuration when configuring the traffic analysis device 10 using software (computer).

[0027] The traffic analysis device 10 shown in Figure 3 has a computer 200 on which a program (including the traffic analysis program of this embodiment) is installed as a hardware component. The computer 200 may be a computer dedicated to the traffic analysis program, or it may be configured to be shared with programs for other functions.

[0028] The computer 200 shown in Figure 3 includes a CPU 201, a primary storage unit 202, a secondary storage unit 203, and a VPU 204.

[0029] In this embodiment, the computer 200 is a computer used for so-called edge computing, and is configured with limited computing resources from the standpoint of power consumption and cost. In other words, the computer 200 is required to efficiently utilize its limited computing resources. However, since the traffic analysis device 10 includes AI analysis processing with a heavy processing load (for example, processing of deep neural networks, etc.), in addition to the CPU 201, it is equipped with a VPU 204 that can efficiently execute AI (especially DNN) related processing. For example, Intel's Movidius® can be used as the VPU 204, but various other VPUs may also be used. In addition, the computer 200 may replace the VPU 204 with another device (chip) if it is possible to efficiently perform AI related processing.

[0030] The primary storage unit 202 is a storage means that functions as the CPU 201's working memory, and can be a high-speed memory such as DRAM (Dynamic Random Access Memory).

[0031] The secondary storage unit 203 is a storage means for recording various data such as the OS (Operating System) and program data (including data for the traffic analysis program according to the embodiment), and can be a non-volatile memory such as FLASH® memory, HDD, or SSD.

[0032] In the computer 200 of this embodiment, when the CPU 201 starts up, it reads the OS and programs (including the traffic analysis program according to the embodiment) recorded in the secondary storage unit 203, loads them onto the primary storage unit 202, and executes them. As a result of operating according to the traffic analysis program, the CPU 201 may load and execute AI-related modules (software) on the VPU 204.

[0033] It should be noted that the specific configuration of computer 200 is not limited to the configuration shown in Figure 3, and various configurations can be applied. For example, if the primary storage unit 202 is non-volatile memory (e.g., FLASH memory), the secondary storage unit 203 may be omitted. Also, for example, the VPU 204 may be omitted from the computer 200.

[0034] Next, we will explain the overview of each component of the internal configuration (functional configuration) of the traffic analysis device 10 using Figure 1.

[0035] The traffic capture unit 11 has means for capturing communication traffic data based on communication traffic flowing through the edge network and for extracting flow information from the communication traffic data.

[0036] A "flow" is an element of communication traffic identified by a set of source IP address, source port number, destination IP address, destination port number, and protocol number. Hereafter, information that can identify a flow (for example, a set of source IP address, source port number, destination IP address, destination port number, protocol number, etc.) will also be referred to as "flow identification information." Hereafter, the monitored device that is the source of each flow's packets will also be referred to as the "source device." In the traffic analysis device 10, flow identification information is associated with each flow information (that is, the flow identification information corresponding to each flow information is kept in a state where it can be obtained).

[0037] "Flow information" refers to information indicating the communication status of a flow, and includes, for example, statistical information obtained from the communication traffic data of the flow (such as packet size and packet arrival interval). The traffic analysis device 10 analyzes this flow information to make a judgment regarding abnormalities in each monitored device 30. The traffic capture unit 11 notifies the control unit 12 of the flow information.

[0038] The control unit 12 controls each element within the traffic analysis device 10 to manage and control the analysis process for each monitored device 30.

[0039] The AI ​​analysis unit 17, following the control of the AI ​​analysis kicker unit 15, performs analysis processing on flow information (hereinafter also referred to as "AI analysis processing") using a machine learning model.

[0040] The non-AI analysis unit 14 performs "non-AI analysis processing" (analysis processing without using AI) on one or more items of the flow information.

[0041] The learning unit 13 performs machine learning processing to obtain a learning model (machine learning model) to be applied to the AI ​​analysis unit 17, in accordance with the control of the control unit 12. The learning unit 13 may also perform processing to calculate parameters used for non-AI analysis in the non-AI analysis unit 14 (for example, statistical thresholds and pattern matching rules used for non-AI analysis processing; hereinafter referred to as "analysis parameters"). In this embodiment, the learning unit 13 can obtain a machine-learned learning model (autoencoder learning model) by having an autoencoder learn from data based on flow information in the steady state of each monitored device 30 (data obtained by vectorizing the features of the flow information). The AI ​​analysis unit 17 then performs processing to analyze the state of the monitored device 30 by inputting the flow information of the monitored device 30 (for example, newly collected flow information) into the autoencoder, which is set with the machine-learned learning model (learned using steady-state flow). In this embodiment, the AI ​​analysis unit 17 will be described as using a VPU 204 as a hardware resource when performing AI analysis processing using a deep neural network (DNN) such as the autoencoder described above.

[0042] The AI ​​analysis kicker unit 15 evaluates the results of the non-AI analysis processing performed by the non-AI analysis unit 14 for each flow and determines the priority for performing AI analysis processing for each flow based on the evaluation results. The AI ​​analysis kicker unit 15 then controls the AI ​​analysis unit 17 to perform AI analysis processing in the order determined by the priority. As described above, the AI ​​analysis unit 17 performs AI analysis processing using the VPU 204, but the number of AI analysis processes that the VPU 204 can execute simultaneously is limited. Therefore, the traffic analysis device 10 is configured to manage the queue (a queue of flows waiting for AI analysis processing) by providing the AI ​​analysis kicker unit 15 to prioritize AI analysis processing for flows with high priority (e.g., security urgency). In this embodiment, it is explained that the number of AI analysis processes that the VPU 204 can execute simultaneously (i.e., the number of learning models that can be loaded simultaneously) is one, but it may be two or more.

[0043] The notification / visualization unit 18 is an output means that outputs the final analysis results (judgment results) for each flow. The specific means (notification method and visualization means) and content of the notification / visualization unit 18 outputting the analysis results are not limited, and various configurations can be applied. The notification / visualization unit 18 may output all analysis results, or it may output only some of the analysis results. In this embodiment, the notification / visualization unit 18 will be described as outputting analysis results only for flows that have been determined to be abnormal (flows that have been determined not to be in a steady state). Specifically, when the notification / visualization unit 18 receives a flow that has been determined to be abnormal (information that identifies the flow) from the AI ​​analysis unit 17, it performs a process of notifying the administrator and visualizing the flow. The notification / visualization unit 18 may, for example, send information regarding the flow that has been determined to be abnormal (hereinafter referred to as "notification information") by email to a predetermined address (for example, the network administrator's email address). Furthermore, the notification / visualization unit 18 may display the flow that has been detected as an anomaly as a web page that can be displayed on a web browser (for example, it may display a web screen (GUI screen) for terminals that have accessed the traffic analysis device 10 via a web browser (e.g., http, etc.)).

[0044] Next, the detailed configuration of the control unit 12 will be described.

[0045] The control unit 12 manages the status related to the analysis process (hereinafter also referred to as "device status") for each monitored device 30. Here, the device status managed by the control unit 12 is described as including at least three states: "unregistered state," which is the state of being unregistered (unmanaged) because it is the monitored device 30 that has received flow information for the first time (the first device to receive flow information as a source device); "learning state," which indicates that the learning unit 13 is in the process of steady-state machine learning; and "analysis state," which is the state in which a steady-state machine learning model has been acquired and analysis processing is possible (or is in the process of analysis processing).

[0046] The control unit 12 manages the machine learning status of the steady communication pattern for each monitored device 30 (machine learning status by the learning unit 13), and distributes the flow information received from the traffic capture unit 11 to the other units according to the learning status of the source device of the flow. The control unit 12 also treats a certain period of time from when the monitored device 30 corresponding to the received flow information is connected (for example, from when the flow information is first received) as the learning state of the steady communication pattern. In the learning state, the control unit 12 notifies the learning unit 13 of the flow information. When the learning state period for a certain monitored device 30 has elapsed, the control unit 12 requests the learning unit 13 to calculate analysis parameters to be used in the non-AI analysis processing by the non-AI analysis unit 14, and to construct a machine learning model to be used for anomaly detection by the AI ​​analysis unit 17, described later. When the control unit 12 receives notification from the learning unit 13 that learning is complete, it treats the monitored device 30 as being in the analysis state. In the analysis state, the control unit 12 notifies the non-AI analysis unit 14 and the AI ​​analysis unit 17 of the received flow information.

[0047] When the learning unit 13 completes various learning tasks, it notifies the control unit 12 that the learning is complete.

[0048] Next, we will describe the detailed configuration of the non-AI analysis unit 14.

[0049] When the non-AI analysis unit 14 receives flow information from the control unit 12, it obtains analysis parameters corresponding to the source device of the flow information from the storage unit 16 and performs processing to extract communications suspected of being abnormal from the flow information using non-AI analysis methods (for example, statistical or pattern matching methods). The non-AI analysis unit 14 is equipped with multiple types of programs (hereinafter referred to as "detection engines" or "detection processing units") that perform analysis processing to detect abnormalities based on flow information. The multiple types of detection engines equipped in the non-AI analysis unit 14 each perform processing to extract communications suspected of being abnormal from different perspectives. Each detection engine equipped in the non-AI analysis unit 14 is capable of operating independently and concurrently. The analysis methods and perspectives used by the detection engines are not limited.

[0050] For example, the non-AI analysis unit 14 may be equipped with a detection engine that performs the following: a process to detect whether the monitored device 30 is communicating with a communication partner it does not normally communicate with (for example, a communication partner it does not normally communicate with) (hereinafter referred to as a "rare communication state") (hereinafter referred to as a "rare communication state"), a process to detect whether the amount of communication from the monitored device 30 is statistically higher than normal (hereinafter referred to as a "communication volume anomaly state") (hereinafter referred to as a "communication volume anomaly detection process"), and a process to detect whether the device is in a state where it is suspected of being "under attack by port scanning" or "carrying a port scanning attack by malware, etc." (hereinafter referred to as a "scan suspicion state") (hereinafter referred to as a "scan suspicion detection process"). The non-AI analysis unit 14 notifies the AI ​​analysis kicker unit 15 of any flow that has been determined to be abnormal by any of the detection engines in the detection engine group, along with the name representing the detection engine (hereinafter referred to as the "detection engine name"). In this embodiment, the detection engine name is described as information corresponding to the anomaly item (index). In this embodiment, the non-AI analysis unit 14 is described as comprising at least a detection engine that performs rare communication detection processing (detection engine name: rare communication), a detection engine that performs communication volume anomaly detection processing (detection engine name: communication volume anomaly), and a detection engine that performs scan suspicion detection processing (detection engine name: scan suspicion). However, the types and combinations of detection engines installed in the non-AI analysis unit 14 are not limited.

[0051] Next, we will describe the detailed configuration of the learning unit 13.

[0052] When the learning unit 13 receives flow information from the control unit 12, it stores the flow information in the storage unit 16. Furthermore, when the learning unit 13 receives a learning request for the monitored device 30 from the control unit 12, it retrieves the flow information of the monitored device 30 from the storage unit 16 and constructs a machine learning model for use in the AI ​​analysis unit 17. When the learning unit 13 constructs the machine learning model, it may utilize the VPU 204 as a hardware resource, or it may be implemented using a program running on the CPU 201.

[0053] Furthermore, when the learning unit 13 receives a learning request for the monitored device 30 from the control unit 12, it may also perform a process to calculate the analysis parameters (analysis parameters for the monitored device 30) to be used by each detection engine of the non-AI analysis unit 14 as part of the learning process.

[0054] The learning unit 13 can perform the same processing as various communication security detection engines to acquire the analysis parameters (analysis parameters of the monitored device 30) used by each detection engine. For example, it may perform the following processing: For a detection engine that detects a "rare communication state," for example, it may store the address or hostname of a communication partner that communicates in a steady state as an analysis parameter, and detect a rare communication state when communication starts with a communication partner other than those stored. For a detection engine that detects an "abnormal communication volume state," for example, it may store the communication volume in a steady state (for example, the amount of data or number of packets per minute) as an analysis parameter, and store a threshold of steady state communication volume + α (where α is, for example, about 50% of steady state communication volume) as an analysis parameter, and detect an abnormal communication volume state when the communication volume exceeds the stored threshold. Furthermore, the detection engine for detecting a "suspected scan state" may, for example, store the pattern of a packet sequence under a scan attack (the pattern of flow information constituting the packet sequence; the rule for pattern matching) as an analysis parameter, and detect a suspected scan state when a packet sequence matching that pattern occurs. Since packet sequences in a scan attack state cannot be stored in the steady state, the learning unit 13 may pre-set patterns for each device type or common to all devices in the detection engine for detecting a suspected scan state. The learning unit 13 may also use the number of flows generated per unit time in the steady state of the monitored device 30 as a parameter for the steady state. In this case, for each monitored device 30, a suspected scan can be detected if the number of flows generated per unit time from itself deviates significantly from the steady state value, or if the number of flows generated for itself per unit time deviates significantly from the steady state value.

[0055] Next, we will explain the detailed configuration of the AI ​​analysis kicker unit 15.

[0056] When the AI ​​analysis kicker unit 15 receives a flow that has been judged as abnormal from the non-AI analysis unit 14 and the name of the detection engine that judged the flow as abnormal, it obtains priority information 161 from the storage unit 16, evaluates the priority of the flow based on the contents of the priority information 161, and determines the priority (order) for which the AI ​​analysis unit 17 will perform AI analysis on the flow. The priority information 161 is used to determine which flow to prioritize for AI analysis when there are multiple flows that have been judged as abnormal by one or more detection engines in the non-AI analysis unit 14, and consists of one or more items, with a priority assigned to each item. In other words, the priority information 161 is information that defines a policy for evaluating flows based on the name of the detection engine that judged them as abnormal.

[0057] Figure 4 shows an example of the configuration of priority information 161.

[0058] In this embodiment, the priority information 161 is described as containing information with the configuration shown in Figure 4. Figure 4(a) is a diagram showing the priority list that constitutes the priority information 161.

[0059] In the priority list shown in Figure 4(a), the priority is described for each item name of the flow evaluation criteria. A lower priority value indicates a higher priority for that item name.

[0060] Figure 4(a) shows the criteria (evaluation criteria) for item names, including "number of detection items," "presence or absence of priority detection engines for each device type," and "presence or absence of a priority engine common to all devices." In Figure 4(a), the priorities for "number of detection items," "presence or absence of priority detection engines for each device type," and "presence or absence of a priority detection engine common to all devices" are 1, 2, and 3, respectively. In other words, Figure 4(a) indicates that the item "number of detection items" is the highest priority criterion.

[0061] In the "number of detected items" criterion, flows that have detected anomalies by more detection engines for the same flow are judged to have a higher priority.

[0062] The criteria for determining the "presence or absence of a priority detection engine for each device type" involves preparing a list of detection engines with a high probability of detecting anomalies for each device type (attribute) to which the monitored device 30 belongs. Flows that have been identified as abnormal by the detection engine listed as the priority detection engine for the device type of the flow's source device are given higher priority. For example, in the case of a PC, since it is generally used by users, the communication destinations tend to change daily, so the detection engine that detects rare communications has a low priority. However, if the communication volume is much higher than usual, there is a suspicion of an anomaly, so it is desirable to set the detection engine that detects abnormal communication volume as the priority detection engine. On the other hand, IoT devices usually only communicate with predetermined communication destinations, so it is desirable to set the detection engine that detects rare communications as the priority detection engine.

[0063] Figure 4(b) shows a list of priority detection engine names (item names) for each device type, which are used as criteria for determining the presence or absence of a priority detection engine for each device type. In Figure 4(b), "Communication Volume Anomaly" is set as the priority detection engine name (item name) for the "PC" model type. Also in Figure 4(b), "Rare Communication" is set as the priority detection engine name (item name) for the "IoT Device" model type. In Figure 4(b), one priority detection engine is assigned to each device type, but two or more detection engines may be set. In this case, the AI ​​analysis kicker unit 15 may determine that there is a priority detection engine for each device type if any of the priority detection engine names match for a device type in which multiple detection engines are set. The method by which the AI ​​analysis kicker unit 15 maintains the device type information of each monitored device 30 is not limited. For example, the traffic analysis device 10 may be equipped with means for manually receiving information on the type of equipment for each monitored device 30 by an operator (administrator), or a method may be applied to estimate the type of equipment from the communication traffic using a different AI analysis algorithm.

[0064] The "presence or absence of a common priority detection engine for all devices" criterion uses a common set of detection engine priority information 161 regardless of the device type. Flows detected as abnormal by a detection engine with a higher priority are considered to have a higher priority. Figure 4(c) shows a list of priority detection engine names (item names) used in the "presence or absence of a common priority detection engine for all devices" criterion. In Figure 4(c), the priority is described for each priority detection engine name (item name). A smaller priority value indicates a higher priority for that priority detection engine name (item name). Also, in Figure 4(c), the priorities for "suspected scan," "abnormal communication volume," and "rare communication" are 1, 2, and 3, respectively. In other words, Figure 4(c) shows that the "suspected scan" item is the highest priority criterion.

[0065] As described above, the AI ​​analysis kicker unit 15 reads priority information 161 from the memory unit 16, prioritizes the flows extracted by the non-AI analysis unit 14 based on the priority information 161, and notifies the AI ​​analysis unit 17 of the flows (identification information for identifying flows) in order of priority.

[0066] Next, we will explain the detailed configuration of the AI ​​analysis unit 17.

[0067] When the AI ​​analysis unit 17 receives flow information from the control unit 12, it stores the received flow information in the storage unit 16. Then, when the AI ​​analysis unit 17 receives notification from the AI ​​analysis kicker unit 15 of a flow to be processed by AI analysis (for example, notification of flow identification information), it retrieves the flow information of the corresponding flow from the storage unit 16, retrieves the machine learning model of the source device (monitored device 30) of the flow from the storage unit 16, performs AI analysis, and determines if an anomaly occurs. The AI ​​analysis unit 17 loads the machine learning model, which has learned the steady-state communication pattern of the source device (monitored device 30) of the flow, into the VPU 204, inputs the flow information, and performs the anomaly determination process. The deep learning machine learning model and anomaly determination method used by the AI ​​analysis unit 17 are not limited, but for example, a method using an autoencoder may be applied that converts the flow information of a predetermined number of packets (for example, several tens of packets) at the beginning (latest) of the flow into vector data (vector data composed of feature quantities), compresses the dimension of the feature quantities, and then reconstructs it to the input dimension. In this case, the AI ​​analysis unit 17 may determine that a flow (flow information) is abnormal if there is a large discrepancy between the vector data input to the autoencoder and the vector data output from the autoencoder (for example, if there is a difference of more than a predetermined amount between the input and output sides). The format of the vector data (features) processed by the autoencoder (DNN) can be, for example, a combination of data such as packet size and packet arrival interval obtained from the flow information. Furthermore, the process of acquiring a learning model based on steady-state flow information in the monitored device 30 and using this learning model to determine the latest communication status of the monitored device 30 (whether or not abnormal communication is included) is not limited to the above example and various configurations can be applied.

[0068] Figure 5 shows an example of anomaly detection processing for flow (flow information) using AI (Autoencoder / DNN).

[0069] Figure 5 shows an example of performing AI analysis processing (anomaly detection processing) on ​​flow information based on the flow of a certain monitored device 30 (hereinafter referred to as "device A").

[0070] In the AI ​​analysis process (anomaly detection process) shown in Figure 5, the source data used is a time-series arrangement of flow information pkt1 to pkti for i packets (where i is an integer greater than or equal to 2; i may be a value of several tens, for example) from the beginning of the flow of device A. In the AI ​​analysis process (anomaly detection process) shown in Figure 5, packet size, packet arrival interval, etc., are extracted from the flow information of each packet and converted into feature data, which is obtained as input data fval. In Figure 5, the data corresponding to the flow information pkt1 to pkti are shown as fval1 to fvali. Then, in Figure 5, the input data fval (fval1 to fvali) is input to the autoencoder 171 (input layer), and the output data fval' (fval1' to fvali') is output from the autoencoder 171 (output layer). The data fval1' to fvali' that make up the output data fval' are output data (reconstructed data) that correspond to the data fval1 to fvali that make up the input data fval, respectively. The AI ​​analysis unit 17 may compare the input data fval(fval1~fvali) with the output data fval'(fval1'~fvali') and determine that the flow is in an abnormal state if the difference (loss due to the autoencoder 171; for example, the mean square error) is greater than or equal to a certain value. If the AI ​​analysis unit 17 determines that an abnormality has occurred through AI analysis, it notifies the notification / visualization unit 18 of the flow in which the abnormality has been detected.

[0071] The memory unit 16 is a storage means for storing various information used in the processing of the traffic analysis device 10. The memory unit 16 stores, for example, priority information 161, learning flow information used in the learning unit 13, anomaly detection flow information used in the AI ​​analysis unit 17, analysis parameters used in the detection engine of the non-AI analysis unit 14, and machine learning models used in the AI ​​analysis unit 17. For example, the memory unit 16 may manage information used in the AI ​​analysis processing of the AI ​​analysis unit 17 (for example, anomaly detection flow information) with a time limit (for example, 1 minute), and delete expired flow information sequentially. The memory unit 16 may also delete the flow information of monitored devices 30 that have completed learning in the learning unit 13.

[0072] (A-2) Operation of the first embodiment Next, the operation of the traffic analysis device 10 of the first embodiment having the above configuration (traffic analysis method according to the embodiment) will be described.

[0073] Figure 6 is a diagram illustrating the overview of the analysis process in the traffic analysis device 10.

[0074] In the traffic analysis device 10, the control unit 12 supplies flow information based on communication traffic data to each detection engine of the non-AI analysis unit 14. Each detection engine analyzes its respective items (for example, items related to communications suspected of being abnormal communications such as attacks) to determine whether or not there is a communication anomaly, and the determination result is supplied to the AI ​​analysis kicker unit 15. The AI ​​analysis kicker unit 15 then aggregates the detection results of each detection engine and instructs the AI ​​analysis unit 17 to perform AI analysis processing on flows for which communication anomalies have been determined by one or more detection engines. The AI ​​analysis unit 17 loads a machine learning model of the source device of the flow instructed by the AI ​​analysis kicker unit 15 into the VPU 204. The AI ​​analysis unit 17 then acquires the flow information of the flow instructed by the AI ​​analysis kicker unit 15 (for example, flow information for the first i-packet), inputs it into the machine learning model (autoencoder 171), performs AI analysis processing, and determines whether or not there is a communication anomaly based on the results of the AI ​​analysis processing. The AI ​​analysis unit 17 supplies the determination result to the notification / visualization unit 18. The notification / visualization unit 18 performs processing to output the judgment results, etc., supplied from the AI ​​analysis unit 17 in a predetermined format and by predetermined means (for example, processing of notification to the operator and visualization).

[0075] Next, the detailed operation of the traffic analysis device 10 will be explained using the flowcharts in Figures 7 to 10.

[0076] Figure 7 shows the operation when communication traffic data (hereinafter referred to as "focused communication traffic data") of a flow originating from an arbitrary monitored device 30 (hereinafter referred to as "monitored device 30") is supplied to the traffic capture unit 11.

[0077] First, here we assume that the communication traffic data of a flow with the device of interest as the source device has been received by the traffic capture unit 11 (S101).

[0078] When the traffic capture unit 11 receives communication traffic data, it extracts flow information from the communication traffic data and notifies the control unit 12 of the flow information. The control unit 12 obtains the address information of the device of interest from the received flow information and checks the status of the device of interest (either unregistered, learned, or analyzed) (S102).

[0079] If the device under consideration is the first monitored device 30 to receive communication traffic data, the control unit 12 determines that the device under consideration is unregistered and proceeds to step S103, which will be described later. If the device under consideration is in the learning state, the control unit 12 proceeds to step S106, which will be described later, and if it is in the analysis state, it proceeds to step S111, which will be described later.

[0080] If the status of the device under interest was unregistered in step S102 described above, the control unit 12 sets the status of the device under interest to the learning state (S103), and further sets a period for acquiring the steady communication pattern data necessary for learning for the device under interest (S104). The method of setting this period is not limited, but here it is assumed that the same period (for example, one week) is set for all devices. Then, the control unit 12 notifies the learning unit 13 of the flow information, causing the learning unit 13 to save the flow information to the storage unit 16 (S105), and terminates the processing of the communication traffic data under interest.

[0081] If the state of the device of interest in step S102 described above is in the learning state, the control unit 12 checks whether the acquisition period for the steady communication pattern data of the device of interest has elapsed (S106). If the acquisition period for the steady communication pattern data has not elapsed, the control unit 12 proceeds to step S105 described above; otherwise, it proceeds to step S107 described later.

[0082] If the acquisition period for the steady-state communication pattern data of the device of interest has elapsed in step S106 described above, the control unit 12 checks with the learning unit 13 whether a learning request for the device of interest has been sent (S107). If it has been confirmed, the unit proceeds to step S109 described later; otherwise, the unit proceeds to step S108 described later.

[0083] If it is confirmed in step S107 above that a learning request for the device of interest has not been sent to the learning unit 13, the control unit 12 requests (instructs) the learning unit 13 to start the learning process for the device of interest (S108), and terminates the processing of the communication traffic data of interest. Details of the learning process by the learning unit 13 will be described in detail in the flowchart of Figure 8 below.

[0084] If it is confirmed in step S107 above that a learning request for the device of interest has been sent to the learning unit 13, the control unit 12 checks whether it has received a learning completion notification from the learning unit 13 indicating that the learning process for the device of interest has been completed (S109). If the control unit 12 has received a learning completion notification for the device of interest, it proceeds to step S110 described later; otherwise, it proceeds to step S112 described later.

[0085] If, in step S109 described above, it is confirmed that the learning unit 13 has not received a learning completion notification regarding the device of interest, the control unit 12 discards the received flow information (S112) and terminates processing of the communication traffic data of interest.

[0086] On the other hand, if, in step S109 described above, the control unit 12 confirms that it has received a learning completion notification from the learning unit 13 indicating that the learning process for the device of interest has been completed, the control unit 12 sets the status of the device of interest to the analysis state (S110), notifies the non-AI analysis unit 14 and the AI ​​analysis unit 17 of the flow information, starts the execution of the analysis process (S111), and terminates the processing related to the communication traffic data of interest. Details of the analysis process (details of the processing by the non-AI analysis unit 14, the AI ​​analysis kicker unit 15, and the AI ​​analysis unit 17) will be explained later using the flowchart in Figure 9.

[0087] Figure 8 is a flowchart illustrating the operation of the learning unit 13 when it performs learning processing on the device of interest.

[0088] When the learning unit 13 receives an instruction from the control unit 12 to start the learning process for the monitored device 30 in step S108 described above (S201), it acquires flow information of the device of interest from the storage unit 16 (S202).

[0089] Next, the learning unit 13 uses the acquired flow information to construct a machine learning model used for AI analysis (S203). The machine learning model to be constructed is, for example, a model to be applied to the autoencoder 171 as shown in Figure 5 above. At this time, the learning unit 13 may also use the acquired flow information to calculate statistical outlier thresholds and pattern information (for example, information defining the rules for pattern matching) used in each detection engine of the non-AI analysis unit 14.

[0090] Once the machine learning model for the device of interest has been built, the learning unit 13 stores the acquired machine learning model and the analysis parameters of each detection engine for the device of interest (e.g., threshold information and pattern information) in the storage unit 16, notifies the control unit 12 that the learning process for the device of interest is complete (S204), and terminates the learning process for the device of interest.

[0091] Figure 9 is a flowchart illustrating the operation of the non-AI analysis unit 14, the AI ​​analysis kicker unit 15, and the AI ​​analysis unit 17 when they perform analysis processing on the communication traffic data of interest.

[0092] Here, first, the non-AI analysis unit 14 and the AI ​​analysis unit 17 receive flow information from the control unit 12 (hereinafter, this flow information will be referred to as "focus flow information," and the flow corresponding to the focus flow information will be referred to as "focus flow") (S301).

[0093] The AI ​​analysis unit 17 stores the focus flow information in the storage unit 16 (S302). The non-AI analysis unit 14 inputs the focus flow information into each detection engine, causes each detection engine to perform anomaly determination processing from its respective detection perspective (S303), and checks the determination results of each detection engine (S304). If an anomaly determination is made in one or more detection engines as a result of the supply of focus flow information, the non-AI analysis unit 14 proceeds to step S305 described later; otherwise (if no detection engines have made an anomaly determination), it discards the focus flow information data (S307) and terminates processing related to the focus flow information.

[0094] Furthermore, some detection engines may use multiple flow information to determine anomalies. In such cases, the detection engine may retain flow information within itself until it receives the necessary flow information for determination, and then determine an anomaly once enough flow information has been accumulated to make a determination. An example of a detection engine that uses multiple flow information is a scan suspicion detection engine (for example, a detection engine that determines an anomaly if the number of flow occurrences per unit time of the monitored device 30 is abnormally high). Such a detection engine needs to use flow information received in the past to determine anomalies, so it is necessary to accumulate flow information over a certain period of time.

[0095] In step S304 described above, if an anomaly is detected in one or more detection engines, the non-AI analysis unit 14 notifies the AI ​​analysis kicker unit 15 of the name of the detection engine that detected the anomaly, along with the flow information of interest. Here, in the case of a detection engine that makes an anomaly determination using multiple flow information as described above, it may make an anomaly determination for multiple flows, including flows that have been received in the past. In such cases, the non-AI analysis unit 14 notifies the AI ​​analysis kicker unit 15 of all of these multiple flows.

[0096] When the AI ​​analysis kicker unit 15 receives flow information from the non-AI analysis unit 14, it performs a process to prioritize which flows in the received flow information should be analyzed by the AI ​​analysis unit 17 (S305).

[0097] Then, the AI ​​analysis kicker unit 15 controls the AI ​​analysis unit 17 to perform AI analysis processing on the flow according to the prioritized order (S306), and terminates processing related to the flow information of interest.

[0098] At this time, the content that the AI ​​analysis kicker unit 15 notifies the AI ​​analysis unit 17 only needs to be such that it can identify the flow to be processed by AI analysis, and the specific format is not limited. For example, the AI ​​analysis kicker unit 15 may notify the AI ​​analysis unit 17 of the flow identification information of the flow to be processed by AI analysis next (for example, a set of source IP address, source port number, destination IP address, destination port number, protocol number, etc.). This allows the AI ​​analysis unit 17 to obtain the flow information of the flow for which the flow identification information has been notified (for example, the flow information for the first i-packet) from the storage unit 16.

[0099] It should be noted that, before the AI ​​analysis kicker unit 15 inputs all flows to the AI ​​analysis unit 17, it may receive the next flow information from the non-AI analysis unit 14. In this case, the AI ​​analysis kicker unit 15 may compare the flows waiting for input with the newly received flows using the prioritization method described above, and notify the AI ​​analysis unit 17 of the flow identification information in the newly prioritized order.

[0100] Figure 10 is a flowchart illustrating the AI ​​analysis operation in the AI ​​analysis unit 17 and the display operation in the notification / visualization unit 18.

[0101] When the AI ​​analysis unit 17 receives notification from the AI ​​analysis kicker unit 15 of the next flow to be subjected to AI analysis processing (for example, flow identification information of the flow to be subjected to AI analysis processing) (S401), it obtains flow information (for example, flow information for i packets from the beginning of the flow) of the flow corresponding to that flow (hereinafter also called the "AI analysis target flow") from the storage unit 16, performs AI analysis processing of the AI ​​analysis target flow (determination of whether it is abnormal or not) using this flow information (S402), and confirms the result (S403). At this time, the AI ​​analysis unit 17 performs AI analysis processing of the AI ​​analysis target flow (determination of whether it is abnormal or not / determination of whether it is in a steady state or not) using, for example, an AI analysis process using an autoencoder as shown in Figure 5 above.

[0102] In step S403 described above, if the AI ​​analysis processing result of the AI ​​analysis target flow is abnormal, the AI ​​analysis unit 17 notifies the notification / visualization unit 18 that the AI ​​analysis target flow is in an abnormal state. The notification / visualization unit 18 performs output processing (notification and visualization processing) in accordance with the notification from the AI ​​analysis unit 17 (S404). At this time, the notification / visualization unit 18 may, for example, perform processing to notify the administrator of the abnormality judgment result (the content indicating that the AI ​​analysis target flow is in an abnormal state) (for example, by sending an email), or perform processing to present (visualize; display) the abnormality judgment result on the terminal used by the administrator via a web-based GUI (web screen).

[0103] On the other hand, if the AI ​​analysis processing result of the AI ​​analysis target flow in step S403 described above is not abnormal (i.e., it is in a steady state), the AI ​​analysis unit 17 discards the data related to the AI ​​analysis target flow (S405) and terminates the processing related to the AI ​​analysis target flow.

[0104] Next, we will explain in detail the process by which the AI ​​analysis kicker unit 15 prioritizes the AI ​​analysis target flow in step S305 described above.

[0105] The AI ​​analysis kicker unit 15, based on the priority information 161, compares the criteria (perspectives) with the highest priority and determines the matching priority as the priority for each flow. Here, as described above, the content of the priority information 161 is assumed to be as shown in Figure 4.

[0106] Figure 11 shows a specific example of the process by which the AI ​​analysis kicker unit 15 prioritizes the AI ​​analysis target flow.

[0107] Figure 11 is a table illustrating the device type of the source device and the detection engine (detection engine name) that determined the flow to be abnormal by one or more detection engines in the non-AI analysis unit 14 (flows to be prioritized). In Figure 11, flows A, B, and C are shown, which were determined to be abnormal by one or more detection engines and are set in the waiting queue for AI analysis processing in the AI ​​analysis kicker unit 15.

[0108] In Figure 11, Flow A shows that the source device type is "PC" and that it has been detected as an anomaly by two detection engines: "Suspected Scan" and "Abnormal Communication Volume". In Figure 11, Flow B also shows that the source device type is "PC" and that it has been detected as an anomaly by the "Abnormal Communication Volume" detection engine. Furthermore, in Figure 11, Flow C shows that the source device type is "IoT Device" and that it has been detected as an anomaly by the "Rare Communication" detection engine. In other words, in Figure 11, the number of detection engines for Flows A, B, and C is "2", "1", and "1", respectively.

[0109] Here, as shown in Figure 4(a), the highest priority criterion in the priority information 161 is "number of detected items". In other words, in this case, the AI ​​analysis kicker unit 15 evaluates flows so that those with a larger number of detected items have a higher priority. In this case, as described above, only flow A has 2 detected items and is therefore determined to have the highest priority.

[0110] On the other hand, since both Flow B and Flow C have the same number of detection items (1), they have the same priority in terms of the number of detection items. Therefore, the AI ​​analysis kicker unit 15 evaluates the priority of Flow B and C using the next highest priority criterion after the number of detection items, which is "the presence or absence of a priority detection engine for each type of equipment."

[0111] The criterion for determining "the presence or absence of a priority detection engine for each device type" is that a priority detection engine is set for each device type, and a device is determined to have high priority if an abnormality is detected by the priority detection engine corresponding to the device type of the flow source device.

[0112] Here, as described above, the device type for Flow B is a PC, and the device type for Flow C is an IoT device. In the priority information 161 shown in Figure 4(b), the priority detection engine for device type "PC" is "Abnormal Communication Volume," and the priority detection engine for IoT devices is "Rare Communication." As shown in Figure 11, Flow B is judged as abnormal by the PC's priority detection engine, "Abnormal Communication Volume." Also, as shown in Figure 11, Flow C is judged as abnormal by the IoT device's priority detection engine, "Rare Communication." Therefore, based on the criterion of "Presence or Absence of Priority Detection Engine for Each Device Type," the priorities of Flow B and Flow C are judged to be equal.

[0113] Therefore, the AI ​​analysis kicker unit 15 evaluates the priority of flows B and C using the next highest priority criterion after "presence or absence of a priority detection engine for each type of device," namely "presence or absence of a priority detection engine common to all devices."

[0114] Regardless of the device type, the presence or absence of a common priority detection engine determines the priority of the detection engine, and the higher the priority of the detection engine that detects an anomaly, the higher the priority of the flow. Here, as shown in Figure 4(c), the priority is in the order of suspected scan, abnormal communication volume, and rare communication. And here, as shown in Figure 11, since flow B is detected as "abnormal communication volume" and flow C as "rare communication", according to the list in Figure 4(c), flow B is determined to have a higher priority.

[0115] Based on the above, the AI ​​analysis kicker unit 15 will determine that flow A, flow B, and flow C have the highest priority in that order.

[0116] (A-3) Effects of the first embodiment According to the first embodiment, the following effects can be achieved.

[0117] In the first embodiment, the traffic analysis device 10 includes a non-AI analysis unit 14 and an AI analysis kicker unit 15 before the AI ​​analysis unit 17 performs AI analysis on the communication traffic data. The non-AI analysis unit 14 narrows down the communication traffic data that requires AI analysis, the AI ​​analysis kicker unit 15 prioritizes it, and then inputs it to the AI ​​analysis unit 17 for anomaly detection. In other words, in the traffic analysis device 10 of the first embodiment, before anomaly detection of communication traffic is performed using a machine learning model, a non-AI analysis without machine learning is performed to extract suspicious communications, the communication traffic to be judged as anomaly by the machine learning model is narrowed down based on the results of the non-AI analysis, and the narrowed-down communication traffic is further prioritized. As a result, the traffic analysis device 10 of the first embodiment can capture communication traffic of the edge network with a single device equipped with a VPU 204, and can detect anomalies in real time of non-routine communication patterns such as attack communications using a machine learning model that has been unsupervised to learn the steady communication patterns of 30 monitored devices.

[0118] Furthermore, in the traffic analysis device 10 of the first embodiment, high-priority communication traffic data is appropriately selected and analyzed by the AI ​​analysis unit 17, thus reducing the total number of AI analysis processes. In particular, when AI judgment processing is performed using the VPU 204 as in the traffic analysis device 10, not only is the number of AI analysis processes reduced, but the number of machine learning model switching operations set on the VPU 204 is also reduced, resulting in a significant improvement in the efficiency of AI judgment processing.

[0119] (B) Second Embodiment A second embodiment of the traffic analysis device, traffic analysis program, and traffic analysis method according to the present invention will be described in detail below with reference to the drawings.

[0120] The following describes the differences between the traffic analysis device 10A according to the second embodiment and the first embodiment.

[0121] Figure 12 is a block diagram showing the functional configuration of the traffic analysis device 10A according to the second embodiment. In Figure 12, the same or corresponding parts as those in Figure 1 are denoted by the same or corresponding reference numerals.

[0122] In the traffic analysis device 10 of the first embodiment, the AI ​​analysis kicker unit 15 further prioritized the flows narrowed down by the non-AI analysis unit 14. However, in the traffic analysis device 10A of the second embodiment, the AI ​​analysis kicker unit 15 is not provided.

[0123] For example, in environments where the processing power of the AI ​​analysis unit 17 is slightly insufficient for real-time AI analysis of communication traffic (for example, environments with low communication rates), it is sufficient to simply filter out communication traffic data that does not require AI analysis using the non-AI analysis unit 14, as in the traffic analysis device 10A. The traffic analysis device 10A is configured to directly notify the AI ​​analysis unit 17 of the flows filtered by the non-AI analysis unit 14, and the AI ​​analysis unit 17 then performs the AI ​​analysis.

[0124] In the traffic analysis device 10A of the second embodiment, the system operation load related to flow prioritization processing can be reduced by not providing the AI ​​analysis kicker unit 15.

[0125] (C) Third Embodiment A third embodiment of the traffic analysis device, traffic analysis program, and traffic analysis method according to the present invention will be described in detail below with reference to the drawings.

[0126] The following describes the differences between the traffic analysis device 10B according to the third embodiment and the first embodiment.

[0127] Figure 13 is a block diagram showing the functional configuration of the traffic analysis device 10B according to the third embodiment. In Figure 13, the same or corresponding parts as those in Figure 1 are denoted by the same or corresponding reference numerals.

[0128] In the first embodiment of the traffic analysis device 10, the non-AI analysis unit 14 had analysis parameters for the detection engine group (e.g., statistical thresholds and pattern information), and the AI ​​analysis unit 17 had a learning unit 13 for building the machine learning model. In contrast, the third embodiment of the traffic analysis device 10B has a configuration without a learning unit 13, as shown in Figure 13.

[0129] Generally, building machine learning models for AI analysis is computationally intensive, and it can take a long time for edge devices to train, or even be difficult to perform the training at all, due to the high processing load. Therefore, the traffic analysis device 10B is configured to perform the training process externally (for example, on an edge analysis server installed on the company's network upstream of the edge network) and store the trained machine learning model in the memory unit 16. This allows the traffic analysis device 10B of the third embodiment to shorten the time until analysis processing can begin by reducing the training time, and to reduce the processing load on the traffic analysis device. However, since the processing of acquiring analysis parameters for non-AI analysis is often computationally intensive, the processing of acquiring analysis parameters for non-AI analysis may be performed within the traffic analysis device of the third embodiment (for example, the non-AI analysis unit 14).

[0130] (D) Fourth Embodiment A fourth embodiment of the traffic analysis device, traffic analysis program, and traffic analysis method according to the present invention will be described in detail below with reference to the drawings.

[0131] The following describes the differences between the traffic analysis device 10C according to the fourth embodiment and the first embodiment.

[0132] Figure 14 is a block diagram showing the functional configuration of the traffic analysis device 10C according to the fourth embodiment. In Figure 14, the same or corresponding parts as those in Figure 1 are denoted by the same or corresponding reference numerals.

[0133] As shown in Figure 14, the traffic analysis device 10C of the fourth embodiment applies features from both the second and third embodiments, and is configured without both the learning unit 13 and the AI ​​analysis kicker unit 15. As a result, the traffic analysis device 10C of the fourth embodiment can achieve the effects of both the second and third embodiments.

[0134] (E) Fifth Embodiment A fifth embodiment of the traffic analysis device, traffic analysis program, and traffic analysis method according to the present invention will be described in detail below with reference to the drawings.

[0135] The following describes the differences between the traffic analysis device 10D according to the fifth embodiment and the fourth embodiment.

[0136] Figure 15 is a block diagram showing the functional configuration of the traffic analysis device 10D according to the fifth embodiment. In Figure 15, the same or corresponding parts as those in Figure 14 are denoted by the same or corresponding reference numerals.

[0137] The traffic analysis device 10D of the fifth embodiment differs from the fourth embodiment in that an abnormality determination reason estimation unit 19 is added.

[0138] In the fifth embodiment of the traffic analysis device 10D, the abnormality determination result from the AI ​​analysis unit 17 is associated with the information of the detection engine that was determined to be abnormal by the non-AI analysis unit 14, so that the content (reason) of the abnormality determination can also be output. In the fourth embodiment of the traffic analysis device 10D, the abnormality determination reason estimation unit 19 performs a process to recognize the reason for the abnormality determination (hereinafter referred to as the "abnormality reason determination process").

[0139] Figure 16 shows an example of a table (condition list; hereinafter referred to as the "abnormality reason condition list") used in the abnormality determination reason estimation unit 19 for abnormality reason determination processing.

[0140] In the list of abnormality reason conditions shown in Figure 16, the items are, from left to right, "Name of abnormality judgment item in non-AI analysis," which indicates the name of the detection engine that was judged as abnormal by the non-AI analysis unit 14; "AI analysis judgment result," which indicates the judgment result by the AI ​​analysis unit 17; and "Reason for abnormality."

[0141] In other words, the anomaly reason condition list shown in Figure 16 defines an "anomaly reason" for each combination of the detection engine name that was judged as an anomaly by the non-AI analysis unit 14 and the judgment result by the AI ​​analysis unit 17. The anomaly judgment reason estimation unit 19 can obtain the judgment result of the AI ​​analysis process and the information of the detection engine name that was judged as an anomaly by the non-AI analysis unit 14 for each flow that has been processed by the AI ​​analysis unit 17, apply it to the anomaly reason condition list, and obtain the anomaly reason corresponding to that flow.

[0142] Figure 16 shows a list of conditions for outputting the cause of an anomaly, based on the judgment results from the AI ​​analysis unit 17, for the "communication volume anomaly" detection engine, which detects when the communication volume of the monitored device 30 is statistically higher than usual. For example, the items for the cause of an anomaly in the anomaly judgment cause estimation unit 19 may be stored as pre-written data, or they may be configured to be editable by an operator or the like.

[0143] In the first row of the list of abnormality reason conditions shown in Figure 16, the text "Suspicion of an external attack" is written as the reason for the abnormality when an abnormality is detected by both the non-AI analysis unit 14 due to an abnormal communication volume and the AI ​​analysis unit 17. In this case, an abnormal amount of communication occurred at the source device of the flow (monitored device 30) with a communication pattern that does not normally occur, so it can be justified as a suspicion of an external attack, and the above text is written.

[0144] In the second row of the list of abnormality reason conditions shown in Figure 16, the text "Suspicion of data theft due to internal misconduct" is written as the reason for the abnormality when the non-AI analysis unit 14 detects an abnormality due to an abnormal communication volume, but the AI ​​analysis unit 17 does not detect an abnormality (i.e., it is determined to be normal). In this case, the source device of the flow (monitored device 30) has a normal communication pattern, but the communication volume is abnormally high, so it can be justified as suspicion of data theft due to internal misconduct.

[0145] The abnormality reason (text of the abnormality reason) for each flow (for each source device) obtained by the abnormality determination reason estimation unit 19 is supplied to the notification / visualization unit 18, output in a predetermined manner and format, and presented (notified) to the operator, etc.

[0146] As described above, the traffic analysis device 10D of the fifth embodiment can output information (output by the notification / visualization unit 18) that clarifies the cause of an anomaly by combining the results of the AI ​​analysis unit 17 and the results of the non-AI analysis unit 14.

[0147] Furthermore, in any of the traffic analysis devices 10, 10A, and 10B of the first to third embodiments, an abnormality determination reason estimation unit 19 may be added.

[0148] (F) Other embodiments The present invention is not limited to the embodiments described above, and modified embodiments such as those exemplified below can also be cited.

[0149] (F-1) In the first embodiment, the learning unit 13 calculated thresholds and patterns for the non-AI analysis detection engine group after the learning period had elapsed. However, the received communication traffic data may also be learned sequentially during the learning period. For example, in a detection engine that detects rare communications such as destinations or services that are not normally communicated with, one possible implementation method is to acquire destination and service information from the communication traffic data received during the learning period and store it as a list, and in an anomaly detection, consider destinations or services that are not in the list as anomalies. With such a detection engine, the received communication traffic data can be analyzed sequentially and added to the list, so there is no need to wait until the learning period is over. As a result, the traffic analysis device 10 can distribute the learning processing load by sequentially processing non-AI analysis threshold calculations, etc., and reduce the learning processing load after the learning period is completed.

[0150] (F-2) In the traffic analysis device 10 of the first embodiment, the priority information 161 in the AI ​​analysis kicker unit 15 was set to give higher priority to flows that are likely to be judged as abnormal. However, it is not limited to this, and a perspective that reduces the switching of machine learning models may be incorporated. For example, after the AI ​​analysis kicker unit 15 determines the highest priority flow using the priority information 161 in Figure 4, it may select the next highest priority flow by limiting it to other flows from the source device of that flow, and continue this until there are no more flows from the device of interest. Next, the AI ​​analysis kicker unit 15 may select the highest priority flow from the flows of other monitored devices 30, and similarly prioritize it by limiting it to flows from the source device of the selected flow (prioritizing in an order that reduces the switching of machine learning models in the VPU 204). With this prioritization method, the AI ​​analysis unit 17 will continuously perform AI analysis on flows from the same device, thereby reducing the number of machine learning model switches and improving the throughput of AI analysis.

[0151] (F-3) In the non-AI analysis unit 14 of each of the above embodiments, the detection engine may use a lightweight machine learning algorithm (an algorithm that does not require the use of the VPU204). Examples of lightweight machine learning algorithms include logistic regression models and simple decision tree models.

[0152] (F-4) In the AI ​​analysis kicker unit 15 of each embodiment described above, priority information 161 as shown in Figure 4 is used to prioritize each flow by making decisions in order from the highest priority judgment criteria (item name). However, if a procedure is to be given emphasis on the highest priority judgment criteria (item name) as shown in Figure 4, other prioritization procedures may be applied. For example, the AI ​​analysis kicker unit 15 may calculate an evaluation value based on multiple judgment criteria (item names) and prioritize the flows according to that evaluation value. In this case, the weight of the evaluation value according to the priority (importance) may be changed for each judgment criterion (item name). For example, the weight of the evaluation value for "number of detection items" may be set to "3", the weight of the evaluation value for "presence or absence of a priority detection engine for each device type" may be set to "2", and the weight of the evaluation value for "presence or absence of a priority detection engine common to all devices" may be set to "1". For example, the evaluation value for a flow that only has multiple detection items will be "3", while the evaluation value for a flow that has multiple detection items and also has a "presence or absence of a priority detection engine for each device type" will be "5", and the latter flow will have a higher evaluation value and therefore a higher priority. [Explanation of symbols]

[0153] 10, 10A, 10B, 10C, 10D…Traffic analysis device, 11…Traffic capture unit, 12…Control unit, 13…Learning unit, 14…Non-AI analysis unit, 15…AI analysis kicker unit, 16…Storage unit, 17…AI analysis unit, 18…Visualization unit, 19…Anomaly determination reason estimation unit, 20…Network switch, 30…Monitored device, 40…Gateway, 161…Priority information, 171…Autoencoder, 200…Computer, 201…CPU, 202…Primary storage unit, 203…Secondary storage unit, 204…VPU, N1…Monitored network, N2…Internet

Claims

1. A first analysis means comprising two or more detection processing units that analyze data based on communication traffic data flowing on the network under analysis using different items to detect the presence or absence of abnormal communication, A second analysis means determines whether or not abnormal communication exists by an AI analysis processing unit that performs AI analysis processing using a learning model on the data to be analyzed which has been detected as having abnormal communication by one or more of the detection processing units, The system includes a priority determination process that determines the priority order of the data to be analyzed for which the second analysis means will perform the AI ​​analysis process, and an analysis control means that causes the second analysis means to execute the AI ​​analysis process on the data to be analyzed in the order determined by the result of the priority determination process, The analysis control means determines the priority of the data to be analyzed according to the detection processing unit that has detected abnormal communication by the first analysis means. A traffic analysis device characterized by the following features.

2. The traffic analysis device according to claim 1, characterized in that the analysis control means determines the priority by taking into consideration a first judgment criterion based on the number of detection processing units that have detected abnormal communication in the priority determination process.

3. The traffic analysis device according to claim 2, characterized in that the analysis control means maintains device type priority information describing the detection processing unit to be prioritized for each attribute of the source of the communication traffic data, and determines the priority by also considering a second judgment criterion based on the result of comparing the detection processing unit that detected abnormal communication in the priority determination process with the device type priority information.

4. The traffic analysis device according to claim 3, characterized in that the analysis control means holds common device priority information describing the detection processing unit that should be given priority regardless of the source of the communication traffic data, and determines the priority by also considering a third judgment criterion based on the result of comparing the detection processing unit that detected abnormal communication in the priority determination process with the common device priority information.

5. The second analysis means sets a learning model in the AI ​​analysis processing unit that corresponds to the source of the communication traffic data related to the data to be analyzed, The analysis control means determines the priority of the data to be analyzed in an order that reduces the number of times the learning model set in the AI ​​analysis processing unit is switched. The traffic analysis device according to feature 1.

6. The traffic analysis device according to claim 1, further comprising an abnormal determination reason estimation processing means that performs an abnormal determination reason estimation processing that estimates the reason for the abnormal communication according to the combination of the detection processing unit that detects abnormal communication by the first analysis means and the determination result of the second analysis means.

7. The traffic analysis device according to claim 1, further comprising a learning means for acquiring the communication traffic data in a steady state for each source of the communication traffic data, and for performing machine learning processing based on the acquired communication traffic data to acquire a learning model.

8. Computers, A first analysis means comprising two or more detection processing units that analyze data based on communication traffic data flowing on the network under analysis using different items to detect the presence or absence of abnormal communication, A second analysis means determines whether or not abnormal communication exists by an AI analysis processing unit that performs AI analysis processing using a learning model on the data to be analyzed which has been detected as having abnormal communication by one or more of the detection processing units, The second analysis means performs a priority determination process to determine the priority of the data to be analyzed in order of which to perform the AI ​​analysis process, and functions as an analysis control means to cause the second analysis means to execute the AI ​​analysis process on the data to be analyzed in order according to the result of the priority determination process. The analysis control means determines the priority of the data to be analyzed according to the detection processing unit that has detected abnormal communication by the first analysis means. A traffic analysis program characterized by the following features.

9. In the traffic analysis method performed by the traffic analysis device, The traffic analysis device comprises a first analysis means, a second analysis means, and an analysis control means. The first analysis means includes two or more detection processing units that analyze data based on communication traffic data flowing on the network under analysis using different items to detect the presence or absence of abnormal communication. The second analysis means determines whether or not abnormal communication exists using an AI analysis processing unit that performs AI analysis processing using a learning model on the data to be analyzed which has been detected as having abnormal communication by one or more of the detection processing units, The analysis control means performs a priority determination process to determine the priority order of the data to be analyzed for which the second analysis means will perform the AI ​​analysis process, and causes the second analysis means to perform the AI ​​analysis process on the data to be analyzed in the order determined by the result of the priority determination process. The analysis control means determines the priority of the data to be analyzed according to the detection processing unit that has detected abnormal communication by the first analysis means. A traffic analysis method characterized by the following features.

Images