Abnormal communication detection device, abnormal communication detection method, and abnormal communication detection program

Abstract

To provide an abnormal communication detection device, an abnormal communication detection method, and an abnormal communication detection program capable of detecting many types of abnormal communication at low cost.SOLUTION: An abnormal communication detection device 1 includes a data conversion unit 11 that converts information included in communication flow data into data of a plurality of formats, a feature extraction unit 12 that extracts feature amounts from each of the data of the plurality of formats, a data compression unit 13 that compresses a dimension of the feature amounts for each piece of data of the plurality formats, a contribution ratio setting unit 14 that dynamically sets a contribution ratio for each of the feature amounts on the basis of characteristics of each communication type, and an abnormality detection unit 15 that inputs feature amounts into the abnormality detection model after changing parameters of the abnormality detection model in accordance with the contribution ratios and outputs a determination result.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an apparatus, a method, and a program for detecting abnormal communication.

Background Art

[0002] Conventionally, as techniques used for detecting communication anomalies, in many IDSs (Intrusion Detection Systems) and IPSs (Intrusion Prevention Systems), a signature type and an anomaly type are used in combination. In the signature type, communication that matches a known abnormal pattern is determined to be abnormal, and in the anomaly type, communication other than a normal pattern created by dictionary-based or machine learning is determined to be abnormal.

[0003] The signature type is effective against known anomalies but is vulnerable to changes, and the anomaly type is likely to cause false detections depending on the information used to create the pattern. When generating a pattern in the anomaly type, unsupervised learning of machine learning and deep learning is often used. This is because since many attack methods are generated daily, it is difficult to create labels for various real-world attack methods in general supervised learning, and parameter updates of the model using teacher data cannot be performed.

[0004] Also, generating an appropriate representation is important in constructing a learning device, and there is ensemble learning as an idea for giving diversity to the representation. There are, for example, the following methods for ensemble learning. · Add different noises to certain data. · Integrate the results of learning performed on certain data at different periods or samples. · Apply different parameters to a certain algorithm. · Use a learning device that uses a plurality of different algorithms. Among ensemble learning methods, the concept of integrating and processing different types of information is called multimodal ensemble learning.

[0005] In Patent Document 1, a method is proposed that uses packet feature quantities and payload information to determine the stationarity of a time series, and then uses the determined data for anomaly detection. In Patent Document 2, an anomaly communication detection method is proposed that reduces over-detection by classifying information including the temporal characteristics and payload characteristics in communication data according to rules, and performing model learning and anomaly detection.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] In an anomaly-based IDS, it is often set as one-class classification that performs anomaly detection by setting a threshold for normal or not, but it has been difficult to determine the cause of why it was abnormal and the type of attack method. For example, when under a DDoS attack where the traffic volume increases in a short period, it is difficult to judge only by the information of the payload, and in communication where operations such as sending and receiving command commands to obtain root privileges during malware infection are performed singularly, it is difficult to judge only by statistical quantities.

[0008] With a single type of information, although an ensemble can be performed to give diversity to the expression, it is difficult to obtain expressions from different information sources, so there are also attacks that cannot distinguish abnormal communication. For this reason, it is necessary to use multimodal information even in unknown attacks. On the other hand, by using multimodal information, the number of dimensions increases and the calculation cost has been increasing. Although both Patent Documents 1 and 2 focus on the temporal characteristics and the characteristics of the payload, there is a problem with the calculation cost that increases when handling a plurality of types of information.

[0009] An object of the present invention is to provide an abnormal communication detection device, an abnormal communication detection method, and an abnormal communication detection program that can detect various types of abnormal communication at low cost.

Means for Solving the Problems

[0010] The abnormal communication detection device according to the present invention includes a data conversion unit that converts information included in communication flow data into data of a plurality of formats, a feature extraction unit that extracts feature amounts from the data of the plurality of formats, respectively, a data compression unit that compresses the dimensions of the feature amounts for each of the data of the plurality of formats, a contribution ratio setting unit that dynamically sets a contribution ratio for each of the feature amounts based on the characteristics for each communication type, and an abnormal detection unit that changes the parameters of the abnormal detection model according to the contribution ratio, inputs the feature amounts to the abnormal detection model, and outputs a determination result.

[0011] The data conversion unit may convert information including the aggregated value of the flow data into tabular data.

[0012] The data conversion unit may convert the information of the flow data into graph-formatted data.

[0013] The data conversion unit may convert the payload information of the flow data into image-formatted data.

[0014] The data compression unit may compress the dimensions of the feature amounts extracted from the data of the plurality of formats by different algorithms, respectively.

[0015] The data compression unit may generate a plurality of feature amounts by dimensionally compressing at least one of the feature amounts extracted from the data of the plurality of formats using a plurality of different algorithms.

[0016] The flow data is acquired from a vehicle communication network, and the contribution ratio setting unit may set the contribution ratio according to the vehicle type specified based on the authentication information of the vehicle.

[0017] The flow data is acquired from a vehicle communication network, and the contribution ratio setting unit may set the contribution ratio according to the type of in-vehicle application identified by the flow data.

[0018] The abnormal communication detection method according to the present invention includes: a data conversion step of converting information included in communication flow data into data of a plurality of formats; a feature extraction step of extracting feature amounts from the data of the plurality of formats; a data compression step of compressing the dimensions of the feature amounts for each of the data of the plurality of formats; a contribution ratio setting step of dynamically setting a contribution ratio for each of the feature amounts based on characteristics for each communication type; and an abnormal detection step of changing parameters of an abnormal detection model according to the contribution ratio, inputting the feature amounts into the abnormal detection model, and outputting a determination result, which are executed by a computer.

[0019] The abnormal communication detection program according to the present invention is for causing a computer to function as the abnormal communication detection device.

Effects of the Invention

[0020] According to the present invention, various types of abnormal communications can be detected at low cost.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Best Mode for Carrying Out the Invention

[0022] Hereinafter, an example of an embodiment of the present invention will be described. The abnormal communication detection method of this embodiment classifies the types of abnormal communication by combining and utilizing statistical information, information on the network itself, and payload information as multimodal information. Furthermore, by reducing the amount of these information, the curse of dimensionality that may occur in machine learning and deep learning is avoided, and the necessary computing resources are reduced.

[0023] Here, the information necessary for classifying abnormal communication is defined as "statistical numerical information", "information indicating the adjacency and connection of data", and "semantic information of the communication itself", and the abnormal communication detection device converts communication data into a table format, a graph, and an image. For example, in the case of a table format, attention is paid to statistical quantities such as the number of packets, the number of bytes, and the number of connections. In the case of a graph, attention is paid to the relationships such as the connections between communications in the network and time information. In the case of an image, attention is paid to the message of the payload, etc. The information being focused on is different in each case. In this way, by utilizing a plurality of information in parallel, the abnormal communication detection device can calculate the score of the degree of abnormality and, at the same time, by paying attention to the contribution degree of the utilized information, can determine what kind of abnormality it is.

[0024] In this embodiment, a vehicle communication network is exemplified as the communication network to be detected for abnormalities. In a vehicle communication network, it is assumed that a large amount of communication is performed in a short period of time, and it is required to classify the causes of unknown abnormal communication. For this reason, in this embodiment, by applying a dimensionality reduction method adapted to each information in consideration of the computing resources without losing the information of the communication flow data, the computational cost is reduced while maintaining the data representation.

[0025] FIG. 1 is a diagram showing the functional configuration of the abnormal communication detection device 1 in this embodiment. The abnormal communication detection device 1 is an information processing device that includes, in addition to the control unit 10 and the storage unit 20, input / output devices for various data, communication devices, and the like.

[0026] The control unit 10 is a part that controls the entire abnormal communication detection device 1, and realizes each function in the present embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU. Specifically, the control unit 10 includes a data conversion unit 11, a feature extraction unit 12, a data compression unit 13, a contribution ratio setting unit 14, and an abnormality detection unit 15.

[0027] The storage unit 20 is a storage area for various programs and various data for causing the hardware group to function as the abnormal communication detection device 1, and may be a ROM, a RAM, a flash memory, a hard disk drive (HDD), or the like. Specifically, the storage unit 20 stores a program (abnormal communication detection program) for causing the control unit 10 to execute each function of the present embodiment, and further stores flow data received as a processing target, data of various converted formats, an abnormality detection model, various parameters, and the like.

[0028] The data conversion unit 11 converts the information included in the communication flow data into data of a plurality of formats. The plurality of formats may be, for example, tabular data, graphs, and image data.

[0029] Specifically, the data conversion unit 11 converts the information including the aggregated value of the flow data into tabular data. At this time, in addition to the elements generally used in the flow data, the data conversion unit 11 may add continuous values, category values, and binary values calculated based on an arbitrary aggregation method or rule as feature amounts to the tabular data.

[0030] In addition, the data conversion unit 11 converts the information of the flow data into data in graph format. That is, the data conversion unit 11 constructs edges in the time series direction with the flow data as nodes, or constructs a graph with hosts as nodes using the information of the IP addresses or MAC addresses of the source host and the destination host, and the port numbers.

[0031] Furthermore, the data conversion unit 11 converts the payload information of the flow data into data in image format. Here, the method of conversion into an image is not particularly limited, and for example, a method of mapping into a two-dimensional array based on a predetermined rule as follows can be applied.

[0032] (1) Using Sketch that updates a hash table based on the IP address and traffic volume (see, for example, the following Reference A), represent the array of the hash table as an image of a two-dimensional array. Reference A: S. Chang et al., "A flow-based anomaly detection method using sketch and combinations of traffic features," 2010 International Conference on Network and Service Management, 2010, pp. 302-305.

[0033] (2) Convert ASCII characters into a matrix of RGB values (see, for example, the following Reference B). Reference B: R. Shire et al., "Malware Squid: A Novel IoT Malware Traffic Analysis Framework Using Convolutional Neural Network and Binary Visualisation," Internet of Things, Smart Spaces, and Next Generation Networks and Systems, Springer, 2019, pp. 65-76.

[0034] (3) Convert each byte of information into a matrix of values from 0 to 255 (for example, refer to the following Document C). Document C: P. Parmuval, "Malware Family Detection Approach using Image Processing Techniques: Visualization Technique," International Journal of Computer Applications Technology and Research, Volume 7 Issue 03, 2018, pp. 129-132.

[0035] Note that the matrix in (2) or (3) is generated as a two-dimensional array of a predetermined size (M×N). If there is insufficient data to form an M×N matrix, it is imaged by filling in zeros or the like.

[0036] The feature extraction unit 12 extracts multi-dimensional feature amounts from the data in the converted multiple formats, respectively, by a predetermined algorithm. Note that the algorithm for feature extraction is not limited, and various known methods can be applied according to the format of the data.

[0037] The data compression unit 13 compresses the dimensions of the extracted feature amounts for each of the data in the multiple formats, and generates a plurality of feature amounts with reduced dimensions. Note that the method of dimension compression is not limited, and various known methods such as principal component analysis or UMAP can be applied. Also, the number of dimensions to be compressed is not limited to 2D, and can be any dimension according to the processing capacity or data of the abnormal communication detection device 1.

[0038] Here, the data compression unit 13 may compress the dimensions of the feature amounts extracted from the data in the multiple formats by the same algorithm, or may compress the dimensions by different algorithms. Also, the data compression unit 13 may generate a plurality of feature amounts by compressing the dimensions of at least one of the feature amounts extracted from the data in the multiple formats by a plurality of different algorithms.

[0039] The contribution ratio setting unit 14 dynamically sets contribution ratios for each of a plurality of feature amounts based on the data characteristics for each communication type. For example, in the case of flow data acquired from a vehicle communication network, the contribution ratio setting unit 14 may set the contribution ratio according to the vehicle type or model year, etc. specified based on the authentication information of the vehicle and the IP address or MAC address, or according to the type of in-vehicle application identified by the flow data.

[0040] The abnormality detection unit 15 changes the parameters of the abnormality detection model according to the contribution ratio, inputs a plurality of feature amounts into the abnormality detection model, and outputs a determination result based on the integrated information. Here, when the degree of abnormality (likelihood), which is the determination result, exceeds the threshold value, the abnormality detection unit 15 outputs together which feature amounts the determination result of abnormality contributes to. Thereby, known or assumed abnormal contents can be specified, and the cause of the abnormality can be separated.

[0041] FIG. 2 is a flowchart showing the processing procedure of the abnormal communication detection device 1 in the present embodiment. In step S1, the data conversion unit 11 converts the information that can be acquired from the flow data of the communication into data in a plurality of formats such as a table format, an image, and a graph.

[0042] In step S2, the feature extraction unit 12 extracts feature amounts from each of the plurality of formats of data converted in step S1.

[0043] In step S3, the data compression unit 13 compresses and reduces the dimension of each of the feature amounts extracted in step S2.

[0044] In step S4, the contribution ratio setting unit 14 determines whether the vehicle type or the in-vehicle application type can be acquired from the flow data to be processed. If this determination is YES, the process proceeds to step S5, and if the determination is NO, the process proceeds to step S6.

[0045] In step S5, the contribution ratio setting unit 14 sets the contribution ratios of the data in each format of the table format, image, and graph according to the acquired vehicle type or in-vehicle application type.

[0046] In step S6, the abnormality detection unit 15 sets the weighting parameters according to the contribution ratios of the data in each format for the abnormality detection model, and outputs the determination result obtained by inputting the feature amount lightened in step S3.

[0047] According to the present embodiment, the abnormal communication detection device 1 converts the acquired flow data into data in different formats such as images, graphs, and table formats, and sets the contribution ratio according to the characteristics of each communication type for use in abnormality detection. Further, since the abnormal communication detection device 1 lightens the weight by compressing the dimension of the feature amount extracted from the data in each format, it can retain the relationship of the data lost in a single piece of information, separate the information that causes the abnormality, and perform abnormality detection at low cost.

[0048] The abnormal communication detection device 1 can select an algorithm suitable for each format and suppress the decrease in the amount of information due to compression by dimensionally compressing the feature amounts extracted from the data in a plurality of formats by different algorithms, and can improve the abnormality detection accuracy.

[0049] The abnormal communication detection device 1 can obtain a plurality of expressions from the data in the same format by dimensionally compressing at least one of the feature amounts by different algorithms to generate a plurality of feature amounts, and thus the effect of ensemble learning can be expected.

[0050] Since the abnormal communication detection device 1 sets the contribution ratio of each format according to the vehicle type and in-vehicle application type, in the vehicle communication network, the ratio can be dynamically changed according to the characteristics of different data depending on the application, and higher abnormal detection performance can be obtained.

[0051] Note that, according to the above-described embodiment, for example, since abnormal communication in a network can be efficiently detected, it becomes possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0052] As described above, the embodiments of the present invention have been explained. However, the present invention is not limited to the above-described embodiments. Also, the effects described in the above-described embodiments are merely an enumeration of the most suitable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the embodiments.

[0053] The method for detecting abnormal communication by the abnormal communication detection device 1 is realized by software. When realized by software, the program constituting this software is installed in an information processing device (computer). Also, these programs may be recorded on a removable medium such as a CD-ROM and distributed to users, or may be distributed by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded.

Explanation of Reference Numerals

[0054] 1 Abnormal communication detection device 10 Control unit 11 Data conversion unit 12 Feature extraction unit 13 Data compression unit 14 Contribution ratio setting unit 15 Abnormal detection unit 20 Storage unit

Claims

1. A data conversion unit that converts information included in communication flow data into data of a plurality of formats; A feature extraction unit that extracts feature amounts from the data of the plurality of formats respectively; A data compression unit that compresses the dimensions of the feature amounts for each of the data of the plurality of formats; A contribution ratio setting unit that dynamically sets a contribution ratio for each of the feature amounts based on characteristics for each communication type; An abnormal communication detection device comprising: an abnormal detection unit that changes parameters of an abnormal detection model according to the contribution ratio, inputs the feature amounts into the abnormal detection model, and outputs a determination result.

2. The abnormal communication detection device according to claim 1, wherein the data conversion unit converts information including an aggregated value of the flow data into tabular data.

3. The abnormal communication detection device according to claim 1 or 2, wherein the data conversion unit converts the information of the flow data into graph-formatted data.

4. The abnormal communication detection device according to any one of claims 1 to 3, wherein the data conversion unit converts the payload information of the flow data into image-formatted data.

5. The abnormal communication detection device according to any one of claims 1 to 4, wherein the data compression unit dimensionally compresses the feature amounts extracted from the data of the plurality of formats by different algorithms.

6. The abnormal communication detection device according to any one of claims 1 to 5, wherein the data compression unit dimensionally compresses at least one of the feature amounts extracted from the data of the plurality of formats by a plurality of different algorithms to generate a plurality of feature amounts.

7. The flow data is acquired from a vehicle communication network, The abnormal communication detection device according to any one of claims 1 to 6, wherein the contribution ratio setting unit sets the contribution ratio according to the vehicle type specified based on the authentication information of the vehicle.

8. The flow data is acquired from a vehicle communication network, The abnormal communication detection device according to any one of claims 1 to 7, wherein the contribution ratio setting unit sets the contribution ratio according to the type of in-vehicle application identified by the flow data.

9. A data conversion step of converting information included in communication flow data into data of a plurality of formats; A feature extraction step of extracting feature amounts from the data of the plurality of formats respectively; A data compression step of compressing the dimensions of the feature amounts for each of the data of the plurality of formats; A contribution ratio setting step of dynamically setting a contribution ratio for each of the feature amounts based on characteristics for each communication type; An abnormality detection step of changing parameters of the abnormality detection model according to the contribution ratio, inputting the feature amounts into the abnormality detection model, and outputting a determination result, which is an abnormality communication detection method executed by a computer.

10. An abnormality communication detection program for causing a computer to function as the abnormality communication detection device according to any one of Claims 1 to 8.

Images