Abnormal detection device, abnormal detection method, and abnormal detection program
The anomaly detection device uses BERT to generate pseudo-abnormal packets and set a threshold value based on their abnormality degree, addressing the challenge of inaccurate threshold setting in existing systems and enhancing detection accuracy.
Patent Information
- Application Number
- JP2023567408
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-12-15
- Publication Date
- 2025-07-25
- Estimated Expiration
- 2041-12-15
AI Technical Summary
Existing anomaly detection systems struggle to accurately set a threshold value for detecting abnormal packets due to the lack of actual cyber-attack data, leading to inaccurate detection of abnormal packets.
An anomaly detection device that uses a BERT model to determine the magnitude of Attention for each byte location in normal packets, generates pseudo-abnormal packets by rewriting important byte locations, and sets a threshold value based on the abnormality degree of these pseudo-abnormal packets and normal packets.
Enables accurate detection of abnormal packets by setting an appropriate threshold value, improving detection accuracy.
Smart Images
Figure 0007713146000001 
Figure 0007713146000002 
Figure 0007713146000003
Abstract
Description
Technical Field
[0001] The present invention relates to an abnormality detection device, an abnormality detection method, and an abnormality detection program.
Background Art
[0002] Conventionally, attention has been focused on abnormality detection systems and intrusion detection systems for communication used in industrial and building network control systems. Communication in a control system may lead to a serious accident even if, for example, the set value of the temperature changes by one digit. Therefore, it is necessary to be able to detect without missing even an unauthorized rewrite of 1 byte of the communication content (payload). Thus, in an abnormality detection system for an industrial or building network control system, a precise analysis of the payload content is essential.
[0003] As a technique for performing such analysis, for example, there is a technique of applying natural language processing techniques such as BERT (Bidirectional Encoder Representations from Transformers) to packet analysis to extract information from the payload of any protocol and perform abnormality detection.
Prior Art Documents
Non-Patent Documents
[0004]
Non-Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0005] However, Non-Patent Document 1 mentioned above does not touch on the method of setting a threshold value for the degree of abnormality for issuing an alert for abnormality detection. Generally, the threshold value should be adjusted to a value that divides the degree of abnormality of both normal data and abnormal data. However, it is difficult to prepare actual cyber-attack data (abnormal data). Therefore, for example, a method has been taken in which a value obtained by adding a predetermined value to the average value of the degree of abnormality of normal data is used as the threshold value. However, the threshold value obtained by this method is not necessarily a value that appropriately divides the degree of abnormality of normal data and abnormal data. As a result, there has been a problem that the anomaly detection system cannot accurately detect abnormal packets.
[0006] Therefore, an object of the present invention is to set an appropriate threshold value for detecting abnormal packets.
Means for Solving the Problems
[0007] To solve the above problems, the present invention provides an Attention acquisition unit that inputs a normal packet into a BERT (Bidirectional Encoder Representations from Transformers) model learned using normal packets and acquires the magnitude of Attention for each byte location when encoding the normal packet, a pseudo-abnormal packet generation unit that samples important byte locations of the normal packet based on the magnitude of Attention of each byte location of the acquired normal packet and generates a pseudo-abnormal packet by rewriting the sampled important byte locations with random bytes, and a threshold value determination unit that determines a threshold value for the degree of abnormality for detecting abnormal packets based on the degree of abnormality of the generated pseudo-abnormal packet group and normal packet group.
Effects of the Invention
[0008] According to the present invention, an appropriate threshold value for detecting abnormal packets can be set.
Brief Description of the Drawings
[0009]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Embodiments for Carrying Out the Invention
[0010] Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment.
[0011] [BERT's Attention Mechanism] First, the Attention mechanism of BERT used by the anomaly detection device of this embodiment to determine the threshold value of the abnormality degree of the packet will be briefly described. The Attention mechanism used in BERT etc. assigns weights to each token (for example, corresponding to words in a sentence and bytes in a packet). Here, if Attention works strongly, BERT will prioritize the token where Attention works strongly and perform conversion to a fixed-length vector.
[0012] The Attention mechanism is optimized to assign large weights to important tokens through learning of Masked Language Modeling and Next Sentence Prediction, which are BERT's Pre-training tasks.
[0013] Figure 1 is a diagram showing an example of the strength of Attention for each byte location of a packet input to the BERT model. In Figure 1, the darkness of the hatching indicates the strength of Attention. As shown in Figure 1, byte locations that are likely to be important within the packet, such as the function code and data section of the packet, are strongly Attentioned.
[0014] [Overview] The anomaly detection device uses the above Attention mechanism to generate high-quality pseudo-anomaly packets by focusing on rewriting byte locations (= byte locations that are likely to be important) that are strongly Attentioned in normal packets. Figure 2 shows an example of the procedure for generating pseudo-anomaly packets.
[0015] First, the anomaly detection device performs pre-training of the BERT model using the collected normal packets (S1). Next, the anomaly detection device inputs the normal packets into the BERT model, performs encoding, and obtains the Attention for each byte at that time (S2). Here, the Attention mechanism performs processing such as averaging in the Layer direction and Head direction, for example, to obtain the Attention of a one-dimensional vector having the same dimension as the number of bytes of the normal packet input to the BERT model.
[0016] Next, the anomaly detection device randomly samples byte locations from the normal packets M times with weighted sampling using the Attention weights obtained in S2 (S3). The anomaly detection device rewrites the byte locations randomly sampled in S3 with random bytes (for example, 0x00 - 0xff) (S4). That is, the anomaly detection device generates pseudo-anomaly packets by rewriting the important byte locations sampled from the normal packets.
[0017] After S4, if there are unprocessed normal packets (Yes in S5), it returns to S2, and if there are no unprocessed normal packets (No in S5), the process ends.
[0018] By doing so, the anomaly detection device generates the number of normal packets × M pseudo-anomaly packets. Then, the anomaly detection device determines a threshold for anomaly degree for detecting packet anomalies using the generated pseudo-anomaly packet group and the normal packet group. The method for determining the threshold is, for example, F1 optimization or the like.
[0019] Then, the anomaly detection device detects anomaly packets by setting the threshold determined as described above. For example, when the anomaly degree of the packet to be detected exceeds the above threshold, the anomaly detection device detects the packet as an anomaly packet.
[0020] According to such an anomaly detection device, an appropriate threshold for detecting anomaly packets can be set, so that anomaly packets can be detected with high accuracy.
[0021] [Configuration Example] Next, the anomaly detection device 10 will be described with reference to FIG. 3. Note that the anomaly detection device 10 is arranged in an appropriate unit for each network or device to be detected, for example. And the anomaly detection device 10 holds a BERT model, a VAE model, and normal packets corresponding to that unit in the storage unit.
[0022] The anomaly detection device 10 is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.
[0023] The input unit 11 is an interface that receives input operations from input devices such as a keyboard and a mouse, for example. The output unit 12 is an interface for outputting data to a display device such as a liquid crystal display, a printing device such as a printer, etc., for example.
[0024] The communication control unit 13 is realized by, for example, a NIC (Network Interface Card) or the like, and controls the communication between the control unit 15 and an external device via a telecommunications line such as a LAN (Local Area Network) or the Internet.
[0025] The storage unit 14 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance a processing program for operating the anomaly detection device 10, data used during the execution of the processing program, etc., or temporarily stores them each time processing is performed.
[0026] The storage unit 14 stores, for example, the BERT model and VAE (Variational AutoEncoder) model learned by the control unit 15, normal packets, etc. Also, when a threshold value for detecting abnormal packets is determined by the control unit 15, the storage unit 14 stores the determined threshold value.
[0027] The BERT model is a model that has learned a rule for converting one packet into one fixed-length vector. In other words, the BERT model is a model that has learned frequent patterns such as the order of internal byte sequences in normal packets.
[0028] For example, the BERT model solves an auxiliary task of predicting a byte at a certain position in a packet from the surrounding bytes, thereby acquiring an intermediate representation that reflects the characteristics of the packet, that is, a fixed-length vector. Also, the BERT model acquires a vector representation that reflects the characteristics of the packet by predicting a byte at a certain position in the packet from the surrounding bytes.
[0029] The VAE model is a model that assigns an anomaly degree to the fixed-length vector of the packet converted by the BERT model.
[0030] The control unit 15 is implemented using, for example, a CPU (Central Processing Unit) or the like, and executes a processing program stored in a memory. As a result, as illustrated in FIG. 3, the control unit 15 functions as a learning unit 151, an Attention acquisition unit 152, a pseudo-abnormal packet generation unit 153, a threshold determination unit 154, and a detection unit 155. Note that these functional units may be implemented in different hardware, either individually or partially.
[0031] The learning unit 151 uses normal packets to perform learning (generation, update) of the BERT model and the VAE model. For example, the learning unit 151 performs learning of the BERT model by solving two tasks, Masked Language Modeling and Next Sentence Prediction, using normal packets.
[0032] Note that Masked Language Modeling in this embodiment is a task of randomly masking bytes of a normal packet and predicting the byte before masking. Next Sentence Prediction is a task of inputting two packets and predicting whether they are consecutive packets.
[0033] The Attention acquisition unit 152 inputs a normal packet to the learned BERT model and acquires the magnitude of Attention for each byte location when encoding the normal packet.
[0034] The suspected abnormal packet generation unit 153 generates suspected abnormal packets from normal packets. For example, the suspected abnormal packet generation unit 153 samples (e.g., randomly samples) the important byte locations of the normal packet based on the magnitude of Attention at each byte location of the normal packet acquired by the Attention acquisition unit 152. Then, the suspected abnormal packet generation unit 153 rewrites the sampled important byte locations with random bytes (e.g., bytes from 0x00 to 0xff). And the suspected abnormal packet generation unit 153 sets the packet with the important byte locations of the normal packet rewritten as the suspected abnormal packet.
[0035] In addition, when it is preferable that the number of suspected abnormal packets used for determining the threshold is equal to the number of normal packets, the suspected abnormal packet generation unit 153 adjusts the sampling number so that the number of suspected abnormal packets to be generated is the same as the number of normal packets.
[0036] The threshold determination unit 154 determines a threshold for the degree of abnormality for detecting a packet as an abnormal packet using the normal packet and the suspected abnormal packet generated by the suspected abnormal packet generation unit 153.
[0037] For example, the threshold determination unit 154 calculates the degree of abnormality of the normal packet and the suspected abnormal packet based on the above BERT model and VAE model. Next, the threshold determination unit 154 applies F1 optimization or the like to the calculated degrees of abnormality of the normal packet and the suspected abnormal packet to determine the threshold for the degree of abnormality. Then, the threshold determination unit 154 stores the determined threshold for the degree of abnormality in the storage unit 14.
[0038] The detection unit 155 detects abnormal packets using the threshold value determined by the threshold value determination unit 154. For example, when the detection unit 155 receives an input of a packet to be detected, it converts it into a fixed-length vector reflecting the characteristics of the packet by a BERT model. Next, the detection unit 155 assigns an abnormality degree to each fixed-length vector by a VAE model. Then, when the abnormality degree assigned to the fixed-length vector exceeds the above threshold value, the detection unit 155 detects the packet corresponding to the fixed-length vector as an abnormal packet. After that, the detection unit 155 outputs the detection result of the abnormal packet.
[0039] [Example of processing procedure] Next, an example of the processing procedure performed by the anomaly detection device 10 will be described with reference to FIG. 4. First, the learning unit 151 of the anomaly detection device 10 generates or updates a BERT model and a VAE model using normal packets (S11). After that, the pseudo-abnormal packet generation unit 153 generates a pseudo-abnormal packet group using the normal packet group (S12).
[0040] After S12, the threshold value determination unit 154 calculates the abnormality degrees of the pseudo-abnormal packet group and the normal packet group using the BERT model and the VAE model (S13). Then, the threshold value determination unit 154 determines a threshold value for detecting abnormal packets based on the abnormality degrees of the pseudo-abnormal packet group and the normal packet group calculated in S13 (S14). After that, the detection unit 155 detects abnormal packets using the threshold value determined in S14 (S15). After that, the detection unit 155 outputs the detection result of the abnormal packets.
[0041] According to such an anomaly detection device 10, high-quality pseudo-abnormal packets can be generated from normal packets. Thereby, the anomaly detection device 10 can set an appropriate threshold value for detecting abnormal packets. As a result, the anomaly detection device 10 can accurately detect abnormal packets.
[0042] [Comparison result] Next, with reference to FIG. 5, the comparison results are described between the case where the threshold value determined by the anomaly detection device 10 of the present embodiment is used for detecting abnormal packets and the case where the threshold value determined by the prior art (anomaly degree of normal packets + 3 * standard deviation of the anomaly degree of normal packets) is used (comparative example).
[0043] Note that the anomaly detection device 10 determined the threshold value by F1 optimization of the anomaly degrees of suspected abnormal packets and normal packets. Also, the machine learning model used for evaluation is a model that predicts whether the class of the input packet is Positive (abnormal) or Negative (normal) using a BERT model and a VAE model.
[0044] As shown in FIG. 5, when the machine learning model predicts whether the class of the packet is Positive (abnormal) or Negative (normal), the values of Accuracy, Recall, and F-measure are better when using the threshold value determined by the anomaly detection device 10 than when using the threshold value determined by the prior art. From this, it was confirmed that the threshold value determined by the anomaly detection device 10 is more suitable for detecting abnormal packets than the prior art.
[0045] [System configuration, etc.] Also, each component of each part shown in the figure is a functional concept, and it is not necessarily physically configured as shown in the figure. That is, the specific form of the distribution and integration of each device is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage situations, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program executed by the CPU, or can be realized as hardware by wired logic.
[0046] In addition, among the processes described in the above-described embodiments, all or part of the processes described as being automatically performed can also be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, regarding the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings, they can be arbitrarily changed unless otherwise specified.
[0047] [Program] The above-described abnormality detection device 10 can be implemented by installing a program (abnormality detection program) as package software or online software in a desired computer. For example, by causing the information processing device to execute the above program, the information processing device can function as the abnormality detection device 10. The information processing device mentioned here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System), and further includes terminals such as PDAs (Personal Digital Assistants) within its scope.
[0048] FIG. 6 is a diagram showing an example of a computer that executes an abnormality detection program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.
[0049] Memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System), for example. The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.
[0050] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process executed by the above-described abnormality detection device 10 is implemented as a program module 1093 in which computer-executable code is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing the same processing as the functional configuration in the abnormality detection device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).
[0051] Also, the data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1090 to the RAM 1012 and executes them as necessary.
[0052] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090. For example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a LAN (Local Area Network) or a WAN (Wide Area Network)). Then, the program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.
Explanation of Signs
[0053] 10 Abnormality Detection Device 11 Input Unit 12 Output Unit 13 Communication Control Unit 14 Storage Unit 15 Control Unit 151 Learning Unit 152 Attention Acquisition Unit 153 Spurious Abnormality Packet Generation Unit 154 Threshold Determination Unit 155 Detection Unit
Claims
1. An Attention acquisition unit that inputs a normal packet into a BERT (Bidirectional Encoder Representations from Transformers) model trained using normal packets and acquires the magnitude of Attention for each byte location when encoding the normal packet; A pseudo-abnormal packet generation unit that samples important byte locations of the normal packet based on the magnitude of Attention for each byte location of the acquired normal packet and generates a pseudo-abnormal packet by rewriting the sampled important byte locations with random bytes; A threshold determination unit that determines a threshold for anomaly detection based on the degree of anomaly of the generated group of pseudo-abnormal packets and the group of normal packets An anomaly detection device characterized by comprising the above.
2. The threshold determination unit Determines the threshold for the degree of anomaly by the F1 optimization method The anomaly detection device according to claim 1, characterized by the above.
3. A detection unit that detects a packet as an abnormal packet when the degree of anomaly of the packet to be detected exceeds the threshold The anomaly detection device according to claim 1, further comprising the above.
4. The pseudo-abnormal packet generation unit Samples important byte locations of the normal packet based on the magnitude of Attention for each byte location of the acquired normal packet and generates the pseudo-abnormal packet by rewriting the randomly sampled important byte locations with random bytes The anomaly detection device according to claim 1, characterized by the above.
5. An anomaly detection method executed by an anomaly detection device, comprising: A step of inputting a normal packet into a BERT (Bidirectional Encoder Representations from Transformers) model trained using normal packets and acquiring the magnitude of Attention for each byte location when encoding the normal packet; A step of sampling important byte locations of the normal packet based on the magnitude of Attention for each byte location of the acquired normal packet and generating a pseudo-abnormal packet by rewriting the sampled important byte locations with random bytes; A step of determining a threshold value of abnormality degree for detecting abnormal packets based on the abnormality degrees of the generated pseudo-abnormal packet group and the normal packet group An abnormality detection method characterized by including the above.
6. A step of inputting normal packets into a BERT (Bidirectional Encoder Representations from Transformers) model learned using normal packets, and obtaining the magnitude of Attention for each byte position when encoding the normal packets; Based on the magnitude of Attention for each byte position of the obtained normal packets, sampling important byte positions of the normal packets, and generating pseudo-abnormal packets by rewriting the sampled important byte positions with random bytes; A step of determining a threshold value of abnormality degree for detecting abnormal packets based on the abnormality degrees of the generated pseudo-abnormal packet group and the normal packet group An abnormality detection program for causing a computer to execute the above.