Detection device, detection method, and detection program
The detection device employs BERT-based natural language processing to select similar normal packets and calculate anomaly scores, addressing the challenge of detecting unauthorized changes in packet payload content, particularly for protocols with long packet lengths, and enhancing anomaly detection accuracy.
Patent Information
- Application Number
- PCT/JP2023/025542
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2023-07-11
- Publication Date
- 2024-11-28
AI Technical Summary
Anomaly detection systems for industrial and building network control systems face challenges in accurately detecting unauthorized changes in packet payload content, especially with long packet lengths and protocols that include timestamps or serial numbers, leading to potential false positives and reduced detection accuracy.
A detection device and method that uses a natural language processing model like BERT to select similar normal packets and calculate an anomaly score by comparing them with the evaluation target packet data, enabling accurate detection of abnormalities even in protocols with long packet lengths.
The solution effectively detects anomalies in packet contents with high accuracy, reducing false positives and improving detection performance for protocols with varying packet lengths and complexities.
Smart Images

Figure JP2023025542_28112024_PF_FP_ABST
Abstract
Description
Detection device, detection method, and detection program
[0001] The present invention relates to a detection device, a detection method, and a detection program.
[0002] Attention is being focused on anomaly detection systems and intrusion detection systems (OT-IDS: Operational Technology-Intrusion Detection System) for communications used in industrial and building network control systems. For example, even a one-digit change in the temperature setting can lead to a serious accident. In other words, even a one-byte difference in the communication content (payload) can lead to a serious accident.
[0003] For this reason, an anomaly detection system for industrial and building network control systems must be able to perform detailed analysis of payload contents. In other words, the anomaly detection system must be able to detect even a single byte of unauthorized rewriting of the payload.
[0004] Furthermore, various protocols are used in industrial and building networks, so anomaly detection systems are required to be able to analyze payload content for general-purpose protocols.
[0005] One anomaly detection method involves applying natural language processing techniques such as BERT (Bidirectional Encoder Representations from Transformers) to packet analysis to extract information from the payload of any protocol and perform anomaly detection (Patent Document 1).
[0006] International Publication No. 2022 / 059209
[0007] Yutian Chen, Max Welling, Alex Smola, “Super-Samples from Kernel Herding,” [Retrieved April 26, 2023], Internet <URL: https: / / arxiv.org / ftp / arxiv / papers / 1203 / 1203.3472.pdf> Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi, “BERTSCORE: EVALUATING TEXT GENERATION WITH BERT,” [Retrieved April 26, 2023], Internet <URL: https: / / arxiv.org / abs / 1904.09675.pdf>
[0008] However, in the method described in Patent Document 1, since an averaging process is performed in the packet direction, the detection accuracy may be degraded for protocols with long packet lengths. Also, in the method described in Patent Document 1, there is a problem that false positives may occur if a timestamp or a serial number portion is included in a packet and the learning data does not include these comprehensively.
[0009] The present invention has been made in consideration of the above, and aims to provide a detection device, a detection method, and a detection program that can detect abnormalities that occur in the contents of packets with high accuracy, even for protocols with long packet lengths.
[0010] In order to solve the above-mentioned problems and achieve the objectives, the detection device is characterized by having an extraction unit that extracts a predetermined number of similar normal packet data that have a relatively high similarity to packet data to be evaluated from a plurality of normal packet data based on a natural language processing model, and a detection unit that detects whether the packet data to be evaluated is abnormal based on an anomaly score calculated by comparing the predetermined number of similar normal packet data extracted by the extraction unit with the packet data to be evaluated.
[0011] According to the present invention, even for protocols with long packet lengths, it is possible to detect abnormalities occurring in packet contents with high accuracy.
[0012] FIG. 1 is a diagram illustrating an example of the configuration of a detection system according to the first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a learning device shown in FIG. 1. FIG. 3 is a diagram illustrating a procedure for learning processing performed by the learning device shown in FIG. 2. FIG. 4 is a diagram illustrating an example of the configuration of the detection device shown in FIG. 1. FIG. 5 is a diagram illustrating a flow of processing performed by the detection device shown in FIG. 4. FIG. 6 is a diagram illustrating processing performed by the detection unit shown in FIG. 4. FIG. 7 is a diagram illustrating processing performed by the detection unit shown in FIG. 4. FIG. 8 is a diagram illustrating first calculation processing. FIG. 9 is a diagram illustrating a procedure for detection processing according to the first embodiment. FIG. 10 is a flowchart illustrating a procedure for the first calculation processing shown in FIG. 9. FIG. 11 is a flowchart illustrating a procedure for the second calculation processing shown in FIG. 9. FIG. 12 is a diagram illustrating results of an evaluation experiment. FIG. 13 is a diagram illustrating results of an evaluation experiment. FIG. 14 is a diagram illustrating an example of the configuration of a detection system according to the second embodiment. FIG. 15 is a diagram illustrating an example of the configuration of the learning device shown in FIG. 14. FIG. 16 is a diagram illustrating a procedure for learning processing performed by the learning device shown in FIG. 15. FIG. 17 is a diagram illustrating an example of the configuration of the detection device shown in FIG. 14. Fig. 18 is a diagram illustrating the flow of processing by the detection device shown in Fig. 17. Fig. 19 is a diagram illustrating a processing procedure for detection processing according to embodiment 2. Fig. 20 is a diagram illustrating an example of a computer that implements a learning device and a detection device by executing a program.
[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A detailed description of embodiments of a detection device, a detection method, and a detection program according to the present application will be given below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below.
[0014] First Embodiment In a first embodiment, a detection device that detects anomalies in communication data (payload) will be described.
[0015] The detection device according to the first embodiment extracts data similar to the packet data to be evaluated (similar normal packet data) from normal communication data by natural language processing using BERT (Bidirectional Encoder Representations from Transformers). The detection device according to the first embodiment then calculates an anomaly score indicating the difference between the extracted similar normal packet data and the packet data to be evaluated, and if the anomaly score exceeds a predetermined threshold, detects that the packet data to be evaluated is abnormal and outputs an alert.
[0016] [Detection System] Next, a description will be given of a detection system according to Embodiment 1. Fig. 1 is a diagram showing an example of the configuration of the detection system according to Embodiment 1.
[0017] As shown in FIG. 1, the detection system 1 includes a learning device 10 that performs learning of a natural language processing model used by the detection device 20, and a detection device 20 that detects anomalies in communication data (payload) using the natural language processing model trained by the learning device 10.
[0018] [Learning Device] Fig. 2 is a diagram showing an example of the configuration of the learning device 10 shown in Fig. 1. Fig. 3 is a diagram showing the processing procedure of the learning process executed by the learning device 10 shown in Fig. 2.
[0019] The learning device 10 is implemented by loading a predetermined program into a computer or the like including, for example, a read-only memory (ROM), a random access memory (RAM), a central processing unit (CPU), etc., and having the CPU execute the predetermined program. The learning device 10 also has a communication interface for transmitting and receiving various types of information to and from other devices connected via a network or the like.
[0020] The learning device 10 includes a learning data collection unit 11 , a conversion unit 12 having a conversion model 121 , a learning control unit 13 , a comparison normal data generation unit 14 , and a storage unit 15 .
[0021] The learning data collection unit 11 collects learning packet data for the conversion model 121 (step S1 in FIG. 3).
[0022] The learning control unit 13 performs conversion model learning control to cause the conversion model 121 to learn packet conversion rules using the learning packet data (step S2 in FIG. 3). The model parameters of the conversion model 121 optimized in step S2 are applied to a conversion model 231 (described later) of the detection device 20.
[0023] The conversion unit 12 converts packet data into vector data using a conversion model 121. The vector data is data in which each byte of packet data is associated with a vector that represents the characteristics of the value of each byte. The learning control unit 154 causes the conversion model 121 to learn the value of each byte of the learning packet data.
[0024] The conversion model 121 is, for example, a natural language processing model using BERT. The conversion model 121 converts the value of each byte of each communication packet, which is learning packet data, into a 784-dimensional vector. In step S2 of FIG. 3, the conversion model 121 learns a conversion rule for vector data that associates the position of each byte of the communication packet with the 784-dimensional vector converted from that byte.
[0025] The learning control unit 13 may also control the system to perform preliminary learning by solving auxiliary tasks such as MLM (Masked Language Model) or NSP (Next Sentence Prediction) to determine the characteristics of the value of each byte in the learning packet data. MLM predicts the values of missing bytes in a packet that is missing multiple bytes. NSP determines whether two packet data are consecutive packets. The learning control unit 13 uses these auxiliary tasks to determine the validity of the data in the packet and the validity of consecutive packets, and generates a model that identifies the learning packet data. The auxiliary tasks listed here are merely examples, and the learning control unit 13 may also control the system to learn by solving other auxiliary tasks.
[0026] The comparison normal data generation unit 14 also uses the learning packet data to generate a comparison normal data group 151 (step S3 in FIG. 3). The comparison normal data generation unit 14 sets a packet data group sampled from the learning packet data as the comparison normal data group 151.
[0027] At this time, the comparison normal data generation unit 14 performs sampling using a sampling method such as kernel herding. In this case, random sampling is not performed to prevent performance degradation. Although sampling is desirable from the perspective of processing speed, all learning packet data may be used as the comparison normal data group 151. Furthermore, the greater the number of comparison normal packet data in the comparison normal data group 151, the higher the detection accuracy of the detection device 20, which will be described later.
[0028] The normal comparison data group 151 generated in step S3 of FIG. 3 is stored in the storage unit 15 and is also output to the detection device 20.
[0029] [Detection Device] Next, a description will be given of the detection device 20. Fig. 4 is a diagram showing an example of the configuration of the detection device 20 shown in Fig. 1. Fig. 5 is a diagram explaining the processing flow of the detection device 20 shown in Fig. 4.
[0030] The detection device 20 is implemented by, for example, loading a predetermined program into a computer or the like including a ROM, a RAM, a CPU, etc., and causing the CPU to execute the predetermined program. The detection device 20 also has a communication interface for transmitting and receiving various types of information to and from other devices connected via a network, etc.
[0031] The detection device 20 includes a storage unit 21 that stores a comparison normal data group 151 , an evaluation target data acquisition unit 22 , a conversion unit 23 that has a conversion model 231 , an extraction unit 24 , and a detection unit 25 .
[0032] The evaluation target data acquisition unit 22 acquires one piece of evaluation target packet data (step S11 in FIG. 5).
[0033] The conversion unit 23 converts the evaluation target packet data into vector data (hereinafter referred to as evaluation target vector data) using the conversion model 231 (step S12 in FIG. 5). The conversion model 231 is a natural language processing model using BERT. The conversion model 231 is applied with model parameters of the conversion model 121, which has learned vector conversion rules in the learning device 10.
[0034] The extraction unit 24 extracts M pieces of similar normal packet data that have a relatively high degree of similarity to the packet data to be evaluated from the comparison normal data group 151 (plurality of normal packet data) based on the natural language processing model (step S13 in FIG. 5 ). M is a hyperparameter that is set in advance. For example, M is 100.
[0035] The extraction unit 24 converts each piece of comparison normal packet data in the comparison normal data group 151 into vector data (comparison normal vector data) using the conversion model 231. For each piece of comparison normal vector data, the extraction unit 24 calculates the similarity between the comparison normal vector data and the vector data to be evaluated.
[0036] The extraction unit 24 calculates, for example, a BERTScore as the similarity.
[0037] The extraction unit 24 may also calculate the similarity between the vectors of the evaluation target vector data and the vectors of the comparison normal vector data for each byte of the evaluation target vector data, and calculate the similarity between the evaluation target vector data and the comparison normal vector data from the similarity calculated for each byte. Cosine similarity may be used to calculate the similarity between the vectors of each byte. The similarity between the evaluation target vector data and the comparison normal vector data is, for example, the average of the similarities calculated for each byte. In this case, if the number of vectors in the evaluation target vector data differs from the number of vectors in the comparison normal vector data, the similarity may be calculated based on the smaller number of vectors. The number of vectors in each vector data is the number of bytes of packet data before conversion.
[0038] The extraction unit 24 may calculate the similarity with some of the comparison normal vector data in the comparison normal vector data group. For example, the some comparison normal vector data is obtained by extracting multiple representative packet data from multiple normal packet data using MMD (Maximum Mean Discrepancy)-Critic and converting each of the extracted representative packet data using a model. Alternatively, the some comparison normal vector data is multiple comparison normal vector data obtained by extracting comparison normal packet data having the same packet length as the evaluation target packet data from the comparison normal data group 151 and converting each of the extracted comparison normal packet data using a model.
[0039] The extraction unit 24 extracts M pieces of comparative normal packet data with a relatively high degree of similarity. Hereinafter, the comparative normal packet data extracted by the extraction unit 24 will be referred to as similar normal packet data.
[0040] A relatively high similarity means that the similarity between the evaluation target vector data and a certain comparison normal vector data is higher than the similarity between the evaluation target vector data and another comparison normal vector data. The extraction unit 24 extracts a predetermined number of comparison normal packet data in descending order of similarity, starting with the comparison normal packet data with the highest similarity, as similar normal packet data. Alternatively, the extraction unit 24 may extract M comparison normal packet data from the comparison normal packet data with similarity higher than a threshold value as similar normal packet data.
[0041] The detection unit 25 detects whether the packet data to be evaluated is abnormal based on the anomaly score calculated by comparing the M pieces of similar normal packet data extracted by the extraction unit 24 with the packet data to be evaluated.
[0042] The detection unit 25 extracts, from the similar normal packet data, identical-length packet data having the same packet length as the evaluation target packet data. The detection unit 25 compares the evaluation target packet data with the identical-length packet data byte by byte to calculate a first anomaly score. Alternatively, if the packet length of the similar normal packet data is different from that of the evaluation target packet data, the detection unit 25 calculates a second anomaly score based on the edit distance between the evaluation target packet data and the similar normal packet data.
[0043] 6 and 7 are diagrams illustrating the processing of the detection unit 25 shown in Fig. 4. The detection unit 25 performs calculations by comparing the M pieces of similar normal packet data extracted by the extraction unit 24 with the packet data to be evaluated. The detection unit 25 includes a length comparison unit 251, a first calculation unit 252, and a second calculation unit 253.
[0044] The length comparison unit 251 compares the lengths of the M pieces of similar normal packet data with the evaluation target packet data. The length comparison unit 251 determines whether the number of similar normal packet data having the same length as the evaluation target packet data among the M pieces of similar normal packet data is equal to or greater than a predetermined threshold (step S14 in FIG. 5). Note that similar normal packet data having the same length as the evaluation target packet data is referred to as same-length normal packet data.
[0045] If the length comparison unit 251 determines that the number of normal packet data of the same length as the packet data to be evaluated is greater than or equal to a predetermined threshold (step S14 in Figure 5: Yes), the first calculation unit 252 performs a first calculation process (step S15 in Figure 5).
[0046] In the first calculation process, the first calculation unit 252 performs one-dimensional detection based on a comparison of the numerical values for each byte position between the evaluation target packet data and the same normal packet data, and calculates an anomaly score for each byte position ( FIG. 6 ). The first calculation unit 252 compares each byte (int) (0 to 255) one by one, detects any violations, and calculates an anomaly score. The first calculation unit 252 outputs the sum of the anomaly scores for all byte positions as the first anomaly score.
[0047] If the length comparison unit 251 determines that the number of similar normal packet data having the same length as the packet data to be evaluated is less than a predetermined threshold (step S14 in Figure 5: No), the second calculation unit 253 performs a second calculation process (step S16 in Figure 5).
[0048] As the second calculation process, the second calculation unit 253 calculates the edit distance between the evaluation target packet data and each similar normal packet data ( FIG. 7 ). The second calculation unit 253 outputs the smallest edit distance among the calculated edit distances as the second anomaly score. At the same time, the second calculation unit 253 estimates and detects insertion byte locations suspected of insertion or deletion byte locations suspected of deletion based on the calculated edit distance.
[0049] If the first anomaly score or the second anomaly score exceeds a predetermined threshold, the detection unit 25 detects that the evaluation target packet data is abnormal and issues an alert.
[0050] [First Calculation Process] The first calculation process will be described with reference to Fig. 8.
[0051] Specifically, the first calculation unit 252 directly treats the value of the first byte of the similar normal packet data extracted for comparison as a number between "0" and "255." The first calculation unit 252 calculates the anomaly score by applying one-dimensional anomaly detection using, for example, KDE (Kernel Density Estimator) (Reference 3). The first calculation unit 252 similarly calculates the anomaly scores for the second byte and subsequent bytes. The first calculation unit 252 performs one-dimensional detection based on a comparison of the numerical values for each byte position to calculate the anomaly score for each byte position. Reference 3: Duda, R. and Hart, P. (1973), "PATTERN CLASSIFICATION AND SCENE ANALYSIS," John Wiley & Sons. ISBN 0-471-22361-1.
[0052] The first calculation unit outputs the anomaly score in the range of a minimum of "0" to a maximum of "1." For example, a case where byte position L1 is the detection target will be described ((1) in FIG. 8).
[0053] The first calculation unit 252 calculates a probability density distribution of the numerical value of the byte appearing at byte position L1 of the similar normal packet data for each similar normal packet data. In other words, the first calculation unit 252 calculates a distribution centered on the number of bytes appearing in the similar normal packet and calculates the sum of these distributions ((2) in FIG. 8). For example, among the M similar normal packet data, the probability density distribution D1 of the first similar normal packet data, the probability density distribution D2 of the second similar normal packet data, and the probability density distribution D3 of the third similar normal packet data are calculated as the probability density distribution of the numerical value of the byte appearing at byte position L1.
[0054] The first calculation unit 252 obtains a probability density distribution Dt by taking the sum of each probability density distribution (e.g., probability density distributions D1 to D3) of the similar normal packet data. The first calculation unit 252 observes the probability density of the byte value of the evaluation target packet data on the probability density distribution Dt, and performs a normal / abnormal determination of byte position L1 of the evaluation target packet data ((3) in FIG. 8).
[0055] Specifically, the first calculation unit 252 compares a predetermined threshold (for example, 1 / 1024) with the probability density of the byte value of the packet data to be evaluated, and calculates the anomaly score.
[0056] If the probability density of the byte value of the packet data to be evaluated is greater than or equal to a threshold value (for example, byte values "17" or "21"), the first calculation unit 252 determines that this byte position L1 is normal and assigns an anomaly score of 0.
[0057] On the other hand, if the probability density of the byte value of the packet data to be evaluated is less than the threshold value, i.e., approximately "0" (for example, the byte value is "ff"), the first calculation unit 252 determines that this byte position L1 is abnormal and assigns an anomaly score of 1.
[0058] Furthermore, when the probability density of the evaluation target packet data is equal to or greater than the threshold but is close to the threshold (for example, the byte value "30"), the first calculation unit 252 may interpolate the anomaly score between 0 and 1 according to a predetermined rule. For example, the first calculation unit 252 assigns an anomaly score of 0.3 to the byte value "30".
[0059] In this way, the first calculation unit 252 determines whether there is no violation (normal) or a violation (abnormal) for each byte position based on the probability density distribution Dt ((3) in FIG. 8). Therefore, even if the byte value changes randomly at a byte position, this does not lead to overdetection.
[0060] The first calculation unit 252 then calculates an anomaly score for each byte position and outputs the sum of these anomaly scores as the final anomaly score (first anomaly score) for the packet data to be evaluated.
[0061] The first calculation unit 252 may use any anomaly detection method that can handle one-dimensional data, such as a non-parametric method or a naive Bayes method.
[0062] [Second Calculation Process] Next, the second calculation process will be described. In the second calculation process, the second calculation unit 253 calculates the edit distance between the evaluation target packet data and each similar normal packet data by using dynamic programming.
[0063] The second calculation unit 253 directly outputs the calculated edit distance as the anomaly score. Specifically, the second calculation unit 253 sets the smallest edit distance among the calculated edit distances as the second anomaly score.
[0064] Furthermore, by calculating the edit distance, the second calculation unit 253 can identify the location of, for example, an abnormal byte insertion and / or deletion, and directly identify how many packets have been inserted. Furthermore, if there is no similar normal packet data, the edit distance is considered to be large.
[0065] The second calculation unit 253 acquires the evaluation target packet data and M pieces of similar normal packet data from the length comparison unit 251. Then, the second calculation unit 253 compares the acquired similar normal packet data with the evaluation target packet data in order to identify inserted byte positions or deleted byte positions in the evaluation target packet data.
[0066] Specifically, the second calculation unit 253 calculates the edit distance between the packet data to be evaluated and each of the similar normal packet data by using dynamic programming. By calculating the edit distance, the second calculation unit 253 can identify an insertion byte location where insertion is suspected or a deletion byte location where deletion is suspected.
[0067] If there is similar normal packet data whose edit distance is less than the certain distance, the second calculation unit 253 selects the similar normal packet data whose edit distance is shortest from among the similar normal packet data whose edit distance is less than the certain distance as the final similar normal packet data.The second calculation unit 253 then estimates the insertion / deletion byte locations using the edit distance of the selected final similar normal packet data.
[0068] On the other hand, if the edit distances of all the acquired similar normal packet data are equal to or greater than the certain distance, the second calculation unit 253 processes the packet data to be evaluated as not having the final similar normal packet data corresponding to the packet data to be evaluated. Here, the certain distance is a parameter that can specify, for example, approximately 1 / 3 to 1 / 2 of the packet length.
[0069] [Procedure of Detection Processing] FIG. 9 is a diagram showing a procedure of detection processing according to the first embodiment.
[0070] The detection device 20 acquires one piece of evaluation target packet data (step S21). The conversion unit 23 converts the evaluation target packet data into evaluation target vector data using the conversion model 231 (step S22).
[0071] The extraction unit 24 reads each comparison normal packet data from the comparison normal data group 151 (step S23), and converts each comparison normal packet data into comparison normal vector data using the conversion model 231 (step S24).
[0072] The extraction unit 24 extracts M pieces of normal vector data for comparison that have a relatively high degree of similarity with the vector data to be evaluated as normal vector data for similarity comparison (step S25).
[0073] The extraction unit 24 calculates the similarity between each of the comparison normal vector data and the evaluation target vector data, and extracts M similar normal packet data with relatively high similarity based on the calculated similarity (step S25).
[0074] The length comparison unit 251 compares the lengths of the M pieces of similar normal packet data with the evaluation target packet data, and determines whether the number of similar normal packet data pieces having the same length as the evaluation target packet data among the M pieces of similar normal packet data is equal to or greater than a predetermined threshold (step S26).
[0075] If the number of similar normal packet data having the same length as the packet data to be evaluated is greater than or equal to a predetermined threshold (step S26: Yes), the first calculation unit 252 performs a first calculation process (step S27) and outputs a first anomaly score.
[0076] If the number of quasi-normal packet data having the same length as the packet data to be evaluated is less than a predetermined threshold (step S26: No), the second calculation unit 253 performs a second calculation process (step S28) and outputs a second anomaly score.
[0077] The detection unit 25 determines whether the first anomaly score or the second anomaly score exceeds a predetermined threshold value (step S29).
[0078] If the first anomaly score or the second anomaly score exceeds the predetermined threshold (step S29: Yes), the detection unit 25 detects an abnormality in the evaluation target packet data and issues an alert (step S30).
[0079] If the first anomaly score or the second anomaly score is equal to or less than the predetermined threshold (step S29: No), the detection device 20 ends the detection process for this packet data to be evaluated.
[0080] [First Calculation Process] FIG. 10 is a flowchart showing the processing procedure of the first calculation process shown in FIG.
[0081] The first calculation unit 252 sets a parameter n, which indicates the position of the byte to be compared, to n=1 (step S31).
[0082] The first calculation unit 252 calculates a probability density distribution Dt by summing the values at byte position n of the same normal packet data (step S32). The first calculation unit 252 compares the probability density distribution Dt with the value at byte position n of the evaluation target packet data (step S33). The first calculation unit 252 then compares the value at byte position n of the evaluation target packet data with a predetermined threshold (e.g., 1 / 1024) and determines whether the value at byte position n of the evaluation target packet data is equal to or greater than the threshold (step S34).
[0083] If the numerical value at byte position n of the evaluation target packet data is equal to or greater than the threshold value (step S34: Yes), the first calculation unit 252 assigns 0 as the anomaly score for this byte position n (step S35). The first calculation unit 252 may interpolate the anomaly score between 0 and 1 according to a predetermined rule.
[0084] If the numerical value at byte position n of the evaluation target packet data is less than the threshold value (step S34: No), the first calculation unit 252 assigns 1 as the anomaly score for this byte position n (step S36).
[0085] If byte position n is not the last position (step S37: No), the first calculation unit 252 sets n = n + 1 (step S38), performs the processing from step S32 onwards for the numerical value of the next byte position, and calculates the anomaly score.
[0086] If byte position n is the last position (step S37: Yes), the first calculation unit 252 adds up the anomaly scores of all byte positions (step S39) and outputs the sum of the anomaly scores as the first anomaly score for the packet data to be evaluated.
[0087] [Second Calculation Process] FIG. 11 is a flowchart showing the processing procedure of the second calculation process shown in FIG.
[0088] The second calculation unit 253 selects one unselected piece of similar normal packet data from the predetermined number of similar normal packet data (step S41).
[0089] Next, the second calculation unit 253 calculates the edit distance between the evaluation target packet data and the selected similar normal packet data using dynamic programming (step S42).
[0090] Next, the second calculation unit 253 determines whether or not the selection of all of the predetermined number of similar normal packet data has been completed (step S43). If unselected normal packet data remains among the predetermined number of similar normal packet data (step S43: No), the second calculation unit 253 returns to step S41.
[0091] On the other hand, if selection of all of the predetermined number of similar normal packet data has been completed (step S43: Yes), the second calculation unit 253 determines whether there is similar normal packet data whose edit distance is less than a certain distance (step S44).
[0092] If similar normal packet data whose edit distance is less than the certain distance exists (step S44: Yes), the second calculation unit 253 executes the following process. In this case, the second calculation unit 253 selects the similar normal packet data whose edit distance is shortest among the similar normal packet data whose edit distance is less than the certain distance as the final similar normal packet data. Then, the second calculation unit 253 estimates the insertion / deletion byte location 19 using the edit distance between the selected final similar normal packet data and the packet data to be evaluated (step S45).
[0093] On the other hand, if there is no similar normal packet data whose edit distance is less than the certain distance (step S44: No), the second calculation unit 253 determines that there is no final similar normal packet data (step S46).
[0094] The second calculation unit 253 then outputs the smallest edit distance among the calculated edit distances as the second anomaly score (step S47). The second calculation unit 253 also outputs the results of estimation and detection of insertion byte locations where insertion or deletion byte locations where deletion is suspected.
[0095] [Evaluation Experiment] The results of the evaluation experiment are shown below. 451 pieces of training packet data and 439 pieces of normal packet data for comparison were extracted from BACnet protocol communications (average packet length 93.5, maximum packet length 816) acquired in a small-scale experimental environment. The evaluation target packet data was artificially generated by rewriting random positions of 1 to 5 bytes in the normal packet data for comparison with random bytes. BERT's training data itself was also used as the normal packet data for comparison.
[0096] 12 and 13 are diagrams showing the results of the evaluation experiment. FIG. 12 shows the detection results obtained by the detection device 20. FIG. 13 shows the detection results obtained using the method described in Patent Document 1. The evaluation was performed using AUROC. For detection accuracy, an AUC score of 0.95 or higher is desirable.
[0097] 12 and 13, the detection accuracy of the detection device 20 was higher than that of the method described in Patent Document 1. Furthermore, as shown in Fig. 12, when detected by the detection device 20, the AUC score was significantly higher than 0.95 for data in which three or more bytes had been altered, meaning that detection was almost complete.
[0098] [Effects of First Embodiment] As shown in this evaluation experiment, according to the first embodiment, it is possible to detect anomalies occurring in the contents of packets with high accuracy for any communication protocol with long packets.
[0099] Second Embodiment In a second embodiment, a detection device that detects anomalies in communication data (payload) will be described.
[0100] The detection device according to the second embodiment uses natural language processing using BERT to speed up the process of extracting data similar to packet data to be evaluated from normal communication data.
[0101] For example, when extracting similar normal packet data, the detection device 20 according to the first embodiment calculates the similarity for one evaluation target packet as many times as the number of comparison normal packets included in the comparison normal data group 151. Therefore, the processing cost of the detection device 20 is the cost shown in formula (1).
[0102] Processing cost=(number of packets to be evaluated)×(number of normal packet data for comparison)×(number of bytes of packets to be evaluated)×(number of bytes of normal packet data for comparison) (1)
[0103] The detection device according to the second embodiment efficiently reduces the “number of normal packet data for comparison” included in formula (1), thereby enabling extraction of similar normal packet data with lower processing costs than the processing costs of the detection device 20 according to the first embodiment.
[0104] [Detection System] Next, a description will be given of a detection system according to Embodiment 2. Fig. 14 is a diagram showing an example of the configuration of the detection system according to Embodiment 2.
[0105] As shown in FIG. 14 , the detection system 2 includes a learning device 30 that performs learning of a natural language processing model used by the detection device 40, and a detection device 40 that detects anomalies in communication data (payload) using the natural language processing model trained by the learning device 30.
[0106] [Learning Device] Fig. 15 is a diagram showing an example of the configuration of the learning device 30 shown in Fig. 14. Fig. 16 is a diagram showing the processing procedure of the learning process executed by the learning device 30 shown in Fig. 15.
[0107] The learning device 30 is implemented by, for example, loading a predetermined program into a computer or the like including a ROM, RAM, CPU, etc., and having the CPU execute the predetermined program. The learning device 30 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc.
[0108] The learning device 30 includes a learning data collection unit 31, a conversion unit 32 having a conversion model 321, a learning control unit 33, a comparison normal data generation unit , a clustering processing unit 35, and a storage unit .
[0109] The learning data collection unit 31 collects learning packet data for the conversion model 321 (step S51 in FIG. 16).
[0110] The learning control unit 33 performs conversion model learning control to make the conversion model 321 learn packet conversion rules using the learning packet data (step S52 in FIG. 16 ). The model parameters of the conversion model 321 optimized in step S52 are applied to a conversion model 441 (described later) of the detection device 40.
[0111] The conversion unit 32 converts the packet data into vector data using a conversion model 321. The vector data is data in which each byte of the packet data is associated with a vector that represents the characteristics of the value of each byte. The learning control unit 33 causes the conversion model 321 to learn the value of each byte of the learning packet data.
[0112] The conversion model 321 is, for example, a natural language processing model using BERT. The conversion model 321 converts the value of each byte of each communication packet, which is learning packet data, into a 784-dimensional vector. The conversion model 321 learns a conversion rule for vector data in step S52 of FIG. 16 , which associates the position of each byte of a communication packet with the 784-dimensional vector converted from that byte.
[0113] In addition, similar to the learning control unit 13, the learning control unit 33 may perform control to perform preliminary learning by solving auxiliary tasks such as MLM or NSP to determine the characteristics of the value of each byte in the learning packet data.
[0114] The comparison normal data generation unit 34 uses the learning packet data to generate a comparison normal data group 151 (step S53 in FIG. 16 ). The comparison normal data generation unit 34 sets a packet data group sampled from the learning packet data as the comparison normal data group 151.
[0115] At this time, the comparison normal data generation unit 34 performs sampling using a sampling method such as kernel herding. In this case, random sampling is not performed to prevent performance degradation. Although sampling is desirable from the viewpoint of processing speed, all learning packet data may be used as the comparison normal data group 151.
[0116] In step S53 of FIG. 16, the comparison normal data group 151 generated by the comparison normal data generating unit 34 is stored in the storage unit 36.
[0117] The clustering processor 35 acquires the comparison normal data group 151 stored in the storage unit 36, and extracts low-dimensional features from each comparison normal data included in the comparison normal data group 151 (step S54 in FIG. 16 ). For example, the clustering processor 35 may extract low-dimensional features using techniques such as principal component analysis, linear discriminant analysis, t-SNE (t-Distributed Stochastic Neighbor Embedding), or autoencoder, or may extract low-dimensional features using other conventional techniques.
[0118] The user may manually set low-dimensional features for each piece of comparison normal data included in the comparison normal data group 151. In this case, the clustering processing unit 35 executes the following process using the low-dimensional features set in advance for each piece of comparison normal data.
[0119] The clustering processing unit 35 learns clustering based on the low-dimensional feature values of each comparison normal data (step S55 in FIG. 16 ). For example, the clustering processing unit 35 compares the low-dimensional feature values of each comparison normal data and performs processing to classify comparison normal data with similar feature values into the same cluster. For example, the clustering processing unit 35 performs k-means clustering, hierarchical clustering, density-based clustering, etc. to learn clustering.
[0120] For example, the clustering processing unit 35 generates a comparison normal data cluster group 361 by learning clustering for the comparison normal data group 151 (step S56 in FIG. 16). The comparison normal data cluster group 361 includes comparison normal data clusters 361-1, 361-2, ..., 361-N.
[0121] The comparison normal data cluster group 361 generated in step S56 of FIG. 16 is stored in the storage unit 36 and is also output to the detection device 40.
[0122] [Detection Device] Next, a description will be given of the detection device 40. Fig. 17 is a diagram showing an example of the configuration of the detection device 40 shown in Fig. 14. Fig. 18 is a diagram explaining the flow of processing by the detection device 40 shown in Fig. 17.
[0123] The detection device 40 is implemented by, for example, loading a predetermined program into a computer or the like including a ROM, a RAM, a CPU, etc., and causing the CPU to execute the predetermined program. The detection device 40 also has a communication interface for transmitting and receiving various types of information to and from other devices connected via a network, etc.
[0124] The detection device 40 includes a storage unit 41 that stores a comparison normal data cluster group 361, an evaluation target data acquisition unit 42, a cluster setting unit 43, a conversion unit 44 that has a conversion model 441, an extraction unit 45, and a detection unit 46.
[0125] The evaluation target data acquisition unit 42 acquires one piece of evaluation target packet data (step S61 in FIG. 18 ). The evaluation target data acquisition unit 42 outputs the evaluation target packet data to the cluster setting unit 43 and the conversion unit 44.
[0126] The cluster setting unit 43 extracts low-dimensional features from the packet data to be evaluated, performs clustering, selects one comparison normal data cluster from the comparison normal data clusters 361-1 to 361-N included in the comparison normal data cluster group 361, and sets the selected cluster in the conversion model 441 (step S62 in FIG. 18). In the following description, the comparison normal data cluster set by the cluster setting unit 43 will be referred to as a "set cluster."
[0127] Alternatively, the user may manually set low-dimensional features of the evaluation target packet data, and the cluster setting unit 43 may use such low-dimensional features. Alternatively, the cluster setting unit 43 may extract low-dimensional features of the normal comparison data in the same manner as the clustering processing unit 35 of the learning device 30, or may use low-dimensional features that have been set in advance.
[0128] For example, the cluster setting unit 43 may select a set cluster by performing the following process. The cluster setting unit 43 selects one piece of comparison normal data from each piece of comparison normal data included in the comparison normal data cluster 361-1, extracts low-dimensional features of the selected comparison normal data, and sets the extracted low-dimensional features as the low-dimensional features of the comparison normal data cluster 361-1. The cluster setting unit 43 similarly extracts low-dimensional features for the other comparison normal data clusters 361-2 to 361-N.
[0129] The cluster setting unit 43 compares the low-dimensional feature of the evaluation target packet data with the low-dimensional feature of each of the comparison normal data clusters 361-1 to 361-N and calculates the similarity. The cluster setting unit 43 selects, as the setting cluster, the comparison normal data cluster corresponding to the low-dimensional feature that has the greatest similarity to the low-dimensional feature of the evaluation target packet data.
[0130] The conversion unit 44 converts the evaluation target packet data into vector data (hereinafter referred to as evaluation target vector data) using the conversion model 441 (step S63 in FIG. 18 ). The conversion model 441 is a natural language processing model using BERT. The conversion model 441 is applied with model parameters of the conversion model 321, which has learned vector conversion rules in the learning device 30.
[0131] The extraction unit 45 extracts X pieces of similar normal packet data that have a relatively high degree of similarity to the packet data to be evaluated from the set cluster (plurality of normal packet data) based on the natural language processing model (step S64 in FIG. 18 ). X is a hyperparameter that is set in advance. For example, X is 100.
[0132] The extraction unit 45 converts each comparison normal packet data of the set cluster into vector data (comparison normal vector data) using the conversion model 441. For each comparison normal vector data, the extraction unit 45 calculates the similarity between the comparison normal vector data and the evaluation target vector data.
[0133] The process of calculating the similarity performed by the extraction unit 45 is similar to the process of calculating the similarity performed by the extraction unit 24 of the first embodiment described above, and therefore will not be described again. Note that the extraction unit 24 performs a process of calculating the similarity for each comparison normal packet data in the comparison normal data group 151, whereas the extraction unit 45 performs a process of calculating the similarity only for each comparison normal packet data included in the set cluster. Therefore, the extraction unit 45 of the second embodiment can perform the process of calculating the similarity faster than the extraction unit 24 of the first embodiment.
[0134] The extraction unit 45 extracts X pieces of comparative normal packet data that have a relatively high degree of similarity. Hereinafter, the comparative normal packet data extracted by the extraction unit 45 will be referred to as similar normal packet data.
[0135] A relatively high similarity means that the similarity between the evaluation target vector data and a certain comparison normal vector data is higher than the similarity between the evaluation target vector data and another comparison normal vector data. The extraction unit 45 extracts a predetermined number of comparison normal packet data in descending order of similarity, starting with the comparison normal packet data with the highest similarity, as similar normal packet data. Alternatively, the extraction unit 45 may extract, as similar normal packet data, X comparison normal packet data from the comparison normal packet data with similarity higher than a threshold value.
[0136] The detection unit 46 detects whether the packet data to be evaluated is abnormal based on the anomaly score calculated by comparing the M pieces of similar normal packet data extracted by the extraction unit 45 with the packet data to be evaluated.
[0137] The detection unit 46 extracts, from the similar normal packet data, identical-length packet data having the same packet length as the evaluation target packet data. The detection unit 46 compares the evaluation target packet data with the identical-length packet data byte by byte to calculate a first anomaly score. Alternatively, if the packet length of the similar normal packet data is different from that of the evaluation target packet data, the detection unit 46 calculates a second anomaly score based on the edit distance between the evaluation target packet data and the similar normal packet data.
[0138] The detection unit 46 performs the calculation by comparing the X pieces of similar normal packet data extracted by the extraction unit 45 with the packet data to be evaluated. The detection unit 46 includes a length comparison unit 461, a first calculation unit 462, and a second calculation unit 463.
[0139] The length comparison unit 461 compares the length of the X pieces of similar normal packet data with the length of the evaluation target packet data. The length comparison unit 461 determines whether or not the number of similar normal packet data having the same length as the evaluation target packet data, among the X pieces of similar normal packet data, is equal to or greater than a predetermined threshold (step S65 in FIG. 18 ). Note that similar normal packet data having the same length as the evaluation target packet data is referred to as same-length normal packet data.
[0140] If the length comparison unit 461 determines that the number of normal packet data of the same length as the packet data to be evaluated is greater than or equal to a predetermined threshold (step S65 in Figure 18: Yes), the first calculation unit 462 performs a first calculation process (step S66 in Figure 18).
[0141] In the first calculation process, the first calculation unit 462 performs one-dimensional detection based on a comparison of the numerical values for each byte position between the evaluation target packet data and the same normal packet data, and calculates the anomaly score for each byte position ( FIG. 6 ). The first calculation unit 462 compares each byte (int) (0 to 255) one by one, detects any violations, and calculates the anomaly score. The first calculation unit 462 outputs the total value of the anomaly scores for all byte positions as the first anomaly score.
[0142] If the length comparison unit 461 determines that the number of similar normal packet data having the same length as the packet data to be evaluated is less than a predetermined threshold (step S65 in Figure 18: No), the second calculation unit 463 performs a second calculation process (step S67 in Figure 18).
[0143] As the second calculation process, the second calculation unit 463 calculates the edit distance between the evaluation target packet data and each similar normal packet data ( FIG. 7 ). The second calculation unit 463 outputs the smallest edit distance among the calculated edit distances as the second anomaly score. At the same time, the second calculation unit 463 estimates and detects insertion byte locations suspected of insertion or deletion byte locations suspected of deletion based on the calculated edit distance.
[0144] If the first anomaly score or the second anomaly score exceeds a predetermined threshold, the detection unit 46 detects that the evaluation target packet data is abnormal and issues an alert.
[0145] The first calculation process executed by the first calculation unit 462 is similar to the first calculation process described in embodiment 1. The second calculation process executed by the second calculation unit 463 is similar to the second calculation process described in embodiment 1.
[0146] [Procedure of Detection Processing] FIG. 19 is a diagram showing a procedure of detection processing according to the second embodiment.
[0147] The detection device 40 acquires one piece of evaluation target packet data (step S101). The cluster setting unit 43 extracts low-dimensional features from the evaluation target packet data and performs clustering (step S102). The cluster setting unit 43 sets a normal data cluster for comparison (step S103).
[0148] The conversion unit 44 converts the evaluation target packet data into evaluation target vector data using the conversion model 441 (step S104).
[0149] The extraction unit 45 reads each comparison normal packet data of the set cluster (step S105), and converts each comparison normal packet data into comparison normal vector data using the conversion model 441 (step S106).
[0150] The extraction unit 45 extracts X pieces of normal vector data for comparison that have a relatively high degree of similarity with the vector data to be evaluated as normal vector data for similarity comparison (step S107).
[0151] The extraction unit 45 calculates the similarity between each of the comparison normal vector data and the evaluation target vector data. Based on the calculated similarities, the extraction unit 24 extracts X pieces of similar normal packet data with relatively high similarity (step S107).
[0152] The length comparison unit 461 compares the length of the X pieces of similar normal packet data with the length of the evaluation target packet data, and determines whether the number of similar normal packet data pieces having the same length as the evaluation target packet data among the X pieces of similar normal packet data is equal to or greater than a predetermined threshold (step S108).
[0153] If the number of similar normal packet data having the same length as the packet data to be evaluated is greater than or equal to a predetermined threshold (step S108: Yes), the first calculation unit 462 performs a first calculation process (step S109) and outputs a first anomaly score.
[0154] If the number of quasi-normal packet data having the same length as the packet data to be evaluated is less than a predetermined threshold (step S108: No), the second calculation unit 463 performs a second calculation process (step S110) and outputs a second anomaly score.
[0155] The detection unit 46 determines whether the first anomaly score or the second anomaly score exceeds a predetermined threshold value (step S111).
[0156] If the first anomaly score or the second anomaly score exceeds the predetermined threshold (step S111: Yes), the detection unit 46 detects an abnormality in the evaluation target packet data and issues an alert (step S112).
[0157] If the first anomaly score or the second anomaly score is equal to or less than the predetermined threshold (step S111: No), the detection device 40 ends the detection process for this packet data to be evaluated.
[0158] The processing procedure of the first calculation process shown in step S109 of Fig. 19 is similar to the processing procedure of the first calculation process described in Fig. 10. The processing procedure of the second calculation process shown in step S110 of Fig. 19 is similar to the processing procedure of the second calculation process described in Fig. 11.
[0159] [Effects of Second Embodiment] The extraction unit 45 of the detection device 40 according to the second embodiment executes a process of calculating similarity only for each comparison normal packet data included in the set cluster. Therefore, the extraction unit 45 of the second embodiment can speed up the process of calculating similarity compared to the extraction unit 24 of the detection device 20 according to the first embodiment. For example, the detection device 40 can extract similar normal packet data with lower processing cost than the processing cost of the detection device 20 according to the first embodiment by efficiently reducing the "number of comparison normal packet data" included in formula (1).
[0160] [Application Example] The first and second embodiments can be applied to an anomaly detection system for IoT devices, for example.
[0161] Specifically, a network sensor is placed on the IoT network and packets are captured. The captured packets are then used to train the encoding unit (BERT). After training is complete, normal packet data for comparison is set in the detection device 20 (detection device 40). Packets acquired by the network sensor are then input into the detection device 20, and an anomaly score is calculated for each packet. This anomaly score is used to determine whether the packet to be evaluated is similar to the packet acquired during training (i.e., whether it is a normal packet).
[0162] [System Configuration of the Embodiments] The components of the learning devices 10, 30 and the detection devices 20, 40 are conceptual functional components and do not necessarily need to be physically configured as shown in the drawings. In other words, the specific forms of distribution and integration of the functions of the learning devices 10, 30 and the detection devices 20, 40 are not limited to those shown in the drawings, and all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.
[0163] Furthermore, all or any part of the processes performed by the learning devices 10, 30 and the detection devices 20, 40 may be realized by a CPU and a program analyzed and executed by the CPU. Furthermore, each process performed by the learning devices 10, 30 and the detection devices 20, 40 may be realized as hardware using wired logic.
[0164] Furthermore, among the processes described in the first and second embodiments as being performed automatically, all or part of the processes may be performed manually. Alternatively, all or part of the processes described as being performed manually may be performed automatically using a known method. In addition, the process procedures, control procedures, specific names, and information including various data and parameters described above and illustrated may be changed as appropriate unless otherwise specified.
[0165] 20 is a diagram showing an example of a computer in which a learning device 10 and a detection device 20 are realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.
[0166] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.
[0167] The hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes of the learning devices 10 and 30 and the detection devices 20 and 40 are implemented as program modules 1093 in which code executable by the computer 1000 is written. The program modules 1093 are stored, for example, in the hard disk drive 1031. For example, the program modules 1093 for executing processes similar to the functional configurations of the learning devices 10 and 30 and the detection devices 20 and 40 are stored in the hard disk drive 1031. The hard disk drive 1031 may be replaced by an SSD (Solid State Drive).
[0168] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1031. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1031 into RAM 1012 as necessary and executes them.
[0169] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1031, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.
[0170] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.
[0171] 10, 30 Learning device 11, 31 Learning data collection unit 12, 23, 32, 44 Conversion unit 13, 33 Learning control unit 14, 34 Comparison normal data generation unit 15, 21, 36, 41 Storage unit 20, 40 Detection device 22, 42 Evaluation target data acquisition unit 24, 45 Extraction unit 25, 46 Detection unit 121, 231, 321, 441 Conversion model 151 Comparison normal data group 251, 461 Length comparison unit 252, 462 First calculation unit 253, 463 Second calculation unit 361 Comparison normal data cluster group
Claims
1. A detection device comprising: an extraction unit that extracts a predetermined number of similar normal packet data that have a relatively high similarity to packet data to be evaluated from among a plurality of normal packet data based on a natural language processing model; and a detection unit that detects whether the packet data to be evaluated is abnormal based on an anomaly score calculated by comparing the predetermined number of similar normal packet data extracted by the extraction unit with the packet data to be evaluated.
2. The detection device according to claim 1, characterized in that the detection unit has: a first calculation unit that extracts same-length packet data having the same packet length as the packet data to be evaluated from the similar normal packet data, and compares the packet data to be evaluated with the same-length packet data for each byte to calculate a first anomaly score; and a second calculation unit that, when the packet length of the similar normal packet data is different from that of the packet data to be evaluated, calculates a second anomaly score based on an edit distance between the packet data to be evaluated and the similar normal packet data.
3. The detection device described in claim 2, characterized in that the first calculation unit performs one-dimensional detection based on a comparison of numerical values for each byte position between the packet data to be evaluated and the same-length packet data, calculates an anomaly score for each byte position, and sets the total value of the anomaly scores for all byte positions as the first anomaly score.
4. The detection device described in claim 2, characterized in that the second calculation unit calculates the edit distance between the packet data to be evaluated and the similar normal packet data, and sets the smallest edit distance among the calculated edit distances as the second anomaly score.
5. The detection device according to claim 1, wherein the extraction unit uses Bidirectional Encoder Representations from Transformers (BERT) as the natural language processing model.
6. The detection device according to claim 1, further comprising a cluster setting unit which selects one of a plurality of clusters into which the plurality of normal packet data are clustered based on the features of the plurality of normal packet data and the features of the packet data to be evaluated, and wherein the extraction unit extracts a predetermined number of similar normal packet data having a relatively high similarity to the packet data to be evaluated from the plurality of normal packet data included in the selected cluster based on the natural language processing model.
7. A detection method executed by a detection device, comprising: a step of extracting a predetermined number of similar normal packet data having a relatively high similarity to packet data to be evaluated from among a plurality of normal packet data based on a natural language processing model; and a step of detecting whether the packet data to be evaluated is abnormal based on an anomaly score calculated by comparing the predetermined number of similar normal packet data extracted in the extraction step with the packet data to be evaluated.
8. A detection program for causing a computer to execute the steps of: extracting a predetermined number of similar normal packet data from among a plurality of normal packet data, the similarity of which is relatively high with packet data to be evaluated, based on a natural language processing model; and detecting whether the packet data to be evaluated is abnormal or not, based on an anomaly score calculated by comparing the predetermined number of similar normal packet data extracted in the extraction step with the packet data to be evaluated.
Citation Information
Patent Citations
Detection device, detection method, and detection program
WO2022259317A1
Estimation device, estimation method, and estimation program
WO2023112333A1
