An abnormality detection method based on CAN bus

Abstract

The present invention relates to an anomaly detection method based on CAN bus, belonging to the field of automobile CAN bus data information security. It includes an offline stage, extracting the BID and recording the order relationship between each message and calculating the relevant threshold; an online detection stage, performing real-time anomaly detection; an optimization stage, updating the threshold to improve detection efficiency. The beneficial effects are: different processing methods for periodic messages and non-periodic messages are proposed and applied in the improved LevenshteinDistance, thereby improving the accuracy of anomaly detection. The present invention adopts a genetic algorithm to further optimize the threshold, thereby improving the detection precision, recall and accuracy; it has high accuracy and extremely high precision for different attacks, and can be applied to anomaly detection in a resource-constrained network such as an in-vehicle network; compared with the method based on machine learning, it has lower resource occupancy and less detection time, and is more suitable for deployment in an in-vehicle network.

Description

Technical Field

[0001] The present invention belongs to the field of automobile CAN bus data information security, and in particular relates to a CAN bus-based anomaly detection method, which is applied to anomaly detection of automobile CAN bus information transmission, can detect Dos attacks, fuzzy attacks and injection attacks, and ensure the security of CAN bus data information. Background Art

[0002] Modern cars have become mobile information platforms comprised of numerous interconnected embedded subsystems. Within each subsystem, an Electronic Control Unit (ECU) controls corresponding mechanical components and interconnects via in-vehicle networks such as the Controller Area Network (CAN), LIN, and FlexRay. In recent years, more interfaces and infotainment units have been deployed within these networks, creating new avenues for attack.

[0003] As the CAN network connects the core ECUs of the in-vehicle information system, its security is crucial. However, information security mechanisms were not considered from the outset of its design, making it extremely vulnerable to DoS, sniffing, modification, and injection attacks. At the physical layer, the CAN bus uses twisted-pair or coaxial cables, making it easy to access illegally. At the data link layer, messages are transmitted in plain text, lacking encryption, making them vulnerable to eavesdropping and tampering. At the network layer, messages are simply identified and filtered by ID, meaning any ECU can access core ECUs in the CAN bus network, such as those for the engine and steering system. As can be seen from the above, the security of in-vehicle ECUs directly determines the security of the entire in-vehicle information system. Without the security of in-vehicle ECUs, the security of the vehicle's owner's information system cannot be guaranteed. These vulnerabilities make the CAN bus a vulnerable target for hackers.

[0004] Current approaches to in-vehicle network information security include data encryption, message authentication, and anomaly detection. Data encryption is widely used in various communication channels, but due to limited computing resources in in-vehicle networks and the extra time overhead associated with message encryption, it has remained a challenge. Message authentication also faces challenges with bandwidth resources and message response time. Intrusion detection technology is widely used in current vehicles due to its low bandwidth consumption and ease of deployment.

[0005] Currently, existing CAN intrusion detection methods include feature-based anomaly detection, information theory-based and statistical analysis-based detection methods, and machine learning methods. Feature-based anomaly detection primarily analyzes network characteristics such as clock offset and frequency, offering high accuracy for specific attacks. However, physical characteristics can change dynamically over time. Machine learning-based detection methods offer high detection effectiveness for unknown attacks, but still face challenges with computational complexity and real-time performance. Methods based on information theory and statistical analysis primarily consider the accuracy of data within a time window. Comparing the information entropy or similarity of data within a time window captures the overall information within that time window. However, CAN bus messages can be either event-triggered or time-triggered. A simple time-window-based anomaly detection method is only effective for periodic message transmissions, but is less effective for non-periodic messages. Summary of the Invention

[0006] The present invention provides an anomaly detection method based on a CAN bus, which adopts a statistics-based anomaly detection method to better process periodic and non-periodic messages and perform anomaly detection on each message and messages within the entire event window.

[0007] The technical solution adopted by the present invention comprises the following steps:

[0008] (1) In the offline phase, the BID is extracted and the order relationship between each message is recorded and the relevant threshold is calculated;

[0009] (2) In the online detection phase, real-time anomaly detection is performed;

[0010] (3) In the optimization stage, the threshold is updated to improve the detection efficiency.

[0011] The offline phase of step (1) of the present invention includes:

[0012] (1) Take the recurring periodic ID as the ID benchmark BID, record the before and after information of the ID of the vehicle's normal operation message, and construct the transfer matrix tran ij , where i, j represent ID, tran ij The value represents the ID i Then comes ID j The probability of forward and backward transitions between different messages is constructed by taking each ID as a node V of the graph. A graph G(V,E) is constructed. When the value of the transition matrix is greater than the threshold, an edge E(G)=E(G)∪{(i,j)} is constructed from i to j. The nodes of the subgraphs with the longest and second longest lengths and the same period are taken as BIDs.

[0013] (2) To process non-periodic messages, messages that do not appear, and periodic messages that are not in the BID, and to detect individual message anomalies, the N-gram algorithm is used for estimation, 2-gram is used, and the Modified Kneser-Ney smoothing algorithm is used for smoothing. A certain probability is given to the non-appearing events from the transition matrix, and the word connection ability of each word is calculated to obtain each ID (ID i )'s backoff ability i 、Each ID i ) 1-gram probability P(i), any two IDs (ID i to ID j ) The probability P(ij) of the 2-gram is converted into log form to facilitate the subsequent multiplication operation. The above value will be used as a measure;

[0014] (3) In order to measure the similarity of messages in two time windows and detect the anomaly of the whole message, the N-gram algorithm is combined with LevenshteinDistance. When the similarity is higher than the threshold, it is defined as an anomaly. When the ID is BID, the LevenshteinDistance is calculated according to the normal method. a,b (i, j), where a and b represent the messages in the two time windows respectively, i and j represent the position of the currently calculated ID in a and b respectively, lev a,b (i, j) represents the Levenshtein distance between a from the beginning to the i-th position and b from the beginning to the j-th position, which is calculated as follows:

[0015]

[0016] For non-BID ID frames, the probability of occurrence is calculated by n-gram. When the probability of occurrence is less than the threshold ngram When , it means that the possibility of the ID appearing at the end of the message is small, so it is retained in the ID time window and the lev is calculated using formula (1) a,b (i, j), and when it is greater than the threshold, it means that the ID is likely to appear, and the ID is directly regarded as the correct ID. When calculating the distance, its LevenshteinDistance is the minimum LevenshteinDistance of the previous ID, that is, the message is equivalent to not appearing in the sequence, that is:

[0017]

[0018] Finally, the LevenshteinDistance in the two time windows is obtained as a measure of the similarity of the messages in the two time windows. That is, the similarity of the messages in the two time windows a and b is expressed as formula (3), where length(a) and length(b) represent the lengths of the time windows a and b respectively.

[0019] similarity(a,b)=lev a,b (length(a)+1,length(b)+1) #(3)

[0020] (4) In the normal driving data of the vehicle, the threshold method is used to calculate the threshold of the correct value, that is, the minimum probability of each word conversion calculated in the normal data is used as the threshold of the n-gram algorithm ngram Calculate the overall similarity of the partial driving data using step (3), and use the maximum similarity as the similarity threshold similarity .

[0021] The online detection stage of step (2) of the present invention includes:

[0022] (1) Read CAN messages from real-time vehicle operation data in real time, extract the ID of the CAN message, and reconstruct the ID through the hash table, so that the ID is distributed in a continuous space, saving the space for storing the transfer matrix and the speed of calculating the n-gram algorithm;

[0023] (2) A normal message is used as the initialization message and the similarity is calculated with the initialization message to make a judgment;

[0024] (3) For different IDs, the minimum edit distance between BID and non-BID is calculated using a method similar to the offline stage. When the data length of the time window exceeds the time window data length threshold k thro When it is, report an error directly;

[0025] (4) At this stage, the minimum value of the similarity with the data of the previous time window and the first two time windows will be used as the final minimum value. The first two time windows can cover more than 80% of the messages and can also cover the misaligned time windows:

[0026] (5) When the similarity is less than the similarity threshold similarity When , an update operation is performed, and the data in the current time window is stored as the data of the previous time window, and the data of the current time window is stored as the data of the previous two time windows;

[0027] (6) Finally, after the above detection process, four types of results are obtained: TP means the detection result is a normal message and the actual message is also a normal message; FP means the detection result is a normal message but the actual message is an abnormal message; TN means the detection result is an abnormal message and the actual message is also an abnormal message; FN means the detection result is an abnormal message but the actual message is a normal message; the precision, recall and accuracy are shown in Equations 4, 5 and 6:

[0028]

[0029]

[0030]

[0031] The step (three) optimization stage of the present invention includes:

[0032] (1) Two thresholds are used: and threshold similarity , found during the calculation process that threshold similarity The normal value range is [0,3] and the value must be an integer. The granularity is large, so the similarity is not optimized and only the threshold is used. The value range is large and is a real number, so it is optimized by using the simulated annealing algorithm, a heuristic algorithm in the traditional optimization algorithm;

[0033] (2) Genetic algorithm is used to optimize the abnormal data in the online detection stage, and the optimized threshold is updated as the threshold for the final abnormality detection.

[0034] The objective function optimized in step (3) (1) of the present invention is:

[0035]

[0036] C1, C2, and C3 are constants 1, 0.5, and 1 respectively.

[0037] The beneficial effects of the present invention are:

[0038] The messages that exist periodically in each time window are extracted as BIDs, which serve as the skeleton of the messages in each time window, and the messages outside the BIDs are used as fillers to fill in between the BIDs. The anomaly detection of a single message is performed and recorded according to the previous and next relationship of the BIDs. Finally, the similarity of the messages in the two event windows is compared to perform overall anomaly detection. The N-gram algorithm is used as the metric for individual anomaly detection and the improved LevenshteinDistance is used as the standard metric for overall anomaly detection to perform anomaly detection on CAN bus communication. Compared with other anomaly detection methods, different processing methods for periodic messages and non-periodic messages are innovatively proposed and applied in the improved LevenshteinDistance, thereby improving the accuracy of anomaly detection. The present invention uses a genetic algorithm to further optimize the threshold, thereby improving the precision, recall and accuracy of the detection.

[0039] The present invention has high accuracy and extremely high precision for different attacks and can be applied to anomaly detection in resource-constrained vehicular networks. Compared with machine learning-based methods, the method proposed in the present invention has lower resource usage and shorter detection time, making it more suitable for deployment in vehicular networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is the process of determining the BID of the present invention;

[0041] Figure 2 It is the process of N-gram of the present invention;

[0042] Figure 3 It is a flow chart of online detection of the present invention;

[0043] Figure 4 The figure shows a specific example of the improved shortest edit distance of data in two time windows. The first column on the left is the sequence generated by the reconstruction of the message in the previous time window, and the first row on the top is the sequence of the message in the current time window after reconstruction. In this example, 5, 4, 1, 0, and 9 are BIDs; (5, 3), (4, 20), and (1, 8) exceed the threshold through the n-gram algorithm and can be eliminated; as shown by the downward and left arrows in the figure, (9, 11) is below the threshold through the n-gram algorithm and cannot be eliminated. The final similarity is 1 in the lower right corner.

[0044] Figure 5 It is the test result graph. DETAILED DESCRIPTION

[0045] (1) Offline stage:

[0046] (1) The data used are from the public data set of HCRL laboratory, including normal data, Dos data, Fuzzy data, Gear_injections data and RPM_injection data. In order to extract the recurring periodic ID as the benchmark BID of the ID, the before and after information of the ID of the normal operation message of the vehicle is recorded, that is, the normal data is statistically analyzed, including the following contents: the type, frequency, period and post-probability of the ID, and the transfer matrix tran is constructed. ij , where i, j represent ID, tran ij The value represents the ID i Then comes ID j The probability of forward and backward transitions between different messages is constructed by taking each ID as a node V of the graph and constructing a graph G(V,E). When the value of the transition matrix is greater than the threshold threshold = 0.6, an edge E(G) = E(G)∪{(i,j)} is constructed from i to j. The nodes of the subgraphs with the longest and second longest lengths and the same period are taken as BIDs. Figure 1 For the constructed graph, the BIDs are 130, 131, 140, 316, 18F, 260, 2A0, and 329;

[0047] (2) To process non-periodic messages, messages that do not appear, and periodic messages that are not in the BID, and to detect individual message anomalies, the N-gram algorithm is used for estimation, the 2-gram algorithm is used for normal data, and the Modified Kneser-Ney smoothing algorithm is used for smoothing. A certain probability is given to the non-appearing events from the transfer matrix, and the word connection ability of each word is calculated, that is, the ability of new words to appear later, and each ID (ID i )'s backoff ability i 、Each ID i ) 1-gram probability P(i), any two IDs (ID i to ID j ) The probability P(ij) of the 2-gram is converted into log form to facilitate the subsequent multiplication operation. The above value will be used as a measure;

[0048] (3) In order to measure the similarity of messages in two time windows and detect the anomaly of the whole message, the N-gram algorithm is combined with LevenshteinDistance. When the similarity is higher than the threshold, it is defined as an anomaly. When the ID is BID, the LevenshteinDistance is calculated according to the normal method. a,b(i, j), where a and b represent the messages in the two time windows respectively, i and j represent the position of the currently calculated ID in a and b respectively, lev a,b (i, j) represents the Levenshtein distance between a from the beginning to the i-th position and b from the beginning to the j-th position, which is calculated as follows:

[0049]

[0050] For non-BID ID frames, the probability of occurrence is calculated by n-gram. When the probability of occurrence is less than the threshold When , it means that the possibility of the ID appearing at the end of the message is small, so it is retained in the ID time window and the lev is calculated using formula (1) a,b (i, j), and when it is greater than the threshold, it means that the ID is likely to appear, and the ID is directly regarded as the correct ID. When calculating the distance, its LevenshteinDistance is the minimum LevenshteinDistance of the previous ID, that is, the message is equivalent to not appearing in the sequence, that is:

[0051]

[0052] Finally, the LevenshteinDistance in the two time windows is obtained. LevenshteinDistance is used as a measure of the similarity of the messages in the two time windows. That is, the similarity of the messages in the two time windows a and b is expressed as formula (3), where length(a) and length(b) represent the lengths of the time windows a and b respectively.

[0053] similarity(a,b)=lev a,b (length(a)+1,length(b)+1) #(3)

[0054] (4) Calculate two thresholds in the validation set of normal data, namely the threshold of similarity similafity and the threshold of the n-gram algorithm According to the 3δ law, the threshold is defined as 99.74% of the normal data are within the threshold range. First, calculate The minimum probability is calculated to be 1.93ⅹ10 -8 , and then the calculated Calculate the similarity of data in each time window and use its maximum value as the threshold similafity , obtained

[0055] (2) Online detection stage

[0056] (1) Anomaly detection is performed using Dos data, Fuzzy data, Gear_injections data, and RPM_injection data from the HCRL laboratory's public dataset. CAN messages are read from Dos data, Fuzzy data, Gear_injections data, and RPM_injection data in real time, and the IDs of the CAN messages are extracted. The IDs are reconstructed using a hash table, and are distributed in a continuous space, saving space for storing the transfer matrix and speeding up the calculation of the n-gram algorithm.

[0057] (2) Take a normal message as the initialization message and make a judgment by calculating the similarity with the initialization message;

[0058] (3) For different IDs, the minimum edit distance between BID and non-BID is calculated using a method similar to the offline stage. When the data length of the time window exceeds k thro When it is time to report an error, the improved shortest edit distance is as follows Figure 4 As shown;

[0059] (4) At this stage, the minimum value of the similarity between the data of the previous time window and the first two time windows is used as the final minimum value. The first two time windows can cover more than 80 messages and can also cover the misaligned time windows;

[0060] (5) When the similarity is less than the threshold similarity When , an update operation is performed, and the data in the current time window is stored as the data of the previous time window. At the same time, the data of the current time window is stored as the data of the previous two time windows. The overall process is as follows: Figure 3 ;

[0061] (6) Finally, after the above detection process, four types of results are obtained: TP means the detection result is a normal message and the actual message is also a normal message; FP means the detection result is a normal message but the actual message is an abnormal message; TN means the detection result is an abnormal message and the actual message is also an abnormal message; FN means the detection result is an abnormal message but the actual message is a normal message; the precision, recall and accuracy are shown as follows:

[0062]

[0063]

[0064]

[0065] Table 1 shows that this method has high accuracy, recall and precision for different types of attacks. Figure 5 The detection effect of this method on DoS attacks is visualized;

[0066] Table 1

[0067]

[0068] (3) Optimization stage

[0069] (1) In the online detection stage, the threshold method is used for judgment. The granularity is large, which will still lead to certain false positives and false negatives. The threshold obtained by the normal data set is further optimized to improve the accuracy of detection. Two thresholds are used: and threshold similarity , found during the calculation process that threshold similarity The normal value range is [0,3] and the value must be an integer. The granularity is large, so the similarity is not optimized and only the threshold is used. The value range is large and is a real number, so it is optimized; the simulated annealing algorithm, a heuristic algorithm in the traditional optimization algorithm, is used for optimization, and the optimization objective function used is:

[0070]

[0071] Among them, C1, C2, and C3 are 1, 0.5, and 1 respectively.

[0072] (2) Genetic algorithm is used to optimize abnormal data, and the optimized threshold is updated to detect abnormal data. The updated threshold is

Claims

1. A CAN bus-based anomaly detection method, characterized in that: The following steps are involved: (1) In the offline phase, extract the BID and record the order of each message and calculate the relevant threshold, including: (1) Take the recurring periodic ID as the base BID of the ID, record the information before and after the ID of the message of the normal operation of the vehicle, and construct the transfer matrix , where i and j represent IDs respectively, The value represents After that The probability of forward and backward transfer between different messages, each ID is used as a node in the graph , construct a graph , when the value of the transfer matrix is greater than the threshold, a path from point to edge , the nodes of the subgraphs with the longest and second longest lengths and the same cycle are taken as BIDs; (2) To process non-periodic messages, messages that do not appear, and periodic messages that are not in the BID, and to detect individual message anomalies, the N-gram algorithm is used for estimation, 2-gram is used, and the Modified Kneser-Ney smoothing algorithm is used for smoothing. A certain probability is given to the non-appearing events from the transfer matrix, and the word connection ability of each word is calculated to obtain each The ability to connect words , each The probability of 1-gram , the probability of any two ID 2-grams Convert it into log form to facilitate subsequent multiplication operations; (3) To measure the similarity of messages in two time windows and detect anomalies in the whole message, the N-gram algorithm is combined with LevenshteinDistance. When the similarity is higher than the threshold, it is defined as an anomaly. When the ID is BID, it is calculated according to the normal LevenshteinDistance. , where a and b represent the messages in two time windows respectively, i and j represent the position of the currently calculated ID in a and b respectively, It represents the Levenshtein distance between a from the beginning to the i-th position and b from the beginning to the j-th position. The calculation method is as follows: ； For non-BID ID frames, the probability of occurrence is calculated by n-gram. When the probability of occurrence is less than the threshold When , it means that the ID is less likely to appear at the end of the message, so it is retained in the ID time window and calculated using formula (1) When it is greater than the threshold, it means that the ID is likely to appear, and the ID is directly regarded as the correct ID. When calculating the distance, its LevenshteinDistance is the minimum LevenshteinDistance of the previous ID, that is, the message is equivalent to not appearing in the sequence, that is: ； Finally, the LevenshteinDistance in the two time windows is obtained as the measure of the similarity of the messages in the two time windows, that is, the similarity of the messages in the two time windows a and b is formula (3), where length(a) and length(b) represent the lengths of the time windows a and b respectively; ； (4) In the normal driving data of the vehicle, the threshold value of the correct value is calculated using the threshold method, that is, the minimum probability of each word conversion calculated in the normal data is used as the threshold of the n-gram algorithm , and use step (3) to calculate the overall similarity of the partial driving data, and use the maximum similarity as the similarity threshold ; (2) In the online detection phase, real-time anomaly detection is performed, including: (1) Read CAN messages from real-time vehicle operation data in real time, extract the ID of the CAN message, and reconstruct the ID through the hash table, so that the ID is distributed in a continuous space, saving the space for storing the transfer matrix and the speed of calculating the n-gram algorithm; (2) Take a normal message as the initialization message and make a judgment by calculating the similarity with the initialization message; (3) For different IDs, the minimum edit distance between BID and non-BID is calculated using a method similar to the offline stage. When the data length of the time window exceeds the time window data length threshold, When it is, report an error directly; (4) At this stage, the minimum value of the similarity between the data of the previous time window and the first two time windows will be used as the final minimum value. The first two time windows can cover more than 80% of the messages and can also cover the misplaced time windows: (5) When the similarity is less than the similarity threshold When , an update operation is performed, and the data in the current time window is stored as the data of the previous time window, and the data of the current time window is stored as the data of the previous two time windows; (6) Finally, after the above detection process, four types of results are obtained. TP means the detection result is a normal message and the actual message is also a normal message. FP means the detection result is a normal message but the actual message is an abnormal message. TN means the detection result is an abnormal message and the actual message is also an abnormal message. FN means the detection result is an abnormal message but the actual message is a normal message. The precision, recall and accuracy are shown in Equations 4, 5 and 6: ； ； ； (3) Optimization stage: updating the threshold to improve detection efficiency, including: (1) Two thresholds are used: and , found during the calculation process The normal value range is The value must be an integer, and the granularity is large, so the similarity is not optimized, and only the threshold is used. , The value range is large and is a real number, so it is optimized by using the simulated annealing algorithm, a heuristic algorithm in the traditional optimization algorithm; (2) Genetic algorithm is used to optimize the abnormal data in the online detection stage, and the optimized threshold is updated as the threshold for the final abnormality detection.

2. The CAN bus-based anomaly detection method according to claim 1, characterized in that: The objective function of the optimization is: ； They are constants 1, 0.5, and 1 respectively.

Images