Device and method for can datasets

The method provides comprehensive criteria for evaluating CAN datasets, addressing the inconsistency in IDS performance by ensuring dataset quality and diversity, thereby enhancing the reliability of machine learning-based IDS for automotive networks.

KR103046888B1Active Publication Date: 2026-11-11KOREA UNIV RES & BUSINESS FOUND
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
KR1020240187660
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2023-12-19
Filing Date
2024-12-16
Publication Date
2026-11-11
Estimated Expiration
2044-12-16

AI Technical Summary

Technical Problem

Existing machine learning-based Intrusion Detection Systems (IDS) for automotive Controller Area Networks (CAN) rely heavily on the quality of the datasets used for training and evaluation, which lack standardized evaluation criteria, leading to inconsistent performance.

Method used

A method for evaluating CAN datasets using qualitative criteria such as dataset description, collection environment, effective attack dataset, dataset labeling, and accuracy & consistency, and quantitative criteria like attack coverage, dataset balance, and similarity measurements.

Benefits of technology

Enables the creation of more reliable and trustworthy CAN datasets, allowing researchers to select optimal datasets for IDS development and improve detection performance by ensuring dataset quality and diversity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112024139545186-PAT00014_ABST
    Figure 112024139545186-PAT00014_ABST
Patent Text Reader

Abstract

A device and method for evaluating a CAN (Controller Area Network) dataset are disclosed. The CAN dataset evaluation method is performed by a computing device and includes the steps of: acquiring a target dataset; evaluating the target dataset using a first evaluation index representing the ratio of attack types represented by data included in the target dataset among a predetermined number of attack types targeting a CAN bus; evaluating the target dataset using a second evaluation index representing the balance of CAN messages for normal and attack types; evaluating similarity between CAN ID fields for normal data and data representing attacks included in the target dataset; and evaluating similarity between CAN data fields for normal data and data representing attacks included in the target dataset.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to evaluation criteria and evaluation methods for evaluating the quality of a dataset for a CAN (Controller Area Network) dataset, and more specifically, to qualitative and quantitative evaluation criteria for evaluating a publicly available dataset for the learning and evaluation of an automotive IDS (Intrusion Detection System), and to an apparatus and method for evaluating quality using these criteria. Background Technology

[0002] The CAN network was developed by Bosch in the 1980s as an in-vehicle network and is currently the most widely used in-vehicle network protocol due to its stability and efficiency. Multiple Electronic Control Units (ECUs) control the vehicle by exchanging messages with each other through the CAN network. The ECUs are connected to the CAN bus via wires and communicate using a broadcasting method.

[0003] Because the CAN protocol was designed without security considerations during its initial development, attackers can easily access the CAN bus and control the vehicle by transmitting malicious CAN messages. Various studies have been conducted to address these security vulnerabilities. Automotive Intrusion Detection Systems (IDS) are receiving significant attention because they can be applied directly to vehicles without altering the internal structure or modifying ECU hardware. In particular, machine learning (ML)-based IDS capable of detecting unknown attacks have been extensively researched. Machine learning-based IDS learns the characteristics of CAN traffic under normal or attack conditions and, based on this, can detect malicious CAN messages injected into the CAN bus.

[0004] CAN datasets are required for the development, training, and evaluation of machine learning-based IDS, and various research communities are releasing CAN datasets containing normal and attack data to advance and evaluate IDS technology. In fact, many machine learning-based IDS studies utilize these released datasets to train IDS and evaluate detection performance.

[0005] Meanwhile, the detection performance of machine learning-based IDS is highly dependent on the datasets used for training and evaluation. Therefore, this study aims to propose criteria and methods to evaluate the quality of the datasets based solely on the datasets themselves. Prior art literature

[0006] Republic of Korea Published Patent No. 2011-0064675 (Published June 15, 2011) The problem to be solved

[0007] The technical problem that the present invention aims to solve is to provide a CAN dataset evaluation device and method. means of solving the problem

[0008] A CAN (Controller Area Network) dataset evaluation method according to one embodiment of the present invention is a CAN dataset evaluation method performed by a computing device, comprising: a step of acquiring a target dataset; a first evaluation step of evaluating the target dataset using a first evaluation index representing the ratio of attack types represented by data included in the target dataset among a predetermined number of attack types targeting a CAN bus; a second evaluation step of evaluating the target dataset using a second evaluation index representing the balance of CAN messages for normal and attack types; a third evaluation step of evaluating similarity between CAN ID fields for normal data and data representing attacks included in the target dataset; and a fourth evaluation step of evaluating similarity between CAN data fields for normal data and data representing attacks included in the target dataset. Effects of the invention

[0009] Through the present invention, by presenting qualitative and quantitative criteria for evaluating the quality of the CAN dataset itself, researchers who intend to create CAN datasets can build a more trusted open dataset environment by creating CAN datasets while considering these criteria.

[0010] IDS researchers can select the optimal dataset for the development, training, and evaluation of IDS from among various datasets based on qualitative and quantitative criteria, and can further understand and analyze the evaluation results of IDS. Brief explanation of the drawing

[0011] Detailed descriptions of each drawing are provided to help to more fully understand the drawings cited in the detailed description of the present invention. Figure 1 illustrates the evaluation criteria for dataset documentation. FIG. 2 is a flowchart illustrating an evaluation method according to an embodiment of the present invention. Figure 3 illustrates two access points of the vehicle CAN. Specific details for implementing the invention

[0012] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.

[0013] Since embodiments according to the concept of the present invention may be subject to various modifications and may take various forms, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all modifications, equivalents, or substitutions that fall within the spirit and scope of the present invention.

[0014] Terms such as "first" or "second" may be used to describe various components, but said components should not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present invention, the first component may be named the second component and similarly the second component may be named the first component.

[0015] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions describing the relationships between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0016] The terms used herein are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described herein, and should not be understood as precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0017] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0019] First, qualitative and quantitative evaluation criteria for assessing the quality of CAN datasets are presented. Using these evaluation criteria, the strengths and weaknesses of each CAN dataset can be identified.

[0020] Qualitative evaluation criteria

[0021] 1. Dataset Description

[0022] Documents defining a dataset serve as important criteria for evaluating the performance and usability of an IDS using that dataset. Insufficient information regarding network configuration, data collection procedures, and attack scenarios during dataset collection can impede the usefulness of the dataset. Furthermore, datasets collected from actual vehicles are affected by various environments. Therefore, including vehicle environment information such as driving conditions (driving, waiting), status information (speed, gear position), and dataset characteristics (number of IDs, signal values) in the dataset description will have a significant impact on the development of an IDS capable of making flexible decisions based on the driving environment. Accordingly, the present invention proposes four major criteria for evaluating a dataset description as shown in FIG. 1.

[0023] 2. Dataset Collection Environment

[0024] Dataset collection for automotive IDS is classified into two types: dataset synthesis through simulation and collection from actual vehicles. While datasets synthesized in a simulation environment offer the advantage of being easily controlled and customized, they carry a high probability of generating unrealistic data. Therefore, to create an ideal automotive dataset that includes both normal and abnormal conditions, it is crucial to consider network characteristics of actual vehicles, such as arbitration processes, transmission delays, and network latency. For this reason, datasets obtained from actual vehicles provide various meaningful information that can be utilized for classifying normal and abnormal data. Consequently, actual vehicle datasets hold significant value for research and analysis.

[0025] 3. Effective Attack Dataset

[0026] There are two methods to create datasets that accurately represent attacks on vehicles. First, real-time data is collected by executing various attack scripts on actual vehicles. This approach provides realistic attack scenarios and collects actual results regarding the vehicle's internal network. Second, various attack datasets are generated by synthesizing abnormal samples based on previously collected normal datasets. This method allows for the easy generation of diverse attack scenarios. However, manually modifying CAN message timestamps for synthetic attacks can result in unrealistic datasets because factors such as transmission cycles and priorities are not taken into account. Furthermore, the quality of the dataset may degrade as it fails to reflect the complex behaviors of the CAN protocol, such as message timing, content, and event variations. Since it is difficult to physically verify the results of synthetic attacks, it is crucial to consider the meaning and accuracy of events occurring within the vehicle's internal network during IDS training and evaluation. Performing attacks on actual vehicles yields high-quality datasets that reliably reflect the actual results of the attack and changes within the vehicle's internal network.

[0027] 4. Dataset Labeling

[0028] Dataset labels serve as tags that provide meaningful and useful information about the data, defining its context. Samples in a dataset possess one or more labels indicating a normal or aggressive state, and these labels are essential learning elements for machine learning-based IDS. The accuracy and semantics of labels significantly impact IDS performance, and analysis results derived from datasets with incorrect labels lack validity and reliability. Therefore, it is crucial for high-quality datasets to provide accurate labels.

[0029] 5. Accuracy and Consistency

[0030] Datasets must be provided in standardized formats, such as text or CSV, to ensure compatibility with machine learning frameworks, as these formats facilitate tasks such as feature selection and normalization. Furthermore, all messages within the dataset must adhere to the correct format and be error-free. Format inconsistencies or corrupted messages require researchers to perform additional analysis or preprocessing. Therefore, providing datasets in a consistent format and ensuring the absence of corrupted data enables the smooth progress of the research process and promotes reliable analysis.

[0031] 6. Data Diversity

[0032] Dataset diversity indicates whether the data was collected from diverse environments at the time of generation. These environments include various vehicle types, driving conditions, and the participation of multiple drivers. If an IDS is trained solely on a dataset derived from limited environments, it may not operate effectively in real-world situations. Therefore, to ensure reliability and universality, high-quality datasets must include data obtained from various actual driving conditions, diverse vehicle types, multiple drivers, and various attack scenarios.

[0033] Quantitative evaluation criteria

[0034] 1. Attack coverage

[0035] The present invention proposes an attack coverage, which is an indicator of the attack types included in a dataset. This represents the number of attack types specifically targeting the CAN bus and is given by Equation 1. In Equation 1, A is the set of attack types included in the dataset. To this end, the dataset may include a label indicating whether a CAN message belongs to a normal type or to a specific attack type. N corresponds to a set of eight possible attack types, such as Denial of Service (DoS), Fuzzing, Replay, Spoofing, Diagnostic, Bus-off, Suspension, and Masquerade attacks. Therefore, attack coverage can be understood as the ratio of attack types included in the dataset to a predetermined number of attack types (e.g., eight attack types). However, the scope of the present invention is not limited to a predetermined number of attack types, and the attack types may vary depending on the embodiment. High-quality datasets should include various types of CAN attacks.

[0036] [Mathematical Formula 1]

[0037]

[0038] 2. Dataset Balance

[0039] The present invention proposes an indicator representing the balance between the number of CAN messages for normal and attack types. In the field of machine learning, it is generally desirable to have nearly equal class sizes. However, existing vehicle datasets consist largely of normal messages compared to attack messages. Such unbalanced datasets cause problems in IDS development. Therefore, to evaluate this dataset balance issue, Shannon entropy, as shown in Equation 2, is used as an indicator to evaluate the balance between normal and attack messages included in the dataset.

[0040] In Equation 2, n is the total number of messages in the dataset, and k represents the number of other types included therein (which may mean the number of normal and attack types). Each type is labeled with identifier i, and C i represents the number of messages belonging to the corresponding type (or the i-th type). If the dataset consists only of normal messages, the dataset balance becomes 0, indicating imbalance. On the other hand, if the balance of normal and attack messages is equal, the dataset balance will be log k, considering that the size of the dataset for each type is n / k.

[0041] [Mathematical Formula 2]

[0042]

[0043] 3. Attack Complexity

[0044] Attack complexity is defined as a metric that focuses on the ability of machine learning-based IDS to detect attacks. The level of similarity between an injected attack and a normal behavioral pattern directly affects the detection performance of the IDS. Similarity measurements are used to assess this attack complexity, comparing the similarity between normal and attack datasets. Similarity is calculated as the ratio of the intersection of the normal and attack sets to their union. If the two sets are identical, the similarity is 1; a value of 0 indicates no common elements between the two sets and signifies that the attack deviates from the normal pattern.

[0045] The present invention presents a method for measuring similarity using the ID field and Data field of a CAN data frame. Through this, the similarity between normal and attack datasets can be evaluated, and if the similarity value obtained from the attack dataset is close to the value derived from the normal dataset, it indicates that the attack successfully mimics the normal operation pattern.

[0046] 3.1 Similarity between CAN ID fields

[0047] In the present invention, Jaccard similarity calculation is performed using a sliding window method as shown in Equation 3. The sliding window method is a technique that performs specific operations while continuously moving data using a window of a fixed size; by utilizing this method in similarity calculation, the patterns or characteristics of each data can be measured more accurately. In Equation 3, N represents the set of CAN IDs of the normal dataset, and A i Windows W of a dataset containing specific attack types iThis represents the set of CAN IDs shown in. Through this analysis method, the similarity between two sets can be measured more accurately, making it possible to evaluate similarity within the attack dataset. According to Equation 3, measurement results for each window can be derived. In this way, by analyzing the measurement results for each window, i.e., the similarity distribution, the attack complexity of the dataset can be determined.

[0048] [Mathematical Formula 3]

[0049]

[0050] 3.2 Similarity between CAN data fields

[0051] The concept of similarity between CAN data fields is used to evaluate the similarity between the data frame of an injected attack message and the data frame of a normal message, and the formula for calculating similarity is Equation 4.

[0052] [Mathematical Formula 4]

[0053]

[0054] In mathematical formula 4, represents the set of data associated with CAN ID i in the normal dataset, and represents a set of data associated with CAN ID i in a dataset for a specific attack type. In an attack scenario, if an attacker injects malicious messages different from normal data, the attack can be easily detected because the IDS monitors signal values ​​within data fields that have not been previously observed for that CAN ID. Therefore, a similarity value close to 0 indicates that the attack is likely to be detected using simple methods, which suggests that the attack was performed without considering normal data patterns.

[0055] According to mathematical formula 4, measurement results for each type of attack can be derived. In this way, by analyzing the measurement results for each type of attack, that is, the similarity distribution, the attack complexity of the dataset can be determined.

[0056] FIG. 2 is a flowchart illustrating an evaluation method according to an embodiment of the present invention.

[0057] Referring to FIG. 2, the evaluation method may be performed by a computing device comprising at least a processor and / or memory. Accordingly, at least some of the steps constituting the evaluation method may be understood as the operation of a processor included in the computing device, and the computing device may be referred to as the evaluation device. The computing device may include a PC (Personal Computer), a server, a laptop computer, a tablet PC, etc. In the following description of the evaluation method, specific details regarding content that overlaps with the preceding description will be omitted.

[0058] First, a target dataset is obtained (S110). The target dataset may be a CAN dataset. The target dataset may be received from a server that provides the target dataset through a predetermined wired or wireless communication network, or received from a storage device such as a USB memory device through a predetermined input / output interface, or may be stored in advance in a storage device included in the computing device.

[0059] A first evaluation is performed using a first evaluation indicator (S120). The first evaluation indicator may represent an attack range included in the target dataset. At this time, a first evaluation result for the target dataset can be derived using Equation 1.

[0060] A second evaluation is performed using a second evaluation indicator (S130). The second evaluation indicator is the degree of balance of the target dataset, and balance may mean class balance. At this time, a second evaluation result for the target dataset can be derived using Equation 2.

[0061] A third evaluation is performed using a third evaluation indicator (S140). The third evaluation indicator is an indicator for measuring attack complexity and may include similarity between CAN ID fields and similarity between CAN data fields. Similarity between CAN ID fields is evaluated using Equation 3, and similarity between CAN data fields can be evaluated using Equation 4. According to the embodiment, in the case of similarity between CAN ID fields, the average of similarity for each window may be calculated to derive the final CAN ID field similarity evaluation result. Similarly, in the case of similarity between CAN data fields, the average of similarity for each attack type may be calculated to derive the final CAN data field similarity evaluation result.

[0062] An evaluation result is output (S150). The evaluation result may be generated as a file having a predetermined format or displayed through a display device included in a computing device. According to an embodiment, a weighted sum obtained by assigning a predetermined weight to each of the first to third evaluation results and summing them may be derived as the final evaluation result, and the derived final evaluation result may be output. According to this, the sum of the value to which the first weight is assigned to the first evaluation result, the value to which the second weight is assigned to the second evaluation result, the value to which the third weight is assigned to the final CAN ID field similarity evaluation result, and the value to which the fourth weight is assigned to the final CAN data field similarity evaluation result may be derived as the final evaluation result and output.

[0063] The following describes attacks targeting CAN bus communication.

[0064] Generally, CAN attacks can be classified according to the network layer where the attack is executed. Examples include the application layer or the data link layer. At the application layer, adversarial access to the in-vehicle network allows for the transmission and reception of messages without any restrictions on ID or payload. Figure 3 illustrates two access points for a vehicle CAN: OBD-II-based physical access and telematics device-based remote access. In the absence of security features such as encryption or authentication, an attacker who gains access to the CAN bus can control safety-sensitive functions related to essential driving functions, such as braking, acceleration, and steering. Most current research focuses on detecting application layer attacks by utilizing machine learning (ML)-based Intrusion Detection Systems (IDS). Therefore, representative attack scenarios that can occur at the application layer are described.

[0065] 1. DoS attack (or Flooding Attack)

[0066] DoS attacks are the easiest way to carry out an attack on the CAN bus. CAN messages with a CAN ID of 0x000 and an arbitrary payload are injected at high frequency. Because 0x000 has the highest priority in arbitration decisions, other CAN messages with lower priority are delayed or not delivered. The result of such attacks is disruption of vehicle operations and can lead to system malfunctions or even temporary loss of control.

[0067] 2. Fuzzing Attack

[0068] In fuzzing attacks, CAN IDs and data payloads are randomly selected and injected at high frequency. Unlike DoS attacks, the injected messages may include CAN IDs that appear in normal traffic. Consequently, ECUs receiving these CAN messages perform functions based on the payload. By analyzing the vehicle's response, attackers can obtain additional information about the target. This type of attack can compromise vehicle safety and security and provide attackers with vulnerabilities and potential points of exploitation.

[0069] 3. Replay Attack

[0070] In the case of a replay attack, the attacker carefully observes a sequence of messages during a specific time interval and stores the observed sequence of messages. Subsequently, all or part of this legitimate message sequence is injected. Each payload contains a valid CAN control message. Therefore, it can cause potential damage or unexpected vehicle behavior. A successful replay attack can lead to unauthorized control of the vehicle system, manipulation of critical functions, or threaten the safety of vehicle occupants.

[0071] 4. Spoofing Attack (or Fabrication Attack)

[0072] If an attacker knows the CAN ID information used for a specific vehicle function, they can achieve the desired result by sending a forged message to the bus using that CAN ID. In a study by Miller et al., a spoofing attack was performed on a Ford Escape, resulting in a "Door Ajar" notification being displayed on the dashboard even when the doors were closed. However, since the legitimate ECU remains active and transmits the correct message, the attacker must force the receiving ECU to respond to the conflicting message. To achieve this, the frequency of spoofed messages is much higher than that of the correct messages (typically up to 100 times). A successful spoofing attack can cause incorrect information to be displayed or the vehicle system to operate, potentially leading to dangerous situations or the manipulation of critical functions.

[0073] 5. Diagnostic Attack

[0074] Unified Diagnostic Services (UDS), defined in ISO-14229, is a diagnostic communication protocol used over the CAN protocol for vehicle diagnostics. Diagnostic attacks aim to obtain information about ECUs or vehicles connected to the CAN bus using UDS. The range of identifiers used in diagnostic messages is known to be between 0x700 and 0x7FF. Miller et al. transmitted certain diagnostic messages to Ford and Toyota to obtain information such as stopping the engine or controlling the fuel gauge. Lee et al. forced specific ECUs to switch to diagnostic mode by transmitting diagnostic messages. The consequences of a successful diagnostic attack range from unauthorized access to sensitive vehicle data to manipulation of critical functions, and can threaten the safety and security of the vehicle.

[0075] 6. Bus-Off Attack

[0076] A bus-off attack aims to switch a specific ECU to bus-off mode by exploiting its error-handling mechanism. To cause the target ECU's TEC (Transmission Error Counter) to exceed 255, the attacker generates bit errors while the target ECU is transmitting messages. Although the ECU in bus-off mode recovers to error-active mode and resumes message transmission after a certain period, bus-off mode can cause unexpected vehicle behavior. A successful bus-off attack can temporarily suspend communication with the target ECU, disrupt critical functions, and compromise the reliability of the entire system.

[0077] 7. Suspension Attack

[0078] A stop attack attempts to interrupt or halt the transmission of messages by a legitimate ECU. The effect of a stop attack is similar to that of a DoS attack. However, while a DoS attack significantly increases message traffic, a stop attack does not increase the bus load. Instead, message traffic decreases to the same level as the stopped transmission. Lee et al. demonstrated a stop attack utilizing diagnostic services and bus-off attacks. A successful stop attack can disrupt the normal operation of the target ECU, affecting critical vehicle functions or compromising system reliability.

[0079] 8. Masquerade Attack

[0080] Identity spoofing attacks can be viewed as a combination of stop attacks and spoofing attacks. An attacker interrupts message transmission from a specific ECU and maintains the existing CAN bus load by sending manipulated messages to the CAN bus at the same frequency. Lee et al. have released the first dataset of identity spoofing attacks performed on actual vehicles by exploiting diagnostic services and bus-off attacks. A successful identity spoofing attack can lead to unauthorized control of vehicle systems, manipulation of critical functions, and threats to the safety and security of vehicle occupants.

[0081] The device described above may be implemented as a set of hardware components, software components, and / or a combination of hardware components and software components. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an Arithmetic Logic Unit (ALU), a Digital Signal Processor (DSP), a microcomputer, a Field Programmable Array (FPA), a Programmable Logic Unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an Operation System (OS) and one or more software applications executed on said OS. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0082] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0083] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROMs; RAMs; flash memory; and hardware devices specifically configured to store and execute program instructions. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0084] The present invention has been described with reference to the embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the appended claims.

Claims

Claim 1 A method for evaluating the quality of a CAN (Controller Area Network) dataset performed by a computing device, comprising: a step of acquiring a target dataset; a first evaluation step of evaluating the target dataset using a first evaluation indicator representing the ratio of attack types represented by data included in the target dataset among a predetermined number of attack types targeting a CAN bus; a second evaluation step of evaluating the target dataset using a second evaluation indicator representing the balance of CAN messages for normal and attack types; a third evaluation step of evaluating the similarity between CAN ID fields for normal data and data representing attacks included in the target dataset; and a fourth evaluation step of evaluating the similarity between CAN data fields for normal data and data representing attacks included in the target dataset. Claim 2 A CAN dataset quality evaluation method according to claim 1, wherein the predetermined number of attack types are Denial of Service (DoS), Fuzzing, Replay, Spoofing, Diagnostic, Bus-off, Suspension, and Masquerade attacks. Claim 3 In paragraph 2, the first evaluation indicator (attack coverage) is defined by the first mathematical formula, and the first mathematical formula A CAN dataset quality evaluation method, wherein A is a set of attack types included in the target dataset, and N is a set of a predetermined number of attack types. Claim 4 In paragraph 3, the second evaluation indicator (dataset balance) is defined by the second mathematical formula, and the second mathematical formula and, above, n is the number of CAN messages included in the target dataset, above, k is the number of data types including normal types and attack types, and above, C i A CAN dataset quality evaluation method representing the number of CAN messages included in the i-th data type. Claim 5 In paragraph 4, the third evaluation step comprises: a step of measuring similarity between CAN ID fields per window having a predetermined size using a third mathematical formula; and a step of deriving similarity between final CAN ID fields by averaging the similarity between CAN ID fields per window, wherein the third mathematical formula And, the above J a is the similarity between CAN ID fields in the a-th window, the above L is the set of CAN IDs included in the normal data, and the above B a A CAN dataset quality evaluation method, which is a set of CAN IDs for data representing an attack. Claim 6 In claim 5, the fourth evaluation step comprises: a step of measuring similarity between CAN data fields by CAN ID using a fourth mathematical formula; and a step of deriving similarity between final CAN data fields by averaging the similarity between CAN data fields by CAN ID, wherein the fourth mathematical formula and, the above represents a set of data related to CAN ID i in the normal dataset included in the above target dataset, and A CAN dataset quality evaluation method representing a set of data associated with CAN ID i in a dataset for attack types included in the above target dataset. Claim 7 A CAN dataset quality evaluation method according to claim 6, further comprising the step of deriving a final evaluation result for the target dataset by assigning a predetermined weight to each of the results of the first to fourth evaluation steps and summing them.

Citation Information

Patent Citations

  • CAN-FD anomaly detection method based on time sequence content attention and long and short term memory network

    CN117176421A