Vehicle-mounted CAN intrusion detection model training data quality evaluation method and device

Abstract

The invention relates to the technical field of data processing, and discloses a vehicle-mounted CAN intrusion detection model training data quality evaluation method and device, and the method comprises the steps: obtaining a vehicle-mounted CAN network intrusion detection model training reference data set, and dividing the training reference data set into a variable CAN ID data set and a fixed value type CAN ID data set; extracting a reference reconstruction error of the variable CAN ID data set; inputting to-be-evaluated training data into the change rule feature learning model to obtain a to-be-evaluated reconstruction error, and calculating a change type overall quality evaluation index; constructing a reference library for the fixed value type CAN ID data set, calculating a statistical deviation of the fixed value type CAN ID data set relative to the reference library, and then calculating a fixed value type overall quality evaluation index; and carrying out product aggregation on the variation type total quality evaluation index and the fixed value type total quality evaluation index to obtain a comprehensive quality evaluation index, and analyzing data quality. According to the invention, the data quality evaluation accuracy and efficiency can be improved.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a method and apparatus for evaluating the quality of training data for an in-vehicle CAN intrusion detection model. Background Technology

[0002] With the rapid development of intelligent connected vehicles, the security of in-vehicle CAN (Controller Area Network) bus networks is receiving increasing attention. Training efficient in-vehicle network intrusion detection models typically requires massive amounts of benign in-vehicle CAN network communication data. However, due to the high cost and limited scenario coverage of data collection in real-world vehicle environments, generating benign in-vehicle CAN network data using generative models has become a mainstream trend in the industry. In this process, how to automatically screen and evaluate the generated in-vehicle CAN network data with high quality has become a crucial step in ensuring the training effectiveness of intrusion detection models.

[0003] Currently, the existing technologies in the industry for processing and quality evaluation of training data for vehicle CAN intrusion detection models mainly focus on two levels: basic data preprocessing and traditional data quality evaluation. In terms of data preprocessing, mainstream solutions typically use tools such as Python or MATLAB, which can only perform superficial operations such as data cleaning, CAN ID grouping, timestamp sorting, and basic field extraction. Regarding data quality evaluation, existing conventional methods largely rely on manual rule verification and simple basic statistical analysis.

[0004] However, when dealing with the complex and ever-changing real-world communication patterns of automotive CAN buses, the aforementioned existing technologies reveal numerous significant shortcomings in their overall implementation logic, resulting in extremely low feasibility for practical engineering applications. Automotive CAN network messages simultaneously contain both "fixed-value CAN IDs" carrying the basic communication protocol and "variable CAN IDs" carrying dynamic physical states. Existing technologies typically employ a single standard for unified evaluation, failing to differentiate between them in the evaluation logic. This leads to a severe imbalance between the refinement and rigor of the evaluation standards, resulting in low evaluation accuracy and quality differentiation. Furthermore, in existing technologies, data preprocessing, feature extraction and learning, and quality inference are all independent processes, lacking end-to-end design. Manual intervention is often required between these processes, which not only easily introduces human error but also results in long evaluation times per batch, leading to high costs. Summary of the Invention

[0005] This invention provides a method and apparatus for evaluating the quality of training data for an in-vehicle CAN intrusion detection model, which can improve the accuracy and efficiency of evaluating the quality of training data for an in-vehicle CAN intrusion detection model.

[0006] To achieve the above objectives, this invention provides a method for evaluating the quality of training data for an onboard CAN intrusion detection model, comprising: Obtain the training benchmark dataset for the vehicle CAN network intrusion detection model, and then divide the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed value CAN ID dataset according to the data change characteristics of each CAN identifier ID in a continuous period. For the variable CAN ID dataset, a model based on the variation pattern feature learning of the long short-term memory network autoencoder is used to extract the baseline reconstruction error of the variable CAN ID dataset; The training data of the vehicle CAN intrusion detection model to be evaluated is obtained, and the variable CAN ID dataset to be evaluated in the training data is input into the variation law feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain the quality score. According to the preset information entropy strategy, each variable CAN ID data in the variable CAN ID dataset to be evaluated is assigned a weight and weighted aggregation is performed to obtain the overall quality evaluation index of the variable CAN ID. For fixed-value CAN ID datasets, a benchmark library for fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark library is calculated. The overall quality evaluation index of fixed-value CAN ID datasets is calculated using binarization judgment and cumulative multiplication logic. The variable overall quality evaluation index and the fixed-value overall quality evaluation index are multiplied and aggregated to obtain the comprehensive quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated is then analyzed based on the comprehensive quality evaluation index.

[0007] To address the aforementioned problems, the present invention also provides a device for evaluating the training data quality of an in-vehicle CAN intrusion detection model, the device comprising: The data preprocessing module is used to obtain the training benchmark dataset for the vehicle CAN network intrusion detection model. Then, based on the data change characteristics of each CAN identifier ID in a continuous period, the training benchmark dataset for the vehicle CAN network intrusion detection model is divided into a variable CAN ID dataset and a fixed value CAN ID dataset. The quality evaluation index calculation module is used to extract the baseline reconstruction error of the variable CAN ID dataset using a long short-term memory network autoencoder-based change pattern feature learning model for variable CAN ID datasets; acquire the training data of the vehicle CAN intrusion detection model to be evaluated, and input the variable CAN ID dataset to be evaluated from the training data into the change pattern feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain a quality score. Weights are assigned to each variable CAN ID data in the variable CAN ID dataset to be evaluated according to a preset information entropy strategy, and weighted aggregation is performed to obtain the overall quality evaluation index for variable CAN ID datasets. For fixed-value CAN ID datasets, a baseline library for fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the baseline library is calculated. The overall quality evaluation index for fixed-value CAN ID datasets is calculated using binarization determination and cumulative multiplication logic. The quality evaluation module is used to multiply and aggregate the variable overall quality evaluation index and the fixed-value overall quality evaluation index to obtain a comprehensive quality evaluation index, and to analyze the data quality of the training data of the vehicle CAN intrusion detection model to be evaluated based on the comprehensive quality evaluation index.

[0008] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described method for evaluating the training data quality of the vehicle CAN intrusion detection model.

[0009] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described method for evaluating the training data quality of an in-vehicle CAN intrusion detection model.

[0010] This invention divides the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed-value CAN ID dataset based on the data change characteristics of each CAN ID within a continuous period. This enables differentiated classification processing of vehicle CAN messages, allowing for differentiated evaluation standards for all CAN IDs and improving the accuracy of data quality evaluation. Furthermore, for the variable CAN ID dataset, a feature learning model based on the change pattern of a long short-term memory network autoencoder is used to extract the benchmark reconstruction error of the variable CAN ID dataset, making the reference features for data quality evaluation more scientific and no longer reliant on manual and statistical features. Moreover, according to a preset information entropy strategy, weights are assigned to each variable CAN ID in the dataset to be evaluated, and weighted aggregation is performed to obtain the overall quality evaluation index for the variable CAN ID. This preset information entropy strategy highlights variable CAN IDs with higher information content and stronger discriminative value, thereby improving the accuracy of data quality evaluation results. In addition, for the fixed-value CAN ID dataset, a benchmark library for the fixed-value CAN ID dataset is constructed, and the fixed-value CAN IDs in the training data of the vehicle CAN intrusion detection model to be evaluated are calculated. By analyzing the statistical deviation of the ID dataset relative to the benchmark database and using binarization and cumulative multiplication logic to calculate the fixed-value overall quality evaluation index, the accuracy of the fixed-value CAN ID dataset evaluation can be improved. Finally, the variable-value overall quality evaluation index and the fixed-value overall quality evaluation index are multiplied and aggregated to obtain a comprehensive quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated is analyzed based on the comprehensive quality evaluation index, which can improve the accuracy and efficiency of the data quality evaluation of the training data of the vehicle CAN intrusion detection model to be evaluated. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a method for evaluating the training data quality of an in-vehicle CAN intrusion detection model according to an embodiment of the present invention. Figure 2 A model architecture diagram of a variation law feature learning model for a training data quality evaluation method for an in-vehicle CAN intrusion detection model provided in an embodiment of the present invention; Figure 3 This is a functional block diagram of a vehicle-mounted CAN intrusion detection model training data quality evaluation device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the method for evaluating the training data quality of a vehicle-mounted CAN intrusion detection model, as provided in an embodiment of the present invention.

[0012] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0014] This application provides a method for evaluating the quality of training data for an in-vehicle CAN intrusion detection model. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0015] Reference Figure 1 The diagram shown is a flowchart illustrating a method for evaluating the training data quality of an in-vehicle CAN intrusion detection model according to an embodiment of the present invention. In this embodiment, the method for evaluating the training data quality of the in-vehicle CAN intrusion detection model includes: S1. Obtain the training benchmark dataset for the vehicle CAN network intrusion detection model. Then, based on the data change characteristics of each CAN identifier ID in a continuous period, divide the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CANID dataset and a fixed-value CAN ID dataset.

[0016] Understandably, an in-vehicle CAN network refers to an internal vehicle communication bus system that follows the Controller Area Network (CAN) protocol, serving as the core physical link for data exchange between in-vehicle electronic control units (ECUs). In an in-vehicle CAN network, each node broadcasts a message containing a specific ID and data segment.

[0017] Understandably, the CAN identifier (ID) is a field used to uniquely identify the identity and priority of a CAN message. The ID determines the meaning of the physical signal carried by the message (e.g., steering angle, vehicle speed, engine speed) and its arbitration priority in bus contention. For example, a fixed-value CAN ID means that the data load it carries remains constant throughout the observation period, usually corresponding to key communication benchmarks such as synchronization signals, heartbeat messages, or configuration parameters; a variable-value CAN ID means that the data load it carries fluctuates with changes in the dynamic physical state of the vehicle, usually corresponding to physical quantities with significant timing characteristics such as sensor feedback and actuator commands.

[0018] Understandably, the training benchmark dataset for the vehicle CAN network intrusion detection model refers to a set of original CAN message sequences that are specifically used for research on vehicle CAN network security and record the vehicle's normal driving procedures and benign communication status.

[0019] Specifically, before dividing the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed-value CAN ID dataset based on the data change characteristics of each CAN identifier ID within a continuous period, the process further includes preprocessing the training benchmark dataset for the vehicle CAN network intrusion detection model: Read the training benchmark dataset of the vehicle CAN network intrusion detection model and delete redundant RTR data in the training benchmark dataset of the vehicle CAN network intrusion detection model. Iterate through each CAN data message. When the valid data segment carried by the data length code corresponding to a certain CAN ID is less than eight bytes, fill the empty fields in the data segment of the message with hexadecimal zeros and output a set of primary processing messages containing a complete eight-byte hexadecimal data segment. The complete eight-byte hexadecimal data segment carried in the primary processing message set is uniformly converted into a decimal integer, and the timestamp field corresponding to the primary processing message set is converted into a floating-point number; For each independent CAN ID contained in the primary processing message set, decimal integer messages belonging to the same CAN ID are stacked sequentially based on the corresponding floating-point timestamp. After stacking to form the message order relationship, the timestamp field is removed to obtain the ordinary time sequence of each specific CAN ID; Analyze the changes in the data fields of each CAN ID in all consecutive cycles in a normal time sequence: If the decimal integers of a certain CAN ID are exactly the same in all consecutive periods, it is determined to be a fixed-value CAN ID; otherwise, it is determined to be a variable-value CAN ID. For fixed-value CAN IDs, eight fixed decimal data segments are directly extracted from their corresponding ordinary time sequence to construct a two-dimensional matrix-style fixed-value reference library; For the variable CAN ID, a set of positive integers is constructed as sequence identifiers, and the corresponding ordinary timing sequence is output to the normalization processing stage. The ordinary timing sequence of the variable CAN ID is obtained, and the maximum and minimum values ​​of the current sequence are calculated on the eight data segments corresponding to the ordinary timing sequence. A small constant is introduced to avoid division by zero. The min-max normalization method is used to linearly map each decimal integer in the sequence to a closed interval between zero and one, and the normalized CAN timing sequence is output. The normalized CAN timing sequence is truncated in chronological order using a sliding window with a length of twenty communication cycles and a sliding step size of one, resulting in a CAN sequence sample set consisting of multiple 20x8 numerical matrices. The CAN sequence sample set is divided into a training set and a test set according to the chronological order.

[0020] Understandably, RTR (Remote Transmission Request Data) refers to the control data represented by the remote transmission request bit (RTR bit) in the CAN message control segment of the vehicle CAN network communication protocol.

[0021] For example, preprocessing the benchmark dataset for training the intrusion detection model of the vehicle CAN network can be achieved using the following implementation steps: First, read the CSV file of the training benchmark dataset carhacking dataset normal for vehicle CAN network intrusion detection model for vehicle CAN intrusion detection; then delete the redundant RTR data in the CAN data, because the RTR data does not participate in the subsequent research, and the RTR data in this dataset is a constant value: 000. Therefore, deleting the RTR identifier and its carried data information will not affect the quality evaluation of the generated benign CAN data for intrusion detection, and can avoid the extraction of invalid data features caused by data redundancy; then assign values ​​to the empty positions of the data fields. When the valid value in the data segment carried by a certain CAN ID is less than 8, that is, DLC is less than 8, it indicates that there is a situation where the data segment DATA[0]-DATA[7] carried by the CAN ID is empty. In this case, the empty fields are filled with the hexadecimal number 00. Since the hexadecimal data will be converted to decimal data later, after the data is normalized, the number involved in the calculation at this position is always 0. The 0 at this position will not affect the subsequent CAN ID classification will not affect the update of model weights and the calculation of subsequent reconstruction errors; it is also necessary to perform base conversion on the byte data. Decimal data can more clearly express the progression of the cycle, and the subsequent data normalization is preprocessed in advance. Therefore, the 8 bytes of hexadecimal data in the DATA[0] to DATA[7] data segments are converted into decimal integers for subsequent tasks; finally, the timestamp field is converted into a floating-point number for subsequent time series sorting.

[0022] CAN ID classification categorizes CAN IDs into two types based on the data variation characteristics of each CAN ID's data segments: variable and fixed-value. The classification process primarily involves analyzing the changes in the data fields of each CAN ID across all cycles: if the data fields of a CAN ID are identical across all consecutive cycles, it is classified as a fixed-value CAN ID; otherwise, it is classified as a variable CAN ID. Since fixed-value CAN IDs exhibit no data variation, they are unsuitable for time-series feature learning. Therefore, a separate LSTM encoder-based time-series feature learning model is not constructed for fixed-value CAN IDs. Instead, in subsequent reconstruction error calculations, values ​​are assigned individually based on data matching rules and the range of standard deviation to ensure that the reliability evaluation of all CAN IDs is expressed within a unified and reasonable framework.

[0023] Based on the CAN ID classification, each CAN data message is sequentially stacked according to its timestamp for each CAN ID. After stacking, time-series data for a specific CAN ID is formed. Then, the timestamp field and its data are removed, as the CAN message order relationship is only constructed using it, thus forming ordinary time-series data based on CAN ID. Its data shape is × ,in This represents the sequence number of the variant CAN ID, indicating which CAN ID it is in the study. This represents the number of times this CAN ID appears in the car hacking dataset normal dataset, that is... The number of CAN data entries included; This represents the number of data segments carried in the CAN message. Here, since each CAN message carries 8 data segments after preprocessing, its value is fixed at 8. For fixed-value CAN IDs, a fixed-value CAN ID benchmark library is constructed. (Fixed value type CAN ID reference library: FvCANLib), its data shape is as follows × Two-dimensional matrix ( Indicates the number of fixed-value CAN IDs. The size is 8 (representing the length of the data segment), which stores the decimal data corresponding to the fixed 8 data segments of the CAN ID. This facilitates the subsequent construction of a quality score function based on the standard deviation range for the fixed-value CAN ID. The construction order identifier for different CAN IDs is also included. And according to CAN ID, construct a set of positive integers respectively. and To represent the sequence identifier of variable CAN IDs and the sequence identifier of fixed-value CAN IDs, where The value range starts from 1 and increases sequentially; The range of values ​​is within After the statistics are completed, start from the next integer and continue until the statistics for all CAN IDs are finished.

[0024] To ensure stable model training, data normalization is also necessary. In standard automotive CAN network data, the data in a CAN message segment is represented in hexadecimal, with values ​​ranging from 0 to 255 in decimal. However, the "Car Hacking Dataset Normal" dataset collects automotive CAN data from a specific driving scenario and does not cover the entire range of values ​​in a data segment. Therefore, in this scenario, the data fields for different CAN IDs will have extremely narrow numerical distributions. Directly inputting this data into the model will result in an extremely unbalanced distribution of values ​​across different segments, leading to unstable training and ultimately unreliable model performance. Therefore, normalizing the data in the "Car Hacking Dataset Normal" dataset effectively reflects the driving scenario and data distribution characteristics of the dataset. The normalization process, based on the ordinary CAN ID time sequence constructed in the previous step, applies a minimum-maximum normalization method to each data byte of each variant CAN ID, linearly mapping the values ​​to the [0,1] interval, thus forming the normalized CAN ID time sequence. This effectively improves training stability. Normalization parameters. and In the The ordinary timing sequence data corresponding to each CAN ID The maximum and minimum values ​​were obtained by calculating the maximum and minimum values ​​on the eight data segments respectively. Elements The normalization calculation method is as follows: ; in express The Middle The first message Normalized data for each data field. To avoid small constants that divide by zero, normalized timing CAN data for specific CAN IDs was constructed. .

[0025] The main purpose of constructing the training benchmark dataset for the vehicle CAN network intrusion detection model, which includes variant CAN ID sequence samples and sequence sample sets (hereinafter referred to as CAN sequence samples and CAN sequence sample sets), is to generate long-term time-series CAN data for variant CAN IDs. The samples are sequentially converted into CAN time series samples that conform to the input of subsequent models using the sliding window method. The process, and based on this, constructs a CAN sequence sample set of a single variant CAN ID. The sliding window size w is set to 20 communication cycles, i.e., w=20. This is because it can cover the typical dynamic response time of vehicle electronic control systems (200ms-2s) while also taking into account the effective memory length of the LSTM network for historical information, achieving a balance between fully capturing the continuous periodic variation pattern of onboard CAN data and efficient learning. The sliding window step size s is set to 1, meaning there is only one cycle offset between adjacent sequence samples. This construction method maximizes data utilization while maintaining the continuity of the time series. After the above processing, each CAN sequence sample in the CAN sequence sample set... It has a uniform format: a numerical matrix with a shape of 20×8, where 20 represents the sliding window size and 8 represents the number of data bytes carried by each CAN message. Even if there are empty data bytes, after preprocessing and assignment, there will still be data with a value of 0.

[0026] The dataset will be divided into the CAN sequence sample sets constructed above based on each CAN ID. The first 70% of the CAN sequence samples in s are allocated as the training set for subsequent model training. The remaining 30% of the CAN sequence samples were divided into a test set to test the model's learning ability. This is used to construct the dataset needed for subsequent models.

[0027] After preprocessing, the benchmark dataset for training the intrusion detection model of the vehicle CAN network (car hacking datasetnormal) is transformed into a dataset (training set) suitable for subsequent model training and testing. and test set ).

[0028] S2. For the variable CAN ID dataset, the baseline reconstruction error of the variable CAN ID dataset is extracted by using a feature learning model based on the variation pattern of the autoencoder network of long short-term memory network.

[0029] Understandably, a Long Short-Term Memory (LSTM) network autoencoder is a deep learning architecture that combines the temporal processing capabilities of LSTM with the unsupervised compression and reconstruction characteristics of an autoencoder.

[0030] Specifically, a baseline reconstruction error for the variable CAN ID dataset is extracted using a learning model based on the variation pattern characteristics of a long short-term memory network autoencoder, including: The training set is used to input the change pattern feature learning model for feature learning, and the mean square error between the input sequence and the reconstructed sequence is used as the target to train the change pattern feature learning model. After training, the trained change pattern feature learning model is obtained. Input the test set into the trained variation pattern feature learning model, and calculate the reconstruction error of each sequence sample in the test set; Extract the mean of the reconstruction error of the test set and sort the mean from smallest to largest. Then, extract the 95th percentile of the mean after sorting from smallest to largest as the baseline reconstruction error. Save the trained feature learning model of change pattern and its corresponding baseline reconstruction error to the feature learning model library.

[0031] For example, the construction process of a learning model based on the variation pattern features of a long short-term memory network autoencoder can be implemented using the following steps: Long Short-Term Memory (LSTM) networks are a special type of recurrent neural network that effectively solves the gradient vanishing problem of traditional RNNs by introducing a gating mechanism, enabling them to learn long-term temporal dependencies. An LSTM autoencoder is a deep learning model that combines an LSTM network with an autoencoder structure, capable of learning compressed representations of time-series data and reconstructing the original sequence. A feature learning model for CAN ID data variation patterns based on an LSTM autoencoder is designed for each variant CAN ID. The input to FLM-CAN during the training phase is a preprocessed set of CAN data sequence samples from the car hacking dataset normal. During the quality evaluation phase, it is a preprocessed set of CAN data sequence samples to be evaluated from the training data of a benign vehicle CAN intrusion detection model generated by an existing data generation model based on the car hacking dataset normal. Both have a shape of 20×8. The output of the FLM-CAN model is a reconstructed sequence with the same shape as the input, also 20×8. The learning objective of FLM-CAN is to minimize the difference between the input sequence and the reconstructed sequence, thereby learning the temporal characteristics of data changes of each CAN ID in benign CAN data in the vehicle CAN intrusion detection scenario.

[0032] FLM-CAN consists of an encoder and a decoder. The encoder contains two LSTM layers. The first LSTM layer maps the input sequence to the hidden state space and contains 64 hidden units. The second LSTM layer further compresses the features and outputs a fixed-length latent vector with a dimension of 16. The encoding process for each CAN sequence sample can be represented as: , , ; , ; in, It is the input CAN sequence sample. It is the hidden state of the first LSTM output. It is the latent vector after feature compression.

[0033] The decoder also contains two LSTM layers, but functions in the opposite way to the encoder. The latent vector z is expanded into a sequence form through repeated vector operations and then input into the first LSTM layer of the decoder. The first LSTM layer maps the latent vector sequence back to the hidden state space, and the second LSTM layer further processes and outputs a reconstructed sequence with the same shape as the original sequence. The decoding process can be represented as: , ; , ; , ; in, It is a repeating latent vector sequence with a data shape of 20×16. It is the hidden state output by the first layer LSTM of the decoder, with a data shape of 20×64. It is the reconstructed sequence output by the second layer LSTM of the decoder, with a data shape of 20×8, which is consistent with the shape of the CAN sequence sample input to the FLM-CAN model.

[0034] The training objective of the model is to minimize the loss function, which is the input CAN sequence samples. With reconstructed CAN sequence samples Mean Squared Error (MSE) between the two. Loss Function Defined as: ; in =32 represents the batch size. =20 represents the number of CAN messages in each CAN sequence sample, and 8 represents the 8 bytes of data carried in each CAN message. For the first In the nth sample, the nth The first message The original normalized value of 1 byte, This is the corresponding reconstructed value output by the decoder.

[0035] Understandably, this involves adjusting the window size. The setting of 20 is based on the fact that the dynamic response time of typical electronic control systems (such as ESP and throttle control) in the vehicle CAN network is usually between 200ms and 2s. 20 communication cycles can effectively cover a complete physical action evolution process. At the same time, the feature dimension is set to 8 to fully preserve all the byte information of the CAN protocol data field, ensuring that the model can capture cross-byte related features.

[0036] The model was trained using the Adam optimizer with a learning rate of 0.001 and a batch size of 32. An early stopping strategy was employed during training: training was stopped and the optimal model weights were restored if the validation set loss did not improve within 10 consecutive epochs. This resulted in the [number of epochs]. The best model for a variant CAN ID .

[0037] After obtaining the optimal model weights for CAN ID, the test set was constructed. The sequence is reconstructed to count the number of digits. CAN ID exist The baseline reconstruction error distribution is calculated, and the mean reconstruction error is determined. and 95th percentile The calculation formula is as follows: ; ; in, The number of samples in the test set. Indicates the first One CAN ID, For the first Reconstruction error per sample : .

[0038] in, These are the input data points in the test set.

[0039] Understandably, the feature learning model library is a core component, primarily maintaining an independent FLM-CAN for each variable CAN ID. The main reason for this "one CAN ID corresponds to one FLM-CAN" design is as follows: In automotive CAN messages, different CAN IDs correspond to different physical signals, and their data variation patterns differ significantly. Building an independent FLM-CAN for each CAN ID allows for more accurate learning of the specific temporal data variation patterns of each CAN ID.

[0040] Furthermore, the construction of the feature learning model library includes the following implementation steps: The model library construction process is as follows: In this stage, the model construction in the model library is only for variant CAN IDs; then, a separate FLM-CAN is built for each variant CAN ID, and this CAN ID is used for training on the sequence sample training set; next, according to the training strategy, the FLM-CAN for each variant CAN ID is trained on its respective training sequence samples, with the goal of minimizing the reconstruction error on the test samples, forming independent models for the specified variant CAN ID. Finally, each independently trained model and its benchmark statistics, including the mean reconstruction error on the training set, are analyzed. and 95th percentile Save it to the model library.

[0041] For the training data of the benign vehicle CAN intrusion detection model to be evaluated, the FLM-CAN matching strategy in the model library is as follows: First, the training data of the vehicle CAN intrusion detection model to be evaluated is classified according to the CAN ID types defined by the data preprocessing strategy in Section 4.3.1, and sequence samples of all variant CAN IDs are extracted. For each CAN ID, the corresponding FLM-CAN is matched in the model library based on its CAN ID. Then, the model is used to evaluate the quality of the benign vehicle CAN intrusion detection model training data to be evaluated. The model matching process can be formally represented as follows: Let the model library be... = ,in It is the first FLM-CAN with CAN IDs. For the set of CAN IDs in the data to be evaluated... = The valid matching set is: ; in, , Representation Model The corresponding CAN ID.

[0042] S3. Obtain the training data of the vehicle CAN intrusion detection model to be evaluated, and input the variable CAN ID dataset to be evaluated from the training data into the variation law feature learning model to obtain the reconstruction error to be evaluated. Calculate the relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error to obtain the quality score. According to the preset information entropy strategy, assign weights to each variable CAN ID data in the variable CAN ID dataset to be evaluated and aggregate them to obtain the overall quality evaluation index of the variable CAN ID.

[0043] Understandably, the pre-defined information entropy strategy refers to a technical means that uses the uncertainty measurement principle in information theory to adaptively assign evaluation weights to each variant CAN ID based on the degree of dispersion of different CAN identifier IDs in the quality score distribution.

[0044] Specifically, the variable CAN ID dataset to be evaluated from the training data of the vehicle CAN intrusion detection model to be evaluated is input into the variation law feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain the quality score, including: Extract the dataset of variable CAN IDs to be evaluated from the training data of the vehicle CAN intrusion detection model to be evaluated, and match the trained variable pattern feature learning model corresponding to each CAN ID in the feature learning model library; The variable CAN ID dataset to be evaluated is input into the matching variation feature learning model to reconstruct the sequence, and the average mean square error between the input sequence and the reconstructed sequence is calculated and extracted as the reconstruction error to be evaluated. The relative error is obtained by dividing the reconstruction error to be evaluated by the mean of the corresponding baseline reconstruction error. A nonlinear mapping function is used to map the relative error to a closed interval between 0 and 1 to obtain the mapped value, which is then used as the quality score of the corresponding variant CAN ID to be evaluated.

[0045] For example, the quality score is obtained by calculating the relative ratio of the reconstruction error to the baseline reconstruction error, which is achieved by the following implementation steps: For evaluating the quality of variable CAN ID data, the mean square error between the input sequence and the reconstructed sequence, i.e., the reconstruction error, is first calculated. In the training data sequence sample of the vehicle CAN intrusion detection model corresponding to the CAN ID, the th The reconstruction error of each sample is calculated as follows: ; The mean reconstruction error is calculated for all samples in the sample set, using the following formula: ; in For the first The number of samples in the training data sequence sample set corresponding to each CAN ID for the vehicle CAN intrusion detection model.

[0046] To quantitatively evaluate the quality of training data for the vehicle-mounted CAN intrusion detection model, a relative error metric is defined. Relative error is the ratio of the reconstruction error of the vehicle-mounted CAN intrusion detection model training data to the reconstruction error of the baseline data. Let the... The average reconstruction error of each CANID reference data is The average reconstruction error of the corresponding vehicle CAN intrusion detection model training data is The relative error is: ; in, .

[0047] The closer the relative error R is to 1, the closer the reconstruction error of the vehicle CAN intrusion detection model training data is to the baseline data, and the higher the data quality. The larger the value of R, the more significant the reconstruction error of the vehicle CAN intrusion detection model training data is to the baseline data, and the lower the data quality.

[0048] Based on relative error, a quality fraction is defined. , representing the first training data of the vehicle CAN intrusion detection model The quality evaluation metric for each CAN ID. The quality score uses a variant of the sigmoid function to map the relative error to the [0,1] interval: ; when When =1, =1; when When it increases, It monotonically decreases and approaches 0. This non-linear mapping can better distinguish data with similar quality. After obtaining the quality score for each variant CAN ID, the quality scores are aggregated into a comprehensive quality score.

[0049] Furthermore, weights are assigned to each variant CAN ID data in the variant CAN ID dataset to be evaluated according to a preset information entropy strategy, including: Extract the quality scores of all variant CAN IDs in the variant CAN ID dataset to be evaluated, and normalize the quality scores to obtain normalized quality scores; The normalized quality score is divided into a preset number of equal-width intervals. The frequency of the quality score of each variant CAN ID falling into each equal-width interval is counted, and the probability distribution of each equal-width interval is calculated based on the frequency. The actual information entropy of each variant CAN ID is calculated based on the probability distribution of equal-width intervals, and the maximum possible information entropy is calculated when the probability distribution is completely uniform across all equal-width intervals. Calculate the relative deviation between the actual information entropy and the maximum possible information entropy for each variant CAN ID, and use the degree of deviation as the difference coefficient; The difference coefficients of all variant CAN IDs are normalized to obtain the weight of each variant CAN ID data.

[0050] For example, assigning weights to each variant CAN ID data in the variant CAN ID dataset to be evaluated according to a preset information entropy strategy can be achieved using the following implementation steps: For the Each CAN ID has a quality fraction of 1. First, the quality scores of all CAN IDs are normalized: ; in, This represents the normalized quality assessment score. and These are the minimum and maximum quality scores for all CAN IDs, respectively. It is a small constant added to avoid division by zero.

[0051] The normalized quality fraction interval [0,1] is divided into... A number of equal-width intervals ( This section is processed using a commonly used interval partitioning strategy, namely... =10), statistic of the first Calculate the probability distribution of the frequency with which the quality score of each CAN ID falls into each interval. The probability distribution of each interval is defined as the proportion of samples within that interval to the total number of samples. The calculation logic is as follows: ; in, The total number of test samples, It is a counting function.

[0052] Based on the above probability distribution, calculate the... Information entropy of the quality score of a CAN ID Information entropy measures the uniformity of the quality score distribution: a higher entropy value indicates a more uniform distribution and a weaker distinguishing ability for the CAN ID; a lower entropy value indicates a more concentrated distribution and a stronger distinguishing ability. The calculation formula is as follows: ; When the quality score is uniformly distributed across all intervals, the information entropy reaches its maximum value. At this point, the probability of each interval is 1 / : ; In order to quantify the first The distinguishability of each CAN ID, defining the difference coefficient. The relative deviation between the actual entropy and the maximum entropy is calculated using the following formula: ; The closer the difference coefficient is to 1, the more concentrated the quality fraction distribution of the CAN ID is, and the stronger its distinguishing ability; the closer it is to 0, the more uniform the distribution is, and the weaker its distinguishing ability is.

[0053] The difference coefficients of each CAN ID are normalized to obtain the final weights. : ; in, Let be the total number of variant CAN IDs participating in the evaluation, and let the weights satisfy the following formula: This means ensuring the reasonableness of the weighted average.

[0054] Overall data quality evaluation metrics based on variable CAN ID: Among them, the comprehensive quality evaluation index The quality scores of each CAN ID are aggregated by weighted average: ; Among them, the aggregation formula is through Ensure that the timing conformity of each variant CAN ID is accurately quantified; through Ensure that information-rich CAN IDs (such as high-frequency critical control signals) play a greater role in the final evaluation; ultimately A continuous quality mapping from 1.0 to 0.0 was achieved.

[0055] S4. For the fixed-value CAN ID dataset, construct a benchmark library for the fixed-value CAN ID dataset, calculate the statistical deviation of the fixed-value CAN ID dataset relative to the benchmark library in the training data of the vehicle CAN intrusion detection model to be evaluated, and calculate the overall quality evaluation index of the fixed-value dataset using binarization judgment and cumulative multiplication logic.

[0056] Specifically, the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark database is calculated, and the overall quality evaluation index of the fixed-value dataset is calculated using binarization determination and cumulative multiplication logic, including: Compare the data sequence of fixed-value CAN ID in the training data of the vehicle CAN intrusion detection model to be evaluated with the benchmark data in the benchmark library, and calculate the mean standard deviation between the two as the statistical bias. When the standard deviation mean is zero and the data field of the CAN ID in the training data of the vehicle CAN intrusion detection model is consistent with the baseline data, the quality score of the fixed-value CAN ID is assigned as 1. When the mean standard deviation is greater than zero, indicating a change in the training data of the vehicle CAN intrusion detection model, the quality score of the fixed-value CAN ID is assigned to 0. Extract the quality scores of all fixed-value CAN IDs and perform a cumulative multiplication operation. Output the result of the cumulative multiplication as a fixed-value overall quality evaluation index.

[0057] For example, for a fixed-value CAN ID dataset, a benchmark library of fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark library is calculated. The overall quality evaluation index of the fixed-value dataset is then calculated using binarization judgment and cumulative multiplication logic. This can be achieved through the following implementation steps: The quality score of its fixed-value CAN ID dataset The calculation formula is as follows: ; in, , The mean of the baseline data. The mean of the fixed-value CAN ID dataset to be evaluated. The CAN ID data field in the training data for the vehicle CAN intrusion detection model is compared with the fixed-value CAN ID benchmark library. The mean standard deviation of the baseline data corresponding to the CAN ID The calculation formula is as follows: ; in, The first in the training data of the vehicle CAN intrusion detection model The total number of data in a CAN sequence constructed from a fixed-value CAN ID. The first in the training data of the vehicle CAN intrusion detection model to be evaluated The first message The decimal value of each data byte (DATA[0]-DATA[7]). The first in the fixed-value CAN ID reference library The ID corresponding to the first The standard reference value is 1 byte.

[0058] For fixed-value CAN IDs (such as CAN IDs whose data at the beginning of a CAN cycle is a constant value), changes to their 8-byte data segment severely affect the consistency of CAN data across consecutive cycles. Therefore, stricter requirements should be placed on the data carried by these types of CAN IDs. Thus, the overall quality evaluation index for the data patterns of fixed-value CAN IDs... The definition is as follows: ; in, For multiplication operators, This is the set of sequential identifiers constructed for a fixed-value CAN ID. In the above formula... The formula accumulates the quality score for each fixed-value CAN ID. Therefore, if the data carried by any fixed-value CAN ID changes, this situation will be forcibly defined by this function as not conforming to the data change pattern. Thus, the overall quality evaluation index of all fixed-value CAN IDs is set to 0 to ensure that the quality evaluation result conforms to an extreme binary distribution.

[0059] S5. Multiply and aggregate the variable overall quality evaluation index and the fixed-value overall quality evaluation index to obtain a comprehensive quality evaluation index, and analyze the data quality of the training data of the vehicle CAN intrusion detection model to be evaluated based on the comprehensive quality evaluation index.

[0060] Specifically, the comprehensive quality evaluation index is obtained by multiplying and aggregating the variable-type overall quality evaluation index and the fixed-value type overall quality evaluation index, including: The comprehensive quality evaluation index is obtained by multiplying the value of the variable overall quality evaluation index with the value of the fixed overall quality evaluation index. When the fixed-value overall quality evaluation index is 1, the value of the comprehensive quality evaluation index is equal to the value of the variable-value overall quality evaluation index. When the fixed-value overall quality evaluation index is 0, a veto mechanism is triggered, forcibly setting the value of the comprehensive quality evaluation index to 0.

[0061] For example, the comprehensive quality evaluation index is obtained by multiplying and aggregating the variable overall quality evaluation index and the fixed-value overall quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated can be analyzed based on the comprehensive quality evaluation index. The following implementation steps can be adopted: Multiplying the value of the variable overall quality evaluation index by the value of the fixed overall quality evaluation index yields the comprehensive quality evaluation index, the variable CAN ID overall data quality score. and fixed-value CAN ID overall data quality score ; Construct the following comprehensive index for evaluating the training data quality of a benign vehicle CAN intrusion detection model with continuous cycles. The aggregation method for the overall quality evaluation indicators of the two categories is as follows: · ; Understandably, the above formula not only satisfies the requirement that when the fixed-value CAN ID conforms to its fixed data distribution, it does not affect the evaluation of the data change pattern of the variable CAN ID, thus forming a data quality evaluation strategy that is more biased towards the variable CAN ID, but also allows for a veto of the overall data quality when the fixed-value CAN ID shows data changes, thus forming a data quality evaluation strategy that is more biased towards the fixed-value CAN ID, and can also give an extremely low negative evaluation when the overall quality evaluation of the two types of CAN ID fails to meet the standards.

[0062] This invention divides the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed-value CAN ID dataset based on the data change characteristics of each CAN ID within a continuous period. This enables differentiated classification processing of vehicle CAN messages, allowing for differentiated evaluation standards for all CAN IDs and improving the accuracy of data quality evaluation. Furthermore, for the variable CAN ID dataset, a feature learning model based on the change pattern of a long short-term memory network autoencoder is used to extract the benchmark reconstruction error of the variable CAN ID dataset, making the reference features for data quality evaluation more scientific and no longer reliant on manual and statistical features. Moreover, according to a preset information entropy strategy, weights are assigned to each variable CAN ID in the dataset to be evaluated, and weighted aggregation is performed to obtain the overall quality evaluation index for the variable CAN ID. This preset information entropy strategy highlights variable CAN IDs with higher information content and stronger discriminative value, thereby improving the accuracy of data quality evaluation results. In addition, for the fixed-value CAN ID dataset, a benchmark library for the fixed-value CAN ID dataset is constructed, and the fixed-value CAN IDs in the training data of the vehicle CAN intrusion detection model to be evaluated are calculated. By analyzing the statistical deviation of the ID dataset relative to the benchmark database and using binarization and cumulative multiplication logic to calculate the fixed-value overall quality evaluation index, the accuracy of the fixed-value CAN ID dataset evaluation can be improved. Finally, the variable-value overall quality evaluation index and the fixed-value overall quality evaluation index are multiplied and aggregated to obtain a comprehensive quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated is analyzed based on the comprehensive quality evaluation index, which can improve the accuracy and efficiency of the data quality evaluation of the training data of the vehicle CAN intrusion detection model to be evaluated.

[0063] Reference Figure 2 The diagram shown is a model architecture diagram of the variation law feature learning model of a training data quality evaluation method for vehicle CAN intrusion detection model provided in an embodiment of the present invention.

[0064] like Figure 3 The diagram shown is a functional block diagram of a vehicle-mounted CAN intrusion detection model training data quality evaluation device provided in an embodiment of the present invention.

[0065] The vehicle-mounted CAN intrusion detection model training data quality evaluation device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the vehicle-mounted CAN intrusion detection model training data quality evaluation device 100 may include a data preprocessing module 101, a quality evaluation index calculation module 102, and a quality evaluation module 103.

[0066] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0067] In this embodiment, the functions of each module / unit are as follows: The data preprocessing module 101 is used to obtain the training benchmark dataset of the vehicle CAN network intrusion detection model, and then divide the training benchmark dataset of the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed value CAN ID dataset according to the data change characteristics of each CAN identifier ID in a continuous period.

[0068] The quality evaluation index calculation module 102 is used to extract the baseline reconstruction error of the variable CAN ID dataset using a long short-term memory network autoencoder-based change pattern feature learning model for variable CAN ID datasets; acquire the training data of the vehicle CAN intrusion detection model to be evaluated, and input the variable CAN ID dataset to be evaluated from the training data of the vehicle CAN intrusion detection model to be evaluated into the change pattern feature learning model to obtain the reconstruction error to be evaluated, calculate the relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error to obtain the quality score, and assign weights to each variable CAN ID data in the variable CAN ID dataset to be evaluated according to a preset information entropy strategy and perform weighted aggregation to obtain the overall quality evaluation index of variable CAN ID datasets; for fixed value CAN ID datasets, construct a baseline library of fixed value CAN ID datasets, calculate the statistical deviation of the fixed value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the baseline library, and calculate the overall quality evaluation index of fixed value CAN ID datasets using binarization judgment and cumulative multiplication logic.

[0069] The quality evaluation module 103 is used to multiply and aggregate the variable overall quality evaluation index and the fixed value overall quality evaluation index to obtain a comprehensive quality evaluation index, and to analyze the data quality of the training data of the vehicle CAN intrusion detection model to be evaluated based on the comprehensive quality evaluation index.

[0070] like Figure 4 The diagram shown is a structural schematic of an electronic device that implements a method for evaluating the training data quality of an in-vehicle CAN intrusion detection model, according to an embodiment of the present invention.

[0071] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a program for evaluating the quality of training data for an in-vehicle CAN intrusion detection model.

[0072] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing a method for evaluating the quality of training data for a vehicle-mounted CAN intrusion detection model), and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0073] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a method for evaluating the quality of training data for a vehicle-mounted CAN intrusion detection model, but also to temporarily store data that has been output or will be output.

[0074] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0075] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0076] Figure 4 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 4 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0077] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0078] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0079] The program for evaluating the quality of training data for an on-board CAN intrusion detection model, stored in the memory 11 of the electronic device, is a combination of multiple instructions. When run in the processor 10, it can achieve the following: Obtain the training benchmark dataset for the vehicle CAN network intrusion detection model, and then divide the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed value CAN ID dataset according to the data change characteristics of each CAN identifier ID in a continuous period. For the variable CAN ID dataset, a model based on the variation pattern feature learning of the long short-term memory network autoencoder is used to extract the baseline reconstruction error of the variable CAN ID dataset; The training data of the vehicle CAN intrusion detection model to be evaluated is obtained, and the variable CAN ID dataset to be evaluated in the training data is input into the variation law feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain the quality score. According to the preset information entropy strategy, each variable CAN ID data in the variable CAN ID dataset to be evaluated is assigned a weight and weighted aggregation is performed to obtain the overall quality evaluation index of the variable CAN ID. For fixed-value CAN ID datasets, a benchmark library for fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark library is calculated. The overall quality evaluation index of fixed-value CAN ID datasets is calculated using binarization judgment and cumulative multiplication logic. The variable overall quality evaluation index and the fixed-value overall quality evaluation index are multiplied and aggregated to obtain the comprehensive quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated is then analyzed based on the comprehensive quality evaluation index.

[0080] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0081] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0082] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following: Obtain the training benchmark dataset for the vehicle CAN network intrusion detection model, and then divide the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed value CAN ID dataset according to the data change characteristics of each CAN identifier ID in a continuous period. For the variable CAN ID dataset, a model based on the variation pattern feature learning of the long short-term memory network autoencoder is used to extract the baseline reconstruction error of the variable CAN ID dataset; The training data of the vehicle CAN intrusion detection model to be evaluated is obtained, and the variable CAN ID dataset to be evaluated in the training data is input into the variation law feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain the quality score. According to the preset information entropy strategy, each variable CAN ID data in the variable CAN ID dataset to be evaluated is assigned a weight and weighted aggregation is performed to obtain the overall quality evaluation index of the variable CAN ID. For fixed-value CAN ID datasets, a benchmark library for fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark library is calculated. The overall quality evaluation index of fixed-value CAN ID datasets is calculated using binarization judgment and cumulative multiplication logic. The variable overall quality evaluation index and the fixed-value overall quality evaluation index are multiplied and aggregated to obtain the comprehensive quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated is then analyzed based on the comprehensive quality evaluation index.

[0083] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0084] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0085] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0086] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0087] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0088] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0089] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0090] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for evaluating the quality of training data for an onboard CAN intrusion detection model, characterized in that, The method includes: Obtain the training benchmark dataset for the vehicle CAN network intrusion detection model, and then divide the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed value CAN ID dataset according to the data change characteristics of each CAN identifier ID in a continuous period. For the variable CAN ID dataset, a model based on the variation pattern feature learning of the long short-term memory network autoencoder is used to extract the baseline reconstruction error of the variable CAN ID dataset; The training data of the vehicle CAN intrusion detection model to be evaluated is obtained, and the variable CAN ID dataset to be evaluated in the training data is input into the variation law feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain the quality score. According to the preset information entropy strategy, each variable CAN ID data in the variable CAN ID dataset to be evaluated is assigned a weight and weighted aggregation is performed to obtain the overall quality evaluation index of the variable CAN ID. For fixed-value CAN ID datasets, a benchmark library for fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark library is calculated. The overall quality evaluation index of fixed-value CAN ID datasets is calculated using binarization judgment and cumulative multiplication logic. The variable overall quality evaluation index and the fixed-value overall quality evaluation index are multiplied and aggregated to obtain the comprehensive quality evaluation index. The data quality of the training data of the vehicle CAN intrusion detection model to be evaluated is then analyzed based on the comprehensive quality evaluation index.

2. The method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in claim 1, characterized in that, Before dividing the training benchmark dataset for the vehicle CAN network intrusion detection model into a variable CAN ID dataset and a fixed-value CAN ID dataset based on the data change characteristics of each CAN identifier ID within a continuous period, the process also includes preprocessing the training benchmark dataset for the vehicle CAN network intrusion detection model: Read the training benchmark dataset of the vehicle CAN network intrusion detection model and delete redundant RTR data in the training benchmark dataset of the vehicle CAN network intrusion detection model. Iterate through each CAN data message. When the valid data segment carried by the data length code corresponding to a certain CAN ID is less than eight bytes, fill the empty fields in the data segment of the message with hexadecimal zeros and output a set of primary processing messages containing a complete eight-byte hexadecimal data segment. The complete eight-byte hexadecimal data segment carried in the primary processing message set is uniformly converted into a decimal integer, and the timestamp field corresponding to the primary processing message set is converted into a floating-point number; For each independent CAN ID contained in the primary processing message set, decimal integer messages belonging to the same CAN ID are stacked sequentially based on the corresponding floating-point timestamp. After stacking to form the message order relationship, the timestamp field is removed to obtain the ordinary time sequence of each specific CAN ID; Analyze the changes in the data fields of each CAN ID in all consecutive cycles in a normal time sequence: If the decimal integers of a certain CAN ID are exactly the same in all consecutive periods, it is determined to be a fixed-value CAN ID; otherwise, it is determined to be a variable-value CAN ID. For fixed-value CAN IDs, eight fixed decimal data segments are directly extracted from their corresponding ordinary time sequence to construct a two-dimensional matrix-based fixed-value reference library; For the variable CAN ID, a set of positive integers is constructed as sequence identifiers, and the corresponding ordinary timing sequence is output to the normalization processing stage. The ordinary timing sequence of the variable CAN ID is obtained, and the maximum and minimum values ​​of the current sequence are calculated on the eight data segments corresponding to the ordinary timing sequence. A small constant is introduced to avoid division by zero. The min-max normalization method is used to linearly map each decimal integer in the sequence to a closed interval between zero and one, and the normalized CAN timing sequence is output. The normalized CAN timing sequence is truncated in chronological order using a sliding window with a length of twenty communication cycles and a sliding step size of one, resulting in a CAN sequence sample set consisting of multiple 20x8 numerical matrices. The CAN sequence sample set is divided into a training set and a test set according to the chronological order.

3. The method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in claim 1, characterized in that, The method of extracting the baseline reconstruction error of the variable CAN ID dataset using a learning model based on the variation pattern features of a long short-term memory network autoencoder includes: The training set is used to input the change pattern feature learning model for feature learning, and the mean square error between the input sequence and the reconstructed sequence is used as the target to train the change pattern feature learning model. After training, the trained change pattern feature learning model is obtained. Input the test set into the trained variation pattern feature learning model, and calculate the reconstruction error of each sequence sample in the test set; Extract the mean of the reconstruction error of the test set and sort the mean from smallest to largest. Then, extract the 95th percentile of the mean after sorting from smallest to largest as the baseline reconstruction error. Save the trained feature learning model of change pattern and its corresponding baseline reconstruction error to the feature learning model library.

4. The method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in claim 1 or 3, characterized in that, The process involves inputting the variable CAN ID dataset from the training data of the vehicle CAN intrusion detection model to be evaluated into the variation feature learning model to obtain the reconstruction error to be evaluated, and calculating the relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error to obtain a quality score, including: Extract the dataset of variable CAN IDs to be evaluated from the training data of the vehicle CAN intrusion detection model to be evaluated, and match the trained variable pattern feature learning model corresponding to each CAN ID in the feature learning model library; The variable CAN ID dataset to be evaluated is input into the matching variation feature learning model to reconstruct the sequence, and the average mean square error between the input sequence and the reconstructed sequence is calculated and extracted as the reconstruction error to be evaluated. The relative error is obtained by dividing the reconstruction error to be evaluated by the mean of the corresponding baseline reconstruction error. A nonlinear mapping function is used to map the relative error to a closed interval between 0 and 1 to obtain the mapped value, which is then used as the quality score of the corresponding variant CAN ID to be evaluated.

5. The method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in claim 1, characterized in that, The process of assigning weights to each variant CAN ID data in the dataset to be evaluated according to a preset information entropy strategy includes: Extract the quality scores of all variant CAN IDs in the variant CAN ID dataset to be evaluated, and normalize the quality scores to obtain normalized quality scores; The normalized quality score is divided into a preset number of equal-width intervals. The frequency of the quality score of each variant CAN ID falling into each equal-width interval is counted, and the probability distribution of each equal-width interval is calculated based on the frequency. The actual information entropy of each variant CAN ID is calculated based on the probability distribution of equal-width intervals, and the maximum possible information entropy is calculated when the probability distribution is completely uniform across all equal-width intervals. Calculate the relative deviation between the actual information entropy and the maximum possible information entropy for each variant CAN ID, and use the degree of deviation as the difference coefficient; The difference coefficients of all variant CAN IDs are normalized to obtain the weight of each variant CAN ID data.

6. The method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in claim 1, characterized in that, The calculation of the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the benchmark library, and the calculation of the overall quality evaluation index of the fixed-value dataset using binarization determination and cumulative multiplication logic, includes: Compare the data sequence of fixed-value CAN ID in the training data of the vehicle CAN intrusion detection model to be evaluated with the benchmark data in the benchmark library, and calculate the mean standard deviation between the two as the statistical bias. When the standard deviation mean is zero and the data field of the CAN ID in the training data of the vehicle CAN intrusion detection model is consistent with the baseline data, the quality score of the fixed-value CAN ID is assigned as 1. When the mean standard deviation is greater than zero, i.e., when the training data of the vehicle CAN intrusion detection model changes, the quality score of the fixed-value CANID is assigned to 0. Extract the quality scores of all fixed-value CAN IDs and perform a cumulative multiplication operation. Output the result of the cumulative multiplication as a fixed-value overall quality evaluation index.

7. The method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in claim 1, characterized in that, The process of multiplying and aggregating the variable-type overall quality evaluation index with the fixed-value-type overall quality evaluation index to obtain the comprehensive quality evaluation index includes: The comprehensive quality evaluation index is obtained by multiplying the value of the variable overall quality evaluation index with the value of the fixed overall quality evaluation index. When the fixed-value overall quality evaluation index is 1, the value of the comprehensive quality evaluation index is equal to the value of the variable-value overall quality evaluation index. When the fixed-value overall quality evaluation index is 0, a veto mechanism is triggered, forcibly setting the value of the comprehensive quality evaluation index to 0.

8. A device for evaluating the quality of training data for a vehicle-mounted CAN intrusion detection model, characterized in that, The apparatus is used to implement the method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in any one of claims 1 to 7, the apparatus comprising: The data preprocessing module is used to obtain the training benchmark dataset for the vehicle CAN network intrusion detection model. Then, based on the data change characteristics of each CAN identifier ID in a continuous period, the training benchmark dataset for the vehicle CAN network intrusion detection model is divided into a variable CAN ID dataset and a fixed value CAN ID dataset. The quality evaluation index calculation module is used to extract the baseline reconstruction error of the variable CAN ID dataset using a long short-term memory network autoencoder-based change pattern feature learning model for variable CAN ID datasets; acquire the training data of the vehicle CAN intrusion detection model to be evaluated, and input the variable CAN ID dataset to be evaluated from the training data into the change pattern feature learning model to obtain the reconstruction error to be evaluated. The relative ratio of the reconstruction error to be evaluated to the baseline reconstruction error is calculated to obtain a quality score. Weights are assigned to each variable CAN ID data in the variable CAN ID dataset to be evaluated according to a preset information entropy strategy, and weighted aggregation is performed to obtain the overall quality evaluation index for variable CAN ID datasets. For fixed-value CAN ID datasets, a baseline library for fixed-value CAN ID datasets is constructed, and the statistical deviation of the fixed-value CAN ID dataset in the training data of the vehicle CAN intrusion detection model to be evaluated relative to the baseline library is calculated. The overall quality evaluation index for fixed-value CAN ID datasets is calculated using binarization determination and cumulative multiplication logic. The quality evaluation module is used to multiply and aggregate the variable overall quality evaluation index and the fixed-value overall quality evaluation index to obtain a comprehensive quality evaluation index, and to analyze the data quality of the training data of the vehicle CAN intrusion detection model to be evaluated based on the comprehensive quality evaluation index.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the vehicle CAN intrusion detection model training data quality evaluation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for evaluating the training data quality of the vehicle-mounted CAN intrusion detection model as described in any one of claims 1 to 7.

Images