A method, system and application of vehicle-mounted Ethernet intrusion detection and defense based on a Transformer encoder

Abstract

The application discloses a kind of vehicle Ethernet intrusion detection and defense method based on Transformer encoder, comprising the following steps: constructing vehicle Ethernet intrusion detection model, extracting sample data from the database containing normal communication message and / or attack type message pre-set, optimizing the training of the intrusion detection model;Real-time acquisition of real vehicle Ethernet message data, real-time inference is carried out to real data by constructing optimized intrusion detection model, whether the flow is abnormal is judged, and potential network attack or intrusion behavior is identified;According to the intrusion detection result, response unit carries out intrusion alarm and / or intrusion response.The method of the application realizes efficient sequence data processing capability, accurate anomaly detection capability and real-time response capability;At the same time, in order to maintain effective defense to new attack, the system updates the intrusion detection model periodically.The application also discloses a system for implementing the above method, which has a wide range of application scenarios.

Description

Technical Field

[0001] This invention relates to the field of vehicle network security, and in particular to a method, system and application for vehicle Ethernet intrusion detection and defense based on a Transformer encoder. Background Technology

[0002] The demand for high-speed data transmission in connected cars has spurred the rapid development of automotive networks, giving rise to automotive Ethernet, which has become one of the fastest-growing in-vehicle network technologies. However, while improving connectivity within the vehicle, the use of automotive Ethernet also expands the attack surface, thereby increasing the vulnerability of the in-vehicle network to network attacks.

[0003] The investigation revealed that data flows in vehicular Ethernet systems exhibit high temporal sequence characteristics. The Transformer model excels in sequence modeling tasks due to its self-attention mechanism, enabling the model to establish dependencies between different locations without step-by-step processing of the sequence. Therefore, it maintains high performance when processing long sequences and can capture long-distance dependencies. Its parallel computing capability, as the output at each location can be computed independently without being limited by sequence length, significantly accelerates large-scale data processing. Furthermore, its multi-head attention mechanism allows the model to simultaneously attend to different aspects of the input sequence in different representation subspaces. Each attention head can focus on different locations or features in the sequence, thus capturing the relationships between inputs more comprehensively. Therefore, introducing the Transformer model as an intrusion detection model for vehicular Ethernet better adapts to complex data patterns and dynamic network environments, improving the accuracy, coverage, and speed of intrusion detection. Summary of the Invention

[0004] The purpose of this invention is to propose an in-vehicle Ethernet intrusion detection and defense method based on a Transformer encoder. This method aims to provide an efficient, reliable, and secure in-vehicle network protection solution to address the increasingly severe network security challenges. It utilizes a Transformer encoder for detection to achieve efficient intrusion detection and defense. The intrusion detection task in this invention uses only the encoder part of the Transformer model, which helps the system extract useful feature representations from the raw input data without requiring sequence generation or other complex decoding operations. This simplifies the model structure and computational requirements while effectively handling large-scale input data.

[0005] This invention provides a method for in-vehicle Ethernet intrusion detection and prevention based on a Transformer encoder, the method comprising the following steps:

[0006] Step 1: Construct an in-vehicle Ethernet intrusion detection model by extracting sample data from a pre-defined database containing normal communication messages and / or attack type messages, and optimizing and training the intrusion detection model.

[0007] Step 2: Collect real vehicle Ethernet packet data in real time, preprocess and infer the data in real time by building an optimized intrusion detection model, determine whether the traffic is abnormal, and identify potential network attacks or intrusion behaviors.

[0008] Step 3: Based on the intrusion detection results, the response unit performs intrusion alarm and / or intrusion response.

[0009] The intrusion detection model construction in step 1 of this invention is based on the transformer encoder model, and the specific process is as follows:

[0010] Step 1.1: Obtain the vehicle Ethernet packet sample data from the database. Each sample data consists of network connection features and network connection tags (normal / attack). Features are separated by commas, and features and tags are separated by commas.

[0011] Step 1.2: Preprocess the message sample data: cleaning, standardization, and serialization, etc.

[0012] Step 1.2.1, Data Cleaning: Process the data samples to ensure the quality of the data used for training and detection. Cleaning includes removing irrelevant or noisy data, imputing missing values, and removing duplicate data to improve the training effect and detection accuracy of the model.

[0013] Step 1.2.2, Data Standardization Processing: The category features (such as protocols, service types, etc.) are tokenized using label encoding, mapping each category to a numerical value; then the numerical features (such as port numbers, data volume, etc.) and the mapped values ​​of the category features are normalized to ensure that they have similar scales.

[0014] Step 1.2.3, Serialization Processing: The cleaned and standardized data are arranged in chronological order to form a sequence of data packets. In one specific implementation, each sequence contains a fixed 60 time steps (i.e., data packets). For sequences with fewer than 60 data packets, zero padding can be used to achieve a fixed length; for sequences with more than 60 data packets, a sliding window method is used to divide the long sequence into multiple data packet sequences of length 60 to obtain more data fragments. Each window can contain partially overlapping data, thus preserving more information and ensuring a uniform length of the input sequence. Sequence labeling uses a multi-label classification method, where each sequence can have multiple labels, ultimately represented by a multi-hot encoded vector; labels include whether the sequence is normal and the specific attack type for abnormal entries.

[0015] Step 1.3, Data Partitioning: Partition the serialized data into a training set, a validation set, and a test set.

[0016] Step 1.4, Model Training and Optimization: Model training and optimization require the following specific steps:

[0017] Step 1.4.1, Input Embedding: Each time step (data packet) represents the data state at a fixed time point in the input sequence. Perform an embedding process on each time step, that is, convert the input data packet into a numerical vector that can be processed by the model.

[0018] Step 1.4.2, Position Embedding: Represented by PE, the calculation formula is as follows:

[0019] PE(pos, 2i) = sin(pos / (n 2i / d )), PE(pos, 2i + 1) = cos(pos / (n 2i / d )), where: pos: the position of the object in the input sequence, 0 ≤ pos < L - 1, L is the sequence length; d: the dimension of the output embedding space; PE(pos, j): the position function, used to map the position pos in the input sequence to the index (pos, j) of the position matrix; n: a user-defined scalar, usually chosen as 10000 to adjust the frequency range; i: used to map the column index to a value where 0 ≤ i < d / 2, and a single i value is mapped to both the sine and cosine functions.

[0020] Step 1.4.3, Input and Output of the Encoder:

[0021] Among them,

[0022] Encoder Input: Each element x_i in the input sequence X is converted into an embedding vector. The embedding vector obtained according to the input sequence is set as xe_i, and added to the position vector PE(pos) to obtain the position encoding. The calculation formula is as follows: PC(x_i) = xe_i + PE(pos). To prevent the embedding vector xe_i from being minimized by the position vector, that is, to avoid the value of the position encoding being too large and the influence of the embedding vector almost disappearing, resulting in the model being unable to effectively learn the representation of the embedding vector, it is necessary to ensure their consistency in scale. By combining the element embedding vector and the position vector of the input sequence, ensure their numerical scale consistency. The formula is expressed as follows: PC(x_i) = xe_i * math.sqrt(d) + PE(pos), where xe_i represents the embedding vector of the i-th element in the input sequence X, d represents the dimension of the embedding vector, and PE(pos) represents the position vector. The input X of the encoder is processed into the embedding vector PC(x_i) of each element, and these vectors contain the semantic information and relative position information of the elements, enabling the model to effectively process sequence data.

[0023] The encoder layer includes a multi-head attention layer, a pre-feedback layer, and a residual connection and layer normalization layer, which encodes the input position and outputs an encoded information matrix.

[0024] The multi-head attention layer employs a self-attention mechanism: An input matrix X is obtained, representing the input sequence, with each row representing a sequence element. The query matrix Q, key matrix K, and value matrix V are calculated using a linear transformation matrix (Qw, Kw, Vw), as follows: Q = X * Qw, K = X * Kw, V = X * Vw, where each row of X, Q, K, and V represents a sequence. In attention calculation, the query matrix Q, key matrix K, and value matrix V are used to calculate the attention-weighted output matrix Z, using the following formula: [Calculation of attention weights] Calculate the output matrix Z = A*V, where d k It is the dimension of the key matrix K.

[0025] In one specific implementation, the multi-head attention layer is composed of multiple self-attention mechanisms. First, the input X is passed to h different self-attention layers, and h output matrices Z_i are calculated. The multi-head attention layer concatenates these output matrices together, and then passes them through a linear layer to obtain the final output Z of the multi-head attention layer.

[0026] The multi-head attention layer is followed by a residual connection and a layer normalization layer (Add&Norm layer): The Add&Norm layer consists of two parts. The residual connection Add part adds the output and input of the multi-head attention layer, and then performs layer normalization, which is calculated as follows: LayerNorm(X+MultiHeadAttention(X)), where X represents the input of the multi-head attention layer Multi-HeadAttention, and MultiHeadAttention(X) represents the output; then it is input to the pre-feedback layer.

[0027] Feedforward Layer: A network consisting of two fully connected layers. The first layer uses the ReLU activation function, calculated as max(0, XW1+b1). The second layer does not use an activation function, calculated as (max(0, XW1+b1))W2+b2. Here, X is the input matrix, and the final output matrix of the feedforward layer has the same dimensions as X; W1 is the weight matrix of the first fully connected layer, and b1 is the bias term of the first fully connected layer; W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer.

[0028] The feedforward layer is also connected to a residual connection and a layer normalization layer (Add&Norm layer): the Add&Norm layer consists of two parts, where the residual connection Add part adds the output and input of the feedforward layer, and then performs layer normalization, the calculation formula of which is as follows: LayerNorm(X+FeedForward(X)), where X represents the input of the feedforward layer and FeedForward(X) represents the output;

[0029] Layer normalization used after the multi-head attention layer and the pre-feedback layer will transform the input of each neuron into data with the same mean and variance, thus speeding up convergence.

[0030] Encoder Output: Using the multi-head attention layer, pre-feedback layer, and Add&Norm described above, an Encoderblock (a basic building block of the Transformer encoder) can be constructed. The Encoderblock receives the input matrix X and outputs a matrix. Multiple Encoderblocks stacked together form an Encoder; the input to the first Encoderblock is a sequence of single data packet representation vector matrices, the input to subsequent Encoderblocks is the output of the previous Encoderblock, and the output matrix of the last Encoderblock is the encoded information matrix C.

[0031] Step 1.4.4, Linear Layer: The function of the linear layer is to perform a linear transformation on the input matrix C (i.e., the output of the encoder in the previous step) through weights W and biases b. The calculation formula is y = W * C + b. Here, each vector x_i in C is mapped to y_i = Wx_i + b, where x_i is the i-th vector in C, W represents the weight, and b represents the bias.

[0032] Step 1.4.5, Softmax layer: The Softmax layer transforms the result y of the linear transformation W*C+b into a probability distribution such that each element y_i in y is between 0 and 1, and the sum of all elements is 1.

[0033] Step 1.4.6: Input the output vector y and the training target value into the loss function. Here, the minimum cross-entropy loss function is used to calculate the error between the model prediction and the true label.

[0034] Step 1.4.7: Take the derivative of the loss function with respect to the model parameters, and then use gradient descent or its variants to optimize the model parameters to minimize the loss function, thereby improving the model's performance on the training data.

[0035] In step 2 of this invention, the vehicle Ethernet packet data collected in real time is preprocessed and inferred in real time through intrusion detection.

[0036] Step 2.1: Read the vehicle Ethernet packet data stream in real time in the actual environment.

[0037] Step 2.2: Preprocess the read real-time data to form a message sequence arranged in chronological order for subsequent model inference.

[0038] Step 2.3: Use the trained model to infer the serialized data and determine whether the data belongs to normal traffic or a possible attack type.

[0039] In step 3 of this invention, the response unit performs intrusion alarm and / or intrusion response based on the intrusion detection result.

[0040] Specifically, after determining whether the data flow belongs to normal traffic or a possible attack type, the response unit will take corresponding measures to deal with potential security threats. When abnormal traffic is detected and the attack type is obtained, intrusion alarms and intrusion responses will be initiated.

[0041] 1) Intrusion Alert: When the detection system identifies abnormal traffic or known attack patterns, it immediately triggers an alarm mechanism. This typically includes sending alert messages to network administrators and notifying relevant personnel via email, SMS, audio, or visual cues. This ensures timely detection and response to potential security threats, enabling further defensive measures to be taken.

[0042] 2) Intrusion Response: This involves more specific countermeasures. Upon confirming an attack, the system will automatically execute a series of operations based on the set protection policies, such as traffic control, traffic filtering and blocking, and device isolation. The goal is to minimize the damage caused to the network system and restore normal operation as quickly as possible.

[0043] Based on the judgment results of the intrusion detection model in actual use testing, if abnormal behavior is detected, the system will also update the database, including the attack type. The update may be performed periodically or irregularly, depending on the situation.

[0044] The present invention further provides an intrusion detection and defense system capable of implementing the above method. The system includes several key modules: a network packet data acquisition module, a database, an intrusion detection module, and a response unit module.

[0045] The main responsibility of the network packet data acquisition module is to collect in-vehicle Ethernet packet data.

[0046] The database stores information including normal and abnormal judgment information and attack type information, and can be updated according to the intrusion detection results. If abnormal behavior is detected, the database will be modified accordingly.

[0047] The intrusion detection module is responsible for data preprocessing, model training, and model detection. Preprocessing includes data cleaning, standardization, and serialization to provide high-quality data input for subsequent processing steps. Model training involves preprocessing the database to build or periodically update a Transformer-based intrusion detection model. During this process, the model's parameters are continuously optimized to improve its accuracy and efficiency in practical applications. Model detection involves preprocessing the collected network packet data for real-time model inference to determine whether traffic is abnormal, thereby identifying potential network attacks or intrusions.

[0048] The response unit module: When the system detects an attack, this module generates a detailed detection report and triggers the corresponding intrusion alarm and / or response mechanism to ensure the security and stability of the network.

[0049] Through the coordinated operation of these modules, the intrusion detection and prevention system provided by this invention can effectively protect the vehicle network from various network attacks, ensuring the safe and stable operation of the vehicle system.

[0050] The present invention also provides the application of the above-mentioned intrusion detection and defense methods, or the above-mentioned intrusion detection and defense systems, in fields including intelligent connected vehicles, vehicle network security systems, and intelligent transportation systems.

[0051] The beneficial effects of this invention include, but are not limited to: efficient sequence data processing capabilities, accurate anomaly detection, and real-time response capabilities. Specifically, the efficient sequence data processing capability, achieved using the Transformer encoder model, excels in processing sequence data such as vehicular Ethernet packets, capturing complex temporal and spatial dependencies to effectively detect potential intrusion behaviors. Accurate anomaly detection, through a well-trained intrusion detection model, can accurately distinguish between normal network traffic and different types of attack patterns, including DoS attacks, intrusion detection, and unauthorized access. Real-time response capability, due to the efficient inference capabilities of the Transformer model, allows the system to quickly detect abnormal activity in real-time streams and issue timely alerts, facilitating rapid response when attacks occur. Adaptability and flexibility, the model can be trained and adjusted according to different vehicular network environments and specific attack methods, improving the system's adaptability and defense capabilities. Self-learning and optimization, the system can continuously optimize and update its detection algorithm to cope with emerging security threats and attack methods. Data privacy protection, this detection method does not require deep analysis of packet content but rather uses feature extraction and pattern recognition for detection, helping to protect the data privacy of users and the vehicular system.

[0052] In summary, the in-vehicle Ethernet intrusion detection and defense method based on Transformer encoders can not only improve the security and reliability of in-vehicle networks, but also effectively reduce security risks and protect critical information from attacks and unauthorized access threats. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of the system framework of the present invention.

[0055] Figure 2 This is a schematic diagram of the Transformer encoder structure of the present invention.

[0056] Figure 3 This is a schematic diagram of the process flow of the present invention.

[0057] Figure 4 This is a schematic diagram of sample data in the dataset of this invention. Detailed Implementation

[0058] The invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the specific details mentioned below, the processes, conditions, and experimental methods for implementing this invention are all common knowledge and general knowledge in the art, and this invention does not have any particular limitations.

[0059] This invention provides a method for in-vehicle Ethernet intrusion detection and defense based on a Transformer encoder. It acquires sample data from a database of attacked in-vehicle Ethernet packets via the network, preprocesses the data, and uses this preprocessed data to train an in-vehicle Ethernet intrusion detection model based on the Transformer encoder. After training, the model is used to perform real-time intrusion detection on in-vehicle Ethernet packets, determining and statistically analyzing network data types and attack types. The response unit executes intrusion alarms and / or response measures based on the detection results. This invention applies a Transformer encoder to the in-vehicle Ethernet intrusion detection process, achieving efficient sequence data processing capabilities, accurate anomaly detection capabilities, and real-time response capabilities. Furthermore, to maintain effective defense against new types of attacks, the system periodically updates the intrusion detection model. This invention also provides a system for implementing the above method, with broad application scenarios.

[0060] The training and defense process of the vehicle Ethernet intrusion detection model based on the Transformer encoder are described in detail below:

[0061] Based on the above findings, we can combine Transformer encoder detection to achieve efficient intrusion detection and defense.

[0062] Step 1: Construct an in-vehicle Ethernet intrusion detection model. Extract sample data from a pre-defined database containing numerous normal communication packets and / or packets of various attack types. Preprocess this data to train and optimize the intrusion detection model.

[0063] Step 2: Collect real vehicle Ethernet packet data in real time, preprocess and infer the data in real time by building an optimized intrusion detection model, determine whether the traffic is abnormal, and thus identify potential network attacks or intrusion behaviors.

[0064] Step 3: Based on the intrusion detection results, the response unit performs intrusion alarm and / or intrusion response.

[0065] Based on the judgment results of the intrusion detection model in actual use testing, if abnormal behavior is detected, the system will also update the database, including the attack type. The update may be performed periodically or irregularly, depending on the situation.

[0066] Example 1

[0067] The intrusion detection model in step 1 of this invention is based on a transformer encoder model, and the specific process of its construction, training, and optimization is as follows:

[0068] Step 1.1: Obtain sample data of vehicle Ethernet packets from the database. Each sample data consists of network connection features and network connection tags (normal / attack). Features are separated by commas, and features and tags are separated by commas. Example data is shown below. Figure 4 As shown.

[0069] Step 1.2, Data Preprocessing: Cleaning, Standardization, Serialization.

[0070] Step 1.2.1, Data Cleaning: Process the data samples to ensure the quality of the data used for training and detection. Cleaning includes removing irrelevant or noisy data, imputing missing values, and removing duplicate data to improve the training effect and detection accuracy of the model.

[0071] Step 1.2.2: Data Standardization Processing: Categorical features (such as protocols, service types, etc.) are tokenized using label encoding, mapping each category to a numerical value. Then, the mapped values ​​of numerical features (such as port numbers, data volume, etc.) and categorical features are normalized to ensure similar scale. Specifically:

[0072] Category feature tokenization processing is performed, such as converting the three protocol types into numeric identifiers, and converting the protocol types ('tcp', 'udp', 'icmp') in the input list into their corresponding numeric identifiers (indices). If a protocol exists in the predefined protocol list, its index is returned; otherwise, -1 is returned.

[0073] Numerical normalization is necessary to eliminate the influence of dimensions between indicators and to ensure comparability between data indicators. In a specific implementation of the method of this invention, min-max normalization is used to achieve a linear transformation of the original data, mapping the result values ​​to the range [0,1].

[0074] Step 1.2.3, Serialization Processing: Arrange the cleaned and standardized data in chronological order to form a sequence of data packets. In a specific implementation, each sequence contains a fixed 60 time steps (i.e., data packets). For sequences with fewer than 60 data packets, zero padding can be used to achieve a fixed length. For sequences with more than 60 data packets, a sliding window method is used to divide the long sequence into multiple data packet sequences of length 60 to obtain more data fragments. Each window can contain partially overlapping data, thus preserving more information and ensuring a uniform length of the input sequence. Sequence label processing: Multi-label classification method. Each sequence can have multiple labels, which are ultimately represented by a multi-hot encoded vector. The category of a label is 1 if it appears and 0 if it does not appear. For example, all labels all_labels = ['normal', 'DoS', 'Probe', 'R2L', 'U2R'], sequence labels sequence_labels = ['normal', 'DoS', 'R2L']. Output multi-hot encoded labels, such as [1.1.0.1.0.].

[0075] Step 1.3, Data partitioning: Divide the serialized data into training set, validation set and test set.

[0076] The primary purpose of splitting the dataset is for model development and evaluation. The training set is used to train the model, the validation set is used to tune the model's hyperparameters, and the test set is used to finally evaluate the model's performance. The training set comprises 80%, the validation set 10%, and the test set 10%.

[0077] Step 1.4, Model Training: Model training and optimization require the following specific steps:

[0078] Step 1.4.1 Input Embedding: Each time step (data packet) represents the data state at a fixed point in time in the input sequence. Each time step is embedded, that is, the input data packet is converted into a numerical vector that can be processed by the model.

[0079] Step 1.4.2, Location Embedding: Represented by PE, the calculation formula is as follows:

[0080] PE(pos,2i)=sin(pos / (n 2i / d ),PE(pos,2i+1)=cos(pos / (n 2i / d), where: pos: the position of the object in the input sequence, 0 ≤ pos < L - 1; d: the dimension of the output embedding space; PE(pos, j): the position function, used to map the position pos in the input sequence to the index (pos, j) of the position matrix; n: a user-defined scalar, usually chosen as 10,000 to adjust the frequency range; i: used to map the column index to a value of 0 ≤ i < d / 2, and a single i value is mapped to both the sine and cosine functions.

[0081] Step 1.4.3, Input and Output of the Encoder:

[0082] Among them,

[0083] Encoder Input: Each element x_i in the input sequence X is converted into an embedding vector. The embedding vector obtained according to the input sequence is set as xe_i, and added to the position vector PE(pos) to obtain the position encoding. The calculation formula is as follows: PC(x_i) = xe_i + PE(pos). To prevent the embedding vector xe_i from being minimized by the position vector, that is, to avoid the value of the position encoding being too large and the influence of the embedding vector almost disappearing, resulting in the model being unable to effectively learn the representation of the embedding vector, it is necessary to ensure their consistency in scale. By combining the element embedding vector and the position vector of the input sequence, ensure their numerical scale is consistent. The formula is expressed as follows: PC(x_i) = xe_i * math.sqrt(d) + PE(pos), where xe_i represents the embedding vector of the i-th element in the input sequence X, d represents the dimension of the embedding vector, and PE(pos) represents the position vector. The input X of the encoder is processed into the embedding vector (PC(x_i)) of each element, and these vectors contain the semantic information and relative position information of the elements, enabling the model to effectively process sequence data.

[0084] The encoder layer contains a multi-head attention layer, a pre-feedforward layer, and a residual connection and layer normalization layer, and outputs the input position encoding as a coding information matrix;

[0085] Among them, the self-attention mechanism is applied in the multi-head attention layer: Obtain the input matrix X representing the input sequence, and each row represents a sequence element. Through the linear transformation matrices (Qw, Kw, Vw), calculate the query matrix Q, the key matrix K, and the value matrix V as follows: Q = X * Qw, K = X * Kw, V = X * Vw, where each row of X, Q, K, and V represents a sequence. In the attention calculation, use the query matrix Q, the key matrix K, and the value matrix V to calculate the attention-weighted output matrix Z. The formula is as follows: Calculate the attention weights Calculate the output matrix Z = A * V, where d k is the dimension of the key matrix K.

[0086] In a specific implementation, a multi-head attention layer is composed of multiple self-attention mechanisms. First, the input X is passed to h different self-attention layers, and h output matrices Z_i are calculated. The multi-head attention layer concatenates these output matrices together, and then passes them through a linear layer to obtain the final output Z of the multi-head attention layer.

[0087] The multi-head attention layer is followed by a residual connection and a layer normalization layer (Add&Norm layer): The Add&Norm layer consists of two parts. The residual connection Add part adds the output and input of the multi-head attention layer, and then performs layer normalization, which is calculated as follows: LayerNorm(X+MultiHeadAttention(X)), where X represents the input of the multi-head attention layer Multi-HeadAttention, and MultiHeadAttention(X) represents the output; then it is input to the pre-feedback layer.

[0088] Feedforward Layer: A network consisting of two fully connected layers. The first layer uses the ReLU activation function, calculated as max(0, XW1+b1). The second layer does not use an activation function, calculated as (max(0, XW1+b1))W2+b2. Here, X is the input matrix, and the final output matrix of the feedforward layer has the same dimensions as X; W1 is the weight matrix of the first fully connected layer, and b1 is the bias term of the first fully connected layer; W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer.

[0089] The feedforward layer is also connected to a residual connection and a layer normalization layer (Add&Norm layer): the Add&Norm layer consists of two parts, where the residual connection Add part adds the output and input of the feedforward layer, and then performs layer normalization, the calculation formula of which is as follows: LayerNorm(X+FeedForward(X)), where X represents the input of the feedforward layer and FeedForward(X) represents the output;

[0090] Layer normalization used after the multi-head attention layer and the pre-feedback layer will transform the input of each neuron into data with the same mean and variance, thus speeding up convergence.

[0091] Encoder Output: Using the multi-head attention layer, pre-feedback layer, and Add&Norm described above, an Encoderblock can be created. The Encoderblock receives the input matrix X and outputs a matrix. Multiple Encoderblocks stacked together form an Encoder; the input to the first Encoderblock is a sequence of single data packet representation vector matrices, the input to subsequent Encoderblocks is the output of the previous Encoderblock, and the output matrix of the last Encoderblock is the encoded information matrix C.

[0092] Step 1.4.4, Linear Layer: The function of the linear layer is to perform a linear transformation on the matrix C output from the previous step using weights W and biases b. The calculation formula is y = W * C + b. Here, each vector x_i in C is mapped to y_i = Wx_i + b, where x_i is the i-th vector in C, W represents the weights, and b represents the bias.

[0093] Step 1.4.5, Softmax layer: The Softmax layer transforms the result y of the linear transformation W*C+b into a probability distribution such that each element y_i in y is between 0 and 1, and the sum of all elements is 1.

[0094] Step 1.4.6: Input the output vector y and the training target value into the loss function. Here, the minimum cross-entropy loss function is used to calculate the error between the model prediction and the true label.

[0095] Step 1.4.7: Take the derivative of the loss function with respect to the model parameters, and then use gradient descent or its variants to optimize the model parameters to minimize the loss function, thereby improving the model's performance on the training data.

[0096] The in-vehicle Ethernet intrusion detection model built based on the Transformer encoder is as follows: Figure 2As shown. Parameter settings: Number of Transformer layers is set to 8, Number of Attention Heads is set to 16, Embedding Dimension is set to 256, Feedforward Network Dimension is set to 1024, Dropout Rate is set to 0.3, Learning Rate is set to 0.0001, and Batch Size is set to 64. Parameter optimization uses Bayesian optimization, which evaluates different hyperparameter combinations by building a probabilistic model, enabling more intelligent exploration of optimal parameters. For computationally expensive models like the Transformer, this method can find the optimal hyperparameters with limited computational resources. Model training and performance evaluation metrics include Accuracy, Precision, Recall, and F1 Score. To optimize parameters and performance, and to prevent overfitting, the model can be stopped early and training can be stopped. The generalization ability of the model can be further improved by adjusting the dropout rate, and the model that performs best on the test set can be found.

[0097] The detection performance of different intrusion detection models on the test set is shown in Table 1 below:

[0098] Table 1. Detection performance of different intrusion detection models on the test set.

[0099]

[0100] As shown in Table 1, the Transformer encoder achieves an accuracy of 98.5%, significantly outperforming SVM's 85.2%, and surpasses RNN and LSTM (92.3% and 95.0%, respectively) in complex time-series data. This indicates that Transformer can better capture the time-series patterns of automotive Ethernet traffic and effectively distinguish between normal traffic and various attack behaviors. The Transformer encoder's 96% precision and 97.5% recall rate outperform other models, demonstrating its high reliability in identifying abnormal traffic. High precision ensures low false positives for normal data streams, while high recall guarantees comprehensive coverage of potential anomalies. The model effectively reduces false positives and false negatives, providing accurate and stable intrusion detection for automotive systems. The high F1 score reflects the model's accuracy and rapid response capability.

[0101] Example 2

[0102] In step 2 of this invention, the real-time data of the read vehicle Ethernet packets is preprocessed and intrusion detection is performed.

[0103] Step 2.1: Read the vehicle Ethernet packet data stream in real time.

[0104] Step 2.2: Preprocess the read real-time data to form a message sequence arranged in chronological order for subsequent model inference.

[0105] Step 2.3: Use the trained model to infer the serialized data and determine whether the data belongs to normal traffic or a possible attack type.

[0106] In the online detection phase, the trained intrusion detection model is used to perform Ethernet packet detection. Based on the classification results (detection results) of the intrusion detection model, it is determined whether the current vehicle operation status has been subjected to malicious information security attacks. This embodiment uses Jetson Xavier NX for real-time testing. The total intrusion detection time includes two parts: data preprocessing time and model inference time. Table 2 calculates the total time for the average single packet sequence, attack type, model confidence, and model score.

[0107] Table 2 below shows the detection performance of different intrusion detection models for different attack types during the online phase:

[0108] Table 2. Detection performance of different intrusion detection models for different attack types.

[0109]

[0110]

[0111] As shown in Table 2, the Transform encoder exhibits the highest model confidence, especially in detecting normal traffic and various attacks (e.g., a confidence level of 99.0% for attack-free traffic). This indicates that the Transform encoder has strong sensitivity and accuracy in anomaly detection. The SVM, RNN, and LSTM models generally have lower confidence, especially in complex attack combinations (e.g., DoS+Probe+U2R), where SVM has the lowest confidence at only 79.0%, suggesting that traditional models may be limited in handling complex attacks. Regarding total latency, the Transform encoder's total latency is significantly lower than other models, especially when processing simple and normal traffic (e.g., the total latency for normal traffic is 11.0 ms). This makes it more advantageous in real-time applications. SVM also performs well in terms of total latency, but compared to the Transform encoder, its latency increases significantly when processing complex attacks (e.g., the latency for DoS+Probe+U2R is 18.0 ms). The overall latency of RNNs and LSTMs is generally high, especially when dealing with complex attacks (LSTM latency can reach 19.5ms), which may limit their application in real-time detection. Therefore, the Transform encoder can effectively handle a variety of attack types and maintain high accuracy and low latency in complex attack scenarios, demonstrating good adaptability.

[0112] Example 3

[0113] In the implementation of this invention, in step 3, if abnormal behavior is detected based on the judgment result of the intrusion detection model, the system updates the database.

[0114] If the system detects a significant anomaly in network traffic at a certain intersection, with frequent high-volume data packets and most packet characteristics matching the pattern of a DDoS (Distributed Denial-of-Service) attack, the intrusion detection model classifies this traffic as abnormal and identifies a potential attack threat. The system automatically generates anomaly event records for this DDoS attack in the database as data for subsequent analysis, supporting model optimization and improving the model's ability to identify similar attacks.

[0115] This concludes the implementation of the automotive Ethernet intrusion detection and defense solution for the Transformer encoder.

[0116] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0117] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0121] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.

Claims

1. A method for vehicle-mounted Ethernet intrusion detection and prevention based on a Transformer encoder, characterized in that, It includes the following steps: Step 1: Construct a vehicle-mounted Ethernet intrusion detection model, extract sample data from a pre-set database containing normal communication packets and / or attack type packets, and optimize and train the intrusion detection model; Step 2: Real-time collect real vehicle-mounted Ethernet packet data, perform preprocessing and real-time inference on the real data through the constructed optimized intrusion detection model, judge whether the traffic is abnormal, and identify potential network attacks or intrusion behaviors; Step 3: According to the intrusion detection results, the response unit conducts intrusion alarms and / or intrusion responses.

2. The method according to claim 1, characterized in that, In Step 1, each vehicle-mounted Ethernet packet sample data in the database includes network connection features and corresponding network connection labels, separated by commas; perform cleaning, standardization, and serialization preprocessing on the vehicle-mounted Ethernet packet sample data, and divide the preprocessed data into data sets; The cleaning includes removing irrelevant or noise data, filling in missing values, and removing duplicate data; The standardization refers to performing tokenization conversion on categorical features, using label encoding to map each category into a numerical value; then normalizing the numerical features and the mapped values of categorical features; the categorical features include protocols and service types, and the numerical features include port numbers and data volumes; The serialization refers to arranging the data after cleaning and standardization in chronological order to form a data packet sequence; Use the multi-label classification method to mark whether the sequence data is normal, and label the attack types for the sequence data marked as abnormal.

3. The method according to claim 1, characterized in that, In Step 1, the vehicle-mounted Ethernet intrusion detection model includes an input embedding layer, a position embedding layer, an encoder layer, a linear layer, and a softmax layer; 4. The method according to claim 3, characterized in that, Convert the vehicle-mounted Ethernet packet data into a serialized embedding vector through the input embedding layer and the position embedding layer, capture the long-distance dependencies in the data through the self-attention mechanism and the multi-head self-attention mechanism, and further extract features using the pre-feedforward layer; the vehicle-mounted Ethernet intrusion detection model gradually refines the deep-level information through one or more encoder layers, finally performs category mapping in the linear layer, and uses the Softmax layer to generate a probability distribution for classification. In the input embedding layer, each element in the input sequence X is converted into an embedding vector; and / or PE(pos,2i)=sin(pos / (n 2i / d )),PE(pos,2i+1)=cos(pos / (n 2i / d )), In the position embedding layer, the position embedding is represented by PE, and the calculation formula is as follows: where pos represents the position of the object in the input sequence, 0 ≤ pos < L - 1, L is the sequence length; d represents the dimension of the output embedding space; PE(pos, j) represents the position function, which is used to map the position pos in the input sequence to the index (pos, j) of the position matrix; n represents a user-defined scalar; i represents the value used to map the column index to 0 ≤ i < d / 2, and a single i value is simultaneously mapped to the sine and cosine functions; and / or Add the embedding vector and the position vector to obtain the position encoding, denoted as PC(x_i) = xe_i + PE(pos), and input it into the encoder layer; where xe_i represents the embedding vector and PE(pos) represents the position vector; and / or The encoder layer includes a multi-head attention layer, a pre-feedback layer, a residual connection, and a layer normalization layer, which encodes the input position and outputs an encoded information matrix. And / or, The linear layer performs a linear transformation on the input encoded information matrix through weights W and bias b; And / or, The Softmax layer transforms the linear transformation result into a probability distribution such that each element y_i in the linear transformation result is between 0 and 1, and the sum of all elements is 1.

5. The method according to claim 4, characterized in that, The multi-head attention layer contains one or more self-attention layers. The output matrices of the input X in multiple different self-attention layers are concatenated and passed through a linear layer to obtain the final output of the multi-head self-attention layer. And / or, The pre-feedback layer consists of two fully connected layers. The first layer uses the ReLU activation function, while the second layer does not. It performs further nonlinear transformation and feature extraction on the data processed by the multi-head attention layer, represented as (max(0,XW1+b1))W2+b2; where X is the input matrix, and the final output matrix of the pre-feedback layer has the same dimension as X; where W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer; W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer. And / or, The residual connection and layer normalization layer add the outputs and inputs of the multi-head attention layer and the front feedback layer, respectively, and then perform layer normalization, as shown in the following formula: LayerNorm(X+MultiHeadAttention(X)), LayerNorm(X+FeedForward(X)), In this context, LayerNorm(X+MultiHeadAttention(X)) represents adding the input X of the multi-head attention layer to its output and then normalizing the result; LayerNorm(X+FeedForward(X)) represents adding the input X of the feedforward layer to its output and then normalizing the result.

6. The method according to claim 1, characterized in that, The minimum cross-entropy loss function is used to predict the error between the model and the true label, and the model's performance on the training data is improved by taking the derivative.

7. The method according to claim 1, characterized in that, In step 2, the real-time read vehicle Ethernet packet data is preprocessed and inferred in real time; And / or, In step 3, the response unit refers to the unit that, after determining whether the data flow belongs to normal traffic and / or a possible attack type, will take corresponding measures to deal with possible security threats, including intrusion alarms and / or intrusion responses. When abnormal traffic or known attack patterns are detected, an alarm mechanism is triggered to trigger an intrusion alarm. After confirming the attack, protection strategies including traffic control, traffic filtering and blocking, and device isolation are automatically executed to respond to the intrusion.

8. The method according to claim 1, characterized in that, When the intrusion detection model detects abnormal behavior, it updates the database, including the attack type.

9. An intrusion detection and prevention system implementing the method as described in any one of claims 1-8, characterized in that, The system includes: a network packet data acquisition module, a database, an intrusion detection module, and a response unit module; The network packet data acquisition module is responsible for collecting in-vehicle Ethernet packet data; The database stores information including normal / abnormal judgment information and attack type information, and can be updated. The intrusion detection module is responsible for data preprocessing, model training, and model detection; it continuously updates and optimizes the intrusion detection model, and after preprocessing the vehicle Ethernet packet information, it judges and checks whether the network traffic is normal and / or the attack type. The response unit module is used to generate detection reports and trigger intrusion alarms and / or response mechanisms.

10. The application of the method according to claims 1-8 or the system according to claim 9 in fields including intelligent connected vehicles, vehicle network security systems, and intelligent transportation systems.

Images