Transform-based self-supervised vehicle-mounted intrusion detection method

Abstract

The invention relates to the technical field of Internet of Vehicles network security and intelligent intrusion detection, and provides a self-supervision vehicle-mounted intrusion detection method based on Transform. Preprocessing the vehicle-mounted network flow data and constructing a time sequence feature sequence by adopting a sliding window mode; carrying out self-supervision pre-training on normal traffic by using an auto-encoder so as to learn spatial-temporal feature representation of vehicle-mounted network communication; introducing a multi-task joint optimization mechanism of multi-classification detection, dichotomy detection and comparative learning, and performing fine tuning training on the model by using a small amount of labeled samples; and realizing anomaly detection and attack type identification of the vehicle-mounted network traffic through the trained model. According to the method, dependence on attack annotation data is remarkably reduced, the long-range time sequence dependency relationship in vehicle-mounted network communication is effectively modeled, the accuracy, robustness and generalization ability of intrusion detection are improved, and the method is suitable for complex and dynamic vehicle networking safety protection scenes.

Description

A Transformer-based self-supervised vehicle intrusion detection method Technical Field

[0001] This invention relates to the fields of network security and machine learning technology, specifically to a self-supervised vehicle intrusion detection method based on Transformer. Background Technology

[0002] With the rapid development of the Internet of Vehicles (IoV), vehicles, roadside units, and cloud platforms are highly interconnected via wireless and in-vehicle communication networks, providing crucial support for intelligent transportation and autonomous driving. However, the Controller Area Network (CAN) bus commonly used in in-vehicle networks was designed primarily for real-time performance and reliability, lacking security mechanisms such as authentication, data encryption, and access control. This makes them vulnerable to various cybersecurity threats in open, heterogeneous communication environments, including denial-of-service attacks, forgery attacks, and data tampering. Therefore, deploying an Intrusion Detection System (IDS) has become a key technological means to ensure the security and reliability of the IoV.

[0003] Existing intrusion detection technologies for vehicular networks mainly include rule-based methods and data-driven methods. Among them, machine learning-based intrusion detection methods are widely studied and applied due to their ability to handle large-scale network traffic data. Related research typically achieves attack identification by manually designing features and combining them with classifiers, such as weighting or statistically modeling key traffic features to improve detection performance. However, this type of method relies heavily on manual feature engineering, making it difficult to adapt to the constantly evolving and diversified attack patterns in vehicular networks. It also increases the complexity of data preprocessing and limits the system's generalization ability.

[0004] To overcome these problems, researchers have further introduced deep learning techniques, utilizing neural networks for automatic feature extraction. Some intrusion detection models based on convolutional neural networks (CNNs) or recurrent neural networks (RNNs) can learn the spatial or temporal features of traffic data to some extent, improving detection accuracy. However, existing methods mostly focus on static data packets or flow-level features, and are insufficient in modeling the inherent temporal correlations and long-term dependencies in vehicular network communication; in particular, models based on long short-term memory networks (LSTMs) still suffer from problems such as vanishing gradients, slow training convergence speed, and difficulty in parallel computing, making it difficult to meet the real-time and efficiency requirements of vehicular network applications.

[0005] In recent years, Transformer-based intrusion detection methods have attracted attention. These methods utilize self-attention mechanisms to process sequential data in parallel and effectively model long-range dependencies, offering advantages in temporal feature learning and computational efficiency. However, existing Transformer-based IDS typically rely on large amounts of high-quality labeled data for supervised training. In real-world connected vehicle environments, acquiring attack samples and manually labeling them is costly and difficult, severely hindering their engineering applications. Furthermore, some existing methods fail to fully integrate mechanisms such as contrastive learning to improve the model's robustness and adaptability in complex, dynamic network environments.

[0006] In summary, existing vehicular network intrusion detection technologies still have significant shortcomings in terms of labeled data dependence, temporal feature modeling capabilities, and model adaptability. There is an urgent need for an intrusion detection technology solution that can effectively learn the spatiotemporal features of vehicular networks under conditions of a small number of labeled samples, and has high detection accuracy and robustness. Summary of the Invention

[0007] This invention proposes a self-supervised vehicle intrusion detection method and system based on Transformer to solve the above-mentioned problems. This invention adopts a two-stage learning paradigm of "data preprocessing—self-supervised pre-training—contrastive multi-task fine-tuning—intrusion detection inference," models the temporal features of vehicle network communication data through a self-attention mechanism, and combines self-supervised learning and contrastive learning to achieve high-precision and robust intrusion detection with only a small number of labeled samples.

[0008] The self-supervised vehicle intrusion detection system based on Transformer provided by this invention includes a data preprocessing module, a Transformer-based autoencoder pre-training module, a contrastive multi-task fine-tuning module, and an intrusion detection inference module. These modules work together to achieve real-time security protection for vehicle networks.

[0009] The self-supervised vehicle intrusion detection method based on Transformer provided by this invention includes the following steps:

[0010] (i) Preprocess the collected vehicle network traffic data, including outlier cleaning, missing value removal and numerical normalization, to ensure the numerical stability of the data.

[0011] (ii) The raw traffic data is converted into time-series samples using a sliding window serialization method. Let the sliding window length be L and the step size be s. By segmenting continuous traffic records, a time-ordered data sequence is constructed:

[0012]

[0013] Where B is the batch size, L is the sequence length, and D is the feature dimension of a single traffic stream. This serialization representation can explicitly preserve the temporal dependencies in vehicular network communication, providing a foundation for subsequent temporal feature modeling.

[0014] During the pre-training phase, the training set contains only normal traffic data; during the fine-tuning phase, a small number of labeled attack samples are introduced for supervised optimization.

[0015] (III) Self-supervised pre-training method based on Transformer:

[0016] 1. Transformer autoencoder structure

[0017] This invention employs a Transformer-based autoencoder as the core structure for feature learning. For the input sequence, it is first mapped to a high-dimensional embedding space through linear mapping and positional encoding:

[0018]

[0019] in, For embedding matrix, It is a learnable location encoding.

[0020] The Transformer encoder consists of N stacked encoding blocks, each containing a multi-head self-attention module and a feedforward network (FFN). The structure of a single Transformer encoder block is shown in Figure 1. For the nth block, the multi-head self-attention (MHA) mechanism aggregates contextual information from multiple representation subspaces. The computational form of MHA is as follows:

[0021]

[0022] Where h is the number of attention heads. This is the output projection matrix, used to aggregate the outputs of all heads. Each independent attention head captures different aspects of the dependencies in the sequence. The output of the i-th head is calculated as follows:

[0023]

[0024] in It is a projection of the query (Q), key (K), and value (V), each represented by a learnable projection matrix. Definition. The scaling dot product attention mechanism is defined as follows:

[0025]

[0026] scaling factor It was introduced to prevent large values ​​from causing the dot product to become too large, thus keeping the gradient of the softmax operation within a reasonable range.

[0027] The attention output is then processed by a normalization layer following a residual connection. Subsequently, a two-layer feedforward network further transforms the intermediate representation:

[0028]

[0029] in This represents the output of the multi-head attention sublayer, and FFN(·) represents a two-layer feedforward network containing ReLU activations. The final output of the nth block of the encoder. It captures the comprehensive temporal dependencies and statistical patterns of the input sequence, which are then mapped back to the original feature space for sequence reconstruction via a linear projection layer.

[0030] 2. Self-supervised objectives based on reconstruction error

[0031] During the pre-training phase, only benign traffic sequences are used to learn representations of normal network behavior. The reconstruction objective is to minimize the mean squared error (MSE) between the original input sequence and its reconstructed output.

[0032]

[0033] Where X represents the input sequence, The reconstructed output is represented by L, where L is the sequence length and D is the feature dimension. This reconstruction goal encourages the encoder to capture, but not amplify, informative representations in benign traffic patterns, thereby achieving a faithful reconstruction of normal network behavior.

[0034] Through this reconstruction task, the encoder learns salient spatiotemporal features representing normal behavior, providing a solid foundation for the subsequent fine-tuning stage. The pre-trained encoder weights are then transferred to initialize the downstream multi-task model, thereby achieving effective knowledge transfer from unsupervised pre-training to supervised learning.

[0035] (iv) Comparison of multi-task fine-tuning methods:

[0036] After completing self-supervised pre-training, the Transformer encoder parameters obtained from the pre-training are transferred to the downstream detection model to build a multi-task joint optimization framework, which simultaneously performs multi-class detection, binary detection and contrastive learning tasks.

[0037] 1. Multi-class classification detection task

[0038] For the specific attack type identification problem, Focal Loss is adopted as the multi-class loss function. Unlike the traditional cross-entropy loss, Focal Loss reduces the focus on easily classified samples, concentrating the training process on difficult samples. Its definition is:

[0039]

[0040] Where N is the batch size, It is the true class prediction probability of the i-th sample. and It is an adjustable weighting parameter used to emphasize difficult samples.

[0041] 2. Binary anomaly detection task

[0042] Binary classification, as an auxiliary objective, is used to distinguish between benign and malicious traffic, and can also achieve generalized identification of unknown attacks. Its loss function is defined as:

[0043]

[0044] in, It is a binary classification of real labels. It represents the probability of being predicted as an anomaly. This auxiliary task plays a crucial role in maintaining robustness against new variant attacks.

[0045] 3. Comparison of learning tasks

[0046] The contrastive learning objective encourages representations of similar attack samples to converge in the latent space while distancing samples of different classes. The contrastive learning objective is introduced to enhance feature discriminativeness, and its loss function is:

[0047]

[0048] in, Represents the cosine similarity after L2-normalized projection. This represents the set of positive samples (an enhanced view corresponding to sample i). This is a temperature parameter. The objective is to encourage semantically similar attack samples to form compact clusters in the embedding space.

[0049] 4. Joint Optimization Objective

[0050] The objectives mentioned above aim to distinguish specific attack categories while retaining sensitivity to coarse-grained anomalies. The multi-task learning framework jointly optimizes the three objectives during the fine-tuning phase, and the overall optimization objective function is defined as follows:

[0051]

[0052] in , and Used to balance the contributions of each task.

[0053] (v) Intrusion detection reasoning process:

[0054] During the inference phase, the trained model encodes the input traffic sequence to obtain a contextual representation. This representation is then passed to two task-specific classification heads: a binary head (for coarse-grained anomaly detection) and a multi-class head (for fine-grained attack classification). This hierarchical design supports both fine-grained attack identification and generalized anomaly detection, providing flexibility for different security scenarios. Furthermore, the confidence scores of the binary outputs serve as supplementary evidence, enhancing the reliability and robustness of the final security decision. This two-layer decision mechanism improves the system's reliability in detecting unknown attacks and boundary samples, and enhances the interpretability of the results.

[0055] The present invention has the following advantages over the prior art:

[0056] 1. This invention first preprocesses the vehicle network traffic data and constructs a temporal feature sequence using a sliding window approach; then, it uses a Transformer autoencoder based on a self-attention mechanism to perform self-supervised pre-training on normal traffic to learn the spatiotemporal feature representation of vehicle network communication; on this basis, it introduces a multi-task joint optimization mechanism of multi-class detection, binary detection, and contrastive learning, and uses a small number of labeled samples to fine-tune the model; finally, the trained model is used to achieve anomaly detection and attack type identification of vehicle network traffic.

[0057] 2. While significantly reducing the dependence on attack annotation data, this invention can effectively model long-range temporal dependencies in vehicular network communication, improve the accuracy, robustness and generalization ability of intrusion detection, and is suitable for complex and dynamic vehicle network security protection scenarios. Attached Figure Description

[0058] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0059] Figure 1 is a general framework diagram of the method of the present invention;

[0060] Figure 2 is a schematic diagram of the structure of a single Transformer encoder block;

[0061] Figures 3a-d are schematic diagrams of the confusion matrix of the classification results of the method of the present invention in the intrusion detection task of vehicle network. Figures 3a and 3b show the binary classification and multi-class classification detection results on the NSL-KDD dataset, respectively, and Figures 3c and 3d show the binary classification and multi-class classification detection results on the CICIDS2018 dataset, respectively.

[0062] Figures 4a-b are visualizations of the distribution of vehicular network traffic features learned by the method of this invention. Figures 4a and 4b show the feature distribution on test samples of the NSL-KDD and CICIDS2018 datasets, respectively, with different categories of samples forming relatively independent clustering structures in the feature space. Detailed Implementation

[0063] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1:

[0065] This embodiment provides a Transformer-based self-supervised vehicle intrusion detection method, as shown in Figures 1 and 2. Figure 1 shows the overall architecture and data processing flow of the Transformer-based contrastive self-supervised learning vehicle network intrusion detection method proposed in this paper, which fully embodies the training and inference mechanism of the model from vehicle network data collection to final attack type identification.

[0066] First, in the data source and input phase, the system collects CAN bus data generated during vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication processes in the vehicle-to-everything (V2X) environment. This data reflects the communication behavior of vehicles in real traffic scenarios and serves as the foundational input for subsequent intrusion detection.

[0067] Subsequently, in the data preprocessing stage, the raw CAN bus data is normalized sequentially to eliminate differences in the dimensions of different features; then, the continuous data is segmented through a sliding window mechanism, and a flow feature sequence with a time order relationship is constructed, thereby providing a time-series input for the Transformer model.

[0068] In the self-supervised pre-training phase, the model utilizes a large amount of unlabeled vehicular network traffic data to pre-train a Transformer-based autoencoder. This autoencoder encodes the input sequence through a self-attention mechanism, and the decoder reconstructs the input data. By minimizing the reconstruction error, it learns a general temporal feature representation of normal communication behavior in vehicular networks, providing a high-quality feature foundation for subsequent detection tasks.

[0069] After pre-training is complete, the fine-tuning and weight sharing phase begins. The Transformer encoder parameters obtained from the pre-training are transferred to the downstream intrusion detection task and combined with the shared autoencoder structure. This allows for model fine-tuning training using only a small amount of labeled data, thus significantly reducing the reliance on manually labeled data.

[0070] In the multi-task joint learning phase, the features output by the shared encoder are input to different task branches: on the one hand, a binary classification multilayer perceptron (MLP) is used to distinguish between normal traffic and attack traffic; on the other hand, a multi-class MLP is used to identify specific attack types. Simultaneously, a projection layer is introduced and a contrastive loss is calculated to enhance the discriminative power between different categories of features through a contrastive learning mechanism, thereby improving the model's ability to detect complex and covert attacks.

[0071] Finally, in the results output stage, the model can simultaneously output binary classification results (normal or attack) and multi-classification results, including various attack types such as Probe, R2L, DoS, Brute Force, Infiltration and Web Attack, to achieve refined detection and identification of intrusion behavior in vehicle networks.

[0072] The method includes the following steps:

[0073] (i) Preprocess the collected vehicle network traffic data, including outlier cleaning, missing value removal and numerical normalization, to ensure the numerical stability of the data.

[0074] (ii) The raw traffic data is converted into time-series samples using a sliding window serialization method. Let the sliding window length be L and the step size be s. By segmenting continuous traffic records, a time-ordered data sequence is constructed:

[0075]

[0076] Where B is the batch size, L is the sequence length, and D is the feature dimension of a single traffic stream. This serialization representation can explicitly preserve the temporal dependencies in vehicular network communication, providing a foundation for subsequent temporal feature modeling.

[0077] During the pre-training phase, the training set contains only normal traffic data; during the fine-tuning phase, a small number of labeled attack samples are introduced for supervised optimization.

[0078] (III) Self-supervised pre-training method based on Transformer:

[0079] 1. The Transformer autoencoder structure is used for self-supervised reconstruction learning of the temporal features of vehicular network traffic, thereby extracting the spatiotemporal feature representation of normal communication behavior of vehicular networks.

[0080] This invention employs a Transformer-based autoencoder as the core structure for feature learning. For the input sequence, it is first mapped to a high-dimensional embedding space through linear mapping and positional encoding:

[0081]

[0082] in, For embedding matrix, It is a learnable location encoding.

[0083] The Transformer encoder consists of N stacked encoding blocks, each containing a multi-head self-attention module and a feedforward network (FFN). The structure of a single Transformer encoder block is shown in Figure 1. For the nth block, the multi-head self-attention (MHA) mechanism aggregates contextual information from multiple representation subspaces. The computational form of MHA is as follows:

[0084]

[0085] Where h is the number of attention heads. This is the output projection matrix, used to aggregate the outputs of all heads. Each independent attention head captures different aspects of the dependencies in the sequence. The output of the i-th head is calculated as follows:

[0086]

[0087] in It is a projection of the query (Q), key (K), and value (V), each represented by a learnable projection matrix. Definition. The scaling dot product attention mechanism is defined as follows:

[0088]

[0089] scaling factor It was introduced to prevent large values ​​from causing the dot product to become too large, thus keeping the gradient of the softmax operation within a reasonable range.

[0090] The attention output is then processed by a normalization layer following a residual connection. Subsequently, a two-layer feedforward network further transforms the intermediate representation:

[0091]

[0092] in This represents the output of the multi-head attention sublayer, and FFN(·) represents a two-layer feedforward network containing ReLU activations. The final output of the nth block of the encoder. It captures the comprehensive temporal dependencies and statistical patterns of the input sequence, which are then mapped back to the original feature space for sequence reconstruction via a linear projection layer.

[0093] 2. Self-supervised objectives based on reconstruction error

[0094] During the pre-training phase, only benign traffic sequences are used to learn representations of normal network behavior. The reconstruction objective is to minimize the mean squared error (MSE) between the original input sequence and its reconstructed output.

[0095]

[0096] Where X represents the input sequence, The reconstructed output is represented by L, where L is the sequence length and D is the feature dimension. This reconstruction goal encourages the encoder to capture, but not amplify, informative representations in benign traffic patterns, thereby achieving a faithful reconstruction of normal network behavior.

[0097] Through this reconstruction task, the encoder learns salient spatiotemporal features representing normal behavior, providing a solid foundation for the subsequent fine-tuning stage. The pre-trained encoder weights are then transferred to initialize the downstream multi-task model, thereby achieving effective knowledge transfer from unsupervised pre-training to supervised learning.

[0098] (iv) Comparison of multi-task fine-tuning methods:

[0099] After completing self-supervised pre-training, the Transformer encoder parameters obtained from the pre-training are transferred to the downstream detection model to build a multi-task joint optimization framework, which simultaneously performs multi-class detection, binary detection and contrastive learning tasks.

[0100] 1. Multi-class classification detection task

[0101] For the specific attack type identification problem, Focal Loss is adopted as the multi-class loss function. Unlike the traditional cross-entropy loss, Focal Loss reduces the focus on easily classified samples, concentrating the training process on difficult samples. Its definition is:

[0102]

[0103] Where N is the batch size, It is the true class prediction probability of the i-th sample. and It is an adjustable weighting parameter used to emphasize difficult samples.

[0104] 2. Binary anomaly detection task

[0105] Binary classification, as an auxiliary objective, is used to distinguish between benign and malicious traffic, and can also achieve generalized identification of unknown attacks. Its loss function is defined as:

[0106]

[0107] in, It is a binary classification of real labels. It represents the probability of being predicted as an anomaly. This auxiliary task plays a crucial role in maintaining robustness against new variant attacks.

[0108] 3. Comparison of learning tasks

[0109] The contrastive learning objective encourages representations of similar attack samples to converge in the latent space while distancing samples of different classes. The contrastive learning objective is introduced to enhance feature discriminativeness, and its loss function is:

[0110]

[0111] in, Represents the cosine similarity after L2-normalized projection. This represents the set of positive samples (an enhanced view corresponding to sample i). This is a temperature parameter. The objective is to encourage semantically similar attack samples to form compact clusters in the embedding space.

[0112] 4. Joint Optimization Objective

[0113] The objectives mentioned above aim to distinguish specific attack categories while retaining sensitivity to coarse-grained anomalies. The multi-task learning framework jointly optimizes the three objectives during the fine-tuning phase, and the overall optimization objective function is defined as follows:

[0114]

[0115] in , and Used to balance the contributions of each task.

[0116] (v) Intrusion detection reasoning process:

[0117] During the inference phase, the trained model encodes the input traffic sequence to obtain a contextual representation. This representation is then passed to two task-specific classification heads: a binary head (for coarse-grained anomaly detection) and a multi-class head (for fine-grained attack classification). This hierarchical design supports both fine-grained attack identification and generalized anomaly detection, providing flexibility for different security scenarios. Furthermore, the confidence scores of the binary outputs serve as supplementary evidence, enhancing the reliability and robustness of the final security decision. This two-layer decision mechanism improves the system's reliability in detecting unknown attacks and boundary samples, and enhances the interpretability of the results.

[0118] Example 2:

[0119] This embodiment uses two publicly available intrusion detection benchmark datasets to verify the method provided in Embodiment 1, namely the NSL-KDD dataset and the CICIDS2018 dataset.

[0120] The NSL-KDD dataset contains 42-dimensional network traffic features, and the attack types mainly include denial-of-service (DoS) attacks, probe attacks, remote-to-local (R2L) attacks, and user-to-root (U2R) attacks, which are used to verify the detection capabilities of this invention for traditional vehicular network attack scenarios.

[0121] The CICIDS2018 dataset contains 77-dimensional time-series traffic features, covering various complex attack types such as DDoS, botnet attacks, brute force attacks, web attacks, and infiltration attacks. It is suitable for verifying the time-series modeling capability and generalization performance of the present invention in real complex vehicle network environments.

[0122] By combining the two types of datasets mentioned above, the detection performance of this invention under different attack complexities and feature dimensions can be comprehensively evaluated.

[0123] The experimental environment and model parameters are as follows:

[0124] In this embodiment, the Transformer-based contrastive self-supervised learning intrusion detection model runs on a CPU platform and is implemented using the TensorFlow deep learning framework.

[0125] The structural parameters of the Transformer encoder are set as follows:

[0126] • The number of encoding layers is set to 1 to 4;

[0127] • The number of hidden units is set to 64 to 256;

[0128] • The number of heads for multi-head self-attention is set to 2 to 8;

[0129] • The location encoding method is learnable location encoding;

[0130] • The Dropout ratio is set to be between 0.1 and 0.4.

[0131] Key hyperparameters of the model include embedding dimension, attention width, feedforward network dimension, and learning rate (10). -5 Up to 5×10 -4 The γ parameter of Focal Loss and the contrast learning temperature parameter τ were both optimized and selected using an automatic hyperparameter search tool.

[0132] During model training, the Adam optimizer was used with a batch size of 32 and a maximum training epoch of 50. Early stopping and an adaptive learning rate scheduling strategy were employed to ensure stable convergence of the model. The model parameters that minimized the multi-class classification loss on the validation set were used as the final detection model.

[0133] The intrusion detection performance test results are as follows:

[0134] Under the experimental environment and parameter configurations described above, binary classification and multi-class classification detection experiments were conducted on the NSL-KDD and CICIDS2018 datasets, respectively.

[0135] On the NSL-KDD dataset, the method of this invention achieves recognition accuracy of approximately 97.40% and 98.92% for normal traffic and attack traffic, respectively, in the binary classification detection task (Figure 3a). In the multi-class detection task, various attacks show a clear diagonal distribution in the confusion matrix, with low false positive and false negative rates, indicating that the model can effectively distinguish different attack types (Figure 3b).

[0136] On the CICIDS2018 dataset, the method of this invention achieved an accuracy of approximately 100% for normal traffic and 99.56% for attack traffic in the binary classification detection task (Figure 3c); in the multi-class detection task, it has high recognition accuracy for each attack category, especially in the few-sample attack categories (such as penetration attacks and web attacks) it still maintains good detection performance (Figure 3d).

[0137] The above results demonstrate that the method of the present invention can run stably on datasets of varying complexity and achieve high-precision intrusion detection.

[0138] The following is an example of verifying the feature representation effect:

[0139] To further verify the discriminative ability of the features learned by the present invention, this embodiment performs a visual analysis of the feature representation output by the model.

[0140] The test set features were analyzed using dimensionality reduction visualization methods, and the results are shown in Figures 3a-b: different attack categories form compact and mutually separated cluster structures in the feature space, and there is a clear boundary between normal traffic and attack traffic. In particular, in the CICIDS2018 dataset, minority class attacks can be effectively distinguished from a large number of normal traffic samples.

[0141] The results show that by introducing self-supervised learning and contrastive learning mechanisms, the present invention can learn network traffic feature representations with good semantic discriminative ability.

[0142] Example 3:

[0143] This embodiment comprehensively compares the method proposed in Embodiment 1 with several state-of-the-art methods. Table 1 shows the detection accuracy and related evaluation metrics of each method in intrusion detection tasks, illustrating the improvement of the present invention in terms of detection accuracy, robustness, and overall performance compared to existing technologies. The proposed method demonstrates highly competitive performance in both binary and multi-class classification tasks, consistently outperforming the comparison methods in terms of accuracy and F1 score. On the NSL-KDD dataset, the proposed method achieves 98.19% accuracy and 97.57% F1 score in multi-class classification tasks, significantly outperforming recent methods such as Res-TCN (92.30%), IG (93.89%), and MLP (89.55%). Although the Transformer model achieves a slightly higher accuracy of 99.52% in binary classification tasks, the proposed method still maintains 98.27% accuracy and a considerably high F1 score using significantly less labeled data. On the more challenging CICIDS2018 dataset, the proposed method performs even better, achieving 99.85% and 99.80% accuracy in binary and multi-class classification tasks, respectively, and near-perfect 99.96% accuracy in the multi-class scenario. These results surpass comparable methods such as Graph2vec+RF (99.63%), SDL+RF (94.02%), CNN+LSTM (98.53%), and Res-TCN (99.66%). The method demonstrates excellent consistency across both datasets—especially in the CICIDS2018 multi-class classification task, fully demonstrating its effectiveness, robustness, and reliability in intrusion detection.

[0144] Table 1: Performance comparison results of the method of the present invention and existing vehicle network intrusion detection methods on different datasets.

[0145]

[0146] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation fall within the protection scope claimed by the present invention.

Claims

1. A self-supervised vehicle intrusion detection method based on Transformer, characterized in that, The process includes the following steps: S1. Acquire vehicle network traffic data, clean and normalize the traffic data, and construct a traffic feature sequence with a temporal order by using a sliding window method to divide continuous traffic records; S2. Based on the traffic feature sequence, construct a Transformer-based autoencoder model, and perform self-supervised pre-training on the autoencoder using only normal vehicle network traffic data. By minimizing the reconstruction error between the input sequence and the reconstructed sequence, the spatiotemporal feature representation of normal vehicle network communication behavior is learned. S3. Transfer the Transformer encoder parameters obtained through self-supervised pre-training to the intrusion detection model, construct a multi-task joint optimization model including multi-class detection task, binary classification detection task and contrastive learning task, and fine-tune the model using limited labeled samples; S4. Input the in-vehicle network traffic data to be detected into the trained intrusion detection model, and output the anomaly detection results and the corresponding attack type identification results.

2. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The sliding window method used in step S1 has a window length of L and a step size of s. The resulting flow characteristic sequence is represented as follows: Where B is the batch size, L is the sequence length, and D is the feature dimension of a single traffic stream.

3. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The Transformer encoder performs embedding mapping and positional encoding on the input sequence, and its calculation method is as follows: ,in, For embedding matrix, It is a learnable location encoding.

4. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The Transformer encoder includes a multi-head self-attention layer, and its attention is calculated as follows: Furthermore, long-range temporal dependencies in vehicular network traffic sequences are modeled in parallel using multiple attention heads.

5. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The self-supervised pre-training in step S2 uses mean squared error as the reconstruction loss function, and its expression is: Where X represents the input sequence, This represents the reconstructed output, where L is the sequence length and D is the feature dimension.

6. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The multi-class classification detection task uses Focal Loss as the loss function to alleviate the problem of class imbalance in attack samples. Its expression is: Where N is the batch size, It is the true class prediction probability of the i-th sample. and It is an adjustable weighting parameter used to emphasize difficult samples.

7. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The binary classification detection task uses the cross-entropy loss function to distinguish between normal traffic and attack traffic, and its expression is: ,in, It is a binary classification of real labels. It is the probability of being predicted as an anomaly; this auxiliary task plays a crucial role in maintaining robustness against new variant attacks.

8. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The contrastive learning task enhances the discriminative power of the model's feature representations by narrowing the distance between feature representations of similar samples and widening the distance between feature representations of dissimilar samples. Its expression is as follows: ,in, Represents the cosine similarity after L2-normalized projection. This represents the set of positive samples, corresponding to the enhanced view of sample i. It is a temperature parameter; this objective encourages semantically similar attack samples to form compact clusters in the embedding space.

9. The self-supervised vehicle intrusion detection method based on Transformer according to claim 1, characterized in that, The overall loss function of the multi-task joint optimization model is: ,in 、 and The weighting coefficients for the losses of each task.

10. A vehicle network intrusion detection system based on Transformer-based contrastive self-supervised learning, characterized in that, include: The data preprocessing module is used to clean, normalize, and serialize the vehicle network traffic data. A self-supervised pre-training module is used to perform self-supervised feature learning on normal vehicular network traffic based on the Transformer autoencoder. The comparison multi-task fine-tuning module is used to construct a multi-task joint optimization model of multi-class detection, binary detection, and contrastive learning; the intrusion detection inference module is used to output the anomaly detection results and attack type identification results of the vehicle network traffic; the self-supervised pre-training module and the comparison multi-task fine-tuning module share the same Transformer encoder structure.

Images