Anomaly detection method for Internet of Vehicles based on dual knowledge distillation and federated learning

Abstract

A method for detecting anomalies in the Internet of Vehicles (IoV) based on dual knowledge distillation and federated learning relates to the field of IoV security technology. It addresses existing IoV anomaly detection methods, where network traffic data from different vehicles is not independently and identically distributed (IID)—a problem that affects the convergence and detection accuracy of the global model, as well as the difficulty of the global model fully learning the characteristics of each local anomaly detection model, resulting in limited detection capabilities. This method introduces a dataless knowledge distillation mechanism in the cloud. By training a pseudo-data generator, the global model learns the characteristics of abnormal network traffic from multiple local anomaly detection models, and guides the training of local anomaly detection models through knowledge distillation. By leveraging the mutual learning between local anomaly detection models and the global model in the cloud, this method effectively alleviates the IID problem of local data from different vehicles in the IoV without requiring the vehicle to leave the vehicle, thereby improving IoV security and anomaly detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of connected vehicle (IoV) security technology, specifically to an IoV anomaly detection method based on dual knowledge distillation and federated learning. This method leverages the mutual learning between a local anomaly detection model and a global model in the cloud. Without leaving the vehicle's local data, it effectively mitigates the non-independent and identically distributed (Non-IID) problem of local data from different vehicles in the IoV, thereby improving IoV security and anomaly detection accuracy. Background Art

[0002] With the development of the Internet of Things (IoT) and the Internet of Vehicles (IoV), vehicles are interconnected through vehicle-to-everything (V2X) wireless communication technology, which improves the efficiency of intelligent transportation systems. However, the openness of the IoV makes it vulnerable to cyber attacks, such as Figure 1 As shown in Figure 2, malicious attackers can remotely invade vehicles, causing traffic accidents, privacy leaks, and other security issues. Therefore, anomaly detection technology has become an important research direction to ensure the security of the Internet of Vehicles.

[0003] Currently, anomaly detection methods based on machine learning and deep learning are widely used and are primarily categorized into centralized learning and federated learning. Centralized learning methods rely on the cloud to centrally store and train network traffic data for all vehicles. While this improves detection accuracy, it carries risks of data leakage and high bandwidth consumption. To meet the requirement of "no data leaving the vehicle," federated learning has been introduced to IoV anomaly detection. This allows vehicles to train anomaly detection models locally and only upload parameters to the cloud for aggregation. However, federated learning methods still face the problem of non-independent and identically distributed (Non-IID) data. The distribution of network traffic data from different vehicles varies significantly, impacting the convergence of the global model and detection accuracy. Furthermore, since data is not uploaded to the cloud, the global model struggles to fully learn the characteristics of each local anomaly detection model, limiting detection capabilities.

[0004] To solve the above problems, this paper proposes an Internet of Vehicles anomaly detection method (IoVFD) based on dual knowledge distillation and federated learning. Summary of the Invention

[0005] The present invention aims to solve the problems in existing Internet of Vehicles anomaly detection methods, such as the non-independent and identically distributed problems of network traffic data of different vehicles, which affects the convergence and detection accuracy of the global model; and because the data is not uploaded to the cloud, the global model cannot fully learn the characteristics of each local anomaly detection model, resulting in limited detection capabilities. The present invention provides an Internet of Vehicles anomaly detection method based on dual knowledge distillation and federated learning.

[0006] The IoV anomaly detection method based on dual knowledge distillation and federated learning is implemented by the following steps:

[0007] Step 1: For each vehicle, a local anomaly detection model is trained using the local dataset, and the trained local anomaly detection model is uploaded to the cloud server.

[0008] Step 2: Knowledge distillation in the cloud;

[0009] The server uses the uploaded multiple local anomaly detection models to train a generator, which uses labels to generate pseudo data and uses the pseudo data to distill knowledge from the multiple local anomaly detection models into a global model;

[0010] Step 3: Knowledge distillation on the vehicle side;

[0011] The global model is sent to all vehicles, and each vehicle uses the global model with different anomaly detection model knowledge to guide the training of each local anomaly detection model;

[0012] During the training phase, labeled local data is used for federated training using a federated training mechanism; during the online detection phase, the trained local anomaly detection model detects new network traffic data.

[0013] The beneficial effects of the present invention are as follows: The detection method described herein introduces a data-free knowledge distillation mechanism in the cloud. By training a pseudo-data generator, the global model learns the characteristics of abnormal network traffic from multiple local anomaly detection models, and this knowledge distillation guides the training of the local anomaly detection models. Without requiring data to be transferred from the vehicle, this method enhances the generalization capability of the global model, improves anomaly detection accuracy, and improves the overall security of the Internet of Vehicles. Specifically, it has the following advantages:

[0014] Functionally, this paper addresses the challenges of federated learning and knowledge distillation in the Internet of Vehicles (IoV) environment and proposes the IoVFD method. This method addresses the difficulties existing solutions face in effectively addressing the non-independent and identically distributed (Non-IID) problem of vehicle data and the difficulty in achieving global model convergence. It establishes an efficient knowledge distillation mechanism and improves the accuracy of IoV anomaly detection.

[0015] First, the detection method described in this paper utilizes a generator in the cloud to construct pseudo data. Guided by multiple local detection models, the global model effectively learns the knowledge of different vehicle detection models. By employing a generation strategy based on chaos-based Gaussian noise sampling and diversity loss, the pseudo data not only mimics the distribution of real data but also avoids data homogeneity. This method ensures that the global model accurately captures the network traffic characteristics of different vehicles without relying on the original data.

[0016] Secondly, this invention employs a dual knowledge distillation mechanism on the vehicle side, enabling the local anomaly detection model to learn not only from local data but also from the global model, achieving a balance between personalized distribution and global knowledge. Specifically, while the local anomaly detection model is trained on local network traffic data (local data), it distills knowledge from the global model, effectively alleviating the data's non-IID problem and improving anomaly detection performance and generalization.

[0017] In terms of security, this paper analyzes the following three aspects: (1) the privacy protection mechanism of model and parameter transmission in the federated learning process; (2) the reduction of data leakage risk by the data-free global learning method based on knowledge distillation; (3) the improvement of the reliability of anomaly detection in the IoV environment by the optimization strategy of local and global models.

[0018] In terms of data privacy, this invention utilizes a federated learning architecture to prevent vehicles from directly transmitting raw data to the cloud, effectively protecting user privacy. Furthermore, the smart contract only stores necessary model parameters and pseudo-data labels, ensuring that even if the cloud server is attacked, the vehicle's raw data cannot be directly inferred, thus enhancing the system's privacy and security.

[0019] Regarding global model learning strategies, the data-free knowledge distillation method proposed in this paper enables the cloud to learn the knowledge of each local anomaly detection model without accessing raw vehicle data. Compared to traditional federated learning methods that directly aggregate parameters by averaging, this method uses knowledge distillation to avoid the global model's non-convergence issues caused by the non-IID problem, thereby improving the adaptability and stability of the global model.

[0020] In terms of detection reliability, the dual knowledge distillation mechanism proposed in this paper effectively improves the accuracy of local detection models, enabling them to not only detect known anomalies but also adapt to new attack patterns. Experimental results show that compared with existing baseline methods, this paper significantly improves detection precision, recall, F1 score, and overall accuracy, while also reducing false positives and false negatives, and improving the stability of anomaly detection systems in connected vehicle environments.

[0021] In summary, this paper uses a method that combines federated learning with knowledge distillation to solve the problems of data privacy protection, model convergence, and detection reliability in vehicle network anomaly detection, providing strong guarantees for the security and credibility of intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a diagram of the threat model for V2X remote communication in IoV in the existing technology;

[0023] Figure 2 This is a schematic diagram of the Internet of Vehicles anomaly detection method based on dual knowledge distillation and federated learning according to the present invention;

[0024] Figure 3 This is a schematic diagram of the local anomaly detection model described in the present invention;

[0025] Figure 4 This is a principle block diagram of a generator located in the cloud and capable of generating pseudo data using tags according to the present invention;

[0026] Figure 5 This is an architectural diagram of the pseudo data generator according to the present invention;

[0027] Figure 6 Schematic diagram of the principle of distilling the local anomaly detection model from the global model according to the present invention;

[0028] Figure 7 Schematic diagram of the principle of distilling the global model from the local anomaly detection model of the present invention;

[0029] Figure 8 The effect diagram of comparing the IoVFD method of the present invention with four baseline methods in terms of accuracy;

[0030] Figure 9 The effect diagram of comparing the IoVFD method of the present invention with four baseline methods in terms of recall rate;

[0031] Figure 10 The results of the IoVFD method of the present invention are compared with four baseline methods in terms of F1 score.

[0032] Figure 11 The figure shows the effect of comparing the IoVFD method of the present invention with four baseline methods in terms of accuracy. DETAILED DESCRIPTION

[0033] Specific implementation method 1. Combination Figures 2 to 7 This embodiment describes an Internet of Vehicles (IoVFD) anomaly detection method based on dual knowledge distillation and federated learning. This method optimizes the federated learning process through a dual knowledge distillation mechanism. In the cloud, after multiple local anomaly detection models are uploaded to the server, instead of directly performing traditional parameter aggregation, a data-free knowledge distillation method is used to construct pseudo data to assist the global model in learning the knowledge of each local anomaly detection model, thereby improving the global model's adaptability to diverse vehicle network traffic data distributions. The global model is then distributed to each vehicle, guiding the training of local anomaly detection models locally. This allows the models to inherit the global anomaly detection knowledge while retaining the local, personalized network traffic data characteristics.

[0034] In this implementation, each vehicle first uses its local data to initially train a local anomaly detection model. A global model on the cloud then distills the knowledge from multiple local anomaly detection models. Finally, the global model guides the learning of each vehicle's local anomaly detection model. During the training phase, a federated training mechanism is employed, with the IoVFD using labeled network traffic data for federated training. During the online detection phase, the trained local anomaly detection model detects new network traffic.

[0035] In this implementation, a federated training mechanism is used to train the IoVFD, i.e., each vehicle trains a local anomaly detection model using its own local data. ; The trained local anomaly detection model Sent to the cloud server; the cloud server uses multiple uploaded local anomaly detection models (fixed) to train a generator , the generator can generate pseudo data using labels; use the generated pseudo data to distill the knowledge of multiple local anomaly detection models (fixed) into the global model; send the global model to all vehicles, and each vehicle uses the global model (fixed) with different anomaly detection model knowledge to guide the training of the local anomaly detection model; finally, continuously iterate the training process until the local anomaly detection model converges.

[0036] The vehicle network anomaly detection method described in this embodiment is specifically implemented by three steps: local anomaly detection model training, cloud knowledge distillation, and on-vehicle knowledge distillation.

[0037] Step 1: Train a local anomaly detection model for each vehicle using the local dataset and upload the trained local anomaly detection model to the cloud server. The specific process is as follows:

[0038] Step A1: Initialize the local anomaly detection model;

[0039] In order to make the initialized local anomaly detection model have some knowledge of the local data distribution, the local anomaly detection model of each vehicle Leverage locally stored datasets Conduct training, including , Represents the total number of vehicles in the IoV system. In addition, each local anomaly detection model Includes spatial feature extractor and anomaly detector;

[0040] Step A2: Figure 3 As shown, a spatial feature extractor is constructed;

[0041] The spatial feature extractor includes a linear layer and three stacked spatial feature extraction blocks; in order to increase the dimension of data features, a fully connected neural network is used to extract local data points. Embed and get the original feature embedding vector , the calculation formula is as follows:

[0042] ;

[0043] in, is a linear function, is the dimension of the original feature, is the dimension of the original feature embedding vector, > ; and To learn the parameters.

[0044] Step A3: Input data to the spatial feature extraction block;

[0045] Then, the original features are embedded into the vector Input into the stack of three spatial feature extraction blocks, each of which includes a 1-dimensional convolutional layer (1DCNN layer) and an attention layer;

[0046] Step A31, in this embodiment, the 1DCNN layer can fully extract the local spatial features of local data points; the 1DCNN layer can be decomposed into convolution operation, batch normalization (BN) operation and Relu activation operation; in the convolution operation, the 1D convolution kernel of size q is embedded in the original feature vector Slide with a step size s to intercept the local embedding vector and perform a dot product operation to obtain the local spatial feature vector In position Value , the calculation formula is as follows:

[0047] ;

[0048] in, , is the original feature embedding vector In position The value of and They are convolution kernels and bias In position The value of the convolution kernel and bias are all learnable parameters;

[0049] The BN operation and ReLU activation operation can avoid gradient disappearance and enhance the expression ability of CNN. The calculation formula for obtaining local spatial features is as follows:

[0050] ;

[0051] Among them, LSF is the local spatial feature generated by the 1DCNN layer, and are the mean and variance of a batch of original feature embedding vectors, ε is a positive number, γ and β are the scale factor and bias of the normalization layer respectively;

[0052] Step A32: construct an attention layer;

[0053] The local spatial feature LSF obtained by the 1DCNN layer is input to the attention layer to extract the global spatial feature; the multi-head self-attention mechanism of the attention layer is used to extract the global spatial feature. The weighted combination of self-attention mechanisms obtains global spatial features , the calculation formula is as follows:

[0054] ;

[0055] in, It is a multi-head attention mechanism. express The connection of self-attention heads, is a size of In order to fully capture the global spatial dependency between each local spatial feature point, the spatial correlation (attention score) between each local spatial feature point is calculated. The calculation formula is as follows:

[0056] ;

[0057] in, is the activation function, is the number of attention heads, To satisfy × = An integer parameter , ,and They are query vector, key vector and value vector, which are obtained by Multiply by three learnable weight matrices respectively The calculation formula is as follows: ;

[0058] Step A4: Build an anomaly detector;

[0059] The anomaly detector is used to perform binary classification on local data points to obtain normal and abnormal probability distributions; its structure includes a linear layer and a Softmax layer, and the abnormal probability distribution The calculation formula is as follows:

[0060] ;

[0061] in, , is the normal probability, is the abnormal probability, if , then the local data point is abnormal, otherwise it is normal. and are the learnable parameters of the linear layer.

[0062] Step A5: Optimization target of the training phase;

[0063] During the training phase, in order to measure the difference between the local anomaly detection model prediction and the true label, each vehicle uses binary cross entropy loss to train the local anomaly detection model. , that is, the optimization goal of each vehicle is:

[0064] ;

[0065] in, Represents a local anomaly detection model Parameters, It's a vehicle Total entries of network traffic data, Represents network traffic data points The true category.

[0066] According to the data-free vehicle standard, vehicles can only train local anomaly detection models using local datasets. This results in a loss of a holistic view of all network traffic data, which affects model accuracy. Therefore, to aggregate the knowledge of multiple local anomaly detection models to guide local anomaly detection model training, a data-free knowledge distillation method for multiple local detection models (KDFree) is proposed. This method extracts global knowledge about data distribution without requiring data from the vehicle.

[0067] Step 2: Knowledge distillation in the cloud. The specific process is as follows:

[0068] Step B1: train the generator;

[0069] In order to enable the global model to learn the knowledge of the local anomaly detection models of different vehicles, in this implementation, a generator that can generate pseudo data using labels is introduced in the cloud. , and use the local anomaly detection model (fixed) uploaded to the cloud to assist the generator training, such as Figure 4 As shown, given a noise sampled from a standard Gaussian distribution and a one-hot encoded label , generator Aims to generate labeled pseudo data , so that the pseudo data Can be identified by the local anomaly detection model; the specific process is:

[0070] According to the distribution of labels Sampling one-hot encoded labels , label distribution The label frequencies can be derived from the local training dataset as follows:

[0071] ;

[0072] Among them, the one-hot encoded label Indicates normal, one-hot encoded label Indicates an exception, represents the sum of all vehicle training network traffic data points, represents the sum of all vehicle training normal network traffic data points, Represents the sum of all vehicle training abnormal network traffic data points. In order to generate diverse pseudo data, the single-hot encoding label of the sample Add Gaussian noise z to obtain a unique hot label with Gaussian noise ;as follows:

[0073] ;

[0074] in, represents one-hot labels with Gaussian noise.

[0075] In this embodiment, a Gaussian noise sequence is generated by using a Logistic chaotic system to ensure the randomness of sampling, and the noise sampling sequence Z is calculated; Gaussian noise is added Hot Tags Feed into the generator to get fake data ;

[0076] ;

[0077] in, Represents the model parameters of the generator.

[0078] like Figure 5 As shown in the figure, in this embodiment, the generator includes two linear layers and a three-layer spatial feature decoding block; the first linear layer is used to decode the one-hot label with Gaussian noise. Embedding is performed to increase the label dimension, and the label embedding vector as follows:

[0079] ;

[0080] in, , ; Stacking three layers of spatial feature decoding blocks is used to embed the label vector Decode and obtain the spatial characteristics of pseudo data , each block includes an attention layer and a one-dimensional convolution layer (1DCNN layer); the spatial features of the pseudo data as follows:

[0081] ;

[0082] The second linear layer is used to transform the spatial features of the pseudo data Generate fake data ;

[0083] ;

[0084] Step B2: Distill local anomaly detection model knowledge;

[0085] The pseudo data The fake data is fed into the local anomaly detection model uploaded to the cloud, so that the local anomaly detection model can correctly identify it. The specific process is as follows:

[0086] Compute the overall prediction results of the local anomaly detection model :

[0087] ;

[0088] Among them, Figure 4 As shown, , , They are the corresponding local anomaly detection models , local anomaly detection model , local anomaly detection model The prediction results of Divergence (Kullback-Leibler divergence) loss computes the predictions of the local anomaly detection model and the one-hot labels with Gaussian noise The difference value :

[0089] ;

[0090] in, The function can Converted into normalized probability distribution, in order to improve the diversity of pseudo data, the diversity loss function is introduced To improve the diversity of generated pseudo data;

[0091] ;

[0092] in, represents the batch size, and Represent the ξth and ξth labels in the batch with Gaussian noise respectively. One-hot labels with Gaussian noise, Indicates that the generator passes Generated pseudo data, Indicates that the generator passes Generated pseudo data;

[0093] Calculate the total loss function of the generator:

[0094] .

[0095] Step B3: Figure 6 As shown, global model training;

[0096] Due to the non-IID nature of local data from different vehicles, directly aggregating the parameters of all local anomaly detection models may cause the global model to fail to converge. Therefore, we use the trained generator to construct pseudo data and use multiple local anomaly detection models as teacher models to guide the training of the global model.

[0097] Construct a pseudo dataset based on the one-hot encoded label sampling set and the noise sampling set , where pseudo data is a one-hot encoded label generate, is the size of the pseudo dataset;

[0098] The pseudo data Feed to the global model , get the output classification information ;Introduction of temperature coefficient To soften classified information , obtain the soft target of the global model , as shown below:

[0099] ;

[0100] in, and Respectively represent and Softened classification information of elements, Indicates the The soft target of elements corresponds to the output probability of normal and abnormal, Represents the temperature coefficient, which can control the softening degree of the output probability. hour, Degenerates into The result of the function is A larger value will result in a softer probability distribution. Set to and Obtaining soft targets for the global model and soft targets .

[0101] At the same time, the pseudo data Feed into local anomaly detection model separately and calculate the soft target predicted by the local anomaly detection model .

[0102] Calculate the soft target of the global model using binary cross entropy loss and pseudo data One-hot encoded labels The difference between , that is, the student loss function of the global model , as shown below:

[0103] ;

[0104] in, and are the one-hot encoded labels of the first and second elements respectively, and When When is 1, the soft targets of the first and second elements in the global model;

[0105] Calculate the soft target of the global model using KL divergence loss and soft targets predicted by local anomaly detection models The difference between , that is, the distillation loss function of the global model ;

[0106] ;

[0107] Finally, the loss function for training the global model is for:

[0108] ;

[0109] in, is the scaling factor for the loss of training the global model.

[0110] In this embodiment, since the global model has distilled the knowledge of the local anomaly detection models of different vehicles, the global model is sent to each vehicle to guide the training of the local anomaly detection model. In other words, for each vehicle , which has a network traffic dataset , local anomaly detection model and a global model with fixed parameters sent from the cloud .

[0111] Step 3: If Figure 7 As shown, in this embodiment, the global model As a teacher model, local anomaly detection model As a student model, and use the teacher to guide the student's learning; knowledge distillation on the vehicle side; the specific process is:

[0112] Step C1: processing of local data and guidance of teacher model;

[0113] On the vehicle side, local data is fed into a global model with fixed parameters and temperature coefficients are used. Obtaining soft targets for the global model Similarly, local data is fed to the local anomaly detection model and Set to and Obtaining the soft target of the local anomaly detection model and soft targets ;

[0114] Step C2: Calculate the local student loss function;

[0115] Since local data contains its own individual data distribution, in this implementation, the binary cross entropy loss is used to calculate the soft target of the local anomaly detection model. and the true label The difference between , that is, the local student loss function:

[0116] ;

[0117] in, Indicates the soft target of an element.

[0118] Step C3: Calculate the local distillation loss function;

[0119] The global model has knowledge of other vehicles’ models, for example, knowledge of abnormal traffic that the vehicle has never seen before. Therefore, in this implementation, the knowledge of the global model is distilled into the local detection model to enhance the performance of the local detection model. The Kullback-Leibler divergence loss is used to calculate the soft target of the local anomaly detection model. and the soft target of the global model The difference between , that is, the local distillation loss function :

[0120] ;

[0121] Step C4: Optimize the local anomaly detection model;

[0122] The loss function for training the local anomaly detection model is:

[0123] ;

[0124] in, A scaling factor for the loss of training the local anomaly detection model, used to weigh the impact of the local student loss and the distillation loss.

[0125] Specific implementation method 2: Figures 8 to 11 This embodiment is described as a verification example of the vehicle network anomaly detection method based on dual knowledge distillation and federated learning described in Specific Embodiment 1:

[0126] In this example, to verify the effectiveness of the method proposed in this embodiment, the IoVFD method of the present invention was compared with four baseline methods: Local, ACLFL, GRUFL, and FED-IDS. Local used only local data to train a local anomaly detection model; ACLFL used federated learning to train an attention-based convolutional neural network-long short-term memory network anomaly detection model and used average aggregation to obtain a global model; GRUFL used federated learning to train a gated recurrent unit-based anomaly detection model and used average aggregation to obtain a global model; and FED-IDS used federated learning to train a Transformer-based anomaly detection model and used the percentage of participating vehicle data as the weight for aggregating the local model into the global model. The effectiveness of IoVFD was analyzed using metrics such as precision, recall, F1 score, and accuracy. Precision represents the ratio of the number of correctly predicted anomaly data points to the number of predicted anomaly data points. Figure 8The figure shows a comparison of IoVFD with other baseline methods in terms of average precision. IoVFD achieves an average precision of 97.41%, improving on Local, ACLFL, GRUFL, and FED-IDS by 10.19%, 6.54%, 6.38%, and 4.24%, respectively. Local uses only local data to train the local anomaly detection model. When a vehicle has significantly more normal data than anomaly data, the model tends to overfit the normal data, resulting in incorrect predictions about anomaly data. In extreme cases, it may classify all data as normal. ACLFL, GRUFL, and FED-IDS use FL to train a deep detection model. They aggregate the global model through gradient compression averaging, direct averaging, and the percentage of participating vehicles' data, respectively. However, they ignore the no-IID data caused by the diverse behaviors of different vehicles, hindering the convergence of the global model and leading to inaccurate local detection. In IoVFD, the global model learns the local anomaly detection model knowledge for all vehicles based on knowledge distillation, alleviating the no-IID data issue and using the global model to guide the training of the local anomaly detection model. IoVFD not only learns the distribution of local network traffic data, but also learns the knowledge of network traffic data of other vehicles. Therefore, the accuracy of IoVFD is better than Local, ACLFL, GRUFL and FED-IDS.

[0127] like Figure 9 As shown, in this embodiment, the recall rate represents the ratio of correctly predicted abnormal data points to all actual abnormal data points. In other words, the recall rate represents the ability to detect anomalies. Figure 9The figure shows a comparison of IoVFD with other baseline methods in terms of average recall. IoVFD achieves an average recall of 98.87%, which is 9.44%, 6.56%, 8.76%, and 5.78% higher than Local, ACLFL, GRUFL, and FED-IDS, respectively. IoVFD frequently faces new attacks that generate new anomalous network traffic data. In contrast, Local only observes the local data distribution and has difficulty accurately detecting previously unseen anomalies. Furthermore, while ACLFL, GRUFL, and FED-IDS implement federated training for anomaly detection models using various global model aggregation methods, they ignore the no-IID nature of network traffic data between vehicles. As a result, the global model cannot evenly fit the data distribution of all vehicles, resulting in inaccurate local anomaly detection models updated using the global model. IoVFD employs a data-free knowledge distillation method for multiple local anomaly detection models to effectively transfer knowledge from local anomaly detection models to the global model, mitigating the impact of no-IID data from different vehicles. In addition, IoVFD uses the global model to guide the training of local anomaly detection models, enabling them to capture the individual distribution of local data and the knowledge from other local anomaly detection models. As a result, IoVFD achieves higher recall compared to Local, ACLFL, GRUFL, and FED-IDS.

[0128] like Figure 10 As shown, in this embodiment, the F1 score is the harmonic mean of the model's precision and recall. Figure 10The figure shows a comparison of IoVFD with other baseline methods in terms of average F1 scores. IoVFD achieves an F1 score of 98.13%, which is 9.82%, 6.55%, 7.56%, and 5% higher than Local, ACLFL, GRUFL, and FED-IDS, respectively. Local can only train models using local data, while IoVFD can not only learn local data but also learn knowledge about the distribution of other vehicle data through knowledge distillation. As a result, each local anomaly detection model in IoVFD can accurately detect anomalies. IoVFD achieves a significantly better F1 score than Local. In addition, ACLFL compresses the gradients of the locally trained attention-based CNN-LSTM anomaly detection models and aggregates them into a global model through averaging, while GRUFL directly aggregates the locally trained GRU-based anomaly detection models into a global model through averaging. FED-IDS aggregates the locally trained Transformer-based anomaly detection models by using the percentage of each vehicle's data in the total data as a weight. However, due to the no-IID nature of the traffic data from different vehicles, the global models of ACLFL, GRUFL, and FED-IDS fail to converge, resulting in poor detection performance for the locally updated detection models using the global model. Instead of performing average aggregation, IoVFD introduces a generator in the cloud that generates pseudo data. This generator is used to fully distill the knowledge of multiple local anomaly detection models into the global model, thereby mitigating the impact of no-IID data from different vehicles on the global model. As a result, IoVFD achieves a 5% improvement in F1 score compared to the best baseline method.

[0129] like Figure 11 As shown, in this embodiment, Figure 11 The comparison of IoVFD with other baseline methods in terms of average accuracy is shown. The average accuracy of IoVFD is 98.48%, which is 4.13%, 6.61%, 6.46% and 3.68% higher than Local, ACLFL, GRUFL and FED-IDS respectively. IoVFD has the best detection accuracy for the following reasons: (1) It introduces a pseudo data generator, which ensures the diversity of the generated pseudo data through chaos-based Gaussian noise sampling and diversity loss, and uses cross-entropy loss to ensure that the distribution of the generated pseudo data is similar to the global data distribution. (2) It uses the pseudo data generator to transfer the knowledge of multiple local anomaly detection models to the global model, reducing the impact of data non-independent and identically distributed on the global model. (3) The global model is used to guide the training of each local anomaly detection model, which not only retains the individuality of the data distribution of the local anomaly detection model, but also retains the knowledge of other vehicle detection models, thereby improving the detection accuracy of the local anomaly detection model.

[0130] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. An IoV anomaly detection method based on dual knowledge distillation and federated learning, characterized by: The method is implemented by the following steps: Step 1: For each vehicle, a local anomaly detection model is trained using the local dataset, and the trained local anomaly detection model is uploaded to the cloud server. Step 2: Knowledge distillation in the cloud; The server uses the uploaded multiple local anomaly detection models to train a generator. The generator uses labels to generate pseudo data and uses the pseudo data to distill the knowledge of the multiple local anomaly detection models into the global model. The specific process of knowledge distillation is as follows: Step 21: Train the generator; According to the label distribution p(y), sample the one-hot encoding label pesy, add Gaussian noise z to the sampled one-hot encoding label pesy, and obtain the one-hot label y with Gaussian noise * ; The unique hot label y with Gaussian noise * Send to the generator to get pseudo data Step 22: Distill local anomaly detection model knowledge; The pseudo data Transmitted to the local anomaly detection model uploaded to the cloud, so that the fake data can be correctly identified by the local anomaly detection model; Step 2 and 3: global model training; Use the trained generator to construct pseudo data, and use multiple local anomaly detection models as teacher models to guide global model training; Step 3: Knowledge distillation on the vehicle side; The global model is sent to all vehicles, and each vehicle uses the global model with different anomaly detection model knowledge to guide the training of each local anomaly detection model; During the training phase, the federated training mechanism is used to conduct federated training using labeled local data; In the online detection phase, the trained local anomaly detection model detects new network traffic data; The global model GM is used as the teacher model, and the local anomaly detection model M i Act as a student model and use teachers to guide students in their learning; The specific process is: Step 3.1: Transfer local data to the global model and use the temperature coefficient T = t to obtain the soft target ST(t) of the global model. GM ; When the local data is transmitted to the local anomaly detection model and the temperature coefficient T is set to 1 and t respectively, the soft target ST(1) of the local anomaly detection model is obtained. LM and soft target ST(t) LM ; Step 32: Calculate the local student loss function; Computing soft targets ST for local anomaly detection models using binary cross entropy loss (1) LM and the true label y True The difference, namely the local student loss function, is expressed as follows: Where, and are the soft targets for the first and second elements, respectively; Step 3. Calculate the local distillation loss function; The KL divergence loss is used to calculate the soft target ST(t) of the local anomaly detection model LM and the soft target ST(t) of the global model GM The difference between the local distillation loss function It can be expressed as follows: Step 3 and 4: Optimize the local anomaly detection model; expressed as follows: in, is the loss function of the local anomaly detection model, and λ2 is the scaling factor of the loss for training the local anomaly detection model.

2. The method for detecting anomalies in the Internet of Vehicles based on dual knowledge distillation and federated learning according to claim 1 is characterized by: In step 1, the process of training the local anomaly detection model is as follows: Step 11: Set each local anomaly detection model to consist of a spatial feature extractor and an anomaly detector; the local anomaly detection model of each vehicle is trained using the locally stored dataset; Step 1 and 2: Use the spatial feature extractor to extract local spatial features and obtain global spatial features through the attention layer; Step 13: Use an anomaly detector to perform binary classification on the global spatial features to obtain normal and abnormal probability distributions; Step 14: Use the binary cross entropy loss function to train the local anomaly detection model. The optimization objective for each vehicle is: Where w i is the local anomaly detection model M i Parameters, |D i | is the total number of network traffic data entries for vehicle i, y j ∈{0,1} represents the network traffic data point x j The true category, P normal is the normal probability, P abnormal is the abnormal probability.

3. The method for detecting anomalies in the Internet of Vehicles based on dual knowledge distillation and federated learning according to claim 2 is characterized by: In steps 1 and 2, the spatial feature extractor consists of a linear layer and three stacked spatial feature extraction blocks; Embed the local data point x through the linear layer to obtain the original feature embedding vector E x ; The original features are embedded into the vector E x The input is fed into three stacked spatial feature extraction blocks to extract global spatial features.

4. The method for detecting anomalies in the Internet of Vehicles based on dual knowledge distillation and federated learning according to claim 3 is characterized by: Each spatial feature extraction block consists of a one-dimensional convolutional layer and an attention layer; The one-dimensional convolution layer is used to extract local spatial features of local data points, and perform convolution operations, batch normalization operations, and activation operations to obtain local spatial features; The local spatial features are input into the attention layer, and the multi-head self-attention mechanism of the attention layer is used to ′ A weighted combination of self-attention mechanisms is used to obtain global spatial features.

5. The method for detecting anomalies in the Internet of Vehicles based on dual knowledge distillation and federated learning according to claim 1, characterized in that: In step 21, the generator includes two linear layers and a three-layer stacked spatial feature decoding block; The first linear layer is used to train the one-hot label y with Gaussian noise * Embed and obtain the label embedding vector E y , which can be expressed as follows: AND y =Linear(and * ); Where, Linear() is a linear function; The three-layer spatial feature decoding block is used to embed the label vector E y Decode and obtain the spatial feature ST of the pseudo data; The second linear layer is used to generate pseudo data from the spatial feature ST of pseudo data 6. The method for detecting anomalies in the Internet of Vehicles based on dual knowledge distillation and federated learning according to claim 1, characterized in that: The specific implementation process of step 22 is as follows: First, calculate the overall prediction result Output of the local anomaly detection model LM , which can be expressed as follows: Where w i is the local anomaly detection model M i Parameters of ; KL divergence loss is used to calculate the predictions of the local anomaly detection model and the one-hot labels y with Gaussian noise * The difference value It can be expressed as follows: In the formula, the Softmax() function is used to convert Output LM Convert to normalized probability distribution and introduce diversity loss function Improving the diversity of generated pseudo data can be expressed as follows: Where Q is the batch size, y * ξ and y * η are the ξ-th and η-th unique hot labels with Gaussian noise, Pass y for the generator * ξ Generated pseudo data, Pass y for the generator * η Generated pseudo data; Then, calculate the total loss function of the generator:

7. The method for detecting anomalies in the Internet of Vehicles based on dual knowledge distillation and federated learning according to claim 1, characterized in that: The specific implementation process of steps two and three is as follows: First, the pseudo data Transfer to the global model GM to obtain the output classification information logits GM ; Introduce temperature coefficient T to soften classification information logits GM , and obtain the soft target of the global model, which is expressed as follows: Where, and are the classification information of the i-th and j-th elements respectively, is the soft target of the i-th element in the global model. When T is set to 1 and t respectively, the soft target ST(1) of the global model is obtained GM and soft target ST(t) GM ; At the same time, the pseudo data are transmitted to the local anomaly detection model M i and calculate the soft target ST(t) predicted by the local anomaly detection model LM ; Secondly, the binary cross entropy loss is used to calculate the soft target ST of the global model (1) GM and pseudo data The one-hot encoded label pesy i The difference between , that is, the student loss function of the global model It is expressed as follows: Where, and The one-hot encoded labels pesy for the first and second elements respectively i , and are the soft targets of the first and second elements in the global model when T is 1; Then, the KL divergence loss is used to calculate the soft target ST(t) of the global model GM and the soft target ST(t) predicted by the local anomaly detection model LM The difference between , the distillation loss function of the global model Finally, the loss function for training the global model is for: Where λ1 is the scaling factor for the loss of training the global model.

Images