Vehicle-mounted network anomaly detection method and system based on federal hybrid expert model
Through the adaptive weight allocation technology of the federal hybrid expert model, the privacy protection and detection accuracy of on-board network communication packets in intelligent connected vehicles is solved, and efficient abnormal detection and privacy protection is achieved.
Patent Information
- Application Number
- CN202510507622.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-22
- Publication Date
- 2025-07-01
AI Technical Summary
The prior art has problems in the intelligent connected vehicles that require in-vehicle network communication packet data privacy protection requirements and non-independent and homogeneous data prediction capabilities, resulting in privacy data leakage and poor adaptability of global models.
The federal hybrid expert model is adopted to generate feature maps through sliding window algorithms, segmented into global data sets and local data sets, and adaptive weight allocation is performed in combination with the routing model to realize abnormal detection.
In the federated learning scenario, take into account global generalization capabilities and end-side application capabilities, improve detection accuracy, ensure privacy protection, and reduce detection delays.
Smart Images
Figure CN120238364A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of vehicle networking security, and particularly relates to the in-vehicle network security technology of intelligent connected vehicles, and mainly relates to an in-vehicle network anomaly detection method and system based on a federated hybrid expert model. Background Art
[0002] With the rapid development of intelligent connected vehicles, the number and complexity of in-vehicle electronic control units (ECUs) have increased significantly, and the information interaction between vehicles and the outside world has become more frequent, which makes the communication security problems faced by intelligent connected vehicles more severe. At present, most intelligent connected vehicles are equipped with bus systems represented by FlexRay, Controller Area Network (CAN), Local Interconnect Network (LIN), and in-vehicle Ethernet, which are the core for data interaction between ECUs. However, in-vehicle network communication messages mainly perform high-speed communication in a broadcast manner, and most of them lack basic security mechanisms such as authentication or message encryption. Therefore, external attackers can insert malicious messages into communication messages through in-vehicle gateways such as T-Box, and perform attacks such as Denial of Service (DoS), fuzzing, and spoofing on the vehicle, posing a great threat to the in-vehicle network communication system.
[0003] In order to effectively detect abnormal communication messages in the in-vehicle network, most current methods adopt deep learning methods, using a large number of collected communication messages as a data set to train a deep learning model, so as to be able to predict unknown communication messages. The problems with this method are as follows: In-vehicle network communication messages can directly control the ECUs of the vehicle, so there is a very high privacy protection requirement. Once the communication message data is leaked, the vehicle will face serious security threats. The federated learning training method is an effective solution to achieve privacy protection. It only transmits model parameters without transmitting the original training data, which can ensure that the in-vehicle network communication message data is available but invisible. However, the problem with the federated learning training method is that when a federated client as a training member exits, the prediction performance of the federated global model will decline, and it cannot fully learn the message type distribution characteristics at the vehicle end, and it is not applicable when predicting abnormal communication messages with non-independent and identically distributed vehicles.
[0004] Therefore, aiming at the problem of abnormal detection of in-vehicle network communication messages based on deep learning in intelligent connected vehicles, there is an urgent need for a detection method that takes into account both privacy protection and the ability to predict non-independent distribution data, to solve the problems that the deep learning method of centralized training may lead to privacy data leakage, and the federated learning training method may lead to the global model being unable to adapt to the personalized data distribution of vehicles. Summary of the Invention
[0005] In view of the problems in the prior art, the present invention provides a method and system for on-vehicle network anomaly detection based on a federated mixture-of-experts model. First, in the feature map generation and partitioning stage, each vehicle converts communication messages within a fixed time interval into a feature map as a training data set based on a sliding window algorithm, and further divides the data set into a global data set for federated learning training and a local data set for vehicle-side fine-tuning according to vehicle privacy protection requirements. As a federated client, in each round of federated training, the vehicle uses the global data set on the vehicle side for local training, uploads the model parameters to the central parameter server for parameter aggregation, and then uses the aggregated model parameters to repeat the above process until the preset number of federated training rounds is reached. The vehicle initializes the local expert model with the global model and performs training fine-tuning on the local data set to make the local expert model adapt to the vehicle-side data distribution. Finally, the vehicle initializes a routing model and further fine-tunes the local expert model and the routing model using the full vehicle-side data set, so that the routing model can perform adaptive weight allocation on the input data for the federated global model and the local expert model, obtaining a mixture-of-experts model, thereby realizing anomaly detection for the input communication messages.
[0006] To achieve the above object, the technical solution adopted by the present invention is: An on-vehicle network anomaly detection system based on a federated mixture-of-experts model, including a plurality of federated clients and a central parameter server.
[0007] The federated client: collects on-vehicle network communication messages generated locally, constructs a client data set after feature extraction, performs local training on the client data set using the model parameters sent by the central parameter server, and uploads the trained model parameters to the central parameter server.
[0008] The central parameter server: is used to collect the model parameters uploaded by each client, aggregate the model parameters according to the federated average aggregation algorithm, and distribute the aggregated parameters to each client for the next round of federated training.
[0009] As an improvement of the present invention, the federated client includes at least two isomorphic anomaly detection models and a routing model. The two isomorphic anomaly detection models respectively represent the federated global model for federated training and the local expert model for local fine-tuning, and the routing model is used to perform adaptive weight allocation on the input data.
[0010] The central parameter server fixes an anomaly detection model structure for aggregating and updating the model parameters uploaded from multiple clients.
[0011] The anomaly detection model at least includes a feature extraction module and an output layer. The feature extraction module is composed of non-linear convolutional blocks, and the output layer is composed of a fully connected layer and a softmax function to achieve anomaly category mapping;
[0012] The routing model at least includes a feature extraction module and an output layer. The output layer is composed of a fully connected layer and a sigmoid function, which is used to allocate the prediction weights of the federated global model and the local expert model according to the output of the feature extraction module;
[0013] To achieve the above object, the technical solution adopted by the present invention is also: an on-vehicle network anomaly detection method based on a federated hybrid expert model, including the following steps:
[0014] S1, Feature map generation and division: The federated client extracts features from the on-vehicle network communication messages generated locally, uses a sliding window to convert the fixed-length messages into feature maps, and constructs a feature map data set; according to the vehicle data privacy protection requirements, it is divided into two parts, including a global data set for federated training of the federated client and a local data set for fine-tuning of the federated client;
[0015] S2, Federated training: Each federated client downloads the initialized model weights from the central parameter server, uses the local global data set for local training, and the central parameter server aggregates the model parameters of the federated clients until the number of training rounds reaches the preset value to obtain a federated global model;
[0016] S3, Client fine-tuning: Each federated client freezes the federated global model parameters obtained after the training in step S2, initializes the local expert model with the model parameters, and performs fine-tuning training on the local data set of the client to obtain the local expert model;
[0017] S4, Hybrid expert model fusion: Each federated client initializes a routing model to perform adaptive weight allocation on the federated global model obtained in step S2 and the local expert model obtained in step S3, and performs fusion training to obtain a hybrid expert model, so as to realize anomaly detection of the input communication messages.
[0018] As an improvement of the present invention, in the feature map generation and division in step S1, for each federated client, collect the communication messages of different local anomaly types, fix the window length, map the Payload byte values of the messages in the window to the 0-255 interval, and perform a linear dimension transformation operation to obtain a three-channel feature map, move to the next window, repeat the steps until all the messages collected locally by the federated client are converted into a feature map data set, and according to the vehicle data privacy protection requirements, divide the feature map data set into a global data set for federated training of the federated client and a local data set for fine-tuning of the federated client.
[0019] As another improvement of the present invention, the federated training in step S2 specifically includes the following steps:
[0020] S21: The central parameter server initializes the parameters w of the anomaly detection model g , and the federated client initializes the parameters of the federated global model of the local expert model of as well as the parameters of the routing model r i parameters
[0021] S22: In each round of training, a fixed number of federated clients are randomly selected as the federated training clients for this round. For each federated client, the parameters of the federated global model of the federated client are initialized using the model parameters of the central parameter server Local training is performed using the global dataset of the federated client The parameters are uploaded to the central parameter server; where is the global dataset for federated training;
[0022] S23: The central parameter server uses the federated averaging algorithm to aggregate the model parameters of the federated training clients in this round;
[0023] S24: Repeat steps S22 - S23 until the preset number of federated training rounds is reached, and each federated client saves the same federated global model.
[0024] As yet another improvement of the present invention, in the client fine-tuning in step S3, for each federated client, the parameters of the local expert model are initialized using the parameters of the federated global model The local expert model is fine-tuned using the local dataset The parameters of the local expert models are different from each other.
[0025] As a further improvement of the present invention, in the hybrid expert model fusion in step S4, the federated global model of the client the local expert model i and the routing model r are fixed. For the input feature map x to be predicted, hybrid expert model prediction is performed
[0026] Compared with the prior art, the effective effects of the present invention are:
[0027] (1) The method of the present invention adaptively allocates weights to the federated global model and the local expert model through a routing model, and automatically allocates a more suitable model according to the input data features, which can ensure that the prediction performance of the hybrid expert model is not weaker than any of the two expert models.
[0028] (2) After the number of federated clients participating in the training is reduced, the prediction performance of the hybrid expert model of the method of the present invention does not fluctuate significantly, and it has good applicability to the client local data set.
[0029] (3) The hybrid expert model of the method of the present invention can integrate the global knowledge of the federated global model and the personalized knowledge of the local expert model at the client, and balance the global generalization ability and the end-side applicability in the federated learning scenario. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic structural diagram of the vehicle-mounted network anomaly detection system based on the federated hybrid expert model of the present invention;
[0031] Figure 2 It is a schematic structural diagram of the UCANet model used in Embodiment 2 of the present invention;
[0032] Figure 3 It is a schematic structural diagram of the routing model used in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0033] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.
[0034] Embodiment 1
[0035] The vehicle-mounted network anomaly detection method based on the federated hybrid expert model specifically includes the following steps:
[0036] Step S1, feature map generation and partitioning stage: The federated client extracts the feature fields from the locally generated vehicle-mounted network communication messages, uses a sliding window to convert the fixed-length messages into three-channel feature maps, and constructs a feature map data set. Further, according to the privacy compliance requirements of each client, a part is split as the global data set for client federated training, and the other part is used as the local data set for client fine-tuning.
[0037] S11: For each client client i, collect communication packets of different local anomaly types, fix the window length, map the Payload byte values of the packets in the window to the range of 0 - 255, and perform a linear dimensionality transformation operation to obtain a three-channel feature map; move to the next window and repeat the above steps until all the communication packets collected locally by the client are converted into the feature map dataset D i .
[0038] S12: Further divide the client's feature map dataset D i into a global dataset for federated training and a local dataset for local model fine-tuning Meet
[0039] Step S2, Federated Training Phase: Each federated client downloads the initialized model weights from the central parameter server and performs local training using the local global dataset. After local training is completed, the client uploads the model parameters to the central parameter server and waits for the central parameter server to aggregate the model parameters of the clients participating in the federated training in this round using the federated average aggregation algorithm. Then, it downloads the aggregated model parameters and continues to perform federated training using the global dataset of the client until the number of training rounds reaches the preset value. After federated training is completed, each federated client saves the same federated global model.
[0040] S21: The central parameter server initializes the anomaly detection model parameters w g , and the client initializes the parameters of the federated global model of the local expert model of and the routing model r i parameters
[0041] S22: Randomly select a fixed number of clients in each round of training as the federated training clients in this round; for each client, initialize the federated global model parameters of the client using the model parameters of the central parameter server Perform local training using the global dataset of the client Upload the parameters to the central parameter server;
[0042] S23: The central parameter server uses the federated average algorithm to aggregate the model parameters of the federated training clients in this round;
[0043] S24: Repeat S22 and S23 until the preset number of federated training rounds is reached.
[0044] Step S3, Client Fine-tuning Phase: Each federated client freezes the parameters of the federated global model trained in S2, initializes the local expert model with these model parameters, and performs fine-tuning training on the local dataset of the client. After the training is completed, between each federated client, a federated global model with exactly the same parameters and a local expert model with different parameters are saved respectively.
[0045] For each client i , initialize the local expert model parameters with the federated global model parameters Fine-tune the local expert model using the local dataset
[0046] Step S4, Hybrid Expert Model Fusion Phase: Each federated client initializes a routing model with a single neuron in the output layer, which is used to perform adaptive weight allocation for the federated global model obtained in step S2 and the local expert model obtained in step S3. For each federated client, keep the federated global model parameters in a frozen state, and use the full-scale feature map dataset of the client processed in step S1 to perform fusion training on the local expert model and the routing model. After the fusion training is completed, for the input feature map to be predicted, first calculate the routing weights assigned to the federated global model and the local expert model through the routing model, then input the feature map into the federated global model and the local expert model respectively, multiply with their respective routing weights at the output layer, and finally obtain the prediction output of the hybrid expert model.
[0047] Embodiment 2
[0048] A vehicle network anomaly detection system based on a federated hybrid expert model, the system model diagram is as Figure 1 shown, including a central parameter server and several federated clients, and some clients may withdraw from the federated training throughout the process.
[0049] Federated Client: Collect the in-vehicle network CAN communication messages generated locally, preprocess and package them into the client dataset, perform local training on the client dataset using the model parameters issued by the central parameter server, and upload the trained model parameters to the central parameter server. Central Parameter Server: Responsible for collecting the model parameters uploaded by each client, aggregating the model parameters according to the federated average aggregation algorithm, and distributing the aggregated parameters to each client for the next round of federated training.
[0050] The central parameter server fixes a UCANet (Universal Inverted Bottleneck - Based Network for CAN Messages) model structure for parameter aggregation. Each client deploys 3 models, including two isomorphic UCANet models F g and F s , representing the federated global model for federated training, the local expert model for local fine - tuning, and a routing model r for adaptively allocating weights to the input data respectively.
[0051] In this embodiment, the structure of the anomaly detection model UCANet used is as Figure 2 shown, mainly divided into 3 parts: a non - linear convolution block composed of convolution (Conv), batch normalization (BN), and Gaussian error linear unit activation function (GELU); a universal inverted bottleneck (UIB) block composed of depth - wise convolution (DW - Conv) and multi - layer perceptron (MLP); and finally, a fully - connected layer and a softmax function as the output layer to achieve anomaly class mapping. UCANet as a whole serves as the initialization structure of F g and F s .
[0052] In this embodiment, the structure of the routing model r is as Figure 3 shown, composed of the non - linear convolution block in UCANet, a fully - connected layer, and an output layer composed of a single neuron and a sigmoid function, used to allocate prediction weights for the mixture of experts model.
[0053] In this embodiment, it is assumed that there are 20 federated clients in total, the number of federated training rounds is 10, 5 federated clients participate in each round of training, and the local training period of the federated clients is 3.
[0054] A vehicle - mounted network anomaly detection method based on a federated mixture of experts model includes the following steps:
[0055] Step S1, feature map generation and partitioning stage: For each client client i(i = 1, 2, …, 20) Collect CAN messages of different local abnormal communication types, with a fixed window length. Map the CAN ID and 8-byte Payload value of the CAN messages in the window to the range of 0 - 255, and perform a linear dimensionality transformation operation to obtain a CAN feature map. Then move the window to the next segment of CAN messages and repeat the above process to continue processing, obtaining a dataset D composed of CAN feature maps of different abnormal communication types. i , Split the dataset according to privacy compliance requirements, which are respectively the global dataset for federated training and the dataset for local fine-tuning. Satisfy
[0056] Step S2, Federated training stage: Through multiple rounds of interaction between the central parameter server and federated clients, a federated global model is trained. The specific steps are as follows:
[0057] S21: The central parameter server initializes the UCANet model structure, and each client initializes and as the UCANet model structure, and initializes r i as the aforementioned routing model;
[0058] S22: For the t-th round of federated training, t = 1, 2, …, 10, randomly select 5 clients as the set C of federated clients in the t-th round t ; For each client client t in C j , use the global model parameters of the central parameter server to initialize the global model parameters of the client and train for 3 epochs on the global dataset.
[0059] S23: Each client in C t uploads the model parameters completed in this round of training to the central parameter server, and the central parameter server runs the federated averaging algorithm for aggregation to obtain the global model parameters of this round:
[0060]
[0061] S24: Repeat S22 to S23 until the training round t reaches the preset 10 times, and the federated global model of each client has the same parameter w g , and perform the parameter freezing operation
[0062] Step S3, Client fine-tuning stage: For each client client i, initialize the local expert model parameters using the federal global model parameters Further fine-tune and train for 3 epochs on the local dataset to obtain the fine-tuned local expert model. For clients that do not participate in the federal training throughout the process, they can still obtain the federal global model parameters from the central parameter server to initialize the local expert model.
[0063] Step S4, the hybrid expert model fusion stage. The routing model makes adaptive weight allocation for the federal global model and the local expert model. The specific steps are as follows:
[0064] S41: For each client i , use the local full CAN feature map dataset D i , for the routing model r i and the local expert model Further fine-tune and train for 3 epochs, and finally obtain
[0065] S42: The routing model r i of the client i , the federal global model the local expert model respectively obtain the corresponding model parameters For the input CAN feature map data x, the prediction result is
[0066]
[0067] According to the probability distribution in the prediction vector, the input is determined as the corresponding abnormal category.
[0068] Thus, the training and prediction process of a vehicle network anomaly detection method based on a federal hybrid expert model is completed.
[0069] In summary, the present invention makes adaptive weight allocation for the local expert model and the federal global model through the routing model, selects the appropriate prediction model according to the input data characteristics, and the hybrid expert model combines the global knowledge of the federal global model and the personalized knowledge of the local expert model at the client. It can balance the global generalization ability and the end-side applicability in the federal learning scenario, meeting the high-accuracy detection requirements and privacy protection requirements for vehicle network anomaly detection tasks in intelligent connected vehicles. Compared with traditional federal aggregation methods such as FedAvg, this method significantly improves the detection accuracy of each client in the case of non-independent and identically distributed client data and a large number of clients withdrawing from training, while maintaining a low detection latency, providing security guarantees for vehicle network privacy protection.
[0070] It should be noted that the above content only illustrates the technical idea of the present invention, and the protection scope of the present invention cannot be limited thereby. For those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can still be made, and these improvements and refinements all fall within the protection scope of the claims of the present invention.
Claims
1. The vehicle network anomaly detection system based on the federated hybrid expert model is characterized by: It includes multiple federated clients and a central parameter server. The federated client: collects the in-vehicle network communication messages generated locally, constructs a client data set after feature extraction, performs local training on the client data set using the model parameters sent by the central parameter server, and uploads the trained model parameters to the central parameter server; The central parameter server is used to collect model parameters uploaded by each client, aggregate the model parameters according to the federated average aggregation algorithm, and distribute the aggregated parameters to each client for the next round of federated training.
2. The vehicle network anomaly detection system based on the federated hybrid expert model as claimed in claim 1, characterized in that: The federated client includes at least two isomorphic anomaly detection models and a routing model, wherein the two isomorphic anomaly detection models respectively represent a federated global model for federated training and a local expert model for local fine-tuning, and the routing model is used to perform adaptive weight allocation on input data; The central parameter server fixes an anomaly detection model structure for aggregating and updating model parameters uploaded from multiple clients; The anomaly detection model at least includes a feature extraction module and an output layer, wherein the feature extraction module is composed of a nonlinear convolution block, and the output layer is composed of a fully connected layer and a softmax function to achieve anomaly category mapping; The routing model at least includes a feature extraction module and an output layer, wherein the output layer is composed of a fully connected layer and a sigmoid function, and is used to allocate prediction weights of the federated global model and the local expert model according to the output of the feature extraction module.
3. A method for detecting anomalies in an in-vehicle network based on a federated hybrid expert model using the system as claimed in claim 1, characterized in that: The steps include: S1, feature map generation and partitioning: The federated client extracts features from the locally generated vehicle network communication messages, converts fixed-length messages into feature maps using a sliding window, and constructs a feature map dataset; The feature map dataset is divided into two parts according to the vehicle data privacy protection requirements, including a global dataset for federated training of a federated client and a local dataset for fine-tuning of a federated client; S2, federated training: Each federated client downloads the initialized model weights from the central parameter server and performs local training using the local global data set. The central parameter server aggregates the model parameters of the federated clients until the number of training rounds reaches the preset value, and a federated global model is obtained. S3, client fine-tuning: Each federated client freezes the federated global model parameters obtained through training in step S2, uses the model parameters to initialize the local expert model, and performs fine-tuning training on the client's local data set to obtain the local expert model; S4, hybrid expert model fusion: Each federation client initializes a routing model to perform adaptive weight allocation on the federation global model obtained in step S2 and the local expert model obtained in step S3, and performs fusion training to obtain a hybrid expert model, thereby realizing anomaly detection of input communication messages.
4. The vehicle network anomaly detection method based on the federated hybrid expert model as claimed in claim 3, characterized in that: In the feature map generation and division of step S1, for each federated client, local communication messages of different abnormal types are collected, the window length is fixed, the Payload byte value of the message in the window is mapped to the range of 0-255, and a linear dimensionality transformation operation is performed to obtain a three-channel feature map, move to the next window, and repeat the steps until all the messages collected locally by the federated client are converted into feature map data sets. The data sets consist of two non-intersecting sets, which are used for federated training and client fine-tuning respectively.
5. The vehicle network anomaly detection method based on the federated hybrid expert model as claimed in claim 3, characterized in that: The step S2 of federated training specifically includes the following steps: S21: The central parameter server initializes the anomaly detection model parameters w g , the federated client initializes the federated global model Parameters Local Expert Model Parameters And the routing model i parameter S22: In each round of training, a fixed number of federated clients are randomly selected as the federated training clients of this round. For each federated client, the model parameters of the central parameter server are used to initialize the federated global model parameters of the federated client. Leverage global datasets from federated clients for local training The parameters Upload to the central parameter server; among them, is a global dataset used for federated training; S23: The central parameter server aggregates the model parameters of the clients in this round of federated training using the federated averaging algorithm; S24: Repeat steps S22-S23 until a preset federation training round is reached, and each federation client saves the same federation global model.
6. The vehicle network anomaly detection method based on the federated hybrid expert model as claimed in claim 5, characterized in that: In the client fine-tuning step S3, for each federated client, the local expert model parameters are initialized using the federated global model parameters. Fine-tune and train local expert models using local datasets The parameters of the expert models are different.
7. The vehicle network anomaly detection method based on the federated hybrid expert model as claimed in claim 3 is characterized in that: In the step S4 of hybrid expert model fusion, the client's federated global model Local Expert Model Routing Model i Get fixed, for the input feature map x to be predicted, perform hybrid expert model prediction According to the probability distribution in the prediction vector, the input is judged as the corresponding abnormal category.