Internet of vehicles DDoS attack detection method based on enhanced robust federated learning

Abstract

The invention discloses an Internet of Vehicles DDoS attack detection method based on enhanced robust federated learning, and belongs to the technical field of Internet of Vehicles security. According to the method, a technical means of combining reinforcement learning decision optimization, robust aggregation screening and energy consumption perception scheduling is adopted, a lightweight time sequence feature extraction model (TCN-Attention) is deployed on a vehicle-mounted edge node, and a reinforcement learning agent is introduced at a server side; each vehicle node carries out model training by using the traffic data locally and uploads parameters to the server; the server intelligently decides how to aggregate the models through a reinforcement learning algorithm, and uses a BALANCE robust mechanism to screen out malicious models uploaded by abnormal nodes; and finally, a global detection model which is still stable and reliable in a dynamic attack environment is formed. According to the method disclosed by the invention, the Internet of Vehicles has self-adaptive defense capability when facing distributed denial of service (DDoS) attacks, and the security, robustness and energy efficiency of the system are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of Internet of Vehicles security, and relates to distributed intelligent security protection in the Internet of Vehicles environment, in particular to a DDoS attack detection method for the Internet of Vehicles based on reinforcement robust federated learning. BACKGROUND

[0002] With the rapid development of Internet of Vehicles (IoV) technology, real-time information exchange and collaborative decision-making are carried out between vehicle nodes through wireless communication. However, this open communication environment also brings serious network security risks, especially distributed denial of service attacks (DDoS), which paralyze the communication network through a large number of malicious traffic, leading to traffic control and vehicle system failure, and seriously threatening driving safety.

[0003] Traditional centralized intrusion detection methods need to collect a large amount of raw traffic data for unified analysis, although the detection accuracy is high, but there are the following problems:

[0004] 1. Data centralized upload will cause privacy leakage risk;

[0005] 2. Communication and computing overhead is too large, which cannot meet the real-time detection demand;

[0006] 3. When facing non-independent and identically distributed (non-IID) data, the centralized model has poor generalization ability.

[0007] Federated learning (FL) as a kind of distributed collaborative learning framework, can realize joint training of global model without sharing raw data, effectively solving the problem of privacy protection. However, traditional FL still faces two key challenges in the Internet of Vehicles scenario, one is that the data distribution between nodes is significantly different, which leads to slow model convergence and unstable global performance; the second is that some nodes may be invaded or maliciously poisoned, and the abnormal model updates uploaded by them will seriously affect the global model accuracy and security.

[0008] Therefore, how to realize a federated learning method with robustness, security and energy efficiency in the Internet of Vehicles has become a key problem in the field of DDoS attack detection. SUMMARY

[0009] The present application aims to solve the problems of poor security, low robustness and high energy consumption of traditional federated learning in the vehicle networking environment, and proposes a vehicle networking DDoS attack detection method based on reinforcement robust federated learning, which adopts the technical means of combining reinforcement learning decision optimization, robust aggregation screening and energy consumption perception scheduling, dynamically adjusts the federated aggregation strategy through the reinforcement learning agent, and combines the BALANCE algorithm to robustly screen the uploaded model, so as to realize the technical effect of stable attack detection and defense under the conditions of existing malicious nodes, non-IID data and resource constraints. This method enables the vehicle networking to have adaptive defense capability when facing distributed denial of service (DDoS) attacks, significantly improving the security, robustness and energy efficiency of the system.

[0010] A vehicle networking DDoS attack detection method based on reinforcement robust federated learning, the specific steps are as follows:

[0011] Step 1, first establish the communication relationship between vehicle nodes and edge servers in vehicle networking, initialize the TCN-Attention local model and global model parameters. Deploy reinforcement learning agent on the server, set state space, action space and reward function, initialize balance parameters γ, κ, λ to control the BALANCE screening threshold.

[0012] Step 2: Each vehicle node collects its own communication traffic sequence in a time window manner as the input of the DDoS attack detection model; perform standardization processing on each dimension to ensure that different feature dimensions are on the same scale; each vehicle node uses TCN-Attention to perform local training, extracts multi-scale time sequence features and burst attack behaviors; after training, each vehicle node uploads the updated model parameters to the server.

[0013] Step 3: The server receives the uploaded model of each node According to the BALANCE algorithm, the uploaded model parameters of any two vehicle nodes i and j Calculate the amplitude difference D mag and the direction similarity S dir of (i,j,t) two similarity indexes; if they meet the following conditions simultaneously:

[0014]

[0015] S dir (i,j,t)≤η dir

[0016] , the model of vehicle node j is accepted; the rejected model is considered as a potential abnormal node; τ(t) is an exponential decay threshold function; η dir is the direction threshold.

[0017] For the set of vehicle nodes that passed the screening Perform aggregate weight update:

[0018]

[0019] in, The next round of global model generated by the server for vehicle node i; α∈[0,1] is the confidence coefficient.

[0020] After each aggregation weight update, a stability check is performed. If the stability check condition is met after 5 consecutive aggregation weight updates, the global model after the current aggregation weight update is considered to have converged and enters the final aggregation stage.

[0021] Step 4: The server uses the set of trusted models S filtered by BALANCE. t As input, construct the state vector s t for:

[0022]

[0023] Where, ΔAcc t This represents the improvement in global model accuracy; Var(Acc) t ) represents the variance of node accuracy; For the rejection ratio; Confidence level p for system-level attacks attack,i Take the average; These are the normalized values ​​for energy consumption constraints.

[0024] According to state s t From action set a t = {Adjusting the threshold γ, adjusting the decay rate κ, adjusting the confidence weight α} selects actions and dynamically adjusts the aggregation strategy; simultaneously, the agent optimizes the aggregation strategy and updates the action value through the Q-learning algorithm:

[0025]

[0026] Where η is the learning rate and ζ is the discount factor; R t This is the reward function.

[0027] Based on the state vectors constructed above t Action set a t And the reward function, weights The is:

[0028]

[0029] Among them, u iFor node i, score the unnormalized node, u j Give the same type of rating to other nodes j in set St;

[0030]

[0031] These are the initial score for node i based on accuracy performance and the comprehensive score for node i based on accuracy × energy consumption penalty, respectively; λ E The intensity of the energy consumption penalty is controlled, and ρ is the mixing coefficient for adjusting the weight ratio.

[0032] Step 5: The server updates the global parameters based on the optimal aggregation weights obtained through reinforcement learning optimization, thus obtaining the local update model w for aggregating vehicle nodes of each trusted node in round t. (t+1) :

[0033]

[0034] The updated model is broadcast to each vehicle node for the next round of local training; the system determines convergence and outputs the final model w when the global model reaches any convergence criterion on the validation set. * .

[0035] Step 6: Detect deployment and adaptive updates.

[0036] After training is complete, the server will finally generate the global model w. * The data is distributed to the vehicle nodes; each vehicle node deploys a lightweight detection module to perform real-time detection and local retraining at fixed time intervals to achieve adaptive updates; during the detection process, the vehicle nodes upload abnormal traffic tags to the server to update the global reinforcement learning state (S). t This forms a closed-loop feedback mechanism.

[0037] The advantages of this invention are:

[0038] 1. This invention presents a DDoS attack detection method for connected vehicles based on reinforcement robust federated learning. It combines reinforcement learning with a balance screening mechanism to achieve simultaneous intelligent decision-making and anomaly detection during the model aggregation process. Its advantage lies in the system's ability to adjust aggregation strategies in real time when faced with model poisoning or abnormal nodes, thereby improving security and overall stability.

[0039] 2. This invention presents a DDoS attack detection method for vehicle-to-everything (V2X) networks based on reinforcement robust federated learning. By introducing energy consumption and latency terms into the reinforcement learning reward function, the model reduces communication and computing resource consumption while maintaining high accuracy. This design can significantly improve the energy efficiency and real-time response capability of V2X networks in actual operation, demonstrating engineering feasibility and promotional value.

[0040] 3. This invention presents a DDoS attack detection method for connected vehicles based on reinforced robust federated learning. By deploying filtering modules on both vehicle nodes and the server side, it achieves multi-layered anomaly detection and defense. This improves the overall robustness of the system, avoids the security risks caused by single-point aggregation, and is adaptable to large-scale heterogeneous connected vehicle scenarios. Attached Figure Description

[0041] Figure 1 A flowchart outlining the steps of the DDoS attack detection method for connected vehicles based on reinforced robust federated learning provided by this invention.

[0042] Figure 2 The graph shows a comparison of the detection accuracy of the DDoS attack detection method for vehicle networking based on reinforced robust federated learning provided by this invention with other solutions. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings:

[0044] This invention presents a DDoS attack detection method for connected vehicles based on reinforcement robust federated learning. Each vehicle node trains its model locally using traffic data and uploads parameters. Edge servers intelligently decide how to aggregate models using reinforcement learning algorithms and utilize a robust BALANCE mechanism to filter out malicious models uploaded by abnormal nodes. Ultimately, a stable and reliable global detection model is formed even in dynamic attack environments. Figure 1 As shown, the specific steps are as follows:

[0045] Step 1: The server and the vehicle node establish a secure communication connection via MQTT / UDP protocol to initialize the local and global model parameters of TCN-Attention.

[0046] The server generates the initial global model parameters (w) (0) This is then broadcast to all vehicle nodes; simultaneously, a node identity ID and encryption key are assigned to achieve secure access. The following initial parameters are set during this phase:

[0047] The initial screening threshold γ0 = 1.2; the exponential decay rate κ0 = 0.1; the confidence coefficient α0 = 0.7; and the softmax temperature coefficient τ = 5.0. A reinforcement learning agent (based on Q-learning) is deployed on the server, and the state space, action space, and reward function are set. The balance parameters γ, κ, and λ are initialized to control the subsequent BALANCE screening threshold.

[0048] The Temporal Convolutional Network (TCN) combined with an attention mechanism (TCN-Attention) neural network structure employed in this invention captures long-range dependencies through dilated convolutions and highlights key temporal features using the attention mechanism. This improves detection accuracy while reducing latency and computational overhead, making it suitable for real-time edge learning in vehicle-to-everything (V2X) environments. Specifically, the TCN network utilizes a multi-layer residual dilated convolutional structure.

[0049]

[0050] Wherein, the kernel width k∈[3,5], and the dilation rate r l =2 l-1 The number of channels in the TCN network is C∈[64,256]. Multi-scale temporal patterns are captured through multi-layer stacking. The hidden temporal feature sequence H is output from the last layer of the TCN network. (L) =[h1,h2,…,h T To address this, a time-attention layer is introduced:

[0051] e n =v T tanh(W h h n +b)

[0052]

[0053]

[0054] Among them, h n The output feature at time step n; α n Let z represent the attention weights at each time step, and e be the final weighted representation. n e represents the unnormalized attention score for time step n, reflecting the importance of the model to the features at that time step; j The unnormalized attention score for all time steps j (j = 1...T) is used to calculate the final attention weights using Softmax.

[0055] The aforementioned temporal attention mechanism is responsible for filtering and focusing on key features, and finally completes classification prediction through a fully connected layer and a softmax function.

[0056] Step 2: Local training of vehicle nodes.

[0057] Each vehicle node uses local vehicular network traffic data, employs a TCN-Attention neural network structure to extract temporal features, and trains a DDoS attack detection model to obtain locally updated parameters. The superscript 't' represents the global model epoch at the start of the current training epoch. The superscript '1 / 2' (half an epoch) indicates the intermediate result after the update in the current epoch, which has not yet been aggregated by the server and only reflects the update result of the vehicle node on the local data in the current epoch. Data source: The CICDDoS2019 dataset is used as the data source for vehicle-to-everything (V2X) intrusion detection. 45-dimensional network traffic features are extracted, including packet length, protocol type, inter-packet latency, signal power, and bandwidth utilization, generating 10,000 communication traffic sequences. Each vehicle node stores 10% of the communication traffic sequences for distributed local training.

[0058] Each vehicle node collects its own communication traffic sequence using a time window approach. As input to the DDoS attack detection model:

[0059]

[0060] A sliding window with length T = 50 and step size s = 5 is used. Feature dimension d x This represents the original traffic statistics dimension. Standardization is performed on each dimension:

[0061]

[0062] Ensure that different feature dimensions are on the same scale. The eigenvalues ​​obtained after standardization; x t,j The original flow eigenvalues ​​of time step n on feature dimension j; The mean of the j-th feature dimension within the current window; is the standard deviation of the j-th feature dimension within the current window; ∈ is a very small positive constant used to prevent the denominator from being zero and to ensure numerical stability, usually taken as 10^-6 to 10^-8.

[0063] Each vehicle node employs a TCN-Attention network structure instead of the traditional CNN-LSTM for local training to extract multi-scale temporal features and sudden attack behaviors. The TCN-Attention model structure is as follows:

[0064] h t =ReLU(W f ·x t-d:t +b f )

[0065]

[0066]

[0067] Where d is the expansion coefficient; W a W is the attention weight matrix. f... t-d:t The input segment for the dilated convolution is the set of sampled vectors of the input sequence at time step (td, t-2d, ..., t); b f This is the bias vector corresponding to the TCN convolutional layer; The TCN-Attention mechanism scores the importance of the hidden state at time step t to the overall features. This structure captures multi-layered temporally dependent features while enhancing the response during anomalous periods through the attention mechanism. On the same dataset, the TCN-Attention model reduces computation by approximately 40% compared to CNN-LSTM, while improving accuracy by approximately 5.7%.

[0068] The loss function is cross-entropy:

[0069]

[0070] Among them, y c The true class label of the input sample at class c is represented by one-shot encoding to indicate the true classification result, which is used for supervised training; p c (w) represents the predicted probability of class c output by the model with all learnable parameters w.

[0071] The optimizer uses Adam, with a learning rate η∈

[10] . -4 10 -3 Batch size B∈32,64.

[0072] The training process ends under the following conditions:

[0073] a. Reaching the maximum number of local iteration rounds E max =5;

[0074] b. The accuracy improvement in two consecutive rounds of verification is less than the threshold ∈ acc =0.005.

[0075] After training is complete, each vehicle node will update the model parameters locally. Upload to the server. Upload frequency and model upload strategy are dynamically adjusted by the reinforcement learning module. Each node has ≥1GB of memory, and the time taken for a single batch of forward-backward propagation is ≤200ms. The local model size is ≤10MB to accommodate onboard computing limitations.

[0076] The aforementioned vehicle node local training is completed on the vehicle itself, without uploading raw data, thereby protecting user privacy and reducing communication pressure.

[0077] Step 3: Robust screening based on the BALANCE algorithm by the server.

[0078] The server receives the models uploaded by each node. According to the BALANCE algorithm, the uploaded model parameters are processed for any two vehicle nodes i and j respectively. Calculate the following two similarity metrics to determine whether two objects belong to a trustworthy neighborhood:

[0079] 1. Amplitude difference:

[0080]

[0081] 2. Directional similarity:

[0082]

[0083] Let the exponential decay threshold function be:

[0084] τ(t)=γe -κλ(t)

[0085] Where γ∈[0.5,2.0] is the initial relaxation coefficient; κ∈[0.5,5.0] controls the tightening speed; λ(t)=t / T is the normalized round. If simultaneously:

[0086]

[0087] S dir (i,j,t)≤η dir

[0088] Then the model for vehicle node j is accepted, η dir The directional threshold is defined as [0.05, 0.2]. Rejected models are considered potential anomalous nodes. This step effectively filters out abnormal parameters uploaded by malicious nodes, preventing model poisoning.

[0089] The above method uses an exponential decay threshold γ t By progressively tightening the selection criteria, more sample differences are incorporated in the early training phase, improving aggregation robustness in later stages. If a node's model deviates too much from the mean, it is considered an anomaly and aggregation is rejected. This method can effectively resist poisoning attacks from 10% to 20% of nodes, keeping the global accuracy drop below 2%, while traditional FedAvg experiences a drop of over 8% under the same conditions.

[0090] For the set of vehicle nodes that passed the screening Perform aggregate weight update:

[0091]

[0092] in, The next round of global model generated by the server for vehicle node i; The local update model for vehicle node j generated by the server; α∈[0,1] is the confidence coefficient.

[0093] After each aggregated weight update, a stability check is performed. The criteria are: the accuracy improvement is less than 0.001, and the fluctuation of the global loss function is less than 10^-4. If the stability criteria are met after 5 consecutive rounds of aggregated weight updates, the global model after the current round of aggregated weight updates is considered to have converged, and the model enters the final aggregation stage.

[0094] Step 4: Dynamic aggregation weight optimization based on reinforcement learning agents.

[0095] In the robustly selected model set, the server dynamically assigns aggregate weights to each vehicle node through a reinforcement learning (Q-learning) agent. Simultaneously adjust control parameters such as thresholds to achieve the optimal balance between detection accuracy, robustness and safety, and resource overhead (energy consumption, latency, bandwidth).

[0096] The server uses a set of trusted models S filtered by BALANCE. t As input, construct the state vector s t s t Let the discrete / continuous state vector of the agent in round t be defined as a set of important indicators that reflect the current performance, risk, and resource consumption of the system:

[0097]

[0098] Where, ΔAcc t ΔAcc represents the improvement in global model accuracy, indicating the increase in accuracy of the global model in round t compared to the previous round. t =Acc t -Acc t-1 Use percentages (e.g., 93.5%) or decimals (e.g., 0.935); Var(Acc) t ) represents the variance of node precision. To reject the proportion, Confidence level p for system-level attacks attack,i Take the average. Energy consumption constraints With bandwidth constraints Normalized value. E max D represents the maximum allowable energy consumption threshold of the system. max This is the maximum allowed latency threshold for the system.

[0099] According to state s t From action set at = {Adjusting the threshold γ, adjusting the decay rate κ, adjusting the confidence weight α} to select actions, and dynamically adjust the aggregation strategy. Simultaneously, the agent optimizes the aggregation strategy and updates action values ​​using the Q-learning algorithm.

[0100]

[0101] Where η is the learning rate, ζ is the discount factor, and R0 t Here, a' represents any action among all available actions in the next state, used to calculate the maximum action value that can be obtained in the next step.

[0102] The confidence levels of the above system-level attacks are averaged. The formula is:

[0103]

[0104] Define the mean and standard deviation μ of the population distance distribution. d ,σ d The node attack probability estimation method is (z-score→Sigmoid):

[0105]

[0106] Where, σ sig λ represents the SigMoid activation function; p >0 controls kurtosis, ∈ prevents division by zero. d i Calculate the local anomaly degree (based on similarity / distance) for a single node i:

[0107]

[0108] Where med(.) represents the vector of median of the elements (or the global model of the previous round is selected).

[0109] The above action set a t The construction method is as follows:

[0110] The action space is designed as a discrete set, and a mapping is given:

[0111] a 1 Adjusting γ (threshold size) – three levels: {loose, nominal, strict} corresponding to γ ​​values ​​{γ0·1.3, γ0, γ0·0.7}. γ0 is the default relaxation factor representing the robust aggregation screening threshold, used to control the tightness of the model's upload distance distribution. Its value can be obtained through validation set parameter tuning.

[0112] a 2Adjusting κ (threshold size) – three levels: {decrease, keep, increase} corresponding to κ values ​​{κ0·0.8, κ0, κ0·1.2}. κ0 represents the baseline decay rate of the decay function, used to control the convergence speed of the screening threshold with training rounds, and is a preset positive hyperparameter.

[0113] a 3 Adjust the local confidence weight α – three levels: {low, med, high}, corresponding to α values ​​of {0.55, 0.7, 0.85}.

[0114] a 4 Weight allocation strategy selection—{accuracy-based, energy-aware, hybrid}, where accuracy-based is sorted by local verification Acc and mapped to weights, and energy-aware multiplies the weight of each node by . Correction: The hybrid approach uses a weighted sum strategy, accuracy × (1 - λ * energy).

[0115] The above a 1 a 2 a 3 a 4 There are four types of actions in the action space, and each type of action will adjust different hyperparameters of the global aggregation.

[0116] Combine the actions into a discrete action vector a t =(a 1 ,a 2 ,a 3 ,a 4 If there is too much room for movement, you can select only some of the movements.

[0117] For the above reward function, first give the original reward:

[0118]

[0119] To ensure numerical stability and training effectiveness, we first normalize the terms:

[0120]

[0121] Introducing the normalized terms into the original reward function, we obtain the normalized reward function as follows:

[0122]

[0123] The weights α1 = 1.0, β1 = 0.8, γ1 = 0.5, and δ1 = 0.3 are adjusted according to specific conditions. The normalized reward function is used as the core learning signal in Q-learning to update the Q-value, thereby influencing the agent's next action 'a'. t+1 The choice.

[0124] The aforementioned rewards and constraints (the energy consumption and latency terms related to the reward function) refer to the energy consumption (local training) estimated by vehicle node i (estimating the computational cost required for each node i to complete local training) that needs to be quantified in the preceding text.

[0125] E comp,i =η comp FLOPs i

[0126] Among them, E comp,i For the local training energy consumption estimation of vehicle node i; η comp The energy consumption factor for a single floating-point operation (Joules / FLOPs), FLOPs i It can be estimated by model size × local epochs × batches. Communication energy consumption for parameter upload and download:

[0127] E comm,i =η tx ·Bytes i

[0128] Where, η tx Energy consumption per byte of data transmitted.

[0129] Therefore, the total energy consumption is:

[0130]

[0131] E i =E comp,i +E comm,i

[0132] End-to-end delay estimation D i :

[0133] D i =D up,i +D agg +D down,i

[0134] Among them, D up,i It's an upload delay, D agg It is the server aggregation time, D down,i It's a delay in distribution.

[0135] Based on the state vectors constructed above t Action set at And the reward function, weights The calculation method is as follows:

[0136] Given action a t First, calculate the original node score s. i Then map it to the weight simplex:

[0137]

[0138] Where, λ E ∈[0,1] controls the intensity of energy consumption penalty. These are the initial score of node i based on accuracy performance and the comprehensive score of node i based on accuracy × energy consumption penalty, respectively.

[0139] The final score is weighted (Softmax / Projection) by first calculating the unnormalized score u. i (Depending on the action chosen):

[0140]

[0141] In the formula, ρ is a mixing coefficient used to adjust the weight ratio of "accuracy score" and "energy consumption score".

[0142] Then map to the simple form of the weights (guaranteed to be non-negative and sum to 1):

[0143]

[0144] Where τ>0 is the temperature parameter (the larger the value, the closer it is to winner-take-all). i For node i, score the unnormalized node, u j Give the same type of rating to other nodes j in set S_t.

[0145] This strategy can dynamically balance accuracy and energy efficiency under high-load scenarios. Simulation results show that average energy consumption decreases by 17% and polymerization accuracy improves by 3.4%. The proposed solution also includes a degradation / protection strategy. (If all nodes are eliminated), then a backup strategy is adopted: select the k nodes with the smallest distance from the global model. Perform aggregation, rather than complete interruption. If the p of a node i... attack,i >p th (e.g., p) th If the value is less than or equal to 0.9, it is blacklisted and logged. After M consecutive anomalies (e.g., M=3), it is forcibly isolated. A safety constraint is introduced for action selection in Q-learning: no action may cause... Exceeding [0, 1-∈] (to prevent single-point weight monopoly, ∈ being too small).

[0146] Step 5: Global model update and broadcast.

[0147] The server receives local model parameters from all trusted nodes. Based on the reinforcement learning decision in step 4, an aggregate weight vector is generated. The weighted aggregation update formula is executed to calculate the local update model w of each trusted node in round t. (t+1) :

[0148]

[0149] in, satisfy This global model will be broadcast to all vehicle nodes as the initial model for local training in round t+1.

[0150] Then the updated model parameters are broadcast to all nodes, which receive them, replace their local models, and continue the next round of training.

[0151] The entire training process continues between each node and the server until the global model reaches any of the following convergence conditions on the validation set. To ensure the repeatability and numerical stability of the convergence judgment, this scheme defines convergence criteria at three levels.

[0152] 1. Accuracy Variation Convergence Criterion: When the variation in validation set accuracy between two adjacent rounds is lower than a set threshold, the model accuracy is considered to have stabilized.

[0153] |Acc t -Acc t-1 |<ε acc

[0154] Among them, Acc t Let ε represent the accuracy of the validation set in round t. acc Generally, 10 is taken. -3 ~10 -4 To avoid misjudgments due to instantaneous oscillations, a moving average method can be used for judgment.

[0155]

[0156] This indicates that the average improvement of K consecutively is below the threshold.

[0157] 2. Convergence condition for parameter changes: When the global model parameter update magnitude tends to stabilize, the model is considered to have reached an approximate optimal solution in the optimization space.

[0158]

[0159] Where, ε w This is the threshold for parameter variation, typically set to 10. -3 .

[0160] 3. Reward and Resource Convergence Criteria: For reinforcement learning-driven aggregation systems, if the agent's reward function tends to stabilize over multiple consecutive rounds, the policy is considered to have converged.

[0161] |R t -R t-1 |<ε R

[0162] Meanwhile, the fluctuations in energy consumption and bandwidth overhead remain within the set range.

[0163]

[0164] Where, ε R ,ε E ,ε B The empirical threshold (taken as 10) -3 ~10 -2 ).

[0165] Once the reinforcement learning-driven aggregation strategy converges, the global model obtained from the final aggregation round is denoted as the final model w. * .

[0166] Step 6: Detect deployment and adaptive updates.

[0167] After training is complete, the server will finally generate the global model w. * The data is distributed to vehicle nodes. Each node deploys a lightweight detection module to perform real-time detection and local retraining at fixed time intervals (e.g., every 5 minutes) to achieve adaptive updates. During the detection process, nodes upload abnormal traffic tags to the server to update the global reinforcement learning state (S). t This forms a closed-loop feedback mechanism. This mechanism implements a three-layer adaptive loop of "detection-aggregation-update," effectively improving the long-term stability and intelligent evolution capabilities of the vehicle-to-everything (V2X) security system. The final converged global model is deployed to the V2X edge nodes for real-time DDoS attack detection. During operation, if the detection results are abnormal or the system is attacked, the reinforcement learning module can be restarted to dynamically adjust the screening threshold and aggregation strategy, achieving continuous robust learning.

[0168] like Figure 2 As shown in the figure, the proposed solution was compared with traditional FedAvg and FedProx through simulation experiments. The results are shown in the table below:

[0169]

[0170] The results show that the present invention outperforms existing algorithms in terms of accuracy, convergence speed and energy efficiency, and remains stable even when malicious nodes are present.

Claims

1. A vehicular Internet of Things (IoV) DDoS attack detection method based on reinforcement robust federated learning, characterized in that: The specific steps are: Step 1: First, establish the communication relationship between vehicle nodes and edge servers in the vehicle-to-everything (V2X) network, and initialize the parameters of the TCN-Attention local and global models; deploy reinforcement learning agents on the server, set the state space, action space, and reward function, and initialize the balance parameters γ, κ, and λ to control the BALANCE screening threshold; Step 2: Each vehicle node collects its own communication traffic sequence in a time window manner. The parameters are used as input to the DDoS attack detection model; standardization is performed on each dimension to ensure that different feature dimensions are on the same scale; each vehicle node uses TCN-Attention to perform local training to extract multi-scale temporal features and sudden attack behaviors; after training, each vehicle node updates the model parameters locally. Uploaded to the server; Step 3: , the upload model parameters of any two vehicle nodes i and j are calculated respectively according to the BALANCE algorithm , , the amplitude difference and the direction similarity two similarity indexes; if both are satisfied: then the model of vehicle node j is accepted; rejected models are considered potential outlier nodes; is an exponential decay threshold function; is a directional threshold; To the set of vehicle nodes that pass the screening Perform aggregated weight update: wherein, Gi+1is the next round global model generated by the server for the vehicle node i; [0,1] is a confidence coefficient; Stability judgment is performed after each aggregation weight update. When the stability judgment condition is met after 5 consecutive rounds of aggregation weight update, it is considered that the global model after the current round of aggregation weight update has converged, and the final aggregation stage is entered; Step 4: Server with trusted model set filtered by BALANCE As input, construct state vector is: wherein, is the amount of improvement in global model accuracy; is the node precision variance; is the rejection ratio; is the system-level attack confidence is taken as the average; , are the energy consumption constraint normalized values, respectively; According to the state Select an action from the action set ={adjust threshold , adjust decay rate , adjust confidence weight } to dynamically adjust the aggregation strategy; at the same time, the agent optimizes the action value of the aggregation strategy through the Q-learning algorithm: wherein, is a learning rate, is a discount factor; is a reward function; denotes an arbitrary one of all possible actions in the next state; Based on the state vector constructed above , the action set , and the reward function, the weight is: wherein, is the un-normalized node score for node i, is the same type score for other nodes j in the set St; , are the initial score of node i based on accuracy performance and the initial score of node i based on precision x energy consumption, respectively a combined score of the penalties; controlling the strength of the energy consumption penalty, a mixing coefficient for adjusting the weight proportions Step 5: The server updates the global parameters according to the optimal aggregation weight obtained by reinforcement learning optimization, to obtain the local update model of each trusted vehicle node in the tthround of aggregation : The updated model is broadcast to each vehicle node for the next round of local training; until the global model reaches any one of the convergence conditions on the validation set, the system determines convergence and outputs the final model ; Step 6: detection and adaptive update; After the training is completed, the server will issue the final global model to the vehicle nodes; each vehicle node deploys a lightweight detection module to perform real-time detection and local retraining at a fixed time period, achieving adaptive updating; during the detection process, the vehicle nodes upload the abnormal traffic labels to the server for updating the global reinforcement learning state , forming a closed-loop feedback mechanism.

2. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: The TCN network of the TCN-Attention local model uses a multi-layer residual dilated convolution structure: wherein is the convolution kernel width, dilation rate; the hidden time sequence feature sequence output by the last layer of the TCN network on the top, introduce the time attention layer: wherein, denotes the attention weight for each time step, is the final weighted representation, is the unnormalized attention score for time step reflecting the importance of the feature of this time step to the model; is the unnormalized attention score for all time steps j (j = 1...T) for the Softmax computation of the final attention weight, and finally the classification prediction is completed through the fully connected layer and the Softmax function.

3. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: In step 2, the loss function is cross-entropy: wherein, is the value of the true class label of the input sample at class c, encoded in one-shot, representing the true classification result, used for supervised training; is the total learnable parameters of the model ;​ Optimizer Adam, learning rate , batch size ; The end condition of the training process is as follows: a. maximum number of local iteration rounds reached ; b. the accuracy improvement of two consecutive rounds of verification is less than a threshold .

4. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: In step 4, is: Mean and standard deviation defining the overall distance distribution where the node attack probability estimation is given by wherein, represents a SigMoid activation function; controls the steepness, prevents division by 0; for a single node computes the local anomaly degree: Where med(.) represents the median vector of the elements or the last round global model.

5. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: In step 4, the set of actions The construction method is as follows: The action space is designed as a discrete set, and the mapping is given: : adjust , with three optional values: {loose, nominal, strict} corresponding to ; default relaxation factor for representing robust aggregation filtering threshold : adjust , with three possible values: {decrease, keep, increase} corresponding to values { }; denotes the reference decay rate of the decay function; Adjust local confidence weights , with three optional values: {low, med, high} corresponding to values {0.55, 0.7, 0.85}; Weight allocation strategy selection, including {accuracy-based, energy-aware, hybrid}, accuracy-based sorts and maps to weights by local verification Acc, energy-aware multiplies each node weight by Correction, hybrid adopts weighted sum strategy accuracy x (1 - λ * energy); The above , , , are four types of actions in the action space, each of which adjusts a different hyperparameter of the global aggregation; the actions are combined into a discrete action vector .

6. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: In step 4, the original reward is given: Normalize each item: Introduce the normalized items into the original reward function to obtain the normalized reward function: wherein the weights According to specific adjustments.

7. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: In step 4, the energy consumption is calculated by the estimated vehicle node i: wherein, local training energy estimate for vehicle node i; single floating point operation energy coefficient, may be estimated by model size x local epochs x batchs; for communication energy of parameter upload and download: wherein, is the unit byte transmission energy consumption; And the total energy consumption is: End-to-end delay estimation : wherein, is the upload delay, is the server aggregation latency, is the download delay.

8. The DDoS attack detection method based on reinforcement robust federated learning for V2X according to claim 1, wherein: In step 5, the convergence criteria are defined at three levels: a. Precision change convergence condition: when the accuracy of the validation set changes by less than a set threshold between adjacent two rounds, it is considered that the model accuracy has stabilized: wherein, represents the first wheel validation set accuracy, take ; b. Parameter change convergence condition: when the global model parameter update amplitude tends to be stable, it is considered that the model has reached an approximate optimal solution in the optimization space: wherein is a parameter variation threshold value; c. Reward and resource convergence condition: for reinforcement learning driven aggregation system, if the reward function of the agent tends to be stable for consecutive rounds, it is considered that the strategy has converged: At the same time, the change amplitude of energy consumption and bandwidth overhead is kept within a set range wherein is an empirical threshold value.

Images