A security defense method against edge federation network attacks

Abstract

This invention discloses a security defense method against edge federated network attacks, relating to the field of artificial intelligence security technology. Edge servers and local devices jointly train a global model by exchanging model parameters, ensuring data privacy protection. Each participant trains the model locally and performs anomaly detection using a density-based spatial clustering algorithm. Updated model parameters are uploaded to the edge server, where the global model is optimized and robustly aggregated using a TD3 deep reinforcement learning network, further enhancing the model's defense against potential attacks. Finally, the edge server distributes the optimized global model to all participants, ensuring continuous model updates and system security. This invention, by combining edge computing and deep reinforcement learning technologies, effectively enhances the system's protection against poisoning attacks and adversarial attacks.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to a security defense method against edge federation network attacks. Background Technology

[0002] Edge federated networks are an emerging network architecture that combines the real-time capabilities of edge computing with the privacy-preserving advantages of federated learning. They enable efficient data processing and distributed model training in dynamic and complex environments. By sharing model updates among participants, rather than raw data, they effectively address data privacy concerns and support distributed, seamless collaborative work. Due to the flexibility and efficiency demonstrated by edge federated learning mechanisms across various tasks, they are widely used in fields such as high-performance computing, intelligent analytics, and security protection, providing a unique solution, especially in environments where traditional cloud servers cannot be relied upon.

[0003] However, with the widespread application of edge federated networks, cybersecurity has gradually become a key challenge restricting their development. Due to the distributed nature of edge computing and federated learning models, network security faces severe challenges. Especially in open and dynamic environments, malicious nodes may interfere with the normal operation of the system through various attack methods. These malicious nodes can not only forge data uploads and tamper with model parameters, but may also cause data leaks or a sharp decline in model performance through malicious behavior, or even paralyze the entire system.

[0004] Therefore, in-depth research into attack methods and potential threats in edge federated networks, and the exploration of effective defense strategies, is particularly important. This not only ensures data privacy protection but also enhances the system's robustness and resilience. Especially with the trend towards intelligence and autonomy among heterogeneous nodes with large-scale access, cybersecurity issues will become more complex and diverse. Only through innovative defense mechanisms can the long-term stable operation of edge federated networks in complex environments be ensured. Summary of the Invention

[0005] The technical problem to be solved by this invention is to design defense methods against data poisoning attacks, model poisoning attacks, and adversarial example attacks that exist in the training of edge federated networks. On the one hand, this will enable the edge federated network model to maintain excellent performance, and on the other hand, reduce the impact of external malicious attacks on the expected performance of the model during the training phase, thereby ensuring the robustness of the edge federated network artificial intelligence model.

[0006] The technical solution adopted by this invention to solve the above-mentioned technical problems is to provide a security defense method against edge federation network attacks. This method involves anomaly detection and removal of abnormal data from the local dataset before local training on ground terminal devices. During parameter aggregation, the differences in data quality and quantity among different ground terminal devices are considered, and weights are assigned to the aggregation model parameters of each participating device. Then, deep reinforcement learning is used to enable the system to dynamically adjust the aggregation strategy based on the current network state and attack situation, enhancing its resistance to data poisoning, model poisoning, and adversarial sample attacks. The method includes the following steps:

[0007] S1: Edge server initializes global model parameters Distribute to all participating parties;

[0008] S2: In ground terminal equipment Before local training, a density-based spatial clustering algorithm (DBSCAN) was used on the local dataset. Perform anomaly detection. If an abnormal data point is detected... ( If the data point is indexed, then it should be cleaned or corrected.

[0009] S3: Each participant uses local data to train the model and calculates updates to the model parameters. ;

[0010] S4: Upload the locally updated parameters to the edge node acting as a wireless relay;

[0011] S5: After receiving the model parameters uploaded by the ground terminal equipment, the edge node performs model anomaly detection and calculates the mean of the model parameters in each dimension. and variance ,in For parameter dimensions;

[0012] S6: The edge server uses the parameters uploaded by the edge nodes to train a deep reinforcement learning network based on Siamese Delay Deep Deterministic Policy Gradient (TD3);

[0013] S7: Robustly aggregate the updated parameters using a deep reinforcement learning algorithm to calculate the new parameters of the global model. ;

[0014] S8: The edge server will update the new parameters of the global model. The data is then distributed to all participating parties, who update their local models and proceed to the next round of training.

[0015] This invention fully leverages the dynamic decision-making advantages of deep reinforcement learning by deeply integrating the TD3 algorithm with a federated learning mechanism to construct a robust optimization framework capable of adaptively adjusting weighted aggregation strategies. This method, through an edge server-led reinforcement learning interaction mechanism combined with network environment state awareness, continuous action space mapping, and multi-dimensional reward feedback, significantly improves the stability and accuracy of the global model under adversarial attack scenarios. Specifically, the system maintains model convergence performance during normal training and retains reliable model quality when encountering data poisoning, model poisoning, or adversarial attacks, while continuously optimizing the defense effect through an intelligent dynamic weight allocation mechanism.

[0016] The beneficial effects of this invention are:

[0017] (1) By constructing a deep reinforcement learning framework, the system can dynamically generate weighted aggregation strategies based on real-time network status and attack characteristics, thereby improving the adaptive defense capability in adversarial environments.

[0018] (2) Based on the continuous action space optimization characteristics of the TD3 algorithm, a multi-dimensional state space including model parameter features, equipment status and performance indicators, as well as a continuously weighted coefficient action space that can be finely adjusted, is designed to ensure the flexibility and optimality of the weight allocation strategy.

[0019] (3) By designing a multi-objective reward function that integrates model performance improvement and attack resistance, the agent is guided to systematically enhance its defense capabilities against various poisoning attacks and adversarial examples while ensuring model accuracy. It is applicable to large-scale heterogeneous node access environments under poisoning attacks and adversarial example attacks, and has strong adaptability and scalability.

[0020] (4) Construct a federated learning-reinforcement learning dual-loop architecture dominated by edge servers. While maintaining the distributed advantages of federated learning, the robustness of the global model is enhanced through centralized strategy optimization. Attached Figure Description

[0021] Figure 1 A schematic diagram of a federated network architecture with multiple edge nodes (following drones are edge nodes, and lead drones are edge servers).

[0022] Figure 2 A robust weighted aggregation algorithm based on anomaly detection and deep reinforcement learning. Detailed Implementation

[0023] The technical solution of a security defense method against edge federation network attacks according to the present invention is described in further detail below, but the scope of protection of the present invention is not limited to the following description.

[0024] This embodiment is based on the following scenario: In a multi-edge node federated network architecture, it consists of one lead drone, multiple follower drones, and multiple ground terminal devices. The lead drone is responsible for the final global model aggregation. The follower drones collect model parameters from the ground terminal devices within their respective areas and transmit the aggregated parameters to the lead drone. The ground terminal devices perform local data collection, processing, and local model training.

[0025] The collaborative training process of edge federated networks can be divided into three stages: data collection, model training, and model inference. In edge federated networks, data collection follows the federated learning principle; the dataset does not leave the local machine, primarily involving the collection of local data and preparatory work such as negotiating data formats among users. Malicious attackers can compromise the integrity of the training dataset during the data collection stage. Common methods include dirty label attacks, which tamper with dataset labels to cause the model to learn incorrect information, thus affecting the model's training performance; or a single attacker using multiple collusive identities to join the system, increasing the attack's stealth and effectiveness, often combined with poisoning attacks to disrupt the normal learning process. The model training stage aims to iteratively optimize model parameters to accurately fit the data and uncover its potential value. During this stage, malicious attackers may control the training phase of some ground terminals, tampering with uploaded local models in an attempt to manipulate the global model. To enhance stealth, methods such as transforming the optimization process into an optimization problem may be used to make toxic updates difficult to detect.

[0026] In response to the various attack methods described above, this embodiment provides a security defense method against edge federation network attacks, as follows: Figure 2 As shown, the specific steps include: using a multi-strategy fusion-based anomaly detection and robust aggregation algorithm, which integrates anomaly detection, robust aggregation, and deep reinforcement learning strategies, to ensure the stability and accuracy of edge federated network training.

[0027] S1: The edge server initializes global model parameters, setting global parameters for federated learning, such as the global update round T and the number of terminals participating in training N. A suitable deep learning model (a convolutional neural network CNN is used in this embodiment, but the specific model can be chosen and set according to the specific scenario) is selected as the global model, and its parameters are randomly initialized. The Actor network of the TD3 algorithm is initialized. Critic Network and and its target network , and Set the parameters. Configure hyperparameters such as learning rate, discount factor, policy delay update frequency, noise standard deviation, and target network update coefficients. Set the neighborhood radius for the DBSCAN anomaly detection algorithm. and minimum points parameter;

[0028] S2: Before local training on the ground terminal equipment, a density-based spatial clustering algorithm (DBSCAN) is used to detect anomalies on the local dataset (SVHN dataset in this embodiment). For each ground terminal equipment... Let its local dataset be ,in It is the number of samples in the local dataset. It is the first One sample, For image data, These are the corresponding numerical labels. By calculating the density connectivity between data points, densely connected points are divided into different clusters, and points with a density below a set threshold are considered outliers.

[0029] Specifically, for the SVHN dataset, the density of image data in the feature space is calculated. Among them, the neighborhood , representing all distances Less than or equal to the neighborhood radius The sample, It can be calculated using Euclidean distance or cosine similarity. Identify key points, construct clusters, and mark isolated image samples (such as images with features significantly different from surrounding samples) as anomalies. If anomaly data points are detected... ( If the data point is indexed, then it should be cleaned or corrected.

[0030] S3: Local ground terminal devices use the cleaned local data to train the CNN model and update the local model parameters;

[0031] S4: Upload the updated local model parameters to the edge node (following the drone), such as... Figure 1 As shown;

[0032] S5: After receiving the local model parameters uploaded by the ground terminal equipment, the edge node performs model anomaly detection and calculates the statistical characteristics of each model parameter, such as mean and variance. For the first... Each edge node receives data from a set of ground terminal devices. Model parameters Calculate the mean of the model parameters across all dimensions. and variance ,in For parameter dimensions.

[0033]

[0034]

[0035] If the model parameters uploaded by a certain ground terminal device deviate from the mean by more than a set threshold (such as 3 times the standard deviation) in multiple dimensions, the model is determined to be potentially poisoned, marked as an abnormal model, and cleaned or corrected.

[0036] S6: Train the TD3 deep reinforcement learning network using the uploaded parameters;

[0037] In a robust weighted aggregation scenario within an edge federated network, a state space, action space, and reward function are constructed, with edge servers as the primary components. State Space This includes model parameter features uploaded by edge nodes, information from ground terminal devices, and current model performance metrics; action space. These are continuous weighted aggregation coefficients used to adjust the weights of edge node model parameters in global aggregation; reward function. The algorithm is designed based on the performance improvement of the aggregated model on the validation set and its resistance to attacks, guiding TD3 to learn the optimal weighted aggregation strategy. The specific algorithm flow is as follows:

[0038] S601: State Construction; Before aggregating model parameters, the edge server collects the model parameters uploaded by the edge nodes. Calculate the mean of each dimension of the model parameters. ,variance Statistical characteristics. Obtain the number of ground terminal devices covered by each edge node. Combined with the current global model's accuracy on the validation set. Performance indicators such as loss value Construct state vector As input to the TD3 algorithm.

[0039] S602: Action Generation; The Actor network outputs a deterministic action based on the current state. The action is represented as a set of continuous weighted coefficients. ( , (Number of edge nodes), used for weighted aggregation. To increase exploratory power, a Gaussian-distributed noise is added to the action. The actual action performed is .

[0040] S603: Weighted aggregation; the edge server calculates the aggregation based on the actual actions performed, i.e., the weighting coefficients. Global model aggregation is performed. A weighted aggregation formula is used:

[0041]

[0042] in For the first The parameters of the edge node aggregation model These are the updated global model parameters.

[0043] S604: Reward Calculation; The updated global model is applied to the validation set to calculate the model's accuracy and loss. A reward is calculated based on the model's performance improvement and its effectiveness in resisting attacks. The reward function is designed as follows:

[0044]

[0045] in These are the weighting coefficients. , To update the changes in the model's accuracy and loss value on the validation set before and after the update, Rewards for defending against attacks.

[0046] S605: Experience playback and learning; state ,action ,award and the next state Forming tuples The tuples are stored in the experience replay pool. When the number of tuples in the experience replay pool reaches a certain threshold, a batch of tuples is randomly selected from the experience replay pool for learning.

[0047] Perform Critic network updates and calculate target values. :

[0048]

[0049] in For the target Critic network, For the target Actor network, Here is the entropy regularization coefficient. It's added noise. This is achieved by minimizing the loss function. Update the Critic network parameters using the gradient descent algorithm:

[0050]

[0051] in, This represents the number of samples drawn.

[0052] Perform Actor network updates to maximize the objective function. Update the parameters of the Actor network using the gradient ascent algorithm:

[0053]

[0054] To update the target network, periodically update the parameters of the target Actor network and the target Critic network to gradually approach the parameters of the current network. The update formula is as follows:

[0055]

[0056] in For the target network update coefficients, For the current network parameters, These are the target network parameters.

[0057] S606: Iterative loop;

[0058] Repeat steps S602 to S606. As the number of training rounds increases, TD3 continuously learns and optimizes the weighted aggregation strategy to adapt to different attack situations and network states, achieving robust weighted aggregation and improving the training performance of the edge federated network.

[0059] S7: Robustly aggregate the updated parameters using a deep reinforcement learning algorithm to calculate the new parameters of the global model. In this step, the edge server uses a deep reinforcement learning (DRL) algorithm to robustly aggregate local model parameters uploaded from each participant. First, the edge server compares the model parameters from each participant with the local global model and weights the aggregated model parameters according to each participant's contribution weight (e.g., amount of training data, computing power, etc.). To improve the model's robustness in dynamic environments, the DRL algorithm evaluates the current aggregated model's resistance to potential attacks and adjusts the aggregation strategy based on the reward mechanism in reinforcement learning. This process is implemented using the TD3 algorithm, ensuring that the model continuously optimizes and enhances its defense capabilities over multiple training rounds.

[0060] Specifically, the TD3 algorithm uses an Actor-Critic architecture to evaluate the performance of the current aggregation parameters and update the global model's parameters by optimizing the aggregation strategy. During this process, the model adjusts to environmental feedback (i.e., the model parameters uploaded by participants and the actual attack defense capabilities) and maintains its robustness in constantly changing attack environments. Through repeated training, the deep reinforcement learning algorithm ensures that the global model can better adapt to changes in different scenarios, ultimately calculating the optimal global model parameters and providing strong support for the next step of model updates.

[0061] S8: The edge server will update the new parameters of the global model. The new global model parameters, processed by the deep reinforcement learning algorithm, are transmitted wirelessly to each participant (such as other drones or ground terminal devices). The edge server, acting as the central control unit, ensures the accurate distribution of these parameters to each participant. Upon receiving the new global model parameters, each participant uses them to update its local model for further local training and adjustments. The system automatically identifies the characteristics and needs of different devices, optimizing the global model's performance while ensuring security.

[0062] Once all participants have updated their local models and trained them with the new parameters, the system will enter the next training cycle. Each training cycle optimizes based on the results of the previous cycle to ensure that the system continuously improves its overall performance during training. If, during this process, the system has achieved the predetermined target performance (e.g., meeting set standards in attack / defense capabilities, model accuracy, etc.) through validation set evaluation, the training process will stop, completing the final model optimization. A dynamic learning rate adjustment strategy is employed during parameter updates to optimize convergence speed and stability during training.

[0063] The above description represents preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technical or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A security defense method against edge federation network attacks, characterized in that, Includes the following steps: S1: The edge server initializes global model parameters and distributes them to all participants; S2: Before local training on the ground terminal equipment, a density-based spatial clustering algorithm is used to detect anomalies in the local dataset and clean or correct abnormal data points. S3: Local ground terminal devices use local data to train the model and update the model parameters; S4: Upload the locally updated parameters to the edge node acting as a wireless relay; S5: After receiving the model parameters uploaded by the ground terminal equipment, the edge node performs model parameter anomaly detection and calculates the mean and variance of the model parameters in each dimension. S6: The edge server uses the parameters uploaded by the edge nodes to train a deep reinforcement learning network based on Siamese Delay Deep Deterministic Policy Gradient TD3; S7: Robustly aggregate the updated parameters using a deep reinforcement learning algorithm to calculate the new parameters of the global model; S8: The edge server distributes the new parameters of the global model to each participant, and each participant updates its local model and enters the next round of training. S6 specifically includes the following steps: S601: State Construction; Before aggregating model parameters, the edge server collects the model parameters uploaded by the edge nodes. Calculate the mean of each dimension of the model parameters. ,variance Obtain the number of ground terminal devices covered by each edge node. Combined with the current global model's accuracy on the validation set Loss value Construct state vector ; S602: Action generation; The Actor network outputs actions based on the current state. The action is represented as a set of continuous weighted coefficients. , , The number of edge nodes is used for weighted aggregation; a Gaussian-distributed noise is added to the action. The actual action performed is ; S603: Weighted aggregation; the edge server calculates the aggregation based on the actual actions performed, i.e., the weighting coefficients. Global model aggregation is performed using a weighted aggregation formula: ； in For the first The parameters of the edge node aggregation model These are the updated global model parameters; S604: Reward Calculation; Apply the updated global model to the validation set, calculate the model's accuracy and loss; The reward function is designed as follows: ； in These are the weighting coefficients. , To update the changes in the model's accuracy and loss value on the validation set before and after the update, Rewards for defending against attacks; S605: Experience playback and learning; state ,action ,award and the next state Forming tuples The tuples are stored in the experience replay pool. When the number of tuples in the experience replay pool reaches a threshold, a batch of tuples is randomly selected from the experience replay pool for learning. Update the parameters of the Critic network and the Actor network; periodically update the parameters of the target Actor network and the target Critic network. S606: Iterate continuously until the set conditions are met.

2. The security defense method against edge federation network attacks according to claim 1, characterized in that, S5 is specifically as follows: For the Each edge node receives data from a set of ground terminal devices. Model parameters Calculate the mean of the model parameters across all dimensions. and variance ,in For parameter dimensions: ； ； If the model parameters uploaded by a ground terminal device deviate from the mean by more than a set threshold in multiple dimensions, the model will be marked as an abnormal model.

3. A security defense method against edge federation network attacks according to claim 2, characterized in that, The density-based spatial clustering algorithm is as follows: Calculate the density of local data in the feature space: ； in, The neighborhood is defined as all distances. Less than or equal to the neighborhood radius The sample is calculated as follows: ； in, Represents the j-th data point With the vth data point distance, Indicates ground terminal equipment The local dataset; according to Identify the core elements and construct clusters, where Set the minimum number of points; mark isolated data points as anomalies.

Images