Security defense method for edge federated network attack

Abstract

The invention discloses a security defense method for edge federated network attacks, and relates to the technical field of artificial intelligence security. The edge server and the local device jointly train a global model by exchanging model parameters to ensure protection of data privacy. And each participant carries out data training locally, and carries out anomaly detection through a density-based spatial clustering algorithm. Updated model parameters are uploaded to an edge server, optimization and robust aggregation of a global model are carried out by using a TD3 deep reinforcement learning network, and the defense capability of the model to potential attacks is further improved. And finally, the edge server issues the optimized global model to each participant to ensure continuous updating of the model and security of the system. According to the method, edge calculation and deep reinforcement learning technologies are combined, so that the protection capability of the system on poisoning attacks and confrontation attacks is effectively enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence security, and particularly relates to a security defense method for edge federated network attacks. BACKGROUND

[0002] Edge federated network is a new network architecture that combines the real-time performance of edge computing and the privacy protection advantages of federated learning, enabling efficient data processing and distributed model training in dynamic and complex environments. By sharing model updates among participants rather than raw data, it effectively addresses data privacy issues and supports distributed and seamless collaboration. Due to the flexibility and efficiency of edge federated learning mechanisms in various tasks, it is widely used in efficient computing power, intelligent analysis, security protection and other fields, especially in environments where traditional cloud servers cannot be relied upon, providing a unique solution.

[0003] However, with the widespread application of edge federated networks, network security issues have gradually become a key challenge to its development. Due to the distributed nature of edge computing and federated learning models, network security is under severe test. Especially in an open and dynamic environment, malicious nodes can interfere with the normal operation of the system through various attack methods. These malicious nodes not only can fake data uploads and tamper with model parameters, but also can cause data leakage or model performance to drop sharply through malicious behavior, and even cause the entire system to collapse.

[0004] Therefore, it is particularly important to study the attack methods and potential threats in edge federated networks and explore effective defense strategies. This not only ensures the protection of data privacy, but also enhances the robustness and resistance of the system. Especially in the trend of large-scale access to heterogeneous nodes and intelligent and autonomous nodes, network security issues will be more complex and diverse. Only through innovative defense mechanisms can edge federated networks operate stably in complex environments for a long time. SUMMARY

[0005] The technical problem to be solved by the present application is to design a defense method for data poisoning attacks, model poisoning attacks and adversarial sample attacks in edge federated network training. On the one hand, the edge federated network model maintains excellent performance, and on the other hand, reduces the influence of external malicious attacks on the expected effect of the model in the training stage, and guarantees the robustness of the edge federated network artificial intelligence model.

[0006] The technical scheme adopted by the present application to solve the above technical problems is to provide a security defense method for edge federated network attacks, which performs anomaly detection on the local data set before local training of the ground terminal equipment, and eliminates abnormal data. When aggregating parameters, the differences in data quality and quantity of different ground terminal equipment are considered, and weights are assigned to the aggregated model parameters of each participating training equipment. Then, through deep reinforcement learning, the system dynamically adjusts the aggregation strategy according to the current network state and attack situation, and enhances the resistance to data poisoning, model poisoning and adversarial sample attacks. The method includes the following steps:

[0007] S1: The edge server initializes global model parameters and distributes them to each participant;

[0008] S2: Before local training of the ground terminal equipment , a density-based spatial clustering algorithm (DBSCAN) is used to perform anomaly detection on the local data set . If abnormal data points ( is the index of the data point), the abnormal data points are cleaned or corrected;

[0009] S3: Each participant trains the model using local data and calculates the update of the model parameters;

[0010] S4: Upload the updated local parameters to the edge node as a wireless relay;

[0011] S5: After receiving the model parameters uploaded by the ground terminal equipment, the edge node performs model anomaly detection, calculates the mean and variance of the model parameters in each dimension, where is the parameter dimension;

[0012] S6: The edge server trains a deep reinforcement learning network based on twin-delayed deep deterministic policy gradient (TD3) using the parameters uploaded by the edge node;

[0013] S7: Robust aggregation of updated parameters is performed through a deep reinforcement learning algorithm, and new parameters of the global model are calculated ;

[0014] S8: The edge server distributes the new parameters of the global model to each participant, each participant updates the local model, and enters the next round of training.

[0015] The application fully utilizes the dynamic decision advantage of deep reinforcement learning, and constructs a robust optimization framework capable of self-adaptive adjustment of weighted aggregation strategy by deeply integrating the TD3 algorithm and the federated learning mechanism. Through the edge server-led reinforcement learning interaction mechanism, combined with network environment state perception, continuous action space mapping and multi-dimensional reward feedback, the stability and accuracy of the global model in the face of attack scenarios are significantly improved. Specifically, the system can maintain model convergence performance during normal training, and maintain reliable model quality when encountering data poisoning, model poisoning or adversarial sample attacks, while continuously optimizing the defense effect through an intelligent weight dynamic allocation mechanism.

[0016] The beneficial effects of the application are:

[0017] (1) Through the construction of a deep reinforcement learning framework, the system can dynamically generate a weighted aggregation strategy according to real-time network states and attack characteristics, achieving improved adaptive defense capabilities in adversarial environments.

[0018] (2) Based on the continuous action space optimization characteristics of the TD3 algorithm, a multi-dimensional state space containing model parameter characteristics, device states and performance indicators, and a continuously adjustable continuous weighting coefficient action space are designed, ensuring the flexibility and optimality of the weight allocation strategy.

[0019] (3) Through the design of a multi-objective reward function that combines model performance improvement and attack resistance effect, the agent is guided to systematically enhance the defense capabilities against various poisoning attacks and adversarial samples while ensuring model accuracy. It is applicable in large-scale heterogeneous node access environments under poisoning attacks and adversarial sample attacks, and has strong adaptability and expandability.

[0020] (4) A federated learning-reinforcement learning double-loop architecture led by edge servers is constructed, which, on the basis of maintaining the distributed advantages of federated learning, realizes the robustness enhancement of the global model through centralized policy optimization. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 Schematic diagram of a multi-edge node federated network architecture (the following unmanned aerial vehicle is an edge node, and the leading unmanned aerial vehicle is an edge server).

[0022] Figure 2 Robust weighted aggregation algorithm flow based on anomaly detection and deep reinforcement learning. DETAILED DESCRIPTION

[0023] The technical scheme of a security defense method for edge federated network attacks of the application will be described in further detail below, but the protection scope of the application is not limited to the following description.

[0024] The embodiment is based on the following scenario: in a multi-edge node federated network architecture, composed of one leading UAV, multiple follower UAVs and multiple ground terminal devices. The leading UAV is responsible for the final global model aggregation. The follower UAVs collect model parameters of the ground terminal devices in the areas they are responsible for and transmit the aggregated parameters to the leading UAV. The ground terminal devices perform local data collection, processing and local model training.

[0025] The cooperative training process of the edge federated network can be divided into three stages: data collection, model training and model inference stage. In the edge federated network, data collection follows the federated learning concept, and the data set will not leave the local. The main work is the collection of local data and the negotiation of data formats between users. Malicious attackers can destroy the integrity of the training data set in the data collection stage. The common way is dirty label attack, that is, tampering with the data set label, so that the model learns the wrong information, and then affects the model training effect or a single attacker uses multiple collusion identities to join the system, increases the concealment and effect of the attack, often combined with poisoning attack, to interfere with the normal learning process. The model training stage aims to optimize the model parameters through iteration, so that it can accurately fit the data and mine the potential value of the data. In this stage, malicious attackers control part of the ground terminal training stage and tamper with the uploaded local model to try to manipulate the global model. In order to enhance the concealment, means such as converting the boosting process into an optimization problem will be used to make the poisoned update difficult to be found.

[0026] In view of the various attack means above, a security defense method for edge federated network attack provided by the embodiment is as shown in Figure 2 , and specifically includes the following steps: through the abnormality detection and robust aggregation algorithm based on multi-strategy fusion, the abnormality detection, robust aggregation and deep reinforcement learning strategy are fused to ensure the stability and accuracy of the edge federated network training:

[0027] S1: The edge server initializes the global model parameters and sets the global parameters of the federated learning, such as the global update round T, the number of terminals participating in the training N, etc. Select a suitable deep learning model (in this embodiment, a convolutional neural network CNN is used, and in practice, it can be selected and set according to the specific scene) as the global model, and randomly initialize its parameters. Initialize the parameters of the Actor network , the Critic network and and their target networks , and of the TD3 algorithm. Set the learning rate, discount factor, policy delay update frequency, noise standard deviation, target network update coefficient and other hyperparameters. Set the neighborhood radius and the minimum point number of the DBSCAN abnormality detection algorithm parameter;

[0028] S2: Before local training on the ground terminal device, the density-based spatial clustering algorithm (DBSCAN) is used to perform anomaly detection on the local dataset (SVHN dataset is used in this embodiment). , let its local dataset be ,in is the number of samples in the local dataset, It is samples, is the image data, is the corresponding digital label. By calculating the density connectivity between data points, the density-connected points are divided into different clusters, and the points with density lower than the set threshold are considered as outliers.

[0029] Specifically, for the SVHN dataset, the density of image data in the feature space is calculated , where the neighborhood , indicating that all distances Less than or equal to the neighborhood radius A sample of It can be calculated using Euclidean distance or cosine similarity. Determine the core points, build clusters, and mark isolated image samples (such as images with large feature differences from surrounding samples) as abnormal. ( is the data point index), then clean or correct it;

[0030] S3: Each ground terminal device uses the cleaned local data to train the CNN model and update the local model parameters;

[0031] S4: Upload the updated local model parameters to the edge node (following drone), e.g. Figure 1 As shown;

[0032] S5: After receiving the local model parameters uploaded by the ground terminal device, the edge node performs model anomaly detection and calculates the statistical characteristics of each model parameter, such as mean, variance, etc. edge nodes, receiving data from a set of ground terminal devices Model parameters , calculate the mean of the model parameters in each dimension and variance ,in is the parameter dimension.

[0033]

[0034]

[0035] If the model parameters uploaded by a ground terminal device deviate from the mean in multiple dimensions by more than a set threshold (e.g., 3 times the standard deviation), the model is considered to be potentially poisoned and marked as an abnormal model, which is then cleaned or corrected.

[0036] S6: Train the TD3 deep reinforcement learning network using the uploaded parameters;

[0037] In the robust weighted aggregation scenario of the edge federation network, the state space, action space and reward function are constructed with the edge server as the main body. Covers the model parameter characteristics uploaded by the edge node, the information of the ground terminal equipment, and the performance indicators of the current model; action space is a continuous weighted aggregation coefficient used to adjust the weight of edge node model parameters in global aggregation; reward function The design is based on the performance improvement of the aggregated model on the validation set and the effectiveness of attack resistance, guiding TD3 to learn the optimal weighted aggregation strategy. The specific algorithm process is as follows:

[0038] S601: State construction; the edge server collects the model parameters uploaded by the edge node before aggregating the model parameters . Calculate the mean of each dimension of the model parameters ,variance Obtain the number of ground terminal devices covered by each edge node Combined with the accuracy of the current global model on the validation set , loss value and other performance indicators , construct the state vector As input to the TD3 algorithm.

[0039] S602: Action generation; the Actor network outputs a deterministic action based on the current state , the action is represented as a set of continuous weight coefficients ( , is the number of edge nodes) for weighted aggregation. To increase exploratory power, a noise that follows a Gaussian distribution is added to the action. The actual action performed is .

[0040] S603: Weighted aggregation; the edge server performs weighted aggregation based on the actual action, i.e., the weighted coefficient , perform global model aggregation. Use weighted aggregation formula:

[0041]

[0042] in For the Edge node aggregation model parameters, are the updated global model parameters.

[0043] S604: Reward calculation: Apply the updated global model to the validation set and calculate the model's accuracy and loss. Calculate rewards based on the model's performance improvement and its effectiveness in resisting attacks. The reward function is designed as follows:

[0044]

[0045] in is the weight coefficient, 、 is the change in accuracy and loss value of the model on the validation set before and after the update, Bonus for attack defense.

[0046] S605: Experience playback and learning; ,action ,award and the next state Composing tuples Store them in the experience replay pool. When the number of tuples in the experience replay pool reaches a certain threshold, a batch of tuples are randomly selected from the experience replay pool for learning.

[0047] Update the Critic network and calculate the target value :

[0048]

[0049] in is the target Critic network, is the target Actor network, is the entropy regularization coefficient, is the added noise. By minimizing the loss function , use the gradient descent algorithm to update the Critic network parameters:

[0050]

[0051] in, is the number of samples drawn.

[0052] Update the Actor network to maximize the objective function , use the gradient ascent algorithm to update the parameters of the Actor network:

[0053]

[0054] Target network update is performed to periodically update the parameters of the target Actor network and the target Critic network to gradually approach the parameters of the current network, and the update formula is:

[0055]

[0056] wherein is a target network update coefficient, is a current network parameter, is a target network parameter.

[0057] S606: Loop iteration;

[0058] Steps S602 to S606 are repeated, and as the number of training rounds increases, TD3 continuously learns and optimizes the weighted aggregation strategy to adapt to different attack situations and network states, realizes robust weighted aggregation, and improves the training performance of the edge federated network.

[0059] S7: Robust aggregation of updated parameters through deep reinforcement learning algorithm to calculate new parameters of the global model In this step, the edge server uses a deep reinforcement learning (DRL) algorithm to robustly aggregate the local model parameters uploaded from each participant. First, the edge server compares the model parameters of each participant with the local global model, and weights the aggregation of model parameters according to the contribution weight of each participant (e.g., training data volume, computing power, etc.). In order to improve the robustness of the model in a dynamic environment, the deep reinforcement learning algorithm evaluates the resistance of the current aggregated model to potential attacks, and adjusts the aggregation strategy according to the reward mechanism in reinforcement learning. This process is implemented through the TD3 algorithm, ensuring that the model can be continuously optimized in multiple rounds of training to enhance defense capabilities.

[0060] Specifically, the TD3 algorithm uses an Actor-Critic architecture to evaluate the performance of the current aggregation parameters and updates the new parameters of the global model by optimizing the aggregation strategy. In this process, the model adjusts to the feedback from the environment (i.e., the model parameters uploaded by participants and the actual attack defense capabilities), and maintains its robustness in a changing attack environment. Through repeated training, the deep reinforcement learning algorithm can ensure that the global model can better adapt to changes in different scenarios, and ultimately calculate the optimal global model parameters to provide strong support for the model update in the next step.

[0061] S8: The edge server uploads the new parameters of the global model to the local model of each participant The global model updated by the edge server is transmitted to each participant (e.g., other UAVs or ground terminal devices) through wireless communication. The edge server as the central control unit will ensure that the new global model parameters are accurately transmitted to each participant. After receiving the new global model parameters, each participant updates its local model with the new parameters and performs further local training and adjustment. The system can automatically identify the characteristics and needs of different devices, optimize the performance of the global model while ensuring safety.

[0062] Once all participants complete the update of the local model and train with the new parameters, the system enters the next round of training cycle. Each round of training is optimized based on the results of the previous round to ensure that the system can continuously improve the overall performance in the continuous training process. In this process, if the system has reached the predetermined target effect (e.g., reaches the set standard in terms of attack and defense capabilities, model accuracy, etc.) through the evaluation of the validation set, the training process will stop, and the final model optimization is completed. A dynamic adjustment learning rate strategy is used in the parameter update process to optimize the convergence speed and stability in the training process.

[0063] The above is the preferred embodiment of the present application, it should be understood that the present application is not limited to the form disclosed herein, should not be considered as excluding other embodiments, but can be used in other combinations, modifications and environments, and can be modified within the scope of the concepts described herein, by the above teachings or related art or knowledge. Any modification and change made by those skilled in the art without departing from the spirit and scope of the present application shall be within the scope of protection of the appended claims of the present application.

Claims

1. A security defense method against edge federation network attacks, characterized in that: The following steps are involved: S1: The edge server initializes the global model parameters and distributes them to all participants; S2: Before local training on the ground terminal device, a density-based spatial clustering algorithm is used to detect anomalies in the local data set and clean or correct abnormal data points; S3: Terminal devices on the ground use local data to train the model and update the model parameters; S4: Upload the locally updated parameters to the edge node acting as a wireless relay; S5: After receiving the model parameters uploaded by the ground terminal device, the edge node performs model parameter anomaly detection and calculates the mean and variance of the model parameters in each dimension; S6: The edge server uses the parameters uploaded by the edge node to train a deep reinforcement learning network based on twin delayed deep deterministic policy gradient TD3; S7: Robustly aggregate the updated parameters through the deep reinforcement learning algorithm to calculate the new parameters of the global model; S8: The edge server sends the new parameters of the global model to each participant, and each participant updates the local model and enters the next round of training.

2. A security defense method against edge federation network attacks according to claim 1, characterized in that: The S6 specifically includes the following steps: S601: State construction; the edge server collects the model parameters uploaded by the edge node before aggregating the model parameters , calculate the mean of each dimension of the model parameters ,variance , get the number of ground terminal devices covered by each edge node , combined with the accuracy of the current global model on the validation set , loss value , construct the state vector ; S602: Action generation; Actor network outputs actions based on current state , the action is represented as a set of continuous weight coefficients , , is the number of edge nodes, used for weighted aggregation; add a Gaussian distributed noise to the action The actual action performed is ; S603: Weighted aggregation; the edge server performs the weighted coefficient according to the actual action performed , perform global model aggregation and use the weighted aggregation formula: ； in For the Edge node aggregation model parameters, is the updated global model parameter; S604: Reward calculation: Apply the updated global model to the validation set and calculate the model's accuracy and loss value. The reward function is designed as: ； in is the weight coefficient, 、 is the change in accuracy and loss value of the model on the validation set before and after the update, Bonus for attack defense; S605: Experience playback and learning; ,action ,award and the next state Composing tuples Store them in the experience replay pool. When the number of tuples in the experience replay pool reaches a threshold, a batch of tuples are randomly selected from the experience replay pool for learning. Update the parameters of the Critic network and the Actor network; regularly update the parameters of the target Actor network and the target Critic network; S606: Iterate the loop until the set condition is met.

3. A security defense method against edge federation network attacks according to claim 2, characterized in that: The S5 is specifically as follows: For the edge nodes, receiving data from a set of ground terminal devices Model parameters , calculate the mean of the model parameters in each dimension and variance ,in is the parameter dimension: ； ； If the model parameters uploaded by a ground terminal device deviate from the mean in multiple dimensions by more than the set threshold, the model will be marked as an abnormal model.

4. A security defense method against edge federation network attacks according to claim 3, characterized in that: The density-based spatial clustering algorithm is specifically as follows: Calculate the density of local data in feature space: ； in, Represents the neighborhood, specifically all distances Less than or equal to the neighborhood radius The sample is calculated as follows: ； in, Represents the jth data point With the vth data point distance, Indicates ground terminal equipment local dataset; according to Determine the core points and build clusters, where is the minimum number of points; isolated data points are marked as anomalies.

Images