Data poisoning defense method based on sparse autoencoder in cloud-edge collaborative environment
By using multi-scale feature extraction and ternary gradient quantization of sparse autoencoders, the problem of detecting malicious edge nodes in cloud-edge collaborative environments is solved, achieving efficient identification of malicious gradients and high-performance maintenance of the global model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI ZHENXIN INTERNET TECH CO LTD
- Filing Date
- 2025-11-20
- Publication Date
- 2026-05-15
AI Technical Summary
Existing cloud-edge collaborative environment defense methods are unable to effectively detect and defend against more than half of malicious edge nodes and highly concealed toxic model parameters, resulting in damage to the global model performance.
Anomaly gradient detection is performed using a sparse autoencoder. The sparse autoencoder is trained as an anomaly gradient detector. The model gradient is generated using a small, clean dataset for training. Combined with multi-scale feature extraction, adaptive sparsity constraints, and ternary gradient quantization, malicious gradients are detected and removed. The global model is then updated by weighted aggregation.
It improves the accuracy and robustness of malicious gradient detection, ensuring that the global model can maintain high performance when facing most malicious edge nodes, and enhances the security of the cloud-edge collaborative system.
Smart Images
Figure CN121462287B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial internet information security technology, specifically to a data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment. Background Technology
[0002] As an extension of the Internet of Things (IoT) into the industrial manufacturing sector, the Industrial Internet inherits the massive and heterogeneous data characteristics of IoT. In Industrial Internet scenarios, edge devices can only process localized data and cannot form a global understanding; therefore, cloud computing platforms are needed to achieve information fusion. Cloud-edge collaboration technology compensates for the shortcomings of traditional centralized cloud services. In Industrial Internet using cloud-edge collaboration technology, data is generated by edge devices, and this data is transmitted from edge nodes to cloud servers for further analysis and storage. However, due to weak data security measures at edge nodes, this can lead to serious privacy risks.
[0003] Due to the privacy protection mechanisms and distributed nature of cloud-edge collaboration, cloud servers cannot directly analyze the raw data of edge nodes. This means that cloud servers cannot determine whether the data from edge nodes participating in computation is erroneous. More seriously, malicious edge nodes may upload toxic model parameters to influence the training of the global model. This malicious behavior is called a poisoning attack, which can be further divided into targeted attacks and non-targeted attacks, depending on the attacker's objective. In targeted attacks, the attacker wants the model to make incorrect predictions only for specific inputs while maintaining high prediction accuracy for other inputs. In non-targeted attacks, the attacker attempts to make the global model make incorrect predictions for all inputs.
[0004] Therefore, it is necessary to address the problem of poisoning attacks launched by malicious edge nodes, enabling cloud servers to learn a high-performance global model even when malicious edge nodes exist in the system. Existing cloud-edge collaborative defense methods against poisoning attacks can be mainly divided into robust aggregation-based defense methods and anomaly detection-based defense methods. The main idea of robust aggregation-based defense methods is to mitigate the impact of poisoning attacks on the global model through methods such as pruning, correction, and weighted aggregation. For example, the Median algorithm sorts model parameters in each dimension and then selects the median value in each dimension as the global model update value for that dimension. The main idea of anomaly detection-based defense methods is to use anomaly detection methods such as similarity calculation and clustering to identify and remove outliers in model parameters. For example, the FLTrust algorithm calculates the cosine similarity between the gradient of the cloud server verification model and the gradient of the edge node model to identify and exclude malicious model gradients. However, research has shown that attackers can bypass existing defense methods and affect the performance of the global model by uploading carefully crafted and hidden malicious model parameters, and these defense methods are ineffective when facing poisoning attacks launched by more than half of the attackers in the system. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment. The aim is to accurately detect more than half of the malicious edge nodes and highly concealed poisonous model parameters, and minimize their impact on the global model.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] A data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment includes the following steps:
[0008] S1. Before the cloud-edge collaborative task begins, the cloud server uses a small, clean dataset to train a sparse autoencoder as an anomaly gradient detector.
[0009] S2. After the cloud-edge collaborative task begins, the cloud server reconstructs the local model gradient uploaded by each edge node using the trained sparse autoencoder and calculates the reconstruction error.
[0010] S3. The cloud server calculates the anomaly score for each edge node based on the reconstruction error;
[0011] S4. The cloud server performs weighted aggregation of the gradients of benign edge nodes based on the anomaly scores and updates the global model;
[0012] S5. Simultaneously, the cloud server updates the sparse autoencoder using the ternary gradient parameters of the benign edge nodes.
[0013] Furthermore, training the sparse autoencoder as an anomaly gradient detector includes the following steps:
[0014] S11. The cloud server uses a small, clean dataset to generate model gradients and performs ternary quantization on the model gradients, using the ternary quantized gradient data as training samples.
[0015] S12. Design multiple parallel convolutional branches in the sparse autoencoder, each branch using convolutional kernels of different sizes to capture different gradient features in the training set.
[0016] S13. Perform max pooling dimensionality reduction on the output of each convolutional branch;
[0017] S14. The reduced features are fused through a fully connected layer;
[0018] S15. Use adaptive sparsity constraints to dynamically adjust the sparsity penalty coefficient;
[0019] S16. By using a ternary gradient quantization mechanism, the full-precision gradient is converted into a ternary gradient to improve the autoencoder's sensitivity to malicious gradients.
[0020] Furthermore, the use of convolutional kernels of different sizes to capture different gradient features is represented by the following formulas (1)-(3):
[0021] (1);
[0022] (2);
[0023] (3);
[0024] In formulas (1)-(3): , , These represent convolution kernels with different sizes; This represents the feature vector after the local model gradient of the edge node has been flattened. This represents a two-dimensional convolution operation, which involves using a convolution kernel to perform a convolution operation on the input feature vector to extract the spatial features of the input data. , , These represent feature extraction results at different scales.
[0025] Furthermore, the max-pooling dimensionality reduction of the output of each convolutional branch is expressed by the following formulas (4)-(6):
[0026] (4);
[0027] (5);
[0028] (6);
[0029] In formulas (4)-(6): , , These represent feature extraction results at different scales; This represents the max pooling operation, which performs a downsampling operation on the input features and selects the maximum value within the pooling window as the output. , , These represent the output features after max pooling dimensionality reduction.
[0030] Furthermore, the fusion of the dimensionality-reduced features through a fully connected layer specifically includes the following steps:
[0031] S141. The output feature results after max pooling dimensionality reduction are concatenated and fused along the feature dimension, and the concatenated multi-scale features are mapped to a unified feature space.
[0032] S142. In the encoding stage, the concatenated multi-scale features are reduced to a hidden layer representation.
[0033] S143. In the decoding stage, the compressed features are reconstructed layer by layer through fully connected layers to restore the original data dimension;
[0034] Steps S141-S143 are represented by the following formulas (7)-(9):
[0035] , , (7);
[0036] (8);
[0037] (9);
[0038] In formulas (7)-(9): , , These represent the output features after max pooling dimensionality reduction; This means concatenating features along the feature dimensions to form a comprehensive feature representation that includes multi-scale information; This represents the feature representation after fusion; This represents the fusion weight matrix, which is used to map the concatenated multi-scale features to a unified feature space. This represents the fusion bias term, used to increase the expressive power of the model; its size is consistent with the dimension of the fused features. Indicates the activation function; This represents the encoded hidden layer feature representation; This represents the decoder weight matrix, used to reconstruct the original input from the hidden layer features; Indicates the decoder bias term; This represents the reconstructed output, used to calculate the reconstruction error.
[0039] Furthermore, the dynamic adjustment process of the sparsity penalty coefficient is expressed by the following formulas (10)-(11):
[0040] (10);
[0041] (11);
[0042] In formula (10)-(11): This represents the initial sparsity penalty coefficient; This represents the reconstruction error in the current round, and its value is inversely proportional to the reconstruction effect. This represents the maximum reconstruction error on the training set; This represents the dynamically adjusted penalty coefficient; This represents the feature vector after the local model gradient of the edge node has been flattened. This represents the reconstructed output; The dimension of the feature vector, i.e., the total dimension after the input gradient data is flattened, determines the average range of the error calculation. Represents the first element of the original input vector. One component; The first element of the reconstructed output vector is... Each component.
[0043] Furthermore, the ternary gradient quantization is expressed by the following formulas (12)-(13):
[0044] (12);
[0045] (13);
[0046] In formula (12)-(13): This represents the full-precision local model gradient, which is the original high-dimensional floating-point gradient vector obtained by edge nodes after local training. , indicating that the dimension is The ternary gradient vector; Representing vectors and The inner product of the vector, whose value is the same as that of the vector. Direction and original gradient The directionality is directly proportional to the consistency. Representing vectors Its inner product with itself; This represents the separated scalar, used to preserve the overall magnitude information of the gradient; It is a mathematical operator that represents the parameter that maximizes the function value. ; This represents the optimal ternary gradient vector.
[0047] Furthermore, the anomaly score for each edge node is calculated using the following formula (14):
[0048] (14);
[0049] In formula (14): Represents edge nodes The reconstruction error of the edge local model gradient; This represents the minimum gradient reconstruction error of all edge local models in this iteration; Represents edge nodes Abnormal scores.
[0050] Furthermore, the specific process of weighted aggregation of gradients for benign edge nodes is as follows:
[0051] (15);
[0052] In formula (15): Represents edge nodes Abnormal scores; , representing the smoothing factor, is used to control the impact of outlier scores on the weights; Represents the set of all benign edge nodes, i.e. edge nodes, This represents the mean of the anomaly scores calculated for each edge node; Represents edge nodes Weights during aggregation.
[0053] Furthermore, updating the sparse autoencoder using the ternary gradient parameters of benign edge nodes includes aggregating the local model gradients at the edges and updating the global model parameters in the cloud, which are expressed by the following formulas (16)-(17):
[0054] (16);
[0055] (17);
[0056] In formula (16)-(17): Represents edge nodes Weights during aggregation; Indicates the first The edge node at the ... The scalar separated from the wheel is obtained by the ternary quantization process; Indicates the first The edge node at the ... The ternary quantization gradient of the wheel; Indicates the first Gradient of the local model at the wheel edge; Indicates the first Wheel cloud global model parameters; This represents the global learning rate of the model in the cloud, controlling the step size for model updates; Indicates the first The updated cloud-based global model parameters.
[0057] As can be seen from the above technical solutions, compared with the prior art, the present invention has the following technical advantages:
[0058] 1. This invention improves the accuracy of anomaly detection in model gradient data by using a sparse autoencoder architecture with multi-scale feature extraction and adaptive sparsity constraints.
[0059] 2. This invention improves the sensitivity of the autoencoder to malicious gradients by introducing a ternary gradient quantization mechanism, while also enhancing the robustness of aggregation.
[0060] 3. This invention accurately identifies malicious edge nodes in each training round and removes them from the training process as early as possible. Even when the proportion of malicious edge nodes is high, it can still maintain a high global model accuracy.
[0061] 4. This invention effectively utilizes the gradient information of benign edge nodes through a weighted aggregation mechanism, ensuring the performance of the global model. Attached Figure Description
[0062] Figure 1 This is a schematic diagram of the framework of the data poisoning defense method based on sparse autoencoder in the cloud-edge collaborative environment of the present invention;
[0063] Figure 2 This is a global model accuracy curve under LF attack provided in an embodiment of the present invention;
[0064] Figure 3 This is a global model accuracy curve under the Min-Sum attack provided in an embodiment of the present invention;
[0065] Figure 4 This is a graph showing the global model accuracy under a Min-Max attack, provided in an embodiment of the present invention.
[0066] Figure 5 This is a graph showing the global model accuracy under IPM attack, provided in an embodiment of the present invention. Detailed Implementation
[0067] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.
[0068] like Figure 1 As shown in the preferred embodiment, this method for preventing data poisoning based on a sparse autoencoder in a cloud-edge collaborative environment includes the following steps:
[0069] I. Anomaly Gradient Detector Based on Sparse Autoencoder
[0070] Specifically, before the cloud-edge collaborative task begins, the cloud server uses a small, clean dataset (i.e., trustworthy, uncontaminated data) that it has collected or holds to generate model gradients as training samples. These gradients represent the benign patterns that model updates should exhibit under attack-free conditions. Then, the model gradients are ternary quantized, and the ternary gradient information is used to train the proposed diluted autoencoder model. This enables the autoencoder to learn how to efficiently compress and reconstruct these benign gradients, allowing its internal encoding to capture the essential characteristics and distribution patterns of benign gradients. This lays the foundation for subsequently identifying anomalous information from untrusted edge nodes in a distributed environment.
[0071] Traditional sparse autoencoders identify anomalous data through reconstruction errors, but the high dimensionality, sparsity, and distributional differences among multiple edge nodes of gradient data pose challenges to the autoencoder's performance. Therefore, this preferred embodiment proposes an improved sparse autoencoder structure, which includes multi-scale feature extraction and adaptive sparsity constraints, and introduces a ternary quantization mechanism to improve the autoencoder's sensitivity to anomalous inputs.
[0072] The training of a sparse autoencoder as an anomaly gradient detector in this preferred embodiment includes the following steps:
[0073] Traditional sparse autoencoders typically use fully connected layers for layer-by-layer dimensionality reduction. However, gradient data often exhibits variations at different scales, especially between model gradients in different parameter layers, where the magnitude of information change may differ. Therefore, this preferred embodiment innovatively improves the encoder by using multiple parallel convolutional layers (convolutional branches) instead of traditional fully connected layers as the encoder's front end. The workflow is as follows: input gradient → multiple parallel convolutional branches (for multi-scale feature extraction) → pooling layer → feature fusion fully connected layer → bottleneck layer → decoder (fully connected layer) → reconstructed output.
[0074] Specifically, this preferred embodiment designs three parallel convolutional branches, each using convolutional kernels of different sizes (e.g., 3×3, 5×5, 7×7) to capture different gradient features. Smaller convolutional kernels focus on local, subtle features, while larger kernels capture more global features.
[0075] (1);
[0076] (2);
[0077] (3);
[0078] In formulas (1)-(3): , , These represent convolution kernels with different sizes; This represents the feature vector after the local model gradient of the edge node has been flattened. This represents a two-dimensional convolution operation, which involves using a convolution kernel to perform a convolution operation on the input feature vector to extract the spatial features of the input data. , , These represent feature extraction results at different scales.
[0079] Then, max pooling is performed on the output of each convolutional branch to reduce the size of the feature map, thereby reducing computational complexity and providing a suitable input dimension for subsequent fully connected layer fusion. At the same time, the most significant feature responses at each scale are preserved, enhancing sensitivity to malicious gradient anomaly patterns. Furthermore, the independence of the features of the three branches is maintained, avoiding information mixing and providing rich multi-scale information for subsequent feature fusion.
[0080] (4);
[0081] (5);
[0082] (6);
[0083] In formulas (4)-(6): , , These represent feature extraction results at different scales; This represents the max pooling operation, which performs a downsampling operation on the input features and selects the maximum value within the pooling window as the output. , , These represent the output features after max pooling dimensionality reduction.
[0084] The output features after max pooling dimensionality reduction are concatenated and fused along the feature dimension to form a comprehensive feature representation containing multi-scale information, and the concatenated multi-scale features are mapped to a unified feature space:
[0085] , , (7);
[0086] During the encoding stage, the concatenated multi-scale features are further reduced to hidden layer representations to achieve feature compression and information extraction.
[0087] (8);
[0088] During the decoding stage, the compressed features are reconstructed layer by layer through fully connected layers to restore the original data dimensions:
[0089] (9);
[0090] In formulas (7)-(9): , , These represent the output features after max pooling dimensionality reduction; This means concatenating features along the feature dimensions to form a comprehensive feature representation that includes multi-scale information; This represents the feature representation after fusion; This represents the fusion weight matrix, which is used to map the concatenated multi-scale features to a unified feature space. This represents the fusion bias term, used to increase the expressive power of the model; its size is consistent with the dimension of the fused features. This represents the activation function, introducing a nonlinear transformation to enhance the model's expressive power; This represents the encoded hidden layer feature representation; This represents the decoder weight matrix, used to reconstruct the original input from the hidden layer features; Indicates the decoder bias term; This represents the reconstructed output, used to calculate the reconstruction error.
[0091] Next, an adaptive sparsity constraint is used to dynamically adjust the sparsity penalty coefficient, which controls the sparsity of the activation of hidden layer neurons in the loss function of the sparse autoencoder. The larger the sparsity penalty coefficient, the stronger the sparsity constraint imposed on the model, and more neurons will be suppressed (output close to 0).
[0092] (10);
[0093] (11);
[0094] In formula (10)-(11): This represents the initial sparsity penalty coefficient, which indicates the fixed sparsity penalty strength without adaptive adjustment. This indicates the reconstruction error in the current round; This represents the maximum reconstruction error on the training set, serving as a baseline for comparing the current error. Normalize it to make it a relative value; This represents the dynamically adjusted penalty coefficient; This represents the feature vector after the local model gradient of the edge node has been flattened. This represents the reconstructed output; The dimension of the feature vector, i.e., the total dimension after the input gradient data is flattened, determines the average range of the error calculation. Represents the first element of the original input vector. One component; The first element of the reconstructed output vector is... There are several components; specifically, a small reconstruction error, that is, conforms to the normal distribution learned by the autoencoder, is considered a benign gradient; correspondingly, a large reconstruction error, that is, deviates from the normal distribution pattern, is considered a malicious gradient.
[0095] The parameters of the sparse autoencoder described in this preferred embodiment are shown in Table 1, where batch_size represents the batch size and hidden_size represents the hidden layer size.
[0096] Table 1 Sparse Autoencoder Parameter Table
[0097]
[0098] In the cloud-edge collaborative anomaly edge node detection task, the gradient reconstruction method based on autoencoders relies on the full-precision model gradients uploaded by the edge nodes. However, directly using full-precision model gradient information may have several problems. First, full-precision model gradients contain rich numerical information, which attackers can manipulate by adding perturbations or scaling operations to some of the information, making it difficult for the autoencoder to distinguish between benign and malicious gradients. Second, the numerical range of full-precision model gradients is usually large, and the autoencoder may focus excessively on the absolute value of the gradient during training, neglecting the core structure, thus reducing its sensitivity to malicious gradients. Finally, due to the small number of training samples for the autoencoder, learning full-precision gradient information may lead to overfitting, potentially causing misclassification of benign model gradients. Therefore, this preferred embodiment introduces a ternary gradient quantization mechanism to convert full-precision gradients into ternary gradients to improve the autoencoder's sensitivity to malicious gradients.
[0099] (12);
[0100] (13);
[0101] In formula (12)-(13): This represents the full-precision local model gradient, which is the original high-dimensional floating-point gradient vector obtained by edge nodes after local training. , indicating that the dimension is The ternary gradient vector; Representing vectors and The inner product of the vector, whose value is related to the vector Direction and original gradient The directionality is directly proportional to the consistency. Representing vectors Its inner product with itself; This represents the separated scalar, used to preserve the overall magnitude information of the gradient; It is a mathematical operator that represents the parameter that maximizes the function value. ; This represents the optimal ternary gradient vector.
[0102] Specifically, ternary quantization of gradients restricts the gradient value to 1 / 2. By retaining only the direction information of the gradient and eliminating the influence of numerical scale, the autoencoder focuses more on the structural patterns of the gradient rather than its magnitude. The anomalies of malicious gradients are also reflected in directional differences across specific dimensions. Compared to full-precision gradients, ternary gradients reduce noise interference when projected onto a low-dimensional representation, thus amplifying the directional differences between benign and malicious gradients and improving the autoencoder's ability to distinguish outliers. Furthermore, autoencoders trained using ternary gradients can learn more general gradient structure information, reducing dependence on specific data distributions and improving their adaptability to different tasks and attack scenarios. Because ternary gradients are insensitive to numerical perturbations, attackers need to manipulate the gradient direction more precisely to bypass detection, increasing the difficulty of the attack. Even if an attacker attempts to fine-tune the gradient magnitude, due to the nonlinear mapping characteristics of ternary quantization, these fine-tunings are unlikely to affect the final autoencoder detection result even if the anomaly detector fails to identify the malicious gradient, thus improving the robustness of the aggregation.
[0103] II. Anomaly Detection and Weighted Aggregation Based on Reconstruction Error
[0104] After the cloud-edge collaborative task begins, the model gradients uploaded by each edge node are reconstructed using a pre-trained sparse autoencoder, and the reconstruction error is calculated. Then, based on the reconstruction error, an anomaly score is calculated for each edge node to further quantify the anomaly of the gradients uploaded by each edge node.
[0105] (14);
[0106] In formula (14): Represents edge nodes The reconstruction error of the edge local model gradient; This represents the minimum gradient reconstruction error of all edge local models in this iteration; Represents edge nodes The anomaly score is calculated. Through normalization, the reconstruction error is transformed into an anomaly score metric that is easy to judge. The cloud server calculates the mean of the anomaly scores for each edge node as the anomaly threshold. If the score of an edge node is abnormal... If this is the case, it indicates that the gradient uploaded by the edge node is an abnormal model gradient. The cloud server will remove the local model gradient of this edge node and exclude it from this round of global model gradient aggregation, thereby defending against potential poisoning attacks. (Regarding abnormal scores...) For edge nodes, the cloud server considers the local model gradients uploaded by them to be benign, retains them for global model updates, and further utilizes these gradients to optimize the sparse autoencoder, so that the autoencoder continuously adapts to the gradient distribution of the current round of the task, thereby ensuring the accuracy of anomaly detection by the sparse autoencoder.
[0107] The cloud server reweights and aggregates the gradients of benign models based on the anomaly scores of edge nodes to update the global model:
[0108] (15);
[0109] In formula (15): Represents edge nodes Abnormal scores; , representing the smoothing factor, is used to control the impact of outlier scores on the weights; Represents the set of all benign edge nodes, i.e. edge nodes, This represents the mean of the anomaly scores calculated for each edge node; Represents edge nodes The weights during aggregation. This formula uses a soft maximization approach, transforming outlier scores into a probability distribution that ensures the sum of all weights is 1, while assigning greater weights to edge nodes with smaller reconstruction errors.
[0110] The cloud server aggregates the local model's ternary quantization gradient based on the weights of benign edge nodes and updates the sparse autoencoder.
[0111] (16);
[0112] (17);
[0113] In formula (16)-(17): Represents edge nodes Weights during aggregation; Indicates the first The edge node at the ... The scalar separated from the wheel is obtained by the ternary quantization process; Indicates the first The edge node at the ... The ternary quantization gradient of the wheel; Indicates the first Gradient of the local model at the wheel edge; Indicates the first Wheel cloud global model parameters; This represents the global learning rate of the model in the cloud, controlling the step size for model updates; Indicates the first The updated cloud-based global model parameters.
[0114] The following comparative experiments demonstrate the superior performance of the data poisoning defense method based on sparse autoencoders in the cloud-edge collaborative environment described in this invention:
[0115] The experiments were conducted on two public datasets, CIFAR-10 and MNIST. CIFAR-10 is a widely used dataset in computer vision research, typically for image classification tasks. The CIFAR-10 dataset contains a total of 60,000 32x32 pixel color images from 10 different categories, with 6,000 images per category. These categories include airplanes, cars, birds, cats, deer, dogs, frogs, horses, boats, and vehicles. The MNIST dataset is a classic machine learning and computer vision dataset used for handwritten digit recognition tasks. The MNIST dataset contains approximately 70,000 28x28 pixel grayscale images.
[0116] For the CIFAR-10 dataset, experiments used ResNet18 as the global model architecture. For the MNIST dataset, experiments employed a deep learning model with 21,840 trainable parameters. This model contains two convolutional kernels of size [missing value]. The system consists of a convolutional layer, a dropout layer, and two fully connected layers. The two fully connected layers contain 50 and 10 neurons respectively. The output of the last fully connected layer is processed by a softmax function to obtain the final predicted probability.
[0117] The experiment divided the training dataset into 20 equal-sized subsets and assigned them to 20 edge nodes. Considering the varying convergence difficulty of different datasets, different global model iteration rounds were set for different datasets and models. For the CIFAR-10 dataset, the global model underwent 300 iterations, and for the MNIST dataset, it underwent 200 iterations. Each edge node had a local iteration round of 1.
[0118] To verify the effectiveness and necessity of the sparse autoencoder used in the defense method of this invention, experiments were conducted using the SAE-FLDA-ablation algorithm on the CIFAR-10 and MNIST datasets under an attack scenario with 30% malicious edge nodes. Specifically, SAE-FLDA-ablation replaces the sparse autoencoder used in the SAE-FLDA algorithm with a traditional autoencoder, and does not perform ternary quantization on the model gradient. The experimental results are as follows: Figure 2 , Figure 3 , Figure 4 , Figure 5 As shown in the figure, the ordinary autoencoder has limited ability to identify malicious edge nodes. When faced with highly concealed toxic model gradients uploaded by malicious edge nodes, the global model performance drops significantly. Therefore, it can be concluded that the sparse autoencoder model and the introduced ternary quantization mechanism can effectively improve the algorithm's defense capability.
[0119] The experiment compares SAE-FLDA with the following six baseline algorithms:
[0120] FedAvg: A classic algorithm in federated learning, which updates the global model by weighting the local models uploaded by edge nodes in each iteration (based on the amount of local data at the edge nodes).
[0121] Krum: Calculates the sum of the Euclidean distances between the gradient of any edge node model and the gradients of all other edge nodes models as the score of that edge node, takes the gradient of the edge node model with the lowest score as the global model gradient, and updates the global model.
[0122] Median: Takes the median of the model gradient of each edge node in each dimension, and uses it as the value of the global model gradient in that dimension.
[0123] Trimmed-Mean: After removing the maximum and minimum values of all model gradients in each dimension, the mean of the remaining gradients is calculated as the global model gradient.
[0124] FLTrust: Calculates the cosine similarity between the clean model gradient obtained from cloud server training and the local model gradient uploaded by edge nodes to find abnormal gradients, prunes the local model gradients of edge nodes according to the size of the clean gradient, and finally aggregates the cosine similarity value and the pruned local model gradients to obtain the global model gradient.
[0125] FedDet: Reweights the local model gradients based on the median of the L2 norm of each layer of the local model gradients, and then aggregates them to obtain the global model gradient.
[0126] Final model accuracy of each algorithm when malicious edge nodes are set to 0 on the CIFAR-10 and MNIST datasets:
[0127] Table 2. Final model accuracy of each algorithm when no malicious edge nodes are present on the CIFAR-10 dataset.
[0128]
[0129] Table 3. Final model accuracy of each algorithm when there are no malicious edge nodes on the MNIST dataset.
[0130]
[0131] The experimental results in Tables 2 and 3 show that when there are no malicious edge nodes in the cloud-edge collaborative system, the SAE-FLDA algorithm has almost no negative impact on the global model. Meanwhile, most defense methods maintain good global model accuracy, especially on the MNIST dataset. This may be because the sample complexity in the MNIST dataset is low, making the model easier to fit. However, on both datasets, the FLTrust algorithm impacts the global model performance. This is because FLTrust prunes the local model gradient based on the magnitude of the validation model gradient before aggregation. While this protects the system from significant poisoning attacks, it also slows down the global model convergence. The Krum algorithm's model accuracy on the CIFAR-10 dataset is also significantly lower than the baseline algorithm's accuracy because the Krum algorithm selects only the local model gradient of one edge node as the global model gradient in each iteration, ignoring gradient information generated by most of the dataset.
[0132] Final model accuracy of each algorithm when 30% malicious edge nodes are set on the CIFAR-10 and MNIST datasets:
[0133] Table 4. Final model accuracy of each algorithm when there are 30% malicious edge nodes on the CIFAR-10 dataset.
[0134]
[0135] Table 5. Final model accuracy of each algorithm when there are 30% malicious edge nodes on the MNIST dataset.
[0136]
[0137] The experimental results in Tables 4 and 5 show that the SAE-FLDA algorithm performs exceptionally well on the CIFAR-10 and MNIST datasets when facing 30% malicious edge nodes. Regardless of whether it's LF, Min-Sum, Min-Max, or IPM attacks, SAE-FLDA maintains high model accuracy, significantly outperforming existing defense methods. This is particularly evident in highly stealthy poisoning attacks (such as Min-Max).
[0138] To verify the defensive performance of the SAE-FLDA algorithm when more than half of the attackers are present, and the vulnerability of existing defense methods, the experiment was set up so that 60% of the malicious edge nodes launched poisoning attacks:
[0139] Table 6. Final model accuracy of each algorithm when there are 60% malicious edge nodes on the CIFAR-10 dataset.
[0140]
[0141] Table 7. Final model accuracy of each algorithm when there are 60% malicious edge nodes on the MNIST dataset.
[0142]
[0143] The experimental results in Tables 6 and 7 show that when the proportion of malicious edge nodes in the cloud-edge collaborative system reaches 60%, the final global model accuracy of existing cloud-edge collaborative defense methods is significantly lower than that with 30% malicious edge nodes, especially on the CIFAR-10 dataset. This is because when the number of malicious edge nodes in the system exceeds half, the attacker completely controls the direction of global model convergence. The SAE-FLDA algorithm, however, maintains good defense performance because the proposed defense algorithm, while possessing a root of trust, can distinguish subtle differences between the gradients of malicious and benign models.
[0144] This invention addresses the weakness of traditional defense methods against attacks originating from more than half of the attackers in cloud-edge collaborative systems or against covert poisoning attacks. It proposes a data poisoning defense method based on a sparse autoencoder. The sparse autoencoder architecture uses the gradients of a malicious model to reconstruct facts with large errors using an autoencoder trained with the gradients of a benign model, thus improving the accuracy of anomaly detection on model gradient data. A ternary quantization mechanism is introduced to enhance the accuracy of anomaly detection and the robustness of aggregation by introducing a triplet loss function. Experiments were conducted on the CIFAR-10 and MNIST datasets, comparing the final model accuracy of various algorithms with malicious edge nodes set to 0%, 30%, and 60%, respectively. Ablation experiments were also performed using a traditional autoencoder without ternary quantization of model gradients, comparing the SAE-FLDA-ablation and SAE-FLDA algorithms. The experimental results demonstrate that the proposed sparse autoencoder model and the introduced ternary quantization mechanism effectively improve the algorithm's defense capabilities. When the malicious edge nodes were set to 0%, 30%, and 60% on the CIFAR-10 and MNIST datasets, respectively, the final model accuracy of this algorithm was higher than that of other comparative algorithms.
[0145] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment, characterized in that, Includes the following steps: S1. Before the cloud-edge collaborative task begins, the cloud server uses a clean dataset to train a sparse autoencoder as an anomaly gradient detector. The training of the sparse autoencoder as an anomaly gradient detector includes the following steps: S11. The cloud server uses a clean dataset to generate full-precision model gradients and converts the model gradients into ternary gradients through a ternary gradient quantization mechanism, using the ternary gradients as training samples. S12. Design multiple parallel convolutional branches in the sparse autoencoder, each branch using convolutional kernels of different sizes to capture different gradient features in the training samples. S13. Perform max pooling dimensionality reduction on the gradient features output by each convolutional branch; S14. The gradient features after dimensionality reduction are fused through a fully connected layer; S15. Dynamically adjust the sparsity penalty coefficient using adaptive sparsity constraints; the dynamic adjustment process of the sparsity penalty coefficient is expressed by the following formula: ； ； In the formula: This represents the initial sparsity penalty coefficient; This represents the reconstruction error in the current round, and its value is inversely proportional to the reconstruction effect. This represents the maximum reconstruction error on the training set; This represents the sparsity penalty coefficient after dynamic adjustment. This represents the feature vector after the model gradient is flattened when uploaded from the edge node. This represents the reconstructed output; The dimension of the feature vector, i.e., the total dimension after the input gradient data is flattened, determines the average range of the error calculation. Represents the first element of the original input vector. One component; The first element of the reconstructed output vector is... One component; S2. After the cloud-edge collaborative task begins, the cloud server reconstructs the model gradient uploaded by each edge node using the trained sparse autoencoder and calculates the reconstruction error. S3. The cloud server calculates the anomaly score for each edge node based on the reconstruction error; S4. The cloud server performs weighted aggregation of the model gradients of benign edge nodes based on the anomaly scores and updates the global model. S5. Simultaneously, the cloud server updates the sparse autoencoder using the ternary gradient of the benign edge nodes.
2. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 1, characterized in that, Each branch uses convolutional kernels of different sizes to capture different gradient features in the training samples, as represented by the following formula: ； ； ； In the formula: , , These represent convolution kernels with different sizes; This represents the feature vector after the model gradient is flattened when uploaded from the edge node. This represents a two-dimensional convolution operation, which involves using a convolution kernel to perform a convolution operation on the input feature vector to extract the spatial features of the input data. , , These represent feature extraction results at different scales.
3. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 2, characterized in that, The max-pooling dimensionality reduction of the output of each convolutional branch is expressed by the following formula: ； ； ； In the formula: , , These represent feature extraction results at different scales; This represents the max pooling operation, which performs a downsampling operation on the input features and selects the maximum value within the pooling window as the output. , , These represent the output features after max pooling dimensionality reduction.
4. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 3, characterized in that, The process of fusing the dimensionality-reduced features through a fully connected layer specifically includes the following steps: S141. The output feature results after max pooling dimensionality reduction are concatenated and fused along the feature dimension, and the concatenated multi-scale features are mapped to a unified feature space. S142. In the encoding stage, the concatenated multi-scale features are reduced to a hidden layer representation. S143. In the decoding stage, the compressed features are reconstructed layer by layer through fully connected layers to restore the original data dimension; Steps S141-S143 are represented by the following formulas: ； ； ； In the formula: , , These represent the output features after max pooling dimensionality reduction; This means concatenating features along the feature dimensions to form a comprehensive feature representation that includes multi-scale information; This represents the feature representation after fusion; This represents the fusion weight matrix, which is used to map the concatenated multi-scale features to a unified feature space. This represents the fusion bias term, used to increase the expressive power of the model; its size is consistent with the dimension of the fused features. Indicates the activation function; This represents the encoded hidden layer feature representation; This represents the decoder weight matrix, used to reconstruct the original input from the hidden layer features; Indicates the decoder bias term; This represents the reconstructed output, used to calculate the reconstruction error.
5. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 1, characterized in that, The ternary gradient quantization is expressed by the following formula: ； ； In the formula: Represents the full-precision model gradient; , indicating that the dimension is The ternary gradient vector; Representing vectors and The inner product of the vector, whose value is related to the vector Direction and original gradient The directionality is directly proportional to the consistency. Representing vectors Its inner product with itself; This represents the separated scalar, used to preserve the overall magnitude information of the gradient; It is a mathematical operator that represents the parameter that maximizes the function value. ; This represents the optimal ternary gradient vector.
6. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 1, characterized in that, The anomaly score for each edge node is calculated using the following formula: ； In the formula: Represents edge nodes The reconstruction error of the model gradient; This represents the minimum gradient reconstruction error of all models in this iteration; Represents edge nodes Abnormal scores.
7. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 1, characterized in that, The specific process of weighted aggregation of gradients for benign edge nodes is as follows: ； In the formula: Represents edge nodes Abnormal scores; , representing the smoothing factor, is used to control the impact of outlier scores on the weights; Represents the set of all benign edge nodes, i.e. edge nodes, This represents the mean of the anomaly scores for all edge nodes. Represents edge nodes Weights during aggregation.
8. The data poisoning defense method based on sparse autoencoders in a cloud-edge collaborative environment according to claim 7, characterized in that, Updating the sparse autoencoder using the ternary gradient of benign edge nodes includes aggregating the model gradient and updating the global model parameters, which are expressed by the following formulas: ； ； In the formula: Represents edge nodes Weights during aggregation; Indicates the first The edge node at the ... The scalar separated by the round is obtained by the ternary gradient quantization process; Indicates the first The edge node at the ... The ternary quantization gradient of the wheel; Indicates the first Wheel model gradient; Indicates the first Round global model parameters; This represents the global model learning rate, which controls the step size for model updates; Indicates the first The updated global model parameters.