A federated learning privacy protection method based on multi-flow network feature separation
By employing a federated learning method based on multi-stream network feature separation, lightweight global and fine-grained stream features are trained, and a fully connected layer is used as the network head. This solves the problem of poor image recognition performance caused by data heterogeneity, achieving higher image recognition accuracy and privacy protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUXI UNIV
- Filing Date
- 2025-02-13
- Publication Date
- 2026-07-24
AI Technical Summary
Existing personalized federated learning methods face challenges in dealing with data heterogeneity, including a trade-off between model personalization and global sharing, as well as excessive computational and communication overhead, resulting in poor recognition performance, especially in image recognition tasks.
A federated learning method based on multi-stream network feature separation is adopted. The feature separator and head are transmitted to the server, the client performs iterative training, and the data is aggregated on the server to form a new feature separator and head. Lightweight global and fine-grained stream features are trained, and the fully connected layer is used as the network head for image classification.
Without uploading the original data, better image recognition results were achieved, the risk of sensitive information leakage was reduced, and the accuracy of image recognition was improved.
Smart Images
Figure CN120163209B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence privacy protection, and in particular to a federated learning privacy protection method based on multi-stream network feature separation. Background Technology
[0002] With the rapid development of artificial intelligence and big data technologies, Federated Learning (FL), as an emerging distributed learning framework, has gradually become a research hotspot in the field of privacy-preserving machine learning. Federated Learning allows multiple participants (clients) to collaboratively train a global model while protecting the privacy of their local data, without having to upload their local data to a central server. This decentralized learning model is widely used in fields such as healthcare, finance, and the Internet of Things. However, due to significant differences in data distribution and quality among clients, Federated Learning faces considerable challenges in terms of data heterogeneity.
[0003] Data heterogeneity typically manifests in the following ways: 1. Inconsistent feature distribution; 2. Imbalanced label distribution; 3. Imbalanced sample size; 4. Differences in data quality. Specifically, for example, in IoT applications, sensor data collected by different devices may have different feature ranges and distributions. This inconsistency in feature distribution can lead to insufficient generalization ability of the global model on some clients. In the medical field, case data from different hospitals often focus on specific diseases, which can result in the global model lacking robustness to less common categories. Large enterprises possess large amounts of high-quality labeled data, while small enterprises have limited data. This imbalance in data scale can cause the model training process to be dominated by clients with larger data volumes, neglecting the contributions of clients with smaller but more information-rich data.
[0004] To address these challenges, Personalized Federated Learning (PFL), as an extension of federated learning, has garnered significant attention in recent years. Unlike traditional federated learning, which constructs a single global model, PFL aims to generate personalized models adapted to each client's local data distribution while ensuring global collaboration. By combining global information with local characteristics, PFL demonstrates significant advantages in scenarios with heterogeneous data.
[0005] Currently, the main personalized federated learning techniques used to address data heterogeneity include the following:
[0006] 1. Model decomposition-based approach: Divide the model into a shared layer and a personalized layer. The shared layer enables cross-client collaboration, while the personalized layer adapts to local data distribution.
[0007] 2. Meta-learning-based methods: Utilize meta-learning techniques to quickly adapt to the local distribution of the client, for example, by generating personalized models through a small number of gradient updates.
[0008] 3. Multi-task learning-based approach: The federated learning problem is modeled as a multi-task learning problem, optimizing the personalized performance of each client model and the global collaborative effect.
[0009] While these methods have mitigated the impact of data heterogeneity on federated learning to some extent, some problems remain, such as the trade-off between model personalization and global sharing, and the computational and communication overhead in multi-client collaboration. Summary of the Invention
[0010] The purpose of this invention is to address the shortcomings of existing technologies by providing a federated learning privacy-preserving method based on multi-stream network feature separation. This method aims to train lightweight global and fine-grained stream features without uploading the client's original data, and then use the last layer of the fully connected layer as the network head after feature separation to perform image classification tasks, thereby achieving better image recognition results.
[0011] To achieve the above objectives, the present invention provides the following solution:
[0012] A privacy-preserving method for federated learning based on multi-stream network feature separation includes:
[0013] S1. The server transmits a feature separator and a header to the client; wherein the feature separator includes a global feature separator and a fine-grained feature separator, and the header includes a global feature header and a fine-grained feature header;
[0014] S2. Use the client to train and iterate the feature separator and the head;
[0015] S3. Upload the iterated feature separator and header to the server.
[0016] S4. Based on the server side, the iterated feature separator and header are aggregated to form a new feature separator and header;
[0017] S5. Iterate through S1-S4 until the training of the feature separator and the head is completed; perform federated learning for privacy protection based on the trained feature separator and the head.
[0018] Optionally, the feature separator and header transmitted from the server to the client include:
[0019] The client's backbone is divided into a feature separator and a header;
[0020] Initialize the model parameters on the server side and distribute them to each client participating in the training;
[0021] In each iteration, the server randomly selects a preset group of clients to participate in the training.
[0022] Optionally, the client's backbone can be divided into feature separators including:
[0023] For the global feature separator, the input samples are mapped onto the feature space, and the head maps the extracted features onto the label space.
[0024] For the fine-grained feature separator, the input samples are mapped onto the feature space, and the head maps the features onto the label space.
[0025] Optionally, the training iterations using the client on the headers of the feature separator and the server include:
[0026] Select a dataset and divide it into a training set and a test set;
[0027] Based on a preset loss function, the feature separator and the head are trained using a training set and tested using a test set; wherein, the global feature separator is used for global flow feature separation, and the fine-grained feature separator is used for fine-grained flow feature separation.
[0028] Optionally, the global feature separator includes: an encryptor. With decoder ;
[0029] The global feature separator performs global stream feature separation, including:
[0030] Given an input feature map , The number of elements, The dimension of the input features is determined by the parameters of the global flow feature model. initialization , It serves as an intermediate layer for global flow features. The feature dimension after initialization, i.e. Through linear layer Linear layer The encryptor is obtained through random initialization. ,Right now: Decryptor , Let be the feature dimension of the decryptor, i.e.: ; The input feature information is encoded to generate a unique representation for each input, i.e., encrypted information, so that the input data is unencrypted. Unable to be identified before matching;
[0031] The encrypted information is:
[0032]
[0033] in, For standardization;
[0034] Using encrypted information and Perform matrix multiplication to generate global stream features after the decryptor. .
[0035] Optionally, the fine-grained feature separator is based on an improvement of VGG;
[0036] Improvements to VGG include: simplifying the intermediate network layers of VGG to three branches, two of which are equipped with... Convolutional layers The convolutional layer performs batch normalization only after each convolution. Another branch performs average pooling operations in the H and W directions on the input to extract pixel-level features. At the same time, a selective attention mechanism is introduced in this branch to enable the network to suppress irrelevant features and enhance key information.
[0037] The selective attention mechanism is as follows:
[0038]
[0039]
[0040]
[0041] in, express Convolution, This indicates the average pooling operation. 、 、 These represent flow one, flow two, and flow three, respectively; the characteristics of fine-grained flow are:
[0042]
[0043] Optionally, the head is:
[0044]
[0045]
[0046] in, 、 They represent respectively by 、 Parameterized global headers and fine-grained headers, The logical value representing the global stream characteristic. The logical value for fine-grained flow characteristics.
[0047] Optionally, the preset loss function is:
[0048]
[0049] in, It is a hyperparameter. 、 These represent the losses of the global feature separator and the fine-grained feature separator, respectively. Represented as model parameters, This represents the input from the i-th client. This indicates the corresponding label.
[0050] Alternatively, the loss of the global feature separator is:
[0051]
[0052] in, For the category weighting factor, To focus parameters, This represents the logical value of the m-th sample of class c from the global stream header of client i. It indicates the first The number of classes This represents the number of samples m that belong to category c;
[0053] The loss of the fine-grained feature separator is set as the cross-entropy loss function. .
[0054] Optionally, aggregating each part based on the server-side includes:
[0055] The server aggregates the two types of separators and their respective headers received and performs a weighted average, as shown in the following formula:
[0056]
[0057] in, It is the loss of the i-th client. Let be the weight of the i-th client.
[0058] The beneficial effects of this invention are as follows:
[0059] This invention first transmits a feature separator and its header from the server to the client. Then, the client iteratively trains the feature separator and the server header. Next, the iterated feature separator and server header are uploaded back to the server. Then, the server aggregates each part to form a new feature separator and header. Finally, the above steps are iterated until the training of the feature separator and header is complete. This invention aims to train lightweight global and fine-grained streaming features without uploading the client's original data, and to use the last layer of the fully connected layer as the network header after feature separation for image classification tasks, achieving better image recognition results. Attached Figure Description
[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0061] Figure 1 This is a schematic diagram of a federated learning privacy protection method framework based on multi-stream network feature separation according to an embodiment of the present invention;
[0062] Figure 2 This is a schematic diagram of a federated learning privacy protection method based on multi-stream network feature separation according to an embodiment of the present invention;
[0063] Figure 3 This is a comparison chart of the accuracy of the method in this embodiment of the invention with other advanced federated learning algorithms. Detailed Implementation
[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0065] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0066] like Figures 1-2 As shown, this embodiment proposes a privacy-preserving method for federated learning based on multi-stream network feature separation, including:
[0067] Step 1: The server broadcasts the global and fine-grained feature separators, along with their respective headers (the last fully connected layer of the network), to the client.
[0068] Step 2: Each client updates the global and fine-grained feature separators, as well as the global and fine-grained feature headers, through local training iterations.
[0069] Step 3: The client uploads the locally updated multi-stream network feature separator and header to the server.
[0070] Step 4: Each part on the server side is aggregated to form a new global feature separator, fine-grained feature separator, global header and fine-grained header, and the training is completed by iterating through steps 1-4.
[0071] Privacy is protected through federated learning based on a trained feature separator and a header. The feature separator extracts features from the raw data; these features are typically low-dimensional representations with sensitive information removed. The header is used for task classification. This separation allows clients to upload only the features, rather than the raw data, reducing the risk of sensitive information leakage.
[0072] Furthermore, based on the feature separator transmitted from the server to the client, the server-side header includes:
[0073] The client's backbone is divided into a feature separator and a header;
[0074] Initialize the model parameters on the server side and distribute them to each client participating in the training;
[0075] In each iteration, the server randomly selects a preset group of clients to participate in the training.
[0076] Specifically, in this embodiment, step 1 further includes the following:
[0077] Step 1-1: Divide the client's backbone into feature separators. , and head 、 The difference is that this embodiment designs an MFPN module, which separates the global flow features of the client from the personalized features containing the client's fine-grained flow features; specifically, for the flow features of the client... The parameterized global feature separator, in this embodiment, maps the input samples onto the feature space, i.e. ,Depend on The parameterized head maps the extracted features onto the label space, i.e. For those by Similarly, a parameterized fine-grained feature separator maps input samples onto a feature space. , Depend on The parameterized head maps features to the label space, i.e. ; These represent the dimensions of the input, feature space, and labels, respectively.
[0078] Step 1-2, Initialization: Initialize model parameters on the server side. And distribute it to each client participating in the training.
[0079] Steps 1-3, Client Selection: In each iteration, the server randomly selects a subset of clients. St Participate in training.
[0080] Furthermore, the training iterations using the client on the feature separator and the server-side headers include:
[0081] Select a dataset and divide it into a training set and a test set;
[0082] Based on a preset loss function, the feature separator and the head are trained using a training set and tested using a test set; wherein, the global feature separator is used for global flow feature separation, and the fine-grained feature separator is used for fine-grained flow feature separation.
[0083] Specifically, in this embodiment, step 2 includes the following steps:
[0084] Step 2-1, Select the dataset.
[0085] Step 2-2: Divide the dataset into a training set and a test set.
[0086] To better simulate the heterogeneity of data between different clients, this embodiment uses parameters. The Dirichlet distribution function is used to partition the training dataset; specifically, the parameters... The diversity of data distribution for each client was controlled: Smaller: The data distribution is more uneven, and the data categories differ more on each client. Larger: Data distribution is more even, and the differences in data categories across each client are smaller. For each client k Ten categories of data distribution vectors By parameters Dirichlet distribution Generate. The specific formula is as follows:
[0087]
[0088] in, Distribution parameters for each category, It is the generated data distribution vector.
[0089] Steps 2-3: In the MFPN module designed in this embodiment, different feature separators are designed to perform feature separation. Specifically:
[0090] (1) Global flow feature separation
[0091] Considering the computational performance of edge devices and the dominance of global features in the features of a single client, this embodiment proposes an input-independent model in the global feature separator: the cipher. With decoder .
[0092] Given an input feature map , The number of elements, The dimension of the feature; this embodiment uses a linear layer initialization ,Right now Through linear layer , The encryptor is obtained through random initialization. ,Right now: Decryptor ,Right now: ; The input feature information is encoded to generate a unique representation for each input, so that the input data is not... It cannot be identified before matching, and at the same time, It can adapt to changes in context, flexibly selecting different information to ensure that only correctly matched information can be encrypted and used, thus reducing the risk of information leakage. The encrypted information is defined as:
[0093]
[0094] For standardization; As a decryptor, it is dynamically changing. Each change triggers a different decryption method, making... Provide different information, and only , Activated only upon successful matching To ensure that the final output matches the current output. This consistency reduces information interference and distortion.
[0095] Finally, using the encrypted information and Perform matrix multiplication and generate the result after the decryptor. :
[0096] .
[0097] (2) Fine-grained flow characteristic separation
[0098] Fine-grained flow features are crucial for accurately identifying images and acquiring the information needed to do so, considering the low latency and low overhead requirements of edge devices. Convolutional layers significantly increase communication overhead; this embodiment employs... Convolutional layers Convolutional multi-stream networks process features at different scales, avoiding more sequential processing and greater depth, while fusing features from different streams in the later stages of the network.
[0099] In this embodiment, fine-grained flow features are extracted using three branches, corresponding to flow one, flow two, and flow three, respectively; among them, flow one and flow two use 1 1 convolutional layer 3 The convolutional layer 3 performs the operation, and the other branch performs average pooling operations in the H and W directions on three pairs of inputs respectively to extract pixel-level features, while introducing a selective attention mechanism.
[0100] This embodiment improves upon VGG by redesigning it to better suit edge devices; specifically, this embodiment equips two branches of the VGG intermediate network layer with... Convolutional layers The convolutional layers perform batch normalization after each convolution, extracting effective fine-grained features while reducing parameters. Another branch in this embodiment performs average pooling operations in the H and W directions on the input to extract pixel-level features. Simultaneously, this embodiment introduces a selective attention mechanism (sigmoid) in this branch, enabling the network to suppress irrelevant features and enhance key information. The proposed expression is as follows:
[0101]
[0102]
[0103]
[0104] express Convolution, This indicates the average pooling operation. 、 、 These represent flow one, flow two, and flow three, respectively. Therefore, the characteristics of fine-grained flow are:
[0105]
[0106] Steps 2-4: For the two different splitters, this embodiment connects the last fully connected layer of the network layer to the features separated by the splitter. This last layer is referred to as the network header in this embodiment.
[0107]
[0108]
[0109] 、 They represent respectively by 、 Parameterized global headers and fine-grained headers, Logit (logical value) for global stream characteristics. Logit (logistic value) for fine-grained flow characteristics.
[0110] Steps 2-5: Considering that cross-entropy loss may favor the majority of classes while neglecting the minority, this embodiment uses Focal loss for the global flow feature branch and improves upon it by balancing the number of samples from different classes in the loss calculation. This makes the model focus more on rare classes and improves its performance on imbalanced data. The formula proposed in this embodiment is as follows:
[0111]
[0112] in, For the category weighting factor, This is a focusing parameter used to adjust the attention given to samples with a small number of categories; This represents the logits (logical value) of the m-th sample of class c from the global stream header of client i; in this embodiment, the parameter is added as follows. , parameter, It indicates the first The number of classes is adjusted to normalize the samples by class, thereby improving sensitivity to rare class samples. This represents the number of samples m that belong to category c;
[0113] Since fine-grained flow features are less affected by class imbalance, this embodiment sets its loss as the cross-entropy loss function. :
[0114] In summary, this embodiment proposes:
[0115]
[0116] in, It is a hyperparameter. 、 These represent the losses of the global feature separator and the fine-grained feature separator, respectively. These are represented as model parameters.
[0117] Next, proceed to step 3:
[0118] Step 3: The client uploads the locally updated multi-stream network feature separator and header to the server.
[0119] Further, step 4 includes:
[0120] Step 4-1: The server aggregates the two types of separators and their respective headers received and performs a weighted average. The formula proposed in this embodiment is as follows:
[0121]
[0122] in, It is the loss of the i-th client. Let be the weight of the i-th client.
[0123] Step 4-2: Broadcast the result obtained in step 4-1 to each client, and continue to cycle through steps 1 to 4 until the algorithm converges.
[0124] To comprehensively evaluate the superior performance of the innovative method proposed in this invention in practical applications, the invention selected the specialized experimental dataset CIFAR-100 and compared it with traditional federated learning algorithms according to the evaluation criteria detailed in this section. The experimental environment was set up on the Ubuntu 18.04.3 LTS platform, using an Intel Core i7 13700KF CPU and an NVIDIA GeForce RTX 3060TI GPU (32GB RAM). For the programming language and framework, the deep learning model in this paper was built using Python and the PyTorch framework.
[0125] In this specific embodiment of the invention, the dataset selected is the CIFAR-10 dataset, which is widely used to evaluate the performance of image classification algorithms and models due to its moderate difficulty and wide application. Its diversity and complexity make it a classic benchmark for image classification tasks.
[0126] In the specific implementation plan, the CIFAR-10 dataset is divided into a training set and a test set, with 50,000 images in the training set and 10,000 images in the test set.
[0127] The specific parameter settings in this invention are as follows:
[0128] (1) Dirichlet parameter: The Dirichlet parameter α is set to 0.1 to simulate a high degree of client data heterogeneity.
[0129] (2) Number of clients: The total number of clients is set to 20, of which 10 online clients participate in each round of training. Online clients are selected by random sampling.
[0130] (3) Training of client model parameters: The batch size is set to 64, the learning rate is 0.001, the number of local training rounds on the client is set to 1, and the total number of training rounds is 200.
[0131] To verify the effectiveness of the proposed method, the most conventional accuracy metric is used to measure its effectiveness. In the specific implementation method, the method of this invention is experimentally compared with advanced models such as the classic federated learning algorithm FedAvg. Figure 3 As shown, compared with existing federated learning methods, the method proposed in this invention has significant advantages in performance metrics, improving accuracy by 5.6% compared to FedAvg, and optimizing image recognition performance under non-independent and identically distributed conditions.
[0132] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A federated learning privacy-preserving method based on multi-stream network feature separation, applied to image classification tasks, characterized in that, include: S1. The server broadcasts a feature separator and a header to the client; wherein the feature separator includes a global feature separator and a fine-grained feature separator, and the header includes a global feature header and a fine-grained feature header; S2. Utilize the client to perform local training iterations on the feature separator and the head; including: Select a dataset and divide it into a training set and a test set; Based on a preset loss function, the feature separator and the head are trained using a training set and tested using a test set; wherein, the global feature separator is used for global flow feature separation, and the fine-grained feature separator is used for fine-grained flow feature separation. The global feature separator includes: an encryptor With decoder ; The global feature separator performs global stream feature separation, including: Given an input feature map , The number of elements, The dimension of the input features is determined by the parameters of the global flow feature model. initialization , It serves as an intermediate layer for global flow features. The feature dimension after initialization, i.e. Through linear layer Linear layer The encryptor is obtained through random initialization. ,Right now: Decryptor , Let be the feature dimension of the decryptor, i.e.: ; The input feature information is encoded to generate a unique representation for each input, i.e., encrypted information, so that the input data is unencrypted. Unable to be identified before matching; The encrypted information is: in, For standardization; Using encrypted information and Perform matrix multiplication to generate global stream features after the decryptor. ; The fine-grained feature separator is based on an improvement of VGG; Improvements to VGG include: simplifying the intermediate network layers of VGG to three branches, two of which are equipped with... Convolutional layers The convolutional layer performs batch normalization only after each convolution. Another branch performs average pooling operations in the H and W directions on the input to extract pixel-level features. At the same time, a selective attention mechanism is introduced in this branch to enable the network to suppress irrelevant features and enhance key information. The selective attention mechanism is as follows: in, express Convolution, This indicates the average pooling operation. 、 、 These represent flow one, flow two, and flow three, respectively; the characteristics of fine-grained flow are: ； S3. Upload the iterated feature separator and header to the server. S4. Based on the server side, the iterated feature separator and header are aggregated to form a new feature separator and header; S5. Iterate through S1-S4 until the training of the feature separator and the head is completed; perform federated learning for privacy protection based on the trained feature separator and the head.
2. The privacy-preserving method for federated learning based on multi-stream network feature separation according to claim 1, characterized in that, Based on the server-side transmission of the feature separator and header to the client, including: The server-side backbone is divided into a feature separator and a header; Initialize the model parameters on the server side and distribute them to each client participating in the training; In each iteration, the server randomly selects a preset group of clients to participate in the training.
3. The privacy-preserving method for federated learning based on multi-stream network feature separation according to claim 2, characterized in that, The client's backbone is divided into feature separators, including: For the global feature separator, the input samples are mapped onto the feature space, and the head maps the extracted features onto the label space. For fine-grained feature separators, input samples are mapped onto the feature space, and the head maps the features onto the label space.
4. The privacy-preserving method for federated learning based on multi-stream network feature separation according to claim 1, characterized in that, The head is: in, 、 They represent respectively by 、 Parameterized global headers and fine-grained headers, The logical value representing the global stream characteristic. The logical value for fine-grained flow characteristics.
5. The privacy-preserving method for federated learning based on multi-stream network feature separation according to claim 1, characterized in that, The preset loss function is: in, It is a hyperparameter. 、 These represent the losses of the global feature separator and the fine-grained feature separator, respectively. Represented as model parameters, This represents the input from the i-th client. This indicates the corresponding label.
6. The privacy-preserving method for federated learning based on multi-stream network feature separation according to claim 5, characterized in that, The loss of the global feature separator is: in, For the category weighting factor, To focus parameters, This represents the logical value of the m-th sample of class c from the global stream header of client i. It indicates the first The number of classes This represents the number of samples m that belong to category c; The loss of the fine-grained feature separator is set as the cross-entropy loss function. .
7. The privacy-preserving method for federated learning based on multi-stream network feature separation according to claim 1, characterized in that, Aggregating each part based on the server-side includes: The server aggregates the two types of separators and their respective headers received and performs a weighted average, as shown in the following formula: in, It is the loss of the i-th client. Let be the weight of the i-th client.