A feature-based label recovery method and system in a classifier-free transmission federated learning

Abstract

The application belongs to the field of information technology and relates to a feature-based label recovery method and system in a classifier-free transmission federated learning. The method comprises the following steps: constructing multiple training subsets with different label distributions by using auxiliary data sets and performing local training respectively, generating feature anchor points corresponding to different label distributions, and combining the feature anchor points and the corresponding label distributions to form a meta-training data set; constructing a meta-model, training the meta-model based on the meta-training data set, and learning a bidirectional mapping relationship between the feature anchor points and the label distributions; receiving a feature extractor uploaded by a target client, extracting the feature anchor points of the target client on a proxy data set, predicting the label distribution of the target client by using the trained meta-model, and realizing the recovery of the label distribution of the target client. The application can complete label recovery under the premise of only accessing the feature extractor without relying on the classifier gradient, and solves the problem that the existing federated learning label recovery method is invalid in a classifier invisible scene.

Description

Technical Field

[0001] This invention belongs to the fields of information technology and machine learning technology, and specifically relates to a feature-based label recovery method and system in federated learning without classifier transmission. Background Technology

[0002] Federated learning, as an emerging distributed machine learning paradigm, has been widely applied in recent years in fields with extremely high data security requirements, such as healthcare and finance, due to its advantages in protecting privacy and improving communication efficiency. Under the federated learning framework, multiple clients collaboratively train the model without sharing the original data, participating in the optimization of the global model through local model updates and server parameter aggregation. Although this mechanism is designed to retain data locally, thus avoiding centralized processing of the original data, the local model updates uploaded by the clients still reflect the statistical characteristics of their local data during the training process.

[0003] Label distribution, as one of the core features of training data, plays a crucial role in many tasks. Accurately understanding the client's label structure not only helps evaluate the model's training status in non-independent and identically distributed scenarios but also supports needs such as model transparency analysis, training behavior consistency detection, and system performance tuning. Existing label distribution inference methods generally rely on the explicit relationship between the classifier's gradient and the labels. However, with the increasing adoption of "independent classifier upload" communication mechanisms in federated learning methods (such as personalized federated learning) in recent years, the applicability of these methods has been limited. Therefore, how to accurately estimate the client's training label distribution without access to the classifier has become a critical technical problem that urgently needs to be solved.

[0004] Specifically, the existing technology has the following drawbacks:

[0005] 1) Label recovery methods based on classifier gradients are difficult to apply to new scenarios. Existing label recovery methods mainly rely on the direct correlation between classifier gradients and training labels for inference. These methods typically establish a mapping relationship between gradients and labels to recover the labels of training data. For example, some methods determine the label category by analyzing the sign characteristics of the classifier gradients, while others construct a system of linear equations containing gradients, probabilities, and labels, and accurately reconstruct the sample labels by solving the system. Other research uses Monte Carlo estimation of confidence bias to indirectly recover the label distribution. These methods can, to some extent, obtain the distribution of training labels and assist in achieving more accurate data reconstruction. However, they all rely on an important premise: that the server can access the classifier gradients uploaded by the client. However, in recent years, to address the classifier drift problem caused by data heterogeneity in federated learning, more and more algorithms have adopted a mechanism of independent classifier communication, meaning that the client no longer uploads the gradients of the classifier during local training. While this strategy alleviates the interference of non-independent and identically distributed data on model performance, it also directly renders the aforementioned label recovery methods based on classifier gradients ineffective. Therefore, under this type of communication mechanism, the server can no longer rely on classifier gradients for label inference, and existing recovery methods face structural bottlenecks and are difficult to adapt to actual needs.

[0006] 2) Existing methods lack in-depth exploration of the relationship between feature space and label distribution. Recent studies have shown that models trained with different label distributions, even using the same initial parameters, will extract significantly different feature spaces, especially in non-independent and identically distributed data environments, where this bias is even more pronounced. However, existing work has not fully explored the relationship between the internal structure of the feature space and the training label distribution, mainly focusing on the analysis of classifier parameters while neglecting the information potential carried by the feature extractor itself. Although some studies have observed inconsistencies in the feature space, a systematic method for recovering the label distribution from the feature level has not yet been established. Particularly in the case of communication mechanisms invisible to the classifier, how to infer the training label distribution using only information from the feature extractor remains an under-studied problem, and existing work has significant limitations in terms of methodological effectiveness and applicability. Summary of the Invention

[0007] This invention proposes a feature-based label recovery method and system for federated learning without classifier transmission, aiming to address the failure of existing federated learning label recovery methods in scenarios where the classifier is not visible. In federated learning, an increasing number of algorithms adopt communication mechanisms that only upload feature extractor parameters to mitigate performance degradation caused by data heterogeneity. However, this communication method blocks the server's access to classifier gradients, rendering traditional label recovery methods that rely on classifier information inapplicable and severely limiting the scope of existing label recovery techniques. Therefore, there is an urgent need for an effective technical solution that does not rely on classifier gradients and can complete label recovery with only access to the feature extractor.

[0008] To address this, this invention constructs a proxy dataset and a meta-learning mechanism to learn the implicit mapping relationship between the "feature anchors" extracted by the feature extractor and the training label distribution. Specifically, the server constructs multiple simulated training sets with different label distributions using public data or random noise data. Multiple models are trained under unified initial conditions and training strategies, and their feature anchors on a fixed proxy dataset are extracted, generating a large-scale pair of feature anchors and label distributions. Subsequently, a meta-model is constructed, using an encoder-decoder structure to learn the bidirectional mapping relationship between feature anchors and label distributions. During the recovery phase, when the server receives the feature extractor uploaded by the target client, it can directly calculate its feature anchors from the proxy dataset and predict its label distribution using the pre-trained meta-model, achieving label recovery without the involvement of classifier information.

[0009] The technical solution adopted in this invention is as follows:

[0010] A feature-based label recovery method in federated learning without classifier transmission includes the following steps:

[0011] Multiple training subsets with different label distributions are constructed using auxiliary datasets and trained locally to generate feature anchors corresponding to different label distributions. The feature anchors and their corresponding label distributions together form a meta-training dataset.

[0012] Construct a meta-model, train the meta-model based on the meta-training dataset, and learn the bidirectional mapping relationship between feature anchors and label distribution;

[0013] The system receives the feature extractor uploaded by the target client, extracts its feature anchors on the proxy dataset, and uses the trained meta-model to predict its label distribution, thereby restoring the label distribution of the target client.

[0014] Furthermore, the auxiliary dataset is public data or random noise samples.

[0015] Furthermore, the meta-training dataset is constructed using the following steps:

[0016] From the C-1 dimension unit simplex space Δ C-1 Medium-sampled label distribution vector d dum The Dirichlet distribution is used as the sampling strategy.

[0017] According to d dum Construct a training subset D from the auxiliary dataset. dum And use a unified training configuration for local training to obtain the corresponding feature extractor parameters.

[0018] Using a fixed proxy dataset D p Perform forward computation on the feature extractor to extract the corresponding feature anchor point a. dum , to obtain a set of metadata pairs (a dum ,d dum );

[0019] By repeating the above process, a large-scale meta-training dataset containing multiple label distributions and their corresponding feature anchors is constructed.

[0020] Further, the feature anchor points are extracted using the following steps:

[0021]

[0022] Where a is the feature anchor point, D p Let x be the input sample and θ be the proxy dataset. f Let f(·) be the feature extractor, representing the input sample x after passing through the feature extractor θ. f It is then mapped to a K-dimensional feature representation.

[0023] Furthermore, the meta-model adopts an autoencoder structure, which includes an encoder and a decoder; the meta-model combines the label distribution prediction loss and the feature anchor reconstruction loss to capture the bidirectional mapping relationship between feature anchors and label distribution.

[0024] Furthermore, the meta-model is trained using the following steps:

[0025] Encoder φ(·,w φ Map the input feature anchor point 'a' to the predicted label distribution.

[0026] Decoder ψ(·,w ψ ) Received predicted label distribution Reconstruct the estimated value of the feature anchor point

[0027] The following joint loss function is used for optimization:

[0028]

[0029] in, Herringer distance is used to measure the loss for predicting the label distribution. Similarity to the true label distribution d; As the feature anchor reconstruction loss, cosine similarity is used to measure the difference between the original feature anchor a and the reconstructed feature anchor a. The angle difference between them; the hyperparameter λ is used to balance the weights of the two sub-loss terms, controlling the trade-off between label prediction accuracy and feature representation quality.

[0030] Further, the step of receiving the feature extractor uploaded by the target client, extracting its feature anchors on the proxy dataset, and predicting its label distribution using the trained meta-model includes:

[0031] Received feature extractor θ uploaded by the target client f Then, using the proxy dataset D p Extract the corresponding feature anchor point a t This feature anchor point reflects the overall feature response of the target client on a fixed data distribution;

[0032] a t The encoder φ(·,w) of the trained meta-model is input. φ ), to obtain the predicted label distribution

[0033] A feature-based label recovery system for federated learning without classifier transmission, comprising:

[0034] The metadata building module is used to construct multiple training subsets with different label distributions using public data or random noise samples, and to train them locally. It generates feature anchors corresponding to different label distributions. The feature anchors and their corresponding label distributions together form the meta-training dataset.

[0035] The meta-model training module is used to build meta-models, train meta-models based on meta-training datasets, and learn the bidirectional mapping relationship between feature anchors and label distributions.

[0036] The label inference module is used to receive the feature extractor uploaded by the target client, extract its feature anchor points on the proxy dataset, and use the trained meta-model to predict its label distribution, thereby restoring the label distribution of the target client.

[0037] This invention proposes a label recovery method based on feature representation, which can accurately recover the label distribution of a federated learning client without relying on classifier gradients, overcoming the applicability limitations of existing recovery methods in scenarios where the classifier is not visible. Compared with existing technologies, this invention has the following significant advantages:

[0038] 1. No classifier gradient required, adaptable to a wider range of federated communication mechanisms. This invention is applicable to classifier-independent communication mechanisms (such as FedPer, FedBABU, etc.) that have been widely adopted in recent years. Label recovery can be completed by only obtaining model update information from the feature extractor, significantly broadening the application scope of the recovery method.

[0039] 2. It is independent of the data domain and possesses excellent transferability. This invention constructs "feature anchors" on a proxy dataset and learns the mapping relationship between these anchors and the label distribution. Experiments show that even when using random noise as proxy data, the meta-model can still accurately complete label inference, indicating that it does not depend on the specific semantic domain of the client's real data and possesses strong versatility and robustness.

[0040] 3. High recovery accuracy and stable recovery results. This invention has been systematically evaluated on multiple publicly available image classification datasets (such as CIFAR-10, CIFAR-100, SVHN) and various model architectures (such as 4-layer convolutional networks, LeNet-5, ResNet-20, VGG-16). Experimental results show that the proposed method can stably recover high-quality label distributions in both single-round and continuous recovery scenarios, and its prediction results are significantly better than random guessing baselines in metrics such as Wasserstein distance and Hellinger distance. Attached Figure Description

[0041] Figure 1 This is a feature-based label inference framework diagram.

[0042] Figure 2 These are visualizations of the category proportions of label recovery prediction in a single-round recovery scenario, where (a) is a visualization of the label distribution of the CIFAR-10 dataset and (b) is a visualization of the label distribution of the CIFAR-100 dataset.

[0043] Figure 3 The visualizations are of recovery results under different data heterogeneity in multi-round recovery scenarios, where (a) is the visualization of recovery results for the CIFAR-10 dataset and (b) is the visualization of recovery results for the CIFAR-100 dataset. Detailed Implementation

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0045] like Figure 1As shown, the federated learning label recovery method proposed in this invention consists of three main steps (or modules): metadata construction, meta-model training, and label inference. These three work together to recover the client's training label distribution without acquiring the classifier's gradient. First, the server uses public data or random noise samples to construct multiple training subsets by sampling diverse label distributions and trains them locally, thereby generating feature anchors corresponding to different label distributions. These feature anchors and their corresponding label distributions together form the meta-training dataset. Second, based on the constructed meta-training dataset, the server designs and trains an autoencoder structure meta-model consisting of an encoder and a decoder. This model can not only accurately predict the label distribution from the feature anchors but also has reverse reconstruction capabilities to ensure the complete representation of feature information. Finally, in the actual recovery stage, the server extracts the feature anchors from the feature extractor uploaded by the target client on the proxy dataset and uses the trained encoder module to infer its label distribution, thereby recovering the client's label distribution. This scheme has advantages such as strong versatility, high prediction accuracy, and low computational cost, and is suitable for federated learning communication mechanisms where various classifiers are not visible.

[0046] (I) Basic Framework for Label Recovery in Federated Learning:

[0047] This invention is applicable to typical federated learning training architectures. In this scenario, it is assumed that there are N clients, each client i holding a local private dataset D. i This dataset can be represented as the union D of different categories of data. i =∪ c∈[C] D i,c Where C is the total number of categories, D i,c Let d represent the set of samples belonging to class c in client i. Therefore, the label distribution of each client's local data can be represented by the vector d ∈ Δ. C-1 It means that among them This vector is defined in the C-1 dimensional probability simplex space Δ C-1 The data reflects the proportion of local client data in each category.

[0048] In terms of model structure, models in federated learning systems can typically be divided into feature extractors θ. f With classifier θ h Two parts. The input sample x, after passing through the feature extractor, is mapped to a K-dimensional feature representation. The classifier then generates the corresponding category prediction probability distribution based on this feature.

[0049] In the framework of federated learning, several clients possessing training data interact with a central server in multiple rounds. In each round, the client uploads locally trained model parameter updates to the server, while the server aggregates the parameters and distributes the global model back to the clients. In federated learning communication mechanisms where the classifier is not visible, such as FedPer or FedBABU, the client does not upload the classifier's gradients or parameters during local training; the server only receives local updates from the feature extractor. Specifically, in each round, the i-th client uses its private data D... i After training the model parameters locally, upload the updated feature extractor parameters. The server then aggregates the client models using the following weighted method to update the global model:

[0050]

[0051] Where, n i =|D i | represents the amount of data for client i. This represents the sum of data from all clients.

[0052] Label recovery refers to the server inferring the label distribution d of the client's data by updating the model parameters uploaded by the client. In a "classifier-invisible" communication architecture, the server cannot obtain the client's classifier. Traditional label recovery methods relying on classifier gradients are no longer feasible. Therefore, the server needs to recover label distribution information from a local model containing only the feature extractor without accessing the classifier parameters. To achieve this, the server needs to construct a new label recovery framework that infers the client's label distribution d using only the uploaded feature extractor parameters and auxiliary data, thus realizing label inference. In this scenario, assume the server possesses a proxy dataset D. p In addition, an auxiliary dataset is provided, along with details such as the initial model, training algorithm, learning rate, and batch size for local training on the client side. The proxy dataset refers to a set of publicly available, synthetic, or randomly generated data prepared in advance by the server, used to perform forward inference on the model to obtain feature anchors, hence the name "proxy" dataset.

[0053] (II) Metadata Construction:

[0054] This invention proposes a new path for label recovery based on a key observation: even without access to the classifier, the feature extractor parameters uploaded by the client still contain information closely related to the training label distribution. Specifically, if a fixed proxy dataset D is used... pEach locally trained feature extractor can extract an average feature vector, called a "feature anchor," which is represented as:

[0055]

[0056] This invention discovers that different training label distributions lead to significantly different feature anchors, and this difference is largely independent of the domain of the training data. This property allows the server to infer the label distribution of the training data simply by observing the feature anchors, thereby bypassing classifier dependence and achieving label recovery.

[0057] To achieve the above goals, the server first needs to establish a mapping relationship between "feature anchors and label distributions". Since the client data cannot be directly accessed, the server uses self-constructed metadata to simulate the client training process, thereby constructing the supervised sample pairs required for training.

[0058] First, the server starts from the C-1 dimensional unit simplex space Δ C-1 Medium-sampled label distribution vector d dum Considering the complexity of uniform sampling in high-dimensional space, this invention employs the Dirichlet distribution as the sampling strategy, i.e.: d dum ~Dir(β). By adjusting the value of parameter β, the degree of skewness in the label distribution can be controlled, thereby covering a variety of distribution patterns from balanced to extreme, and enhancing the generalization ability of the model.

[0059] Secondly, the server relies on d dum Construct a training subset D from an auxiliary dataset (such as public data or random noise). dum And use a unified training configuration (including model initialization, optimizer, learning rate, number of local rounds, batch size, etc.) for local training to obtain the corresponding feature extractor parameters.

[0060] Next, the server uses a fixed proxy dataset D. p The feature extractor is computed forward, and its corresponding feature anchor point a is extracted according to formula (2). dum Ultimately, a set of metadata pairs (a dum ,d dum By repeating the above process M times, a large-scale meta-training dataset containing multiple label distributions and their corresponding feature anchors can be constructed.

[0061] The metadata construction process is completed entirely offline on the server side, without relying on actual client data, and allows the use of random noise unrelated to the real task as auxiliary data, thus possessing good operability and versatility.

[0062] (III) Meta-model Training:

[0063] In completing a large-scale metadata dataset Following the initial construction, this invention further designs and trains a meta-model to learn the nonlinear mapping relationship between feature anchor point a and training label distribution d. The core objective of this meta-model is to accurately predict the corresponding training label distribution from the feature anchor point, thereby providing a foundation for subsequent label recovery.

[0064] To enhance the expressive power and stability of the model, this invention employs an autoencoder structure as the meta-model architecture, comprising an encoder module and a decoder module, responsible for forward inference and feature reconstruction, respectively. This structure not only improves the generalization ability of the mapping but also guides the model to extract more effective semantic features through reconstruction constraints.

[0065] Specifically, let the encoder be φ(·,w) φ ), which will input feature anchor points The following formula maps to the predicted label distribution.

[0066]

[0067] The Softmax function guarantees that the output satisfies the probability constraints of the label distribution, w φ This represents the encoder's parameters.

[0068] Subsequently, the decoder ψ(·,w ψ ) Received predicted label distribution Reconstruct the estimated value of the feature anchor point Where w ψ This represents the parameters of the decoder.

[0069] The training objective of this autoencoder is to minimize the difference between the predicted label distribution and the true label distribution, while ensuring the consistency between the reconstructed feature anchors and the original feature anchors. Therefore, the following joint loss function is designed for optimization:

[0070]

[0071] in, Let d represent the distribution of the i-th predicted label. i This represents the distribution of the i-th true label. Let a represent the feature anchor point of the i-th reconstruction. iLet represent the i-th original feature anchor point, and M represent the label distribution or the number of feature anchor points.

[0072] in, The Hellinger distance is used to measure the predicted label distribution loss. The similarity between the true label distribution d and the actual label distribution d is defined as follows:

[0073]

[0074] and As the feature anchor reconstruction loss, cosine similarity is used to measure the difference between the original feature anchor a and the reconstructed feature anchor a. The difference in the included angle between them is defined as:

[0075]

[0076] The hyperparameter λ is used to balance the weights of the two sub-loss terms, controlling the trade-off between label prediction accuracy and feature representation quality.

[0077] After training, the meta-model has the ability to infer the corresponding label distribution from any input feature anchor point, providing a high-precision label recovery foundation for the subsequent label recovery stage.

[0078] (iv) Label Inference:

[0079] After completing the meta-model training, the present invention enters the actual recovery stage, that is, based on the feature extractor parameters uploaded by the client, it infers the label distribution of its local training data.

[0080] Specifically, when the server receives a feature extractor θ uploaded by a client... f Then, the proxy dataset D is used first. p Extract the corresponding feature anchor point a according to formula (2). t This feature anchor reflects the overall feature response of the current client model on a fixed data distribution, indirectly encoding the label structure of its training data. Subsequently, the server will... t Input the encoder part φ(·,w) of the pre-trained meta-model φ The predicted label distribution is obtained according to formula (3). Since the encoder structure has learned the correspondence between feature anchors and label distributions on a large number of simulated sample pairs, the prediction result can approximate the client's true training label distribution d under unsupervised conditions. t .

[0081] Key points of this invention:

[0082] 1. This invention proposes a feature-based label recovery method, called Feature-Based Label Recovery (FLR), which innovatively recovers the client-side training label distribution under a federated learning communication mechanism without classifier gradients. The core idea of ​​FLR is to mine the implicit correlation between the feature extractor and the label distribution, and to construct "feature anchors". The method achieves modeling through a "label distribution" mapping relationship. It comprises three key modules: metadata construction, meta-model training, and label inference. In the metadata construction phase, a training model is generated using public data or random noise under diverse label distributions, and its feature anchors are extracted. In the meta-model training phase, an encoder-decoder structure is introduced to jointly optimize prediction and reconstruction losses, thereby capturing the deep correspondence between features and labels. In the label inference phase, high-precision label recovery is achieved solely based on the uploaded feature extractor and proxy dataset, overcoming the limitation of existing methods that rely on classifiers.

[0083] 2. Experimental results on multiple image datasets, including CIFAR-10, CIFAR-100, and SVHN, demonstrate that the proposed FLR method achieves high-accuracy label distribution recovery even in scenarios where the classifier is not visible, significantly outperforming baseline methods such as random guessing. This method is not strictly dependent on auxiliary data, is applicable to various model structures and communication mechanisms, and possesses good versatility and transferability, providing an effective path for research at the feature space level.

[0084] Technical effects of the present invention:

[0085] Datasets: Experiments were conducted on three standard image classification datasets: CIFAR-10, CIFAR-100, and SVHN. Specifically, the CIFAR-10 dataset contains 50,000 training images and 10,000 test images from 10 categories, all of which are 32×32 color images; the CIFAR-100 dataset has the same image size and number of samples, but contains 100 more granular categories; the SVHN (Street View House Numbers) dataset also consists of color images, with 73,257 images in the training set and 26,032 images in the test set, divided into 10 number categories.

[0086] Experimental Setup: Four common model architectures were selected, including a 4-layer convolutional neural network (4-Layer CNN), LeNet-5, ResNet-20, and VGG-16, to simulate client training at different complexities. In the meta-model part, an autoencoder structure was formed using an encoder and decoder consisting of 3 fully connected layers. During the metadata construction phase, the server used auxiliary data generated by random noise and sampled diverse label distributions from the Dirichlet distribution to construct M = 10,000 pairs of feature anchors and label distributions as training samples. Each simulated training session used a uniform initial model and training strategy, with E = 5 training epochs, a batch size of 32, and an SGD optimizer with a learning rate of 0.01. In the label recovery phase, the recovery performance was evaluated under two communication mechanisms: single-epoch recovery for the FedPer federated learning algorithm and multi-epoch recovery for the FedBABU federated learning algorithm.

[0087] Evaluation metrics: To comprehensively evaluate the recovery performance of this invention, the following three metrics were used: (1) Instance-level Accuracy (iAcc), representing the percentage of successfully predicted labels; (2) Wasserstein distance (WD), measuring the overall deviation between the predicted distribution and the true label distribution; and (3) Hellinger distance (HD), measuring the similarity between the predicted label distribution and the true distribution. In all experiments, each set was run three times, and the average of all metrics was reported.

[0088] Results of Single-Round Recovery Experiments: First, the proposed FLR method was evaluated in a single-round recovery scenario. This experiment, based on the FedPer communication mechanism, was conducted on three datasets: CIFAR-10, CIFAR-100, and SVHN, and compared using four model architectures: 4-layer CNN, LeNet-5, ResNet-20, and VGG-16. Table 1 shows the performance of the FLR method on three metrics: instance-level accuracy (iAcc), Wasserstein distance (WD), and Hellinger distance (HD), while also providing random guessing as a baseline. The experimental results lead to the following conclusions: First, FLR significantly outperforms the random guessing baseline. Across all model architectures and dataset combinations, FLR achieves a significant improvement in the iAcc metric. For example, on CIFAR-10, the recovery accuracy using the VGG-16 model reaches 89.6%, a substantial improvement over random guessing (28.3%); on SVHN, the accuracy using the LeNet-5 model even reaches 92.6%. Secondly, the label distribution predicted by FLR is very close to the actual distribution in terms of WD and HD indicators, indicating that its prediction is not only highly accurate, but also has a good matching distribution structure. Figure 2 The results show the label distribution visualization on CIFAR-10 and CIFAR-100, further confirming FLR's ability to recover the proportion of the original training labels and reflect the true proportion relationship between the categories well.

[0089] Table 1. FLR recovery performance in single-round recovery scenarios.

[0090]

[0091] Results of Multi-Round Recovery Experiments: To verify the applicability of FLR in continuous communication scenarios, multi-round recovery experiments were further conducted within the FedBABU framework. CIFAR-10 and CIFAR-100 datasets were used with a 4-layer CNN model, and Dirichlet distribution parameters β = 0.05 and β = 0.1 were set respectively to simulate different levels of data heterogeneity. In this experiment, after each round of communication, the server performed label distribution inference on the feature extractor uploaded by the client and recorded the Hellinger distance between the predicted results and the true label distribution. Figure 3 As shown, FLR maintains a low Hellinger distance in the early rounds of communication, demonstrating good label recovery performance. However, with the increase in the number of communication rounds, multi-round aggregation leads to gradual alignment of the client's feature space, weakening the discriminative power of feature anchors on label distribution, thus causing a gradual decrease in recovery accuracy. Nevertheless, even after the 10th round, FLR still maintains performance superior to random guessing, indicating that this method has a certain degree of robustness in multi-round communication.

[0092] Impact Analysis of Metadata Quantity: To evaluate the impact of metadata quantity on the performance of the proposed FLR method, systematic experiments were conducted on the CIFAR-10 and CIFAR-100 datasets, with metadata sample numbers set to 1,000, 5,000, 10,000, and 50,000, respectively. The results are shown in Table 2, from which the following conclusions can be drawn:

[0093] (1) On CIFAR-10, the label recovery performance of FLR gradually improves with the increase of metadata. When the amount of metadata increases from 1,000 to 10,000, the instance-level accuracy increases from 84.7% to 87.0%, indicating that the model is more accurate in predicting label distribution. When the amount of metadata is further increased to 50,000, the accuracy reaches 87.6%, and although the performance improvement tends to plateau, it still shows some improvement. This indicates that with a small number of categories, a smaller amount of metadata is sufficient to cover the main label distribution structure, but more metadata can further improve prediction accuracy.

[0094] (2) On the CIFAR-100 dataset, the label distribution space is more complex due to the significantly increased number of classes (100), making it more sensitive to the amount of metadata. When only 1,000 metadata entries are used, the instance-level accuracy of FLR is only 50.5%, while when the metadata is increased to 50,000, the accuracy significantly improves to 70.3%. This shows that more simulated training pairs are needed in high-dimensional label distribution spaces to fully cover the simplex space and ensure that the meta-model can accurately learn the mapping relationship between feature anchors and label distribution.

[0095] Table 2. FLR recovery performance under different amounts of metadata

[0096]

[0097] Another embodiment of the present invention provides a feature-based label recovery system in federated learning without classifier transmission, comprising:

[0098] The metadata building module is used to construct multiple training subsets with different label distributions using public data or random noise samples, and to train them locally. It generates feature anchors corresponding to different label distributions. The feature anchors and their corresponding label distributions together form the meta-training dataset.

[0099] The meta-model training module is used to build meta-models, train meta-models based on meta-training datasets, and learn the bidirectional mapping relationship between feature anchors and label distributions.

[0100] The label inference module is used to receive the feature extractor uploaded by the target client, extract its feature anchor points on the proxy dataset, and use the trained meta-model to predict its label distribution, thereby restoring the label distribution of the target client.

[0101] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding process in the aforementioned method embodiments, and will not be repeated here.

[0102] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0103] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0104] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A feature-based label recovery method in federated learning without classifier transmission, characterized in that, Includes the following steps: Multiple training subsets with different label distributions are constructed using auxiliary datasets and trained locally to generate feature anchors corresponding to different label distributions. The feature anchors and their corresponding label distributions together form a meta-training dataset. Construct a meta-model, train the meta-model based on the meta-training dataset, and learn the bidirectional mapping relationship between feature anchors and label distribution; Receive the feature extractor uploaded by the target client, extract its feature anchor points on the proxy dataset, use the trained meta-model to predict its label distribution, and realize the recovery of the label distribution of the target client. The dataset refers to the image dataset. The meta-training dataset is constructed using the following steps: from 1-dimensional simplex space Medium-sampled label distribution vector The Dirichlet distribution is used as the sampling strategy. in accordance with Construct a training subset from the auxiliary dataset. And use a unified training configuration for local training to obtain the corresponding feature extractor parameters. ; Using a fixed proxy dataset Perform forward computation on the feature extractor to extract the corresponding feature anchors. A set of metadata pairs is obtained. ; By repeating the above process, a large-scale meta-training dataset containing multiple label distributions and their corresponding feature anchors is constructed.

2. The method according to claim 1, characterized in that, The auxiliary dataset is either publicly available data or random noise samples.

3. The method according to claim 1, characterized in that, The feature anchor points are extracted using the following steps: in, As a feature anchor point, For proxy datasets, For the input sample, For feature extractors, Indicates input sample After feature extractor Later mapped to Dimensional feature representation.

4. The method according to claim 1, characterized in that, The meta-model adopts an autoencoder structure, which includes an encoder and a decoder; the meta-model combines label distribution prediction loss and feature anchor reconstruction loss to capture the bidirectional mapping relationship between feature anchors and label distribution.

5. The method according to claim 4, characterized in that, The meta-model is trained using the following steps: encoder Input feature anchors Mapped to predicted label distribution ; decoder Received predicted label distribution Reconstruct the estimated values ​​of the feature anchor points ; The following joint loss function is used for optimization: in, To predict the loss for the label distribution, Herringer distance is used as the metric. i The distribution of predicted labels With the i The distribution of real labels Similarity between them; For feature anchor point reconstruction loss, cosine similarity is used to measure the loss of the first feature anchor point. i Original feature anchor points With the i Reconstructed feature anchor points The difference in the included angle between them; hyperparameters The weights of the two sub-loss terms are used to balance the trade-off between label prediction accuracy and feature representation quality. This indicates the number of label distributions or feature anchors.

6. The method according to claim 4, characterized in that, The process of receiving the feature extractor uploaded by the target client, extracting its feature anchors on the proxy dataset, and predicting its label distribution using the trained meta-model includes: Received feature extractor uploaded by the target client Then, use the proxy dataset. Extract the corresponding feature anchor points This feature anchor point reflects the overall feature response of the target client on a fixed data distribution; Will The encoder of the trained meta-model Obtain the predicted label distribution .

7. A feature-based label recovery system in federated learning without classifier transmission, characterized in that, The method of any one of claims 1 to 6 comprises: The metadata building module is used to construct multiple training subsets with different label distributions using public data or random noise samples, and to train them locally. It generates feature anchors corresponding to different label distributions. The feature anchors and their corresponding label distributions together form the meta-training dataset. The meta-model training module is used to build meta-models, train meta-models based on meta-training datasets, and learn the bidirectional mapping relationship between feature anchors and label distributions. The label inference module is used to receive the feature extractor uploaded by the target client, extract its feature anchor points on the proxy dataset, and use the trained meta-model to predict its label distribution, thereby restoring the label distribution of the target client.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.

Images