A discriminative gap backfilling based federated learning optimization method

Abstract

This invention provides a federated learning optimization method based on discriminative gap backfilling, comprising: Step 1, initialization of the discriminative gap backfilling personalized federated system; Step 2, synchronous broadcasting of client selection and shared information; Step 3, generation of basic discriminative output and calculation of discriminative gaps; Step 4, end-side gap backfilling training under dual-valve control of discriminative margin; Step 5, prediction-induced semantic compression of difficult categories and generation of end-side category summaries; Step 6, server aggregation and inertial fusion update of the category discriminative benchmark vector table; Step 7, repeating steps 2 to 6 until a preset number of training rounds is reached or the convergence condition is met. This invention uses an end-side discriminative gap backfiller to provide targeted compensation for the remaining error in basic discriminative classification, reducing the need for clients to repeatedly learn the complete classification task and improving the accuracy of personalized classification.

Description

Technical Field

[0001] This invention belongs to the fields of federated learning, edge intelligence and privacy-preserving machine learning technology, and specifically relates to a federated learning optimization method based on discriminative gap filling. Background Technology

[0002] In real-world environments where IoT, edge computing, and smart terminals are widely deployed, massive amounts of data are scattered across different terminal devices and edge nodes to support intelligent tasks such as image recognition, device status analysis, abnormal behavior detection, IoT security event classification, and user behavior modeling. Due to differences in deployment environments, data collection conditions, hardware performance, business scenarios, and user behavior habits among terminal devices, the data collected by different clients often varies in category distribution, sample size, feature morphology, and noise levels. Furthermore, this data typically contains user privacy information, device operation logs, business traffic, on-site images, or security-sensitive information. Due to privacy protection, industry compliance, and data security management requirements, it is difficult to centrally upload this data to a central server for unified training.

[0003] Federated learning, as a distributed collaborative modeling method, enables model training without requiring clients to leave their local machines for raw data. The server distributes global model parameters, clients train using their local data and upload updated models, and the server then aggregates these updates, achieving multi-client joint modeling. This approach reduces the privacy risks associated with centralized data collection and is suitable for multi-terminal collaboration, edge intelligence, and privacy-sensitive scenarios. However, in real-world applications, client data often exhibits non-independent and identically distributed characteristics, such as class skewness, missing classes, imbalanced sample sizes, and differences in feature distribution, making it difficult for a single global model to simultaneously adapt to the local data of all clients.

[0004] While existing personalized federated learning methods improve client adaptability through local classification heads, the division of shared and personalized layers, the fusion of global and local models, or local fine-tuning, they still have shortcomings. On one hand, some methods require the client to additionally learn a complete local classifier, which can easily overlap with the common discriminative abilities already learned by the global model, leading to overfitting on small samples or class-skewed clients. On the other hand, existing methods typically add global and local predictions directly or fuse them with fixed weights, lacking differentiation between the discriminative states of different samples, making it easy for personalized outputs to interfere with correctly classified high-confidence samples. Furthermore, accuracy drops in multi-class tasks often occur concentrated among a few easily confused classes, while existing methods based on class representations or prototype information often employ full-class alignment or uniform contrast constraints, making it difficult to provide targeted discriminative enhancement for the most misclassified and difficult classes.

[0005] Therefore, under heterogeneous data conditions across multiple clients, a training method more suitable for personalized federated learning is still needed. This method should prevent the client-side personalization module from simply repeating the entire classification task, but instead perform targeted backfilling around the discrimination gaps in the basic discrimination output that have not yet been correctly processed. Simultaneously, the activation intensity and amplitude range of the backfilling on the client side should be adaptively controlled based on the sample discrimination margin, and easily confused difficult categories should be identified in conjunction with the basic prediction results to enhance the discrimination margin between the true category and the difficult negative category. To this end, it is necessary to propose a federated learning optimization method based on discrimination gap backfilling to improve the classification accuracy of client-side personalization models in privacy-preserving distributed scenarios. Summary of the Invention

[0006] Purpose of the Invention: The purpose of this invention is to address the problems of existing personalized federated learning methods under heterogeneous multi-client data conditions, such as repetitive local classifier learning, lack of sample-level control in personalized correction, and insufficient ability to distinguish easily confused categories. This invention provides a federated learning optimization method based on discriminant gap backfilling. Without uploading the original client data, this invention constructs an end-side discriminant gap backfiller, a dual-valve backfill control mechanism for discriminant margin, and a prediction-induced semantic squeezing mechanism for difficult categories. This transforms the client-side personalization module from learning a complete local classification task to targeted backfilling of basic discriminant gaps, thereby improving the classification accuracy of the client-side personalized model under non-independent and identically distributed data conditions.

[0007] To address the aforementioned technical problems, this invention provides a federated learning optimization method based on discriminative gap backfilling. This method, by constructing a basic discriminative output, an edge-side discriminative gap backfiller, a dual-valve backfilling control mechanism for discriminative margin, and a prediction-induced semantic squeezing mechanism for difficult categories, achieves targeted adaptation and improved classification accuracy of heterogeneous data locally on the client side without uploading the client's original data. It is applicable to tasks such as image classification, state recognition, anomaly detection, and security event classification in IoT terminals, edge devices, intelligent sensing nodes, and privacy-sensitive distributed systems.

[0008] The method includes the following steps:

[0009] Step 1: Determine the initialization of the gap-filling personalized federated system;

[0010] Step 2: The client selects to broadcast synchronously with the shared information;

[0011] Step 3: Generate basic discrimination output and calculate discrimination gap;

[0012] Step 4, training on backfilling end-side gaps under dual-valve control of residual capacity;

[0013] The client predicts probabilities based on the baseline probability. Calculate the sample discriminant margin The sample discrimination margin The maximum class probability in the base prediction probability Probability of the second largest category This is used to measure the certainty of the base discrimination output for the current sample; the client, based on... Generate sample-level backfill activation intensity Preset discrimination margin threshold ,in This represents the discrimination margin threshold used to distinguish between samples with low discrimination margins and samples with high discrimination margins; when When the current sample is identified as a low discriminant margin sample, its basic discrimination result has a high possibility of class confusion. It increases accordingly; when When the current sample is identified as a sample with a high discrimination margin, its basic discrimination results are relatively reliable. It decreases accordingly;

[0014] End-side discrimination gap backfiller Based on sample characterization and basic discrimination output Generate end-side backfill vector Subsequently, the client uses the backfill amplitude limiting valve to... By applying budget constraints, a limited backfill vector is obtained. Finally, the client outputs the basic discrimination results. With the limited backfill vector controlled by dual valves The mixture is fused to obtain the final discriminant output. The client outputs the final judgment. Calculate the supervised learning loss and combine it with the gap filling loss for local training;

[0015] in, Indicates the client number. Indicates client Local input samples, Indicates client For the sample The obtained basic prediction probability vector, Indicates client For the sample The calculated sample discrimination margin, This represents the maximum class probability in the base predicted probabilities. This represents the probability of the second largest category after the category with the highest probability. This represents the sample-level backfill activation strength generated by client i for sample x; Indicates client The end-side detection gap backfill device is set up. Indicates client For the sample Extracted sample characterization, Indicates client For the sample The obtained basic discrimination output, Indicates end-side discrimination gap backfiller For the sample The generated end-side backfill vector;

[0016] Step 5: Prediction-induced difficult category semantic compression and end-side category summarization generation;

[0017] Step 6: Server aggregation and category discrimination benchmark vector table inertial fusion update;

[0018] After the client completes local training, it uploads the updated shared model parameters and the set of edge-side category semantic summary vectors to the server. No raw client data is uploaded; the client-side local path is customized. and end-side discrimination gap backfiller This client-side private customization module is retained locally and does not participate in server aggregation; the server aggregates the parameters related to the shared model uploaded by the client to obtain the parameters for the next round of shared feature extraction model. and shared discrimination header parameters Simultaneously, the server uses the client-side category semantic summary vector set. Category discrimination benchmark vector table Perform unit norm inertial fusion update to obtain the class discrimination benchmark vector table at the (t+1)th iteration. The updated shared model parameters and class discrimination benchmark vector table will continue to be sent to the client in the next round of training.

[0019] in, Indicates the current federal training round. Indicates client In the The set of edge-side category semantic summary vectors generated by the round. Indicates client Retain the local personalized discrimination path. The first one obtained by server aggregation Round-sharing feature extraction model parameters, The first one obtained by server aggregation Round-shared discrimination header parameters, Indicates the server is in the The classification benchmark vector table for wheel maintenance. Indicates the first The first round of client category semantic summary update results in the following: Wheel category discrimination benchmark vector table;

[0020] Step 7: Repeat steps 2 to 6 until the preset number of training rounds is reached or the convergence condition is met.

[0021] Step 1 includes: initializing the shared feature extraction model on the server side. Shared discrimination head and the category discrimination benchmark vector table maintained by the server in round t. Where t represents the federal training round, Used to store cross-client discriminative semantic benchmarks for various categories; client i initializes its local personalized discriminative path. and end-side discrimination gap backfiller Server configuration includes the number of federated training rounds T, the number of clients N, and the set of clients participating in training in round t. Local training rounds E, total number of categories K, and discrimination margin threshold. Backfill activation valve steepness coefficient Backfill amplitude budget limit Number of difficulty categories Category discrimination benchmark vector rolling update coefficients and the weight of the loss term , , ;

[0022] in, This represents the shared feature extraction model initialized on the server side. This indicates the shared discrimination header initialized on the server side. This indicates the number of difficult negative classes selected for each sample.

[0023] Step 2 includes: In the t-th round of federated training, the server selects the participating client set from the entire client set. and to participating clients Distribute shared feature extraction model parameters Shared discrimination header parameters and category discrimination benchmark vector table Category discrimination benchmark vector table The class discrimination reference vector of the k-th class is denoted as After receiving the information from the server, the client will share the feature extraction model. Shared discrimination head Local personalized discrimination path Together, they form the basic discriminant structure, used for subsequent generation of basic discriminant output; among them, Indicates loading the first The shared feature extraction model after round-sharing of shared feature extraction model parameters Indicates loading the first Shared discrimination header after round-shared discrimination header parameters.

[0024] Step 3 includes:

[0025] Client i extracts features from sample x to obtain sample representation. :

[0026] ,

[0027] in This represents the shared feature extraction model in round t.

[0028] The client uses a shared discrimination header. Local personalized discrimination path Generate basic discriminant output :

[0029] ,

[0030] according to Obtain the basic prediction probability :

[0031] ,

[0032] Among them, the normalized exponential function softmax is used to map the basic discrimination output to the predicted probability corresponding to each category;

[0033] The one-hot vector representation of the true label y of the sample is as follows Identify gaps Represented as:

[0034] ,

[0035] in, This indicates that the gradient propagation operation has been stopped; the discrimination gap... Indicates client For local samples The obtained discrimination gap vector is used to describe the remaining discrimination error between the base discrimination output and the true label that has not been correctly processed, and serves as an end-side discrimination gap filler. Targeted learning objectives; This represents the one-hot label vector corresponding to the true category y of the sample.

[0036] Step 4 includes:

[0037] End-side discrimination gap backfiller Based on sample characterization and basic discrimination output Generate end-side backfill vector :

[0038] ,

[0039] Construction of a method to determine backfill loss of gaps :

[0040] ,

[0041] in, This represents the scaling factor used to determine the backfill target of the end-side gap. Indicates client For the sample The obtained discriminant gap vector, This represents the squared norm of a vector.

[0042] Step 4 also includes: constructing a dual-valve backfill control mechanism for discriminant margin, including a discriminant margin activation valve and a backfill amplitude limiting valve;

[0043] The discriminant margin activation valve is used to generate sample-level backfill activation intensity based on the confidence state of the basic discriminant output. The client then uses the basic prediction probability. Take the highest class probability and the probability of the second largest category :

[0044] ,

[0045] ,

[0046] in, Indicates the candidate category number that participates in the probability comparison. Indicates client For the sample Predicted as the first The basic predicted probability of the class Indicates all candidate categories Perform a maximum value selection operation on the corresponding base prediction probability. The category number representing the category with the highest base prediction probability;

[0047] Sample discrimination margin Represented as:

[0048] ,

[0049] Sample-level backfill activation intensity Represented as:

[0050] ,

[0051] in, This represents the sigmoid function, a sigmoid logic function.

[0052] For end-side backfill vector Limiting backfill vector Represented as:

[0053] ,

[0054] in, The smallest constant is used to prevent division by zero; the hyperbolic tangent function tanh is used to smooth the magnitude of the end-fill vector.

[0055] Final judgment output Represented as:

[0056] .

[0057] Step 5 includes:

[0058] Server-side maintenance category discrimination benchmark vector table :

[0059] ,

[0060] in, Let represent the category discrimination benchmark vector of the k-th class in the t-th round;

[0061] The client predicts probabilities based on the baseline probability. Choose the most easily confused one from the non-real categories. Each of the following negative difficulty categories constitutes a set of difficulty categories. :

[0062] ,

[0063] in, This indicates that the true category of the sample is excluded. Then, according to the client For the sample Basic prediction probability Select from largest to smallest Filtering operations for each category of difficult categories; Indicates sample The corresponding set of difficulty categories; Indicates client For the sample Predicted as the first The basic predicted probability of the class;

[0064] Client-side computation of sample representation With the category discrimination benchmark vector similarity :

[0065] ,

[0066] in, Represents the normalized function of the identity norm. Represents cosine similarity or unit vector dot product; Indicates sample The representation and the first Similarity between class-discriminating benchmark vectors; for difficult class k, predict the induced dynamic class margin. Represented as:

[0067] ,

[0068] in, This represents the base category interval in the semantic compression of difficult categories. This represents the difficulty category enhancement coefficient, which adjusts the category interval based on the predicted probability of the difficulty category. Indicates the sample and difficulty categories The generated predicted induced category interval;

[0069] Prediction-induced difficulty category semantic squeeze loss Represented as:

[0070] ,

[0071] in, Indicates sample Representation and Real Category The similarity between the corresponding category benchmark vectors This represents the temperature adjustment coefficient in the semantic squeezing loss of the difficulty category.

[0072] Step 5 also includes:

[0073] During the t-th round of local training, client i maintains a cumulative vector sum and sample count for each class k. For category k, the end-side category semantic summary vector Represented as:

[0074] ,

[0075] Only when When the value is greater than 0, the client generates and uploads the edge category semantic summary vector of category k. The set of edge category semantic summary vectors uploaded by the client. Represented as:

[0076] ,

[0077] Used for updating the category discrimination benchmark vector table on the server side. , Indicates client In the Used to generate the first round of local training The number of samples for semantic summarization Indicates client In the Round targeting the first The generated end-side category semantic summary vector.

[0078] Step 6 includes:

[0079] After receiving the client-side category semantic summary vectors uploaded by the client, the server performs category-level aggregation for each category. For category k, the aggregated category semantic summary... Represented as:

[0080] ,

[0081] The server updates the class discrimination baseline vector using an inertial fusion method:

[0082] ,

[0083] in, Indicates the server is in the Round targeting the first The category semantic summary vector obtained by class aggregation Indicates the first The index of clients participating in the training round. Indicates client In the The round used to generate the first The number of samples for semantic summarization Indicates the first The first round of all participating clients The sum of the number of samples in each class Represents the rolling update coefficients of the category discrimination benchmark vector; the updated category discrimination benchmark vector table. In the next round, the shared model parameters will be broadcast to the client.

[0084] In steps 4 and 5, the joint optimization of each loss term specifically includes:

[0085] Client i jointly optimizes the supervised learning loss, the gap filling loss, the hard category semantic squeezing loss, and the backfilling budget constraint loss during local training, with the total loss function being... Represented as:

[0086] ,

[0087] in, Indicates client The total loss function used during local training. This represents the supervised cross-entropy loss calculated based on the final discriminant output. This indicates the determination of backfilling losses. This represents the prediction-induced difficulty category semantic squashing loss. This represents the backfill budget constraint loss used to limit the magnitude of the end-side backfill vector.

[0088] Among them, backfilling budget constraint loss Represented as:

[0089] ,

[0090] in, This means taking the larger value between 0 and the value within the parentheses. Represents the end-side backfill vector The 2-norm, This indicates the upper limit of the backfill amplitude budget; the client minimizes this value. Update local model parameters.

[0091] Beneficial Effects: This invention provides targeted compensation for residual errors in basic discrimination through an end-side discriminative gap backfiller, reducing the need for clients to repeatedly learn the complete classification task and improving personalized classification accuracy. By adaptively adjusting the personalized backfill intensity through a discriminative margin activation valve and a backfill amplitude limiting valve, samples with low discriminative margins receive more sufficient compensation, while reducing negative interference to high-confidence samples. Through a prediction-induced semantic squeezing mechanism for difficult categories, targeted discrimination enhancement is performed on easily confused categories, widening the discriminative difference between the true category and the difficult negative category. Under the condition that the original client data does not leave the local machine, multi-client collaborative training is achieved by sharing model parameters and category semantic summaries, balancing data privacy protection and personalized classification performance. Attached Figure Description

[0092] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0093] Figure 1 This is a schematic diagram of the method framework of the present invention.

[0094] Figure 2 This is an experimental comparison chart of the method of the present invention (on the CIFAR-10 dataset, under actual settings, the Dirichlet distribution is used to simulate the actual heterogeneous data distribution, and the number of clients is 20). Detailed Implementation

[0095] like Figure 1 As shown, this embodiment of the invention provides a federated learning optimization method based on discriminative gap filling, comprising the following steps:

[0096] Step 1: Determine the initialization of the gap-filling personalized federated system;

[0097] Server-side initialization of shared feature extraction model Shared discrimination head and the category discrimination benchmark vector table maintained by the server in round t. Where t represents the federal training round, Used to store cross-client discriminative semantic benchmarks for various categories. Client i initializes its local personalized discriminative path. and end-side discrimination gap backfiller The server is configured with the number of federated training rounds T, the number of clients N, and the set of clients participating in training in round t. Local training rounds E, total number of categories K, and discrimination margin threshold. Backfill activation valve steepness coefficient Backfill amplitude budget Number of difficulty categories Category discrimination benchmark vector rolling update coefficients and the weights of each loss term , , .

[0098] Step 2: The client selects to broadcast synchronously with the shared information;

[0099] In the t-th round of federated training, the server selects the participating client set from the entire client set. and to participating clients Distribute shared feature extraction model parameters Shared discrimination header parameters and category discrimination benchmark vector table Category discrimination benchmark vector table The class discrimination reference vector of the k-th class is denoted as After receiving the information from the server, the client will share the feature extraction model. Shared discrimination head Local personalized discrimination path Together they form the basic discriminant structure, which is used to generate the basic discriminant output in the subsequent process.

[0100] Step 3: Generate basic discrimination output and calculate discrimination gap;

[0101] Client i performs feature extraction on local sample x to obtain sample representation. Subsequently, the client uses a shared discrimination header. Local personalized discrimination path Generate basic discriminant output ; Indicates end-side discrimination gap backfiller The client then uses the baseline prediction results before applying the algorithm; the client further bases its decisions on these results. Obtain the basic prediction probability And combined with the real labels of the samples One-hot vector Calculate the discriminant gap The discriminative gap Used to represent the remaining discrimination error between the basic discrimination output and the true label that has not been correctly processed, and serves as an end-side discrimination gap filler. Targeted learning objectives.

[0102] Step 4, training on backfilling end-side gaps under dual-valve control of residual capacity;

[0103] The client predicts probabilities based on the baseline probability. Calculate the sample discriminant margin The sample discrimination margin The maximum class probability in the base prediction probability Probability of the second largest category This is used to measure the certainty of the base discrimination output for the current sample. The client, based on... Generate sample-level backfill activation intensity Preset discrimination margin threshold ,in This represents the discrimination margin threshold used to distinguish between samples with low discrimination margins and samples with high discrimination margins; when When the current sample is identified as a low discriminant margin sample, its basic discrimination result has a high possibility of class confusion. It increases accordingly; when When the current sample is identified as a sample with a high discrimination margin, its basic discrimination results are relatively reliable. It decreases accordingly.

[0104] End-side discrimination gap backfiller Based on sample characterization and basic discrimination output Generate end-side backfill vector Subsequently, the client uses the backfill amplitude limiting valve to... By applying budget constraints, a limited backfill vector is obtained. Finally, the client outputs the basic discrimination results. With the limited backfill vector controlled by dual valves The mixture is fused to obtain the final discriminant output. The client outputs the final judgment. Calculate the supervised learning loss and combine it with the gap filling loss for local training.

[0105] Step 5: Prediction-induced difficult category semantic compression and end-side category summarization generation;

[0106] The client predicts probabilities based on the baseline probability. Choosing the current sample from a non-true category is most likely to result in a misclassification. Each of the following negative difficulty categories constitutes a set of difficulty categories. The client uses the category discrimination benchmark vector table issued by the server. Calculate sample representation Compared with the true category baseline vector Difficult Negative Class Benchmark Vector The similarity between them is used to construct a prediction-induced difficult category semantic squeeze loss. This loss enables the sample representations to maintain higher similarity to the true class benchmark vectors, while widening the discriminative margin from the current most easily confused difficult negative class benchmark vectors.

[0107] Meanwhile, during local training, client i accumulates sample representations according to category, and generates a set of edge-side category semantic summary vectors in round t. , Used for updating the category discrimination benchmark vector table on the server side. .

[0108] Step 6: Server aggregation and category discrimination benchmark vector table inertial fusion update;

[0109] After the client completes local training, it uploads the updated shared model parameters and the set of edge-side category semantic summary vectors to the server. No raw client data is uploaded. Client-side local personalized decision path. and end-side discrimination gap backfiller This client-side private customization module is retained locally and does not participate in server aggregation; the server aggregates the parameters related to the shared model uploaded by the client to obtain the parameters for the next round of shared feature extraction model. and shared discrimination header parameters Simultaneously, the server uses the client-side category semantic summary vector set. Category discrimination benchmark vector table Perform unit norm inertial fusion update to obtain The updated shared model parameters and class discrimination benchmark vector table will continue to be sent to the client in the next round of training. Repeat steps 2 to 6 until the preset number of training rounds is reached or the convergence condition is met.

[0110] Furthermore, the meanings of each letter symbol in step 1 are as follows:

[0111] Indicates the client number; Indicates the category number; This indicates a local sample on the client side; Indicates the true label of the sample;

[0112] Step 3 includes:

[0113] Client i extracts features from sample x to obtain sample representation. :

[0114] ,

[0115] The client uses a shared discrimination header. Local personalized discrimination path Generate basic discriminant output :

[0116] ,

[0117] Basic prediction probability Represented as:

[0118] ,

[0119] For the true label y of a sample, its one-hot vector is represented as: Identify gaps Represented as:

[0120] ,

[0121] in, This indicates that the gradient propagation operation is stopped to avoid the back-perturbation of the basic discrimination path by the discrimination gap supervision target. This discrimination gap... This represents the residual discrimination error between the basic discrimination output and the true label.

[0122] Step 4 includes:

[0123] End-side discrimination gap backfiller Based on sample characterization and basic discrimination output Generate end-side backfill vector :

[0124] ,

[0125] To enable the end-side discrimination gap backfiller to focus on learning the residual error of the basic discrimination output, this method constructs a discrimination gap backfill loss. :

[0126] ,

[0127] in, Indicates the scaling factor for the backfill target. This represents the sample-level backfill activation strength. This loss causes the end-side discriminative gap filler to preferentially learn the residual error direction on samples with low discriminative margins and reduces its ineffective corrections to high-confidence samples. Unlike directly training a complete local classifier, this method limits client-side personalized learning to directional backfilling of basic discriminative gaps, thereby reducing the risk of local overfitting.

[0128] Step 4 also includes: constructing a dual-valve backfill control mechanism for discriminant margin, including a discriminant margin activation valve and a backfill amplitude limiting valve;

[0129] The discriminant margin activation valve is used to generate sample-level backfill activation intensity based on the confidence state output by the basic discriminant. The client then uses the basic predicted probability... Take the highest class probability and the probability of the second largest category :

[0130] ,

[0131] ,

[0132] in, This represents the category with the highest base prediction probability;

[0133] Sample discrimination margin Represented as:

[0134] ,

[0135] Sample-level backfill activation intensity Represented as:

[0136] ,

[0137] in, This represents the Sigmoid function. Indicates the valve steepness coefficient. This indicates the threshold for determining the margin. When... When smaller, Increase; when When it is large, Decrease.

[0138] The backfill amplitude limiting valve is used to limit the amplitude of the end-side backfill vector. For the end-side backfill vector... Limiting backfill vector Represented as:

[0139] ,

[0140] in, This indicates the upper limit of the backfill amplitude budget. This represents a very small constant to prevent division by zero. The final output is the decision. Represented as:

[0141] ,

[0142] The key to this mechanism is that the end-side personalized correction is controlled by both the sample discrimination margin and the backfill amplitude budget, which enables low margin samples to obtain stronger local compensation while reducing negative interference to high confidence samples.

[0143] Step 5 includes:

[0144] Server-side maintenance category discrimination benchmark vector table :

[0145] ,

[0146] in, Indicates the first Round The class classification baseline vector. The client predicts probabilities based on this baseline. Choose the most easily confused one from the non-real categories. Each of the following negative difficulty categories constitutes a set of difficulty categories. :

[0147] ,

[0148] Client-side computation of sample representation With the category discrimination benchmark vector similarity :

[0149] ,

[0150] in, Represents the normalized function of the identity norm. This represents cosine similarity or the dot product of unit vectors. For difficult categories... Predict the induced dynamic class interval Represented as:

[0151] ,

[0152] in, Indicates the basic category interval, This represents the difficulty category enhancement coefficient. If... The larger the value, the more likely the basic discrimination result is to misclassify the sample as a category. Then a larger discrimination interval requirement is imposed on the difficulty category.

[0153] Prediction-induced difficulty category semantic squeeze loss Represented as:

[0154] ,

[0155] in, This represents the similarity between the sample representation and the true class discrimination benchmark vector. This represents the temperature coefficient. This loss enables the sample representation to maintain a higher similarity to the true class discrimination benchmark vector, while dynamically separating it from the current most easily confused difficult negative class discrimination benchmark vector.

[0156] Step 5 also includes:

[0157] During the t-th round of local training, client i maintains a cumulative vector sum and sample count for each class k. For category k, the end-side category semantic summary vector Represented as:

[0158] ,

[0159] in, This represents the number of samples of category k for client i in round t. Only when... When the value is greater than 0, the client generates and uploads the edge category semantic summary vector for that category to reduce invalid transmissions and noisy updates. The set of edge category semantic summary vectors uploaded by the client. Represented as:

[0160] ,

[0161] Step 6 includes:

[0162] After receiving the client-side category semantic summary vectors uploaded by the client, the server performs category-level aggregation for each category. For category k, the aggregated category semantic summary... Represented as:

[0163] ,

[0164] The server updates the class discrimination baseline vector using an inertial fusion method:

[0165] ,

[0166] in, This represents the rolling update coefficients of the category discrimination benchmark vector table. When the benchmark vector for a certain category has not yet been initialized, the category semantic summary obtained in this round of aggregation can be used directly. Initialize the class discrimination baseline vector table. In the next round, the shared model parameters will be broadcast to the client.

[0167] In steps 4 and 5, the joint optimization of each loss term is as follows:

[0168] Client i jointly optimizes the supervised learning loss, the gap filling loss, the hard category semantic squeezing loss, and the backfilling budget constraint loss during local training, with the total loss function being... Represented as:

[0169] ,

[0170] in, This represents the supervised cross-entropy loss, used to ensure the basic classification ability of the final discriminant output; This indicates the backfill loss of the discrimination gap, used to enable the end-side discrimination gap backfiller to learn the remaining error direction of the basic discrimination output; This represents the prediction-induced hard category semantic squeeze loss, used to enhance the discriminative distance between the true category and the hard negative category; This indicates the backfill budget constraint loss, used to suppress excessive end-side backfilling. , , This indicates the weight of the loss term.

[0171] Among them, backfilling budget constraint loss Represented as:

[0172] ,

[0173] The client minimizes Update the local model parameters. The key is: the activation strength of the backfill at the same sample level. Simultaneously, it acts on the discrimination gap backfilling learning, final backfilling fusion, and difficult category semantic squeezing process, so that the edge-side personalized learning focuses on samples with uncertain basic discrimination, easy confusion, and easy misjudgment, thereby improving the accuracy of personalized classification under heterogeneous data conditions.

[0174] To comprehensively evaluate the superior performance of the proposed method in practical applications, the CIFAR-10 experimental dataset was selected. The accuracy of the proposed method was compared with two classic federated learning algorithms, with 175 training epochs. The experimental configuration consisted of an Intel Core i5-14600KF (14 cores), 32GB DDR5 6000MHz memory, and an NVIDIA GeForce RTX 5060 Ti 16GB GPU. In the specific implementation, the CIFAR-10 dataset was divided into training and testing sets, with 80% allocated for training and 20% for testing.

[0175] In a specific implementation, the method of this invention is experimentally compared with classic federated learning methods FedDyn, FedROD, and FedProto. Regarding the experimental setup, the CIFAR-10 dataset is used, employing a method with parameters... Dir distribution ( Generate client data partitions to simulate the actual heterogeneous data distribution of multiple IoT terminals. Setting it to 0.1 indicates a strong heterogeneity condition. For example... Figure 2 As shown, under the CIFAR-10 dataset conditions, the method of this invention exhibits faster convergence speed and higher final accuracy during training, with a maximum accuracy of 91.22%, representing a 2.5% improvement over the best results in FedROD, a 4.03% improvement over the best results in FedProto, and a 15.98% improvement over the best results in FedDyn, with significantly less oscillation than the comparison algorithms. These experimental results demonstrate that the method of this invention can improve the average classification accuracy of the client-side personalized model under conditions of strong data heterogeneity by implementing end-side targeted backfilling for the residual errors in the basic discrimination results, controlling the backfilling intensity using the discrimination margin, and simultaneously implementing targeted discrimination enhancement for the currently easily confused categories. In heterogeneous IoT data scenarios, this method can effectively alleviate client drift and training oscillations, and improve the generalization performance and convergence stability of the global model.

[0176] In summary, this invention, without requiring the original client data to leave the local machine, can perform personalized backfilling of residual errors in the basic discrimination results of heterogeneous clients, and enhance the discrimination ability between the true category and easily confused categories, thereby improving the classification accuracy of the personalized client model under non-independent and identically distributed data conditions.

[0177] This invention provides a federated learning optimization method based on discriminative gap backfilling. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A federated learning optimization method based on discriminative gap filling, characterized in that, Includes the following steps: Step 1: Determine the initialization of the gap-filling personalized federated system; Step 2: The client selects to broadcast synchronously with the shared information; Step 3: Generate basic discrimination output and calculate discrimination gap; Step 4, training on backfilling end-side gaps under dual-valve control of residual capacity; The client predicts probabilities based on the baseline probability. Calculate the sample discriminant margin The sample discrimination margin The maximum class probability in the base prediction probability Probability of the second largest category This is used to measure the certainty of the base discrimination output for the current sample; the client, based on... Generate sample-level backfill activation intensity Preset discrimination margin threshold ,in This represents the discrimination margin threshold used to distinguish between samples with low discrimination margins and samples with high discrimination margins; when When, the current sample is identified as a low discrimination margin sample; when When this happens, the current sample is identified as a sample with a higher discrimination margin; End-side discrimination gap backfiller Based on sample characterization and basic discrimination output Generate end-side backfill vector Subsequently, the client uses the backfill amplitude limiting valve to... By applying budget constraints, a limited backfill vector is obtained. ; Finally, the client outputs the basic discrimination results. With the limited backfill vector controlled by dual valves The mixture is fused to obtain the final discriminant output. ; The client outputs the final judgment. Calculate the supervised learning loss and combine it with the gap filling loss for local training; in, Indicates the client number. Indicates client Local input samples, Indicates client For the sample The obtained basic prediction probability vector, Indicates client For the sample The calculated sample discrimination margin, This represents the maximum class probability in the base predicted probabilities. This represents the probability of the second largest category after the category with the highest probability. This represents the sample-level backfill activation strength generated by client i for sample x; Indicates client The end-side detection gap backfill device is set up. Indicates client For the sample Extracted sample characterization, Indicates client For the sample The obtained basic discrimination output, Indicates end-side discrimination gap backfiller For the sample The generated end-side backfill vector; Step 5: Prediction-induced difficult category semantic compression and end-side category summarization generation; Step 6: Server aggregation and category discrimination benchmark vector table inertial fusion update; After the client completes local training, it uploads the updated shared model parameters and the set of edge-side category semantic summary vectors to the server. No raw client data is uploaded; the client-side local path is customized. and end-side discrimination gap backfiller This client-side private customization module is retained locally and does not participate in server aggregation; the server aggregates the parameters related to the shared model uploaded by the client to obtain the parameters for the next round of shared feature extraction model. and shared discrimination header parameters Simultaneously, the server uses the client-side category semantic summary vector set. Category discrimination benchmark vector table Perform unit norm inertial fusion update to obtain the class discrimination benchmark vector table at the (t+1)th iteration. The updated shared model parameters and class discrimination benchmark vector table will continue to be sent to the client in the next round of training. in, Indicates the current federal training round. Indicates client In the The set of edge-side category semantic summary vectors generated by the round. Indicates client Retain the local personalized discrimination path. The first one obtained by server aggregation Round-sharing feature extraction model parameters, The first one obtained by server aggregation Round-shared discrimination header parameters, Indicates the server is in the The classification benchmark vector table for wheel maintenance. Indicates the first The first round of client category semantic summary update results in the following: Wheel category discrimination benchmark vector table; Step 7: Repeat steps 2 to 6 until the preset number of training rounds is reached or the convergence condition is met.

2. The method according to claim 1, characterized in that, Step 1 includes: initializing the shared feature extraction model on the server side. Shared discrimination head and the category discrimination benchmark vector table maintained by the server in round t. Where t represents the federal training round, Used to store cross-client discriminative semantic benchmarks for various categories; client i initializes its local personalized discriminative path. and end-side discrimination gap backfiller Server configuration includes the number of federated training rounds T, the number of clients N, and the set of clients participating in training in round t. Local training rounds E, total number of categories K, and discrimination margin threshold. Backfill activation valve steepness coefficient Backfill amplitude budget limit Number of difficulty categories Category discrimination benchmark vector rolling update coefficients and the weight of the loss term , , ; in, This represents the shared feature extraction model initialized on the server side. This indicates the shared discrimination header initialized on the server side. This indicates the number of difficult negative classes selected for each sample.

3. The method according to claim 2, characterized in that, Step 2 includes: In the t-th round of federated training, the server selects the participating client set from the entire client set. and to participating clients Distribute shared feature extraction model parameters Shared discrimination header parameters and category discrimination benchmark vector table Category discrimination benchmark vector table The class discrimination reference vector of the k-th class is denoted as After receiving the information from the server, the client will share the feature extraction model. Shared discrimination head Local personalized discrimination path Together, they form the basic discriminant structure, used for subsequent generation of basic discriminant output; among them, Indicates loading the first The shared feature extraction model after round-sharing of shared feature extraction model parameters Indicates loading the first Shared discrimination header after round-shared discrimination header parameters.

4. The method according to claim 3, characterized in that, Step 3 includes: Client i extracts features from sample x to obtain sample representation. : ， in This represents the shared feature extraction model in round t. The client uses a shared discrimination header. Local personalized discrimination path Generate basic discriminant output : ， according to Obtain the basic prediction probability : ， Among them, the normalized exponential function softmax is used to map the basic discrimination output to the predicted probability corresponding to each category; The one-hot vector representation of the true label y of the sample is as follows Identify gaps Represented as: ， in, This indicates that the gradient propagation operation has been stopped; the discrimination gap... Indicates client For local samples The obtained discrimination gap vector is used to describe the remaining discrimination error between the base discrimination output and the true label that has not been correctly processed, and serves as an end-side discrimination gap filler. Targeted learning objectives; This represents the one-hot label vector corresponding to the true category y of the sample.

5. The method according to claim 4, characterized in that, Step 4 includes: End-side discrimination gap backfiller Based on sample characterization and basic discrimination output Generate end-side backfill vector : ， Construction of a method to determine backfill loss of gaps : ， in, This represents the scaling factor used to determine the backfill target of the end-side gap. Indicates client For the sample The obtained discriminant gap vector, This represents the squared norm of a vector.

6. The method according to claim 5, characterized in that, Step 4 also includes: constructing a dual-valve backfill control mechanism for discriminant margin, including a discriminant margin activation valve and a backfill amplitude limiting valve; The discriminant margin activation valve is used to generate sample-level backfill activation intensity based on the confidence state of the basic discriminant output. The client then uses the basic predicted probability. Take the highest class probability and the probability of the second largest category : ， ， Where c represents the candidate category number participating in the probability comparison. Indicates client For the sample Predicted as the first The basic predicted probability of the class Indicates all candidate categories Perform a maximum value selection operation on the corresponding basic prediction probability. The category number representing the category with the highest base prediction probability; Sample discrimination margin Represented as: ， Sample-level backfill activation intensity Represented as: ， in, This represents the sigmoid function, a sigmoid logic function. For end-side backfill vector Limiting backfill vector Represented as: ， in, The constant that prevents division by zero is used; the hyperbolic tangent function tanh is used to smooth the magnitude of the end-fill vector. Final judgment output Represented as: 。 7. The method according to claim 6, characterized in that, Step 5 includes: Server-side maintenance category discrimination benchmark vector table : ， in, Let represent the category discrimination benchmark vector of the k-th class in the t-th round; The client predicts probabilities based on the baseline probability. Choose the most easily confused one from the non-real categories. Each of the following negative difficulty categories constitutes a set of difficulty categories. : ， in, This indicates that the true category of the sample is excluded. Then, according to the client For the sample Basic prediction probability Select from largest to smallest Filtering operations for each category of difficult categories; Indicates sample The corresponding set of difficulty categories; Indicates client For the sample Predicted as the first The basic predicted probability of the class; Client-side computation of sample representation With the category discrimination benchmark vector similarity : ， in, Represents the normalized function of the identity norm. Represents cosine similarity or unit vector dot product; Indicates sample The representation and the first Similarity between class-discriminating benchmark vectors; for difficult class k, predict the induced dynamic class margin. Represented as: ， in, This represents the base category interval in the semantic compression of difficult categories. This represents the difficulty category enhancement coefficient, which adjusts the category interval based on the predicted probability of the difficulty category. Indicates the sample and difficulty categories The generated predicted induced category interval; Prediction-induced difficulty category semantic squeeze loss Represented as: ， in, Indicates sample Representation and Real Category The similarity between the corresponding category benchmark vectors This represents the temperature adjustment coefficient in the semantic squeezing loss of the difficulty category.

8. The method according to claim 7, characterized in that, Step 5 also includes: During the t-th round of local training, client i maintains a cumulative vector sum and sample count for each class k. For category k, the end-side category semantic summary vector Represented as: ， Only when When the value is greater than 0, the client generates and uploads the edge category semantic summary vector of category k. The set of edge category semantic summary vectors uploaded by the client. Represented as: ， Used for updating the category discrimination benchmark vector table on the server side. , Indicates client In the The number of samples used to generate the semantic summary of the k-th class during the round of local training. Indicates client In the Round targeting the first The generated end-side category semantic summary vector.

9. The method according to claim 8, characterized in that, Step 6 includes: After receiving the client-side category semantic summary vectors uploaded by the client, the server performs category-level aggregation for each category. For category k, the aggregated category semantic summary... Represented as: ， The server updates the class discrimination baseline vector using an inertial fusion method: ， in, Indicates the server is in the Round targeting the first The category semantic summary vector obtained by class aggregation Indicates the first The index of clients participating in the training round. Indicates client In the The round used to generate the first The number of samples for semantic summarization Indicates the first The first round of all participating clients The sum of the number of samples in each class Represents the rolling update coefficients of the category discrimination benchmark vector; the updated category discrimination benchmark vector table. In the next round, the shared model parameters will be broadcast to the client.

10. The method according to claim 9, characterized in that, In steps 4 and 5, the joint optimization of each loss term specifically includes: Client i jointly optimizes the supervised learning loss, the gap filling loss, the hard category semantic squeezing loss, and the backfilling budget constraint loss during local training, with the total loss function being... Represented as: ， in, Indicates client The total loss function used during local training. This represents the supervised cross-entropy loss calculated based on the final discriminant output. This indicates the determination of backfilling losses. This represents the prediction-induced difficulty category semantic squashing loss. This represents the backfill budget constraint loss used to limit the magnitude of the end-side backfill vector. Among them, backfilling budget constraint loss Represented as: ， in, This means taking the larger value between 0 and the value within the parentheses. Represents the end-side backfill vector The 2-norm, This indicates the upper limit of the backfill amplitude budget; the client minimizes this value. Update local model parameters.

Images