Classification method for distributed non-shared unbalanced data

Abstract

The invention relates to the technical field of data classification, in particular to a distributed unshared unbalanced data-oriented classification method, which comprises the following steps of: firstly, learning a converged basic global model based on a federal learning framework; then, an elastic weight solidification method is utilized to enable each local model to further learn local data distribution under the condition that important information in a previous round of global model is not forgotten so as to solve the problems of extreme class imbalance and general class imbalance data on most clients; the problem that an existing distributed unshared data classification method is basically proposed under a federated learning framework, but model disastrous forgetting is caused due to the fact that local data set end class imbalance exists under the framework and class imbalance data generally exists on most clients is not considered is solved. And a global classification model with strong comprehensive performance is inconvenient to cooperatively train by using the data.

Description

Technical Field

[0001] The present invention relates to the technical field of data classification, and in particular to a classification method for distributed unshared class imbalanced data. Background Art

[0002] At present, most of the methods to solve the problem of class imbalanced data classification are based on centralized machine learning technology. However, due to policy restrictions such as data privacy protection, data in some professional fields are usually stored in a distributed form on various local servers instead of in centralized cloud servers, and cannot be shared between different local servers. In this case, it is challenging to centralize all data in one place to collaboratively train a global classification model. Therefore, most researchers choose federated learning as a solution. Federated learning is a distributed machine learning technology that allows multiple clients (such as terminal devices or edge servers) to collaboratively train a global model under the orchestration of a central cloud server without sharing data. In federated learning, the clients first perform local training calculations using their own local data, and then the central cloud server updates the global model parameters through the local model parameters uploaded by the clients. Therefore, federated learning can mitigate many systemic privacy risks and costs associated with traditional centralized machine learning, such as the risk of transmitting sensitive data over insecure networks or data leakage due to centralized storage methods.

[0003] Federated learning still faces the challenge of class imbalance. Existing centralized methods may not be directly applicable to the class imbalance problem in federated learning, because the data in federated learning is distributed on different edge servers and cannot be directly accessed. Moreover, some classes may not even exist on some edge servers. Therefore, these traditional methods cannot guarantee the training of high-quality global models based on these distributed unshared class imbalanced data in federated learning. To solve this problem, the existing methods for solving the classification problem of class imbalanced data in federated learning can be summarized into two main directions: the first direction focuses on improving local updates by introducing novel local loss functions to alleviate the negative impact of class imbalance. These methods adopt the idea of ​​cost-sensitive learning to balance the influence of majority and minority classes on classification decisions on each local client, thereby improving the classification performance of local models in imbalanced scenarios and enhancing global models. However, these algorithms cannot effectively deal with the challenge of extreme class imbalance, that is, there are no samples of some classes in the local dataset; the second direction focuses on selecting local clients for aggregation to reduce the influence of biased local models on the global model. They select clients with balanced data to protect the global model from the class imbalance problem, and use selection strategies such as reinforcement learning and multi-armed bandit to select clients with high classification performance. However, these algorithms only assume that a small number of clients have class imbalanced data, which makes them unsuitable for solving the challenge of class imbalanced data that is prevalent on most clients.

[0004] In summary, existing distributed non-shared data classification methods are basically proposed under the federated learning framework, but they do not consider the problem of catastrophic forgetting of the model caused by the extreme class imbalance of the local data set under this framework and the widespread class imbalance data on most clients, which makes it inconvenient to use these data to collaboratively train a global classification model with strong comprehensive performance. Summary of the invention

[0005] The purpose of the present invention is to provide a classification method for distributed non-shared class-imbalanced data, aiming to solve the technical problem that the existing distributed non-shared data classification methods are basically proposed under the federated learning framework, but do not consider the catastrophic forgetting of the model caused by the extreme class imbalance of the local data set existing in this framework and the widespread class imbalance data on most clients, which makes it inconvenient to use these data to collaboratively train a global classification model with strong comprehensive performance.

[0006] To achieve the above object, the present invention adopts a classification method for distributed unshared class imbalanced data, which includes the following stages:

[0007] Phase 1: Learning a converged basic global model based on the federated learning framework;

[0008] Phase 2: Based on the first phase, the Elastic Weight Consolidation (EWC) method is used to continuously perform federated learning so that each local model can further learn the local data distribution without forgetting the important information in the previous round of global model.

[0009] Repeat the second stage until the global classification model converges again. The cloud server distributes the latest global parameters to the edge servers to update each local model to complete the model training. After the collaborative training of unbalanced data with unshared classes is realized, the classification for distributed unbalanced data with unshared classes can be realized.

[0010] Among them, federated learning can create a global classification model based on deep neural networks by aggregating the parameters of multiple local models trained on different edge servers;

[0011] Each edge server stores its raw data locally, and the local model is trained based on this data;

[0012] Defining feature space and label space Where C is the total number of class labels;

[0013] The dataset collected from the edge server e is represented as in It is a dataset The number of data samples in

[0014] Among them, the training process of federated learning includes three steps: local update, aggregation and distribution.

[0015] Among them, for the local update in the training process of federated learning: each deep neural network-based classification model focuses on minimizing the local loss function f e (θ e ) to improve the classification performance, the optimization objective is defined as

[0016]

[0017] where θ e are the parameters of the local model; l(·) is a suitable loss function; ξ e From the local dataset The data set is uniformly sampled from ; in each local update phase, the objective function is optimized by the stochastic gradient descent method (SGD), that is,

[0018]

[0019] where η t ∈(0,1) is the learning rate; Represents vector The first derivative of .

[0020] Among them, for the aggregation in the training process of federated learning: federated learning only aggregates the parameters of the distributed local model to update the global model; the parameters θ of the global model g It can be obtained by the following equation:

[0021]

[0022] where w e is the weight coefficient that measures the importance of the model on the edge server e. In traditional federated learning, w e It is related to the size of the data on each edge server, for example: Among them, for the distribution during the training process of federated learning: the latest parameters of the global model will be distributed to the local edge servers to update the local models; at this time, all local models have the same parameters;

[0023] After multiple iterations of the three steps of local update, aggregation, and distribution, a global model with good classification performance for all classes is learned; therefore, its goal is to minimize the global objective function f g (·):

[0024]

[0025] The specific method of using the elastic weight solidification method to enable each local model to further learn the local data distribution without forgetting the important information in the previous round of global model is as follows:

[0026] When all local models have good classification performance, For each local model on the Fisher information matrix I e ; During the global aggregation process, in addition to local parameters The local Fisher information matrix is ​​also uploaded to generate the global Fisher information matrix, namely:

[0027]

[0028] At the same time, keep Parameters and as a copy on each edge server; in a later local update step t(MN g ≤t0≤t≤t0+M), use the minimum h e Instead of using edge servers Minimize the local loss function f e (·), that is, using SGD to minimize the following target h e :

[0029]

[0030] The present invention discloses a classification method for distributed unshared class-imbalanced data, comprising the following stages: a first stage: learning a converged basic global model based on a federated learning framework; a second stage: based on the first stage, continuously performing federated learning using an elastic weight solidification method so that each local model can further learn local data distribution without forgetting important information in the previous round of global models; repeating the second stage until the global classification model converges again, the cloud server distributes the latest global parameters to the edge server to update each local model to complete the model training, and the classification of distributed unshared class-imbalanced data can be achieved after the collaborative training of unshared class-imbalanced data is achieved. In this way, the existing distributed unshared data classification methods are basically proposed under the federated learning framework, but the problem of extreme class imbalance of local data sets existing in this framework and the widespread existence of class imbalanced data on most clients is not considered, resulting in the technical problem that it is inconvenient to use these data to collaboratively train a global classification model with strong comprehensive performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a schematic diagram of the cloud-edge collaborative computing network architecture of the present invention.

[0033] Figure 2 This is an example diagram of the drift of FedAvg under the class imbalance problem of the present invention.

[0034] Figure 3 Schematic diagram of using the EWC of the present invention for federated learning of class-imbalanced data.

[0035] Figure 4 It is a schematic diagram of the multi-stage federated learning framework based on continuous learning of the present invention.

[0036] Figure 5 It is a schematic diagram of the convergence of the six federated learning algorithms of the present invention on the CIFAR-10 dataset.

[0037] Figure 6 Schematic diagram of data distribution of the CIFAR-10 dataset in 10 edge servers of the present invention. DETAILED DESCRIPTION

[0038] Embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0039] See also Figure 1 to Figure 6 ,in Figure 1 is a schematic diagram of the cloud-edge collaborative computing network architecture of the present invention, Figure 2 This is an example diagram of the drift of FedAvg under the class imbalance problem of the present invention. Figure 3 This is a schematic diagram of the EWC of the present invention used for class imbalanced data federated learning. Figure 4 is a schematic diagram of a multi-stage federated learning framework based on continuous learning of the present invention, Figure 5 is a schematic diagram of the convergence of the six federated learning algorithms of the present invention on the CIFAR-10 dataset. Figure 6 It is a schematic diagram of data distribution of the CIFAR-10 dataset of the present invention in 10 edge servers.

[0040] The present invention provides a classification method for distributed unshared class imbalanced data, comprising the following stages:

[0041] Phase 1: Learning a converged basic global model based on the federated learning framework;

[0042] Phase 2: Based on the first phase, the Elastic Weight Consolidation (EWC) method is used to continuously perform federated learning so that each local model can further learn the local data distribution without forgetting the important information in the previous round of global models.

[0043] Repeat the second stage until the global classification model converges again. The cloud server distributes the latest global parameters to the edge servers to update each local model to complete the model training. After the collaborative training of unbalanced data with unshared classes is realized, the classification for distributed unbalanced data with unshared classes can be realized.

[0044] Among them, federated learning can create a global classification model based on deep neural networks by aggregating the parameters of multiple local models trained on different edge servers;

[0045] Each edge server stores its raw data locally, and the local model is trained based on this data;

[0046] Defining feature space and label space Where C is the total number of class labels;

[0047] The dataset collected from the edge server e is represented as in It is a dataset The number of data samples in

[0048] like Figure 1 As shown, in the cloud-edge collaborative computing network, the class distribution of data on each edge server is unbalanced, that is, the number of samples belonging to different classes is different.

[0049] In some extreme cases, some classes may have no samples at all, such as Figure 1 The third edge server in Figure 1 is shown.

[0050] Decompose the joint distribution of feature x and label y into p e (x,y)=p e (x|y)p e (y), where and p e (x|y) is the conditional distribution; p e (y) is the empirical marginal distribution, which may vary across E edge servers. However, from a global perspective, the feature space corresponding to the class labels in the classification task is unknown in advance, but theoretically constant. Therefore, for all In a broad sense, p i (x|y) should equal p j (x|y), regardless of the edge servers Is there a lack of some class labels (extreme class imbalance)? This means that the empirical marginal distribution p i (y) and p j Any difference in (y) is due to class imbalance, which leads to a joint distribution p i (x,y) and p j (x,y) also varies on different edge servers.

[0051] Among them, the training process of federated learning includes three steps: local update, aggregation and distribution.

[0052] Among them, for the local update in the training process of federated learning: each deep neural network-based classification model focuses on minimizing the local loss function f e (θ e ) to improve the classification performance, the optimization objective is defined as

[0053]

[0054] where θ e are the parameters of the local model; l(·) is a suitable loss function; ξ e From the local dataset The data set is uniformly sampled from ; in each local update phase, the objective function is optimized by the stochastic gradient descent method (SGD), that is,

[0055]

[0056] where η t ∈(0,1) is the learning rate; Represents vector The first derivative of .

[0057] Among them, for the aggregation in the training process of federated learning: federated learning only aggregates the parameters of the distributed local model to update the global model; the parameters θ of the global model g It can be obtained by the following equation:

[0058]

[0059] where w e is the weight coefficient that measures the importance of the model on the edge server e. In traditional federated learning, w e It is related to the size of the data on each edge server, for example:

[0060]

[0061] Among them, for the distribution during the training process of federated learning: the latest parameters of the global model will be distributed to the local edge servers to update the local models; at this time, all local models have the same parameters;

[0062] After multiple iterations of the three steps of local update, aggregation, and distribution, a global model with good classification performance for all classes is learned; therefore, its goal is to minimize the global objective function f g (·):

[0063]

[0064] In addition, in order to ensure the convergence of the optimization problem, the edge servers need to be Function f on e The following assumptions are made.

[0065] Assumption 1 (smoothness): For all parameters θ e and θ′ e , f e (θe ) is L-smooth, that is:

[0066]

[0067] Assumption 2 (strong convexity): For all parameters θ e and θ′ e ,f e (θ e ) is μ strongly convex, that is:

[0068]

[0069] Assumption 3 (bounded variance): For all θ e , f e (θ e ) has a bounded variance Right now:

[0070]

[0071] where ξ e From the local dataset The set obtained by random uniform sampling from ;

[0072] Assumption 4 (Uniformly Bounded): For all θ e , f e (θ e ) has a uniformly bounded expected square norm τ 2 ,Right now,

[0073]

[0074] Among them, assumptions 1 and 2 are standard assumptions in some classic classification models, and assumptions 3 and 4 have also been proposed in some literature.

[0075] The class imbalance problem of data collected from geographically distributed edge servers will cause the class distribution of data on each edge server to be skewed (i.e. ), the data exhibits the characteristics of distributed class imbalance. In particular, when faced with the two challenges of extreme class imbalance and the common existence of class-imbalanced data on most clients, this situation will cause the local model to favor the majority class in the data and ignore the minority class, which will have a negative impact on the generalization performance of the local model and the effectiveness of the global model. In addition, from the perspective of model update, class imbalance will also cause local updates to drift.

[0076] like Figure 2As shown in Figure 2, under the setting that the data is independent and identically distributed (IID), the global optimal Close to local optimum and lead to Close to However, in the case of class imbalance, keep away lead to and Far away, making the local optimum and the global optimum inconsistent.

[0077] In each round of local updates, the distribution bias of local data may cause the local model to forget the global information of the previous round, thereby damaging the convergence performance of the global model. The federated learning problem with class-imbalanced data is defined as a case of catastrophic forgetting due to different data distributions. To mitigate the impact of class imbalance in federated learning, the elastic weight solidification method in continuous learning is combined to solve the challenges of extreme class imbalance and the common class-imbalanced data on most clients. Each local model should continuously learn from other data distributions while retaining its own data distribution knowledge to achieve continuous optimization towards the global optimal direction.

[0078] Under this intuition, a penalty should be introduced to enforce near Figure 2 Right Because through continuous learning and federated learning, the model on each edge server can continuously learn from the models of other servers. Specifically, the EWC method is introduced in the local model, which finds the optimization direction of local training by explicitly calculating the importance of global parameters in each round, while avoiding catastrophic forgetting of global knowledge.

[0079] like Figure 3 As shown, and After aggregation, global parameters Datasets 1 and 2 are still not well classified. In addition, without the penalty term, after several local updates, the parameters of each local model will return to the local optimum, which makes it difficult to reach the global optimum. To address this problem, we compute the EWC penalty for each dataset and aggregate them into a global EWC penalty. Then the locally updated training trajectory is further updated to the schematic parameter space. move.

[0080] The specific method of using the elastic weight solidification method to continuously perform federated learning so that each local model can further learn the local data distribution without forgetting the important information in the previous round of global model is as follows:

[0081] When all local models have good classification performance, For each local model on the Fisher information matrix I e ; During the global aggregation process, in addition to local parameters The local Fisher information matrix is ​​also uploaded to generate the global Fisher information matrix, namely:

[0082]

[0083] At the same time, keep Parameters and As a copy on each edge server. In the later local update step t(MN g ≤t0≤t≤t0+M), use the minimum h e Instead of using the b edge server Minimize the local loss function f e (·), that is, using SGD to minimize the following target h e :

[0084]

[0085] The introduction of EWC aims to solve the catastrophic forgetting problem of neural network models.

[0086] To learn a new task, EWC can constrain important parameters to stay close to their old values. Specifically, suppose the neural network has learned the best parameters for task A. EWC can select important parameters for task A. When further learning task B, EWC constrains the parameters to The neural network learns new parameters within a small error range. The performance of tasks A and B can be maintained at the same time.

[0087] From a probabilistic point of view, given a data set, The conditional probability under To optimize the neural network parameters θ. According to Bayes' rule, The logarithm of can be expressed as:

[0088]

[0089] Where p(θ) is the prior probability of parameter θ. Assume that the data set Divided into a dataset related to old task A A dataset related to a new task B And use and To train a classification model.

[0090] The present invention introduces an elastic weight solidification method in the federated learning framework to solve the catastrophic forgetting problem of the model caused by data class imbalance. The solution consists of multiple stages: Figure 4 An overview of the multi-stage federated learning framework based on continuous learning with two stages is shown. First, as in classic federated learning, let the number of local updates between two consecutive aggregations be M and the number of global aggregations be N. Then the total number of iterations is make represents the set of aggregated moments, where and

[0091] In the first stage, the specific implementation example is to use Update the parameters of the local model As mentioned above, l(·) is a suitable loss function, usually the cross entropy loss where y c is the one-hot encoding of class c; p c is the Softmax probability of class c. However, cross entropy may not be the best loss function to use when dealing with class imbalance problems. For example, in the case of extreme class imbalance, the majority class should have a high confidence, which means that the predicted probability p of this class is c should be close to 1. However, even if p c When the loss function is close to 1, the cross entropy loss function may still not be close to 0. This may cause the classification model to fail to focus on the minority class, resulting in poor overall performance. Compared with the cross entropy loss, the focal loss reduces the impact of the majority class samples on the loss function, so that more attention can be paid to the minority class samples that are more difficult to classify. Therefore, the focal loss can be reduced by adding a coordination factor (1-p c ) γ The focus loss is used to solve this problem, which can be written as follows:

[0092]

[0093] Where γ>0 is the focus parameter. After M updates are made locally, the cloud server aggregates all uploaded local model parameters to generate new global parameters It is expressed as:

[0094]

[0095] In the second stage, in order to prevent the local model from catastrophically forgetting the data distribution and important information learned by the global model, EWC is used to penalize the local model to achieve the global optimum. Specifically, when all local models have good classification performance (in N g After global aggregation), the dataset For each local model on the Fisher information matrix I e In the global aggregation process, in addition to local parameters The local Fisher information matrix is ​​also uploaded to generate the global Fisher information matrix, namely:

[0096]

[0097] At the same time, keep Parameters and As a copy on each edge server. In the later local update step t(MN g ≤t0≤t≤t0+M), use the minimum h e Instead of using edge servers Minimize the local loss function f e (·) is to use the stochastic gradient descent method SGD to minimize the following target h e :

[0098]

[0099] Continuous federated learning is implemented in the above two stages until the global classification model converges. The cloud server distributes the latest global parameters to the edge server to update each local model to complete the model training and realize collaborative training of unbalanced data without sharing classes.

[0100] A classification method for distributed unshared class-imbalanced data using the present invention includes the following stages: the first stage: learning a converged basic global model based on a federated learning framework; the second stage: based on the first stage, using an elastic weight solidification method to continuously perform federated learning so that each local model can further learn the local data distribution without forgetting important information in the previous round of global models; repeating the second stage until the global classification model converges again, the cloud server distributes the latest global parameters to the edge server to update each local model to complete the model training, and the classification of distributed unshared class-imbalanced data can be achieved after collaborative training of unshared class-imbalanced data is achieved. In this way, the existing distributed unshared data classification methods are basically proposed under the federated learning framework, but the problem of extreme class imbalance of local data sets existing in this framework and the widespread existence of class-imbalanced data on most clients is not considered, resulting in the technical problem that it is not convenient to use these data to collaboratively train a global classification model with strong comprehensive performance;

[0101] The first and second stages of the present invention constitute a novel MConFL framework to collaboratively train an effective global classification model without sharing data, wherein MConFL is a multi-stage continuous federated learning method (Multi-stage Continual Federated Learning, MConFL). In order to prove the convergence of MConFL, the present invention also provides a mathematical proof of the convergence of MConFL. At the same time, a large number of experiments conducted on various settings have proved the effectiveness and superiority of MConFL. Then, the empirical evaluation on two class imbalanced data sets also verified the correctness of the theoretical analysis of the present invention, and proved that MConFL can significantly alleviate the problems raised. In addition, the convergence performance of MConFL is also improved compared with the five baseline methods. The specific effects of the method are compared as follows:

[0102] Accuracy comparison:

[0103] Table 1 shows the test accuracy of six federated learning algorithms (FedAvg, SCAFFOLD, MOON, FedGen, FedALA, and MConFL) on the CIFAR-10 and CIFAR-100 datasets with different neural network models (CNN, ResNet, ResNet-10, and GoogleNet). For CIFAR-10, MConFL achieves the highest accuracy in three of the four models: 0.8779 for ResNet, 0.886 for ResNet-10, and 0.8512 for GoogleNet. While SCAFFOLD shows an exceptionally high accuracy (0.8112) on CNN, it performs poorly on other architectures. For CIFAR-100, MConFL also shows excellent performance, leading in all four model architectures with accuracies of 0.0128 (CNN), 0.014 (ResNet), 0.0143 (ResNet-10), and 0.0997 (GoogleNet), respectively. The consistent excellent performance of MConFL, especially on the more challenging CIFAR-100 dataset, shows that it is more robust and efficient than other algorithms in handling diverse and complex data distributions. The performance on the CIFAR-100 dataset is low for all algorithms under different models. This may be because the CIFAR-100 dataset has complex characteristics and more classes, and is more sensitive to the imbalanced distribution of classes. The above results show that MConFL can effectively enhance the performance of the model under the federated learning framework by improving the model accuracy under different network depths and dataset complexity.

[0104] Table 1 Test accuracy of MConFL and five baseline algorithms

[0105]

[0106] Also, see Figure 5 , illustrating the convergence of six federated learning algorithms in terms of global aggregate round test accuracy on the CIFAR-10 dataset, using (a) ResNet and (b) ResNet-10 architectures. Figure 5 (a) depicts the performance on the ResNet architecture, with all algorithms showing an overall upward trend in accuracy as the number of aggregation rounds increases. MConFL consistently achieves the highest test accuracy, converges faster, and maintains a higher level of accuracy compared to the other algorithms. Specifically, MConFL converges significantly faster at the beginning of the second phase (after 300 rounds of aggregation). This improved performance can be attributed to the Fisher information matrix, which ensures that the edge servers effectively learn from each other. Figure 5(b) shows a similar trend for the ResNet-10 architecture. This further supports the robustness and effectiveness of MConFL’s performance under different neural network models.

[0107] Table 2 Effect of different λ on MConFL on CIFAR-10 dataset

[0108]

[0109] Table 2 shows the test accuracy of MConFL on the CIFAR-10 dataset for four neural network models (CNN, ResNet, ResNet-10, and GoogleNet) under different important parameter λ values. For the CNN architecture, the test accuracy changes slightly with the change of λ. 3 The highest accuracy is 0.808 when λ=10 2 The lowest accuracy is 0.8024 when λ=10. For ResNet, the improvement in accuracy is more obvious as λ increases. 4 The highest accuracy is observed at λ=10, which is 0.8799. For ResNet-10, a similar trend can be seen, that is, the higher the value of λ, the higher the accuracy. 4 The highest test accuracy is 0.8896 when λ is used. Overall, the table shows that MConFL tends to perform better at higher values ​​of λ, especially for deeper architectures such as ResNet and ResNet-10. The improvement in test accuracy with increasing λ shows that the importance parameter plays an important role in improving the performance of MConFL, especially for more complex models. This analysis highlights the importance of fine-tuning λ for optimizing the accuracy of federated learning algorithms.

[0110] Table 3 Test accuracy of different federated learning algorithms under different α on the CIFAR-10 dataset

[0111]

[0112]

[0113] Table 3 shows the test accuracy of the six federated learning algorithms on the CIFAR-10 dataset at different imbalance ratios. MConFL consistently achieves the highest accuracy, especially when α = 0.5, with results of 0.808 (CNN), 0.8779 (ResNet), and 0.886 (ResNet-10). When α = 0.5 (0.8112), SCAFFOLD performs well on CNN but performs very poorly on other architectures. As α increases, FedAvg, MOON, FedALA, and FedGen show steady improvements on different architectures, but are still lower than MConFL. Comparing these results with Figure 6 The data distribution results shown (distribution of the CIFAR-10 dataset among 10 edge servers: (a) α = 0.1; (b) α = 0.3; (c) α = 0.5) are combined, and it is clear that smaller values ​​of α make the data distribution imbalanced and degrade the performance of all algorithms. These results highlight the robustness and superiority of MConFL in improving test accuracy under different neural network models and imbalance ratios.

[0114] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A classification method for distributed unshared class imbalanced data, characterized in that: The following phases are included: Phase 1: Learning a converged basic global model based on the federated learning framework; Phase 2: Based on the first phase, the elastic weight solidification method is used to continuously perform federated learning so that each local model can further learn the local data distribution without forgetting the important information in the previous round of global model; Repeat the second stage until the global classification model converges again. The cloud server distributes the latest global parameters to the edge servers to update each local model to complete the model training. After the collaborative training of unbalanced data with unshared classes is realized, the classification for distributed unbalanced data with unshared classes can be realized.

2. The classification method for distributed unshared class imbalanced data according to claim 1, characterized in that: Federated learning can create a global classification model based on deep neural networks by aggregating the parameters of multiple local models trained on different edge servers; Each edge server stores its raw data locally, and the local model is trained based on this data; Defining feature space and label space Where C is the total number of class labels; The dataset collected from the edge server e is represented as in It is a dataset The number of data samples in 3. The classification method for distributed unshared class imbalanced data according to claim 2, characterized in that: The training process of federated learning consists of three steps: local update, aggregation, and distribution.

4. The classification method for distributed unshared class imbalanced data according to claim 3, characterized in that: For local updates during the training process of federated learning: Each deep neural network-based classification model focuses on minimizing the local loss function f e (θ e ) to improve the classification performance, the optimization objective is defined as where θ e are the parameters of the local model; l(·) is a suitable loss function; ξ e From the local dataset The data set is uniformly sampled from ; in each local update stage, the objective function is optimized by the stochastic gradient descent method, that is: where η t ∈(0,1) is the learning rate; Represents vector The first derivative of .

5. The classification method for distributed unshared class imbalanced data according to claim 4, characterized in that: For aggregation during the training process of federated learning: Federated learning only aggregates the parameters of distributed local models to update the global model; the parameters θ of the global model g It can be obtained by the following equation: where w e is the weight coefficient that measures the importance of the model on the edge server e. In traditional federated learning, w e It is related to the size of the data on each edge server, for example:

6. The classification method for distributed unshared class imbalanced data according to claim 5, characterized in that: For the distribution during the training process of federated learning: the latest parameters of the global model will be distributed to the local edge servers to update the local models; at this point, all local models have the same parameters; After multiple iterations of the three steps of local update, aggregation, and distribution, a global model with good classification performance for all classes is learned; therefore, its goal is to minimize the global objective function f g (·):

7. The classification method for distributed unshared class imbalanced data according to claim 6, characterized in that: The specific method of using the elastic weight solidification method to continuously perform federated learning so that each local model can further learn the local data distribution without forgetting the important information in the previous round of global model is as follows: When all local models have good classification performance, For each local model on the Fisher information matrix I e ; During the global aggregation process, in addition to local parameters The local Fisher information matrix is ​​also uploaded to generate the global Fisher information matrix, namely: At the same time, keep Parameters and as a copy on each edge server; in a later local update step t(MN g ≤t0≤t≤t0+M), use the minimum h e Instead of using edge servers Minimize the local loss function f e (·), that is, using SGD to minimize the following target h e :

Images