A medical image classification method based on client clustering personalized federated learning

Abstract

The application discloses an image classification method based on client clustering personalized federated learning, and mainly solves the problems of poor local data privacy protection, high communication and calculation in the interactive learning process of the existing federated learning method. The implementation scheme is as follows: obtaining a colon pathological data set from a public data set and classifying the colon pathological data set; processing the data distribution degree according to Dirichlet distribution; selecting a Resnet18 classification network as the backbone network model of the server side and the client side; dividing the client group through the training of the client model; training the server side model through adaptive weight aggregation; updating the client model by using the parameters of the trained server side model; and classifying the local disease image data by using the updated client model. The application can guarantee the privacy and security of data, reduce the communication and calculation, improve the classification accuracy, and can be used for the classification detection of medical images of different diseases in medical institutions.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a medical image classification method, which can be used for classification detection of medical images of different diseases in medical institutions. BACKGROUND

[0002] In traditional deep learning, the training of a medical image classification network cannot be separated from a large amount of data. The lack of training data and the difficulty in obtaining labels make it difficult for deep learning to obtain an effective model, which is a bottleneck problem of the current medical image classification task. However, the privacy problem of medical image data is very sensitive. Unlike ordinary data, medical image data contains personal privacy of patients. Restricted by relevant privacy policies and laws and regulations, the circulation of medical image data is strictly limited. It is unrealistic to collect a large amount of data from various medical institutions to train a model. Moreover, the way in which various medical institutions store and manage medical image data is different, which makes data sharing more difficult, forming a "data island". It is necessary to overcome many barriers to train a deep learning network.

[0003] In order to break the "data island", federated learning FL emerges as the times require. It was proposed by Google in 2016. It usually adopts a client-server basic architecture, uses local private data distributed in various clients for local calculation, and then aggregates the locally trained models by the central server to update the global shared model. In the whole process, the client does not need to upload or share the local data, and always maintains the control right of the local data. Compared with centralized training, this emerging technology avoids the storage cost brought by centralized training data, and at the same time avoids the increasingly serious data privacy and security problems. In addition, although distributed learning can also adopt a client-server architecture to train the model on independent multiple nodes, it requires that the data is independent and identically distributed IID, while federated learning does not have this limitation, and more does not like to deal with non-independent and identically distributed non-IID data.

[0004] These advantages make FL have good application prospects in many fields, especially in smart medical care. FL can jointly train a global model by exchanging gradient information between the client and the server instead of the data itself, that is, the medical image data of each medical institution will not leave the local. It breaks the data barrier in traditional deep learning, and each medical institution can obtain a model combining the data advantages of all parties without sharing the local data, ensuring the privacy and security of patient data. Moreover, compared with traditional deep learning, the performance of the cooperative win-win model of federated learning is better.

[0005] The most classic federated learning algorithm is FedAvg, which aims to train a global model that performs well on most clients. It directly averages the model parameters from all clients without reading local data. This algorithm is simple and has excellent performance. However, it may experience severe performance degradation or even divergence when there is data heterogeneity among clients, i.e., the data at each client is not independent and identically distributed. Moreover, it cannot prevent privacy attacks. However, in the real world, the local data of each medical institution is often not independent and identically distributed due to factors such as region, population, and climate. In medical image classification tasks, there are usually varying degrees of feature bias and label bias. Therefore, how to enable FL to handle non-independent and identically distributed data is a key problem for the successful application of this emerging technology in smart healthcare. In addition to the data heterogeneity problem, FL also needs to handle the system heterogeneity problem in edge computing, which is caused by the differences in computing power of each client device. These problems not only affect the training of the global model, but also affect the performance of the global model on some client's local data, even worse than the local training model of the client, making the affected client reluctant to join the federated learning. Therefore, generating a unified model for each client is not the best solution in actual federated learning, and it cannot achieve personalized medical services.

[0006] In recent years, personalized federated learning (PFL) has attracted great interest from researchers around the world. This technology processes each federated learning client individually by adding a local adaptation step or directly trains an independent personalized model on each client, generating a personalized solution for each client, aiming to solve the above-mentioned data heterogeneity and system heterogeneity problems. Data heterogeneity and system heterogeneity problems are common in medical image classification tasks. By applying personalized federated learning technology, a more personalized model that is better suited to the local data distribution and device computing power can be generated for each medical institution, improving classification accuracy and robustness.

[0007] McMahan B proposed an update method for training model parameters in global network models in his 2017 article Communication-efficient learning of deep networks from decentralized data, i.e., the classic FedAvg method, which directly averages model parameters from all clients. While this may seem reasonable, it has the following three shortcomings:

[0008] First, in actual application, once data heterogeneity occurs, the performance of the model will be significantly reduced or even divergent.

[0009] Second, because the complete model parameters are shared, it is possible to cause gradient leakage and cannot guarantee the privacy and security of the data.

[0010] Third, because communication with all clients is required at each parameter model aggregation, the communication volume and computational volume are very large.

[0011] Li T proposed a FedProx method in the article Federated optimization in heterogeneous networks in 2020, which introduces a regularization term in the local model of the client, balances the difference between the global model and the local model through the regularization term, and makes the local update not too far from the global model. Although this method increases the stability of the overall convergence, it hardly improves the accuracy, and still has the problems of inability to guarantee data privacy and security, excessive communication volume and computational volume. SUMMARY

[0012] The present application aims to overcome the shortcomings of the prior art and provides a medical image classification method based on client clustering personalized federated learning to avoid gradient depth leakage, guarantee the privacy and security of the data, reduce the communication volume and computational volume, and improve the classification accuracy.

[0013] The technical idea for achieving the object of the present application is to avoid gradient depth leakage by sharing only the feature extraction layer parameters of the local model of the client, to reduce the communication volume and computational volume of the global model during training by estimating the image sample ratio of each client to cluster the clients, and to improve the accuracy of the global model by using the accuracy of the public dataset on the local model as the weight for adaptive weight aggregation of the global model.

[0014] According to the above idea, the specific steps of the technical solution of the present application include the following:

[0015] (1) Obtain a public dataset of the server side, N image classification datasets of the clients, and a network model:

[0016] (1a) Obtain the dataset of related diseases from N medical institutions or public datasets, and process the non-independent identically distributed degree according to the Dirichlet distribution to obtain N image classification datasets of the clients, N >= 2;

[0017] (1b) Extract image groups with the same number of samples from the dataset of each medical institution or public dataset to form a public dataset of the server side;

[0018] (1c) Divide the samples in the above image classification datasets and the public dataset into training data and test data according to the proportions of 8:2 or 7:3 or 9:1, respectively;

[0019] (1d) Select the existing Resnet18 classification network as the server-side backbone network model S-NET and the client-side backbone network model C-NET respectively;

[0020] (2) Estimate the class vector of the client using the gradient vector change of the client C-NET model, and complete the group division of the client through the clustering algorithm:

[0021] (2a) Initialize the client C-NET model, use the training data in the image classification data set, and train the C-NET model based on the back propagation method of stochastic gradient descent to obtain the trained client network model C-NET1;

[0022] (2b) Input the training data in the public data set into the client C-NET1 model for retraining to obtain the client network model C-NET2 after secondary training;

[0023] (2c) According to the difference of the gradient vectors of C-NET1 model and C-NET2 model, estimate the ratio of sample categories in each client image classification data set Upload the sample distribution category estimation vector R to the server according to the ratio;

[0024] (2d) The server divides the clients into C groups using the K-center clustering algorithm according to the category estimation vector R of each client;

[0025] (3) Use the personalized federated learning and adaptive weight aggregation method of part of the network to retrain the server-side S-NET model and the C-NET2 model obtained after secondary training of the client:

[0026] (3a) Initialize the S-NET model on the server side;

[0027] (3b) The server randomly selects the client participating in the training in this round from the clients in the divided groups, and sends the feature extraction layer parameters in S-NET, i.e. the parameters of part of the network, to the selected client;

[0028] (3c) After receiving the feature extraction layer parameters, the client updates the feature extraction layer parameters of the C-NET2 model of the client, and then uses the training data in the image classification data set of the client to train it based on the back propagation of stochastic gradient descent until the C-NET2 model converges, obtains the client network model C-NET3 after three times of training, and uploads the feature extraction layer parameters to the server;

[0029] (3d) The server aggregates the adaptive weights of the feature extraction layer of the C-NET3 model uploaded by each client, and updates the feature extraction layer parameter of the S-NET model by using the aggregated feature extraction layer parameter;

[0030] (3e) Repeat steps (3b) to (3d) until the S-NET model converges, and obtain the trained server-side network model S-NET1;

[0031] (3f) Send the feature extraction layer parameter in the trained S-NET1 model to each client, and update the feature extraction layer parameter of the network model by using the received feature extraction layer parameter, and obtain the final C-NET4 model of each client;

[0032] (4) The client inputs the test data in the image classification data set of each client into the C-NET4 model, and obtains the classification result.

[0033] Compared with the prior art, the present application has the following advantages:

[0034] 1. Good data privacy protection.

[0035] The model sharing in federated learning is not secure, which may cause gradient deep leakage. Only the gradient information of the participants can restore the pixel-level accurate image and the label-level matching text. In the present application, only the feature extraction layer parameter of the local model of the client is shared, and the classification layer is always kept locally, so the restoration of the pixel-level accurate image and the label-level matching text can be avoided, and the privacy and security of the data are ensured.

[0036] 2. Smaller communication and calculation amount, and higher model performance.

[0037] In the present application, the client category estimation method is used to select as few and accurate models as possible for training in the training process, which greatly reduces the communication and calculation amount of the central server and the client, and makes the performance of the model optimal.

[0038] 3. Higher model accuracy.

[0039] In the present application, the accuracy of the client model on the global public data set is used as a weight coefficient for weight aggregation, which can effectively measure the model performance and improve the accuracy of the server-side S-NET model. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is the implementation flowchart of the present application.

[0041] Figure 2 is the classification graph of the PathMNIST colon pathological data set obtained in the present application;

[0042] Figure 3 is the accuracy comparison chart of the classification of the PathMNIST dataset by the present application and the existing image classification method;

[0043] Figure 4 is the accuracy comparison chart of the image classification of different numbers of clients by the present application. DETAILED DESCRIPTION

[0044] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, other embodiments obtained by persons skilled in the art without creative labor should be within the protection scope of the present application.

[0045] It should be noted that the step numbers in the specification and claims of the present application are only for clearly describing the embodiments of the present application, and the sequence of the numbers is not limited.

[0046] Referring to Figure 1 , the implementation steps of the present embodiment include the following:

[0047] Step 1: Obtain a colon pathological dataset.

[0048] The colon pathological dataset can be obtained from multiple medical institutions or from a public dataset.

[0049] The present embodiment is a PathMNIST colon pathological dataset obtained from the MedMNIST V2 public dataset, as shown in Figure 2 , which has 107180 colon pathological images for predicting the survival rate of colorectal cancer histological sections, and contains 9 categories, namely adipose tissue ADI, background BACK, debris DEB, lymphocytes LYM, mucous tissue MUC, smooth muscle tissue MUS, normal colon mucosa NORM, tumor-related interstitial tissue STR, and colorectal adenocarcinoma epithelium TUM.

[0050] Step 2: Divide the obtained disease dataset.

[0051] 2.1) Obtain the dataset of the related disease from N medical institutions or public datasets, and process the non-independent identically distributed degree according to the Dirichlet distribution to obtain the image classification dataset of N clients. In this example, the PathMNIST colon pathological dataset is divided into 100 clients, and is processed into image classification datasets with distribution degrees of 0.8 and 0.5 according to the Dirichlet distribution

[0052] 2.2) Extract the same number of images of each class from the data set of each medical institution or public data set to form a public data set, and in this example, 32 images of each class are extracted from the PathMNIST colon pathology data set to form a public data set D s ;

[0053] 2.3) The samples in the above public data set and each client image classification data set are respectively divided into training data and test data according to the ratio of 8:2 or 7:3 or 9:1, and in this example, the ratio of 8:2 is used to divide the training data and test data, that is, the training data accounts for 80% and the test data accounts for 20% in the public data set, and the training data accounts for 80% and the test data accounts for 20% in each client image classification data set.

[0054] Step 3: Obtain a training model

[0055] An existing Resnet18 classification network is selected as the server-side backbone network model S-NET and the client-side backbone network model C-NET;

[0056] The structure of the Resnet18 classification network includes 16 layers of convolutional neural network layers and 2 layers of fully connected layers, and the first 7 layers of convolutional neural network layers have no cross-layer connection, and the 8-16 layers of convolutional neural network layers use cross-layer connection, that is, every other residual block has a cross-layer connection, which is used to add the output of the previous layer feature map to the output of the next layer residual block.

[0057] Step 4: Client clustering.

[0058] 4.1) Train the client model C-NET to obtain a trained client C-NET1 model:

[0059] 4.1.1) Initialize the C-NET model θ k , set the learning rate η to 0.01, the decay rate to 0.99, and the batch input image to 16;

[0060] 4.1.2) Input the local image classification data set to the C-NET model of the client, calculate the cross-entropy loss function by the back propagation algorithm, and update the network parameters using the optimizer SGD;

[0061] 4.1.3) Repeat step 4.1.2) for a total of 10 rounds to obtain the trained C-NET1 model gradient vector L s :

[0062] L s =[L s (w1),L s(w2),...,L s (w r ),...,L s (w C )]

[0063] wherein L s (w r ) is the connection weight of the rth neuron after training, r is 1 to C, C represents the number of sample categories, and C is taken as 9 in the embodiment;

[0064] 4.2) The server sends the public data set D s to each client;

[0065] 4.3) The client C-NET1 model is secondarily trained to obtain a secondarily trained client C-NET2 model:

[0066] 4.3.1) The learning rate η is set to 0.01, the decay rate is 0.99, and one batch of input images is 16;

[0067] 4.3.2) The public data set D s is input to the C-NET1 model of the client, the cross-entropy loss function is calculated through the back propagation algorithm, and the network parameter is updated using the optimizer SGD;

[0068] 4.3.3) Step 4.3.2) is repeated for a total of 10 rounds to obtain a secondarily trained C-NET2 model gradient vector L s :

[0069] L s '=[L s '(w1),L s '(w2),...,L s '(w r ),...,L s '(w C )]

[0070] wherein L s '(w r ) is the connection weight of the rth neuron after secondary training;

[0071] 4.4) The class ratio R i of the ith sample in the image classification data set of each client is calculated, and the sample class vector R of the image classification data set of each client is estimated:

[0072] 4.4.1) The change value of the gradient vector of the C-NET1 model and the C-NET2 model after two times of training is calculated

[0073]

[0074] wherein Lr is the weight of the rth neuron connection of the C-NET2 model s (w r ) is the weight of the rth neuron connection of the C-NET1 model s (w r ) is the difference between the weight of the rth neuron connection of the C-NET1 model

[0075] 4.4.2) Calculate the class ratio R of the ith sample in each image classification data set i :

[0076]

[0077] 4.4.3) Estimate the vector R of sample class in each client image classification data set:

[0078] R = [R1, R2,..., R i ..., R C ]

[0079] 4.5) Upload the vector R of sample class of each client to the central server;

[0080] 4.6) The central server divides the clients into groups:

[0081] 4.6.1) Randomly select k cluster grouping initial center points, k is 9 in this embodiment;

[0082] 4.6.2) Calculate the distance D(i, Ck) of each client i and k center points respectively; k

[0083]

[0084] wherein x(i), y(i), z(i) respectively represent each vector in the sample distribution class estimation vector R of the ith client; x(Ck), y(Ck), z(Ck) respectively represent the coordinate values of the kth cluster center Ck. k k k k

[0085] 4.6.3) Assign each client to the nearest cluster center point;

[0086] 4.6.4) Calculate the center point of each cluster according to the samples in the cluster grouping;

[0087]

[0088] ​​​​​where n is the number of all client objects assigned to the cluster center, X(i) represents the coordinate value of the i-th client with C as the cluster center; k

[0089] 4.6.5) Repeat steps 4.6.2)-4.6.4) until the clustering algorithm converges, obtaining 9 groups of clients.

[0090] Step 5: Personalized federated learning.

[0091] 5.1) The server initializes the S-NET model θ g ;

[0092] 5.2) The central server randomly selects one client in each group to participate in this round of training according to the client clustering results, and sends the feature extraction layer parameters of S-NET to each client;

[0093] 5.3) After receiving the , the client updates the feature extraction layer parameters of the client C-NET2 model θ k ;

[0094] 5.4) Retraining the client C-NET2 model:

[0095] 5.4.1) Set the learning rate η to 0.01, the decay rate to 0.99, and the batch input image to 16;

[0096] 5.4.2) Input the local data set to the client C-NET2 model, calculate its cross-entropy loss function through the back propagation algorithm, and use the optimizer SGD to update the network parameters;

[0097] 5.4.3) Repeat step 5.9.2) for a total of 5 rounds to obtain the client network model C-NET3 after three times of training;

[0098] 5.5) Test the C-NET3 model using the public data set D s , and upload the test accuracy and the feature extraction layer parameters of the C-NET3 model to the server;

[0099] 5.6) The server calculates the predicted accuracy Acc k of the k-th client according to the test accuracy uploaded by the client, and the weight α k of the total predicted accuracy:

[0100]

[0101] ​​5.7) Using the accuracy rate weight of each client C-NET3 model k and feature extraction layer parameters Adaptive weight aggregation is performed, and the aggregated feature parameters are updated to the feature extraction layer of the S-NET model

[0102]

[0103] 5.8) Repeat steps 5.2)-5.7) until the S-NET model converges, and obtain the trained server model S-NET1;

[0104] 5.9) The feature extraction layer of the trained S-NET1 model is sent to each client;

[0105] 5.10) After each client receives the feature extraction layer of the trained S-NET1 model , the feature extraction layer parameters of the respective C-NET3 model are updated to obtain the final C-NET4 model of each client;

[0106] Step 6: Each client inputs the local data set into the C-NET4 model to obtain the classification result, and completes the classification detection task of different diseases.

[0107] The effects of the present application can be further illustrated by the following simulation results.

[0108] I. Experimental environment and data

[0109] The running environment of the experiment is Ubuntu 18.04 and Python 3.7, the hardware configuration is Intel Core i7-8700K CPU, Nvidia RTX 2080Ti GPU and 32GB RAM, and the deep learning framework used is Pytorch 1.4.0.

[0110] The data set used in the experiment is the PathMNIST colon pathological data in the MedMNIST V2 data set, as shown in Figure 2 .

[0111] II. Experimental content

[0112] Experiment 1: The accuracy rate comparison simulation of the present application pFedCM and the existing image classification methods FedAvg, FedProx, FedPer and Fedcir on the PathMNIST data set with distribution degree of 0.8 and 0.5, respectively, the accuracy rate parameters are as shown in Table 1, and the accuracy rate convergence speed curve is as shown in Figure 3 , whereinFigure 3 a represents a curve graph with a distribution degree of 0.8, Figure 3 b represents a curve graph with a distribution degree of 0.5.

[0113] Table 1: Accuracy comparison of different image classification methods

[0114] Method σ = 0.8 σ = 0.5 FedAvg 78.6％ 85.5％ FedProx 78.8％ 85.7％ FedPer 87.5％ 83.4％ Fedcir 75.2％ 81.3％ Invention 90.8％ 87.3％

[0115] From Figure 3 and Table 1, it can be seen that the present application obtains the best performance on data with different distribution degrees, especially when the distribution degree of the data is more serious, that is, sigma = 0.8, the improvement compared with other algorithms is higher, in addition, the convergence speed of the present application is the fastest.

[0116] Experiment 2: The accuracy of classifying different numbers of clients participating in training per round on the PathMNIST dataset with a distribution degree of 0.8 is compared by using the pFedCM of the present application and the existing image classification method FedAvg, the accuracy parameters are as shown in Table 2, and the curve is as shown in Figure 4 .

[0117] Table 2: Accuracy comparison of image classification under different numbers of clients

[0118] Number of sampled clients Conventional method Invention C 78.6％ 90.8％ 2C 83.2％ 91.6％ 3C 85.3％ 92.0％ 4C 86.1％ 92.3％

[0119] From Figure 4 and Table 2, it can be seen that the accuracy of the algorithm of the present application is higher than that of the traditional algorithm under different numbers of sampled clients, it can also be seen that the present application can obtain an algorithm model with higher performance than the traditional algorithm under the condition of consuming less communication amount and calculation amount, for example, the accuracy of the present application when the number of sampled clients is the least C is higher than the accuracy of the traditional method when the number of sampled clients is the maximum 4C.

Claims

1. A medical image classification method based on client clustering personalized federated learning, characterized in that, Comprise: (1) Obtain the server-side public data set, N client image classification data set and network model: (1a) Obtain the data set of related diseases from N medical institutions or public data set, and process the non-independent identically distributed degree according to Dirichlet distribution, obtain N client image classification data set, N >= 2; (1b) Extract the same number of images of each type of sample from the data set of each medical institution or public data set to form the server-side public data set; (1c) Divide the training data and test data of each image classification data set and public data set according to the proportion of 8:2 or 7:3 or 9:1 respectively; (1d) Select the existing Resnet18 classification network as the server-side backbone network model S-NET and the client-side backbone network model C-NET respectively; (2) Estimate the class vector of the client using the gradient vector change of the client C-NET model, and complete the group division of the client through clustering algorithm: (2a) Initialize the client C-NET model, use the training data in the image classification data set, and train the C-NET model based on the back propagation method of stochastic gradient descent to obtain the trained client network model C-NET1; (2b) Input the training data in the public data set into the client C-NET1 model for retraining, and obtain the client network model C-NET2 after secondary training; (2c) estimating the ratio of sample classes in each client image classification dataset based on the difference between the gradient vectors of the C-NET1 model and the C-NET2 model uploading the sample distribution class estimation vector R to the server based on the ratio (2d) The server divides the clients into C groups according to the class estimation vector R of each client using K-center clustering algorithm; (3) Use the personalized federated learning and adaptive weight aggregation method of part of the network to retrain the server-side S-NET model and the C-NET2 model obtained after secondary training of the client: (3a) Initialize the S-NET model on the server side; (3b) The server randomly selects the client participating in the training in this round from the clients in the divided groups, and sends the feature extraction layer parameters in S-NET, that is, the parameters of part of the network to the selected client; (3c) After receiving the feature extraction layer parameters, the client updates the feature extraction layer parameters of the client C-NET2 model, and then uses the training data in the client image classification data set to train it based on the back propagation of stochastic gradient descent until the C-NET2 model converges, obtains the third training client network model C-NET3, and uploads the feature extraction layer parameters to the server; (3d) The server aggregates the feature extraction layer parameters of each client C-NET3 model, updates the feature extraction layer parameters of the S-NET model using the aggregated feature extraction layer parameters; (3e) Repeat steps (3b) to (3d) until the S-NET model converges, and obtain the trained server-side network model S-NET1; (3f) the feature extraction layer parameters in the trained S-NET1 model are sent to each client, and after the client receives the feature extraction layer parameters, the feature extraction layer parameters of the network model are updated to obtain the final C-NET4 model of each client; (4) the client inputs the test data in the respective image classification data set into the C-NET4 model to obtain the classification result.

2. The method of claim 1, wherein, Step (1a) is a non-independent identically distributed degree processing according to the Dirichlet distribution. According to the principle of the same total number of samples and different sample classifications, the proportion of the number of main samples in the image classification data set is processed into 80% or 50%.

3. The method of claim 1, wherein, The selected existing Resnet18 classification network in step (1d) includes 16 layers of convolutional neural network layers and 2 layers of fully connected layers, and the first 7 layers of convolutional neural network layers have no cross-layer connection. The 8-16 layers of convolutional neural network layers use cross-layer connection, that is, every other residual block has a cross-layer connection, which is used to add the output of the previous layer feature map to the output of the next layer residual block.

4. The method of claim 1, wherein, Step (2a) trains the C-NET model based on the back propagation method of the stochastic gradient descent, and the implementation steps are as follows: (2a1) define the ResNet18 model using the deep learning framework PyTorch; (2a2) set the model learning rate to 0.01, the training epoch to 10, and the batch_size to 16; (2a3) calculate the cross-entropy loss function by the back propagation algorithm, and update the parameters using the optimizer SGD; (2a4) repeat (2a3) until the preset epoch.

5. The method of claim 1, wherein, Step (2c) estimates the ratio of sample categories in each client image classification dataset according to the difference between the gradient vectors of the C-NET1 model and the C-NET2 model, denoted as s According to the ratio, a sample distribution category estimation vector R is obtained, and the formula is as follows: R = [R1, R2,..., R C ] wherein R i represents the class ratio of the i-th class sample in each client image classification dataset, represents the difference of the gradient vectors of the C-NET1 model and the C-NET2 model.

6. The method of claim 1, wherein, Step (2d) divides the clients into C groups by using the K-center clustering algorithm according to the category estimation vector R of each client, and the implementation steps include the following: (2d1) randomly select C initial center points of the clustering grouping; (2d2) calculate the distance between each client object and the C center points respectively; (2d3) assign each client object to the nearest clustering center point; (2d4) calculate the center point of each cluster according to the samples in the clustering grouping; (2d5) iteratively execute steps 2d2)-2d4) until the clustering algorithm converges, and C divided groups are obtained.

7. The method of claim 1, wherein, Step (3d) the server aggregates the adaptive weights of the feature extraction layer of the C-NET3 model uploaded by each client, and the implementation steps include the following: (3d1) the server sends the public data set to each client; (3d2) each client tests the C-NET3 model using the test data in the public data set, and feeds back the test accuracy to the server; (3d3) the server aggregates the feature extraction layer parameters of the S-NET model using the test accuracy fed back by each client, and the formula is as follows: where θ(t) represents the feature extraction layer parameters of the S-NET model, θ n (t) represents the feature extraction layer parameters of the nth client, α n represents the weight of the prediction accuracy of the nth client in the overall prediction accuracy, Acc n represents the test accuracy of the C-NET3 of the nth client on the test set of the public dataset, i.e., the percentage of correctly predicted samples in the total samples.

8. The method of claim 6, wherein, The distance of each client object from the C number of centroids is calculated in step (2d2) using the Euclidean distance formula to calculate the distance D(i, C k ) of the i k th client from the cluster center C Wherein, x(i), y(i), z(i) respectively represent the coordinate value of the i-th client in the space, that is, each vector in the sample distribution category estimation vector R. x(C k ), y(C k ), z(C k ) respectively represent coordinate values of the Kth cluster center C k .

9. The method of claim 6, wherein, In step (2d4), the center point of each cluster is calculated according to the samples in the clustering grouping, and the formula is as follows: where n is the number of all client objects, X(i) represents the coordinate value of the i-th client with C k as the cluster center.

Images