A federated learning image recognition method based on model inversion and distillation

Abstract

The application discloses a kind of federal learning image identification methods based on model inversion and distillation, comprising:1. training local image model on each client and upload to federal center server;2. federal center server receives local image model and carries out weighted aggregation;3. federal center server utilizes local image model to construct synthetic dataset by depth inversion;4. federal center server utilizes synthetic dataset to carry out graph element relabeling and knowledge distillation;5. global image model parameters are iteratively optimized, and the global model of optimal parameter is obtained and issued to each client for image recognition.The application adopts depth inversion method to synthesize data, and the graph elements of different information density generated by attention weight are screened, and the graph elements with low information density are relabeled by integrated model in the subsequent knowledge distillation stage, so as to solve the image identification problem of single communication federal learning in the data heterogeneous scene.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and involves technologies such as deep learning, federated learning and knowledge distillation. Specifically, it is a federated learning image recognition method based on model inversion and distillation. Background Art

[0002] With the booming development of big data-driven artificial intelligence (AI), deep learning technology has seen further growth in areas such as image recognition, video surveillance, and personalized recommendations. Key drivers behind these advances lie in the collection and utilization of massive amounts of data. However, in real-world applications, the collection and centralized storage of private data often face significant limitations due to factors such as data security, privacy protection, and legal constraints. The implementation of various privacy protection policies and regulations poses significant challenges to traditional deep learning technologies that rely on centralized data processing and storage. Federated learning, an emerging distributed machine learning framework, has emerged to help multiple organizations leverage data and conduct machine learning modeling while meeting user privacy, data security, and government regulatory requirements. It allows multiple participating clients to train models across distributed data sources without accessing the original data. Federated learning has been widely explored across various industries. Compared to traditional centralized data processing methods, federated learning can fully exploit the value of distributed data while protecting user privacy. However, it faces significant challenges due to communication costs and data heterogeneity across clients.

[0003] While traditional methods can improve the performance of server-side models, the poor quality of the synthesized data limits the extent of performance gains achieved by using this low-quality data in federated distillation. Previous work has primarily focused on training generators and using relatively simple generator model structures to synthesize data. Furthermore, faced with the highly heterogeneous data challenges of federated learning, many generated data labels may not match the original image data. This can cause the global model to learn a large amount of incorrect knowledge during the subsequent distillation process, limiting performance gains. Summary of the Invention

[0004] In order to address the shortcomings of the above-mentioned existing technologies, the present invention proposes a federated learning image recognition method based on model inversion and distillation, so as to train a high-performance global model for image classification under the condition of a single communication between the server and the client, thereby realizing privacy-preserving image recognition based on federated learning.

[0005] The present invention adopts the following technical solutions to solve the technical problems:

[0006] The characteristic of the federated learning image recognition method based on model inversion and distillation of the present invention is that it is applied to In a system consisting of clients and a federated central server, Each client has a private local set of labeled image samples , and Respectively represent Client's local image samples and their corresponding true labels, , is the total number of clients, , For the The total number of local image samples of the client; the federated learning image recognition method is performed in the following steps:

[0007] Step 1: Train the local image model on each client, obtain the trained local image model and upload it to the federation center server;

[0008] Step 2: The federation center server receives The trained local image models uploaded by each client are weighted and aggregated to obtain the global model using formula (2). Parameters ;

[0009] (2)

[0010] In formula (2), Indicates the Client-trained local image model Parameters, express The weight of

[0011] Step 3: The federation center server receives The trained local image model uploaded by the client is used for depth inversion to construct the overall loss function of the model inversion , synthetic dataset and sparse primitive datasets ;

[0012] Step 4: The federation center server uses and right Perform knowledge distillation and graph relabeling to construct the loss function for knowledge distillation , classification loss of sparse primitives and primitive relabeling loss , thus obtaining the loss function ;

[0013] Step 5: The federation center server is based on and Constructing the total loss function , used for global model Perform training to obtain a trained global model And sent to each client for image classification and recognition.

[0014] The federated learning image recognition method based on model inversion and distillation described in the present invention is also characterized in that step 1 includes:

[0015] No. The client will Input local image model Processed in the local image samples Belong to The probability of each category ; , Represents the total number of categories; thus constructing the local image model according to formula (1) The cross entropy loss function , and use the stochastic gradient descent algorithm to optimize the local image model The parameters are updated and When it becomes stable, the trained local image model is obtained And upload it to the federation center server;

[0016] (1)

[0017] In formula (1), is a symbolic function, indicating the Client's local image samples If the category , indicating the local image samples Belong to categories, if , indicating the local image samples Does not belong to categories.

[0018] Furthermore, the step 3 includes:

[0019] Step 3.1: Randomly sample a batch of noise samples with real labels. ,in, represents a set of noise samples, Indicates the noise samples, and conform to the Gaussian distribution, express The true label set of express The true label, , is the total number of samples in a batch;

[0020] Step 3.2, define the current moment as , and initialize ,Will As The set of noise samples with true labels at all times is recorded as ;in, express The noise sample set at time , express The moment noise samples, and conform to the Gaussian distribution, express The true label set of express The true label of

[0021] Step 3.3: The federated central server uses the local image model right Each noise sample in is processed to obtain Moment local prediction probability vector ;in, express right The local prediction probability of

[0022] Step 3.4: Construct according to formula (3) Moment-local image model The classification loss ;

[0023] (3)

[0024] In formula (3), is a symbolic function, indicating The first time in a batch noise samples The real category information, if , indicating the noise samples Belong to categories, if , indicating the noise samples Does not belong to categories;

[0025] Step 3.5: The federated central server uses the global image model right Each noise sample in is processed to obtain The global prediction probability vector at time ;in, express right The global prediction probability of

[0026] Step 3.6: Construct according to formula (4) The moment JS divergence loss ;

[0027] (4)

[0028] In formula (4), represents KL divergence;

[0029] Step 3.7: Calculate according to formula (5) The L2 norm total variation regularization loss ;

[0030] (5)

[0031] In formula (5), Respectively represent the number of channels, height and width of the noise sample; express The difference between adjacent pixels along the width direction, express The difference between adjacent pixels along the height direction, express The difference between adjacent pixels along the diagonal direction from top to bottom and from left to right, express Differences between adjacent pixels along the diagonal direction from top to bottom and from right to left;

[0032] Step 3.8, Perform different linear transformations and get Query vector at time 、 Key vector at time and The value vector at time , and input into the ViT model for processing, we get Attention weight at the moment ; and The first row vector in is denoted as ;

[0033] Step 3.9: Use formula (6) to obtain High information density graphics at all times ;

[0034] (6)

[0035] In formula (6), express Attention weight at the moment The first row vector The mask matrix of

[0036] Step 3.10: Using formula (7), we can get Low information density primitives at the moment ;

[0037] (7)

[0038] Step 3.11: According to formula (8), build a federation center server for the The first model inversion Overall loss function ;

[0039] (8)

[0040] In formula (8), and Represents the scaling factor hyperparameters of JS divergence and regularization loss;

[0041] Step 3.12: Local image model of each client Using loss function right Training, get Synthetic dataset for sub-model inversion and sparse primitive datasets ;

[0042] Step 3.13, Assign to , then return to step 3.3 and execute sequentially until the maximum number of iterations is reached So far, The synthetic data inverted by each client is aggregated to obtain the federation center server Synthetic dataset after sub-model inversion and sparse primitive datasets , and build a federation center server according to formula (9) for The overall loss function of the sub-model inversion :

[0043] (9)

[0044] In formula (9), represents the set of trained local image models, and . Indicates the A local image model trained on a client.

[0045] Furthermore, the step 4 includes:

[0046] Step 4.1: Build an integrated model according to formula (10) , and Process and obtain right The integrated prediction probability ;

[0047] (10)

[0048] In formula (10), represents a set of parameters, and ;

[0049] Step 4.2, As a teacher model, As a student model, the loss function of knowledge distillation is constructed according to formula (11) ;

[0050] (11)

[0051] In formula (11), express right The integrated prediction probability of express right The global prediction probability of

[0052] Step 4.3: The federation center server uses the global image model Compute sparse primitive datasets Medium and high information density graphics elements The predicted probability vector ;in, Representing the global image model right Middle High-information-density graphics elements The predicted probability of

[0053] Step 4.4: Build a global model according to formula (12) The classification loss ;

[0054] (12)

[0055] In formula (12), is a symbolic function, indicating Middle High-information-density graphics elements If the category , indicating the High-information-density graphics elements Belong to category ,like , indicating the High-information-density graphics elements Not in category ;

[0056] Step 4.5: The federation center server uses the global model Compute sparse primitive datasets Low and medium information density graphics elements The predicted probability vector ,in, Representing the global image model right Middle Low-information-density primitives The global prediction probability of

[0057] The federation center server uses an integrated model Compute sparse primitive datasets Low and medium information density graphics elements The predicted probability vector ,in, express right Middle Low information density primitives The integrated prediction probability of

[0058] Step 4.6: Construct the primitive re-labeling loss through formula (13) ;

[0059] (13)

[0060] Step 4.7: Build a global model according to formula (14) The loss function :

[0061] (14)

[0062] In formula (14), and Represents the scaling factor hyperparameter for the supervision loss and the relabeling loss.

[0063] Furthermore, the step 5 includes:

[0064] Step 5.1: Construct the total loss function using formula (15) and formula (16) :

[0065] (15)

[0066] (16)

[0067] In formula (15) and formula (16), Express satisfaction The optimal noise sample set with the minimum time;

[0068] Step 5.2: The federal central server and Alternate optimization to optimize the global image Parameters Train so that the total loss function reaches to the minimum and obtain the optimal parameters Global model .

[0069] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the method, and the processor is configured to execute the program stored in the memory.

[0070] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the method when executed by a processor.

[0071] Compared with the prior art, the beneficial effects of the present invention are embodied in:

[0072] 1、The application first introduces a Vision Transformer (ViT) model into a single-communication federated learning task, and proposes an image recognition method based on model inversion and distillation, that is, performing deep model inversion on the local ViT models trained by multiple clients to obtain pseudo data, and then performing knowledge distillation from an ensemble model to a global model based on the obtained data information, instead of directly training a traditional generator and then synthesizing data, thereby solving the problem of low quality of synthesized image datasets and improving the accuracy and robustness of the image classification recognition model.

[0073] 2、The core of the application is to evaluate the quality of generated images using classification-labeled attention values, thereby realizing weighted distillation. Many previous works use server-side knowledge distillation to improve the global model, mainly focusing on designing loss functions and training generators on the server side to generate data. For a label-heterogeneous environment, under the framework of model distillation using a synthesized dataset, a method of model inversion and graph relabeling is proposed to solve the client drift problem in the extreme label-heterogeneous case, and to improve the label and data mismatch problem of the synthesized image dataset.

[0074] 3、The application comprehensively considers the extreme data heterogeneity problem in federated learning, each local client device independently trains a local model, and the models are uniformly placed on the server side for processing, which can effectively apply an ensemble method to capture global information across clients. This single-communication federated learning is more in line with the current model market environment and is more practical in actual image recognition scenarios with more complex image data. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 is a flowchart of the application. DETAILED DESCRIPTION

[0076] In this embodiment, a federated learning image recognition method based on model inversion and distillation is applied to a system composed of clients and a federated center server, wherein the th client has a private labeled local image sample set , and represent the th local image sample of the th client and its corresponding true label, , is the total number of clients, , is the total number of local image samples of the th client; in this embodiment, public image datasets such as CIFAR10 and OfficeHome are used, , the image dataset is segmented using a method based on Dirichlet distribution , for each label, the label category The data is distributed to the client The ratio is .parameter Control the degree of data imbalance, The smaller the value, the more unbalanced the label distribution among local clients; Figure 1 , the federated learning image recognition method is carried out in the following steps:

[0077] Step 1: Train the local image model on each client and upload it to the federation center server;

[0078] No. The client will Input local image model Processed in the local image samples Belong to The probability of the categories ; , Indicates the total number of categories; in this embodiment, for the dataset CIFAR10, , for the dataset OfficeHome, ; Thus, the local image model is constructed according to formula (1) The cross entropy loss function , and use the stochastic gradient descent algorithm to optimize the local image model The parameters are updated and When it becomes stable, the trained local image model is obtained And upload it to the federation center server; in this embodiment, DeiT / 16-Tiny pre-trained on ImageNet-1K is used as the client's local image model;

[0079] (1)

[0080] In formula (1), is a symbolic function, indicating the Client's local image samples If the category , indicating the local image samples Belong to categories, if , indicating the local image samples Does not belong to categories.

[0081] Step 2: The federation center server receives The trained local image models uploaded by each client are weighted and aggregated to obtain the global model using formula (2). Parameters ;

[0082] (2)

[0083] in, Represents the trained local image model Parameters, express The weight of .

[0084] Step 3: The federation center server receives The trained local image model uploaded by each client is used for depth inversion to construct a synthetic dataset;

[0085] Step 3.1: Randomly sample a batch of noise samples with real labels. ,in, represents a set of noise samples, Indicates the noise samples, and conform to the Gaussian distribution, express The true label set of express The true label, , is the total number of samples in a batch; in this embodiment, .

[0086] Step 3.2, define the current moment as , and initialize ,Will As The set of noise samples with true labels at all times is recorded as ;in, express The noise sample set at time , express The moment noise samples, and conform to the Gaussian distribution, express The true label set of express The true label.

[0087] Step 3.3: The federated central server uses the local image model right Each noise sample in is processed to obtain Moment local prediction probability vector ;in, express right The local prediction probability of .

[0088] Step 3.4: Construct according to formula (3) Moment-local image model The classification loss ;

[0089] (3)

[0090] In formula (3), is a symbolic function, indicating The first time in a batch noise samples The real category information, if , indicating the noise samples Belong to categories, if , indicating the noise samples Does not belong to categories.

[0091] Step 3.5: The federated central server uses the global image model right Each noise sample in is processed to obtain The global prediction probability vector at time ;in, express right The global prediction probability of .

[0092] Step 3.6: Construct according to formula (4) The moment JS divergence loss ;

[0093] (4)

[0094] In formula (4), represents the KL divergence.

[0095] Step 3.7: Calculate according to formula (5) The L2 norm total variation regularization loss ;

[0096] (5)

[0097] In formula (5), Respectively represent the number of channels, height and width of the noise sample; express The difference between adjacent pixels along the width direction, express The difference between adjacent pixels along the height direction, express The difference between adjacent pixels along the diagonal direction from top to bottom and from left to right, express Differences between adjacent pixels along the diagonal direction from top to bottom and right to left.

[0098] Step 3.8, Perform different linear transformations and get Query vector at time 、 Key vector at time and The value vector at time , and input into the ViT model for processing, we get Attention weight at the moment ; and The first row vector is denoted as .

[0099] Step 3.9: Use formula (6) to obtain High information density graphics at all times ;

[0100] (6)

[0101] In formula (6), express Attention weight at the moment The first row vector In this embodiment, the screening ratio of the mask matrix is ​​set to , The role of is to evaluate and sort the importance of sparse primitives, and then stop the inversion of low attention weight primitives in proportion, so as to divide the primitives into high information density primitives and low information density primitives for subsequent primitive re-labeling.

[0102] Step 3.10: Using formula (7), we can get Low information density primitives at the moment ;

[0103] (7)

[0104] Step 3.11: According to formula (8), build a federation center server for the The first model inversion Overall loss function ;

[0105] (8)

[0106] In formula (8), and Represents the scaling factor hyperparameter of JS divergence and regularization loss; in this embodiment, the optimized loss function Using Adam optimizer, learning rate , hyperparameters Setting a negative value encourages synthetic images to induce divergence between the student model and the teacher model and enhance image diversity, , .

[0107] Step 3.12: Local image model of each client Using loss function right Training, get Synthetic dataset for sub-model inversion and sparse primitive datasets ;

[0108] Step 3.13, Assign to , then return to step 3.3 and execute sequentially until the maximum number of iterations is reached So far, in this embodiment, the number of iterations ;Will The synthetic data inverted by each client is aggregated to obtain the federation center server Synthetic dataset after sub-model inversion and sparse primitive datasets , and build a federation center server according to formula (9) for The overall loss function of the sub-model inversion ;

[0109] (9)

[0110] In formula (9), represents the set of trained local image models, and , Indicates the A local image model trained on a client.

[0111] Step 4: The federated central server uses the dataset and right Perform knowledge distillation and primitive re-labeling;

[0112] Step 4.1: Build an integrated model according to formula (10) , and Process and obtain right The integrated prediction probability ;

[0113] (10)

[0114] In formula (10), represents a set of parameters, and .

[0115] Step 4.2, As a teacher model, As a student model, the loss function of knowledge distillation is constructed according to formula (11) ;

[0116] (11)

[0117] In formula (11), express right The integrated prediction probability of express right The global prediction probability of .

[0118] Step 4.3: The federated central server uses the global image model Compute sparse primitive datasets Medium and high information density graphics elements The predicted probability vector ;in, Representing the global image model right Middle High-information-density graphics elements The predicted probability of .

[0119] Step 4.4: Build a global model according to formula (12) The classification loss ;

[0120] (12)

[0121] In formula (12), is a symbolic function, indicating Middle High-information-density graphics elements If the category , indicating the High-information-density graphics elements Belong to category ,like , indicating the High-information-density graphics elements Not in category ;

[0122] Step 4.5: The federated central server uses the global model Compute sparse primitive datasets Low and medium information density graphics elements The predicted probability vector ,in, Representing the global image model right Middle Low-information-density primitives The predicted probability of

[0123] Federated central server utilization integration model Compute sparse primitive datasets Low and medium information density graphics elements The predicted probability vector ,in, express right Middle Low information density primitives The predicted probability of

[0124] Step 4.6: Construct the primitive re-labeling loss through formula (13) ;

[0125] (13)

[0126] Step 4.7: Build a global model according to formula (14) The loss function :

[0127] (14)

[0128] In formula (14), and Represents the scaling factor hyperparameter of the supervision loss and the relabeling loss; in this example, the optimized loss function Using SGD optimizer, learning rate , hyperparameters , the distillation temperature is set to The purpose of this process optimization is to utilize the primitives with different information density generated in the model inversion stage during knowledge distillation, use synthetic labels to assist in supervising the training of the global model for primitives with high information density, and use the integrated model to re-label primitives with low information density.

[0129] Step 5: The federated central server checks the global model Perform training to obtain a trained global model And send it to each client to realize image classification and recognition;

[0130] Step 5.1: Construct the total loss function using formula (15) and formula (16) :

[0131] (15)

[0132] (16)

[0133] In formula (15) and formula (16), Express satisfaction The optimal noise sample set with the minimum time;

[0134] Step 5.2, the federation center server and Alternate optimization to optimize the global image Parameters Train so that the total loss function reaches to the minimum and obtain the optimal parameters Global model .

[0135] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0136] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

[0137] Example:

[0138] In order to verify the effectiveness of the method of the present invention, this embodiment selected commonly used public datasets such as CIFAR10 and OfficeHome, and adopted the test accuracy of the global model as the quantitative evaluation standard.

[0139] In this embodiment, five methods are selected to compare their effects with the method of the present invention. The selected methods are FedAvg, FedFTG, DENSE, Co-Boosting and DeepInversion. The first four are federated learning methods. FedAvg is the basic federated learning algorithm in the paper McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deepnetworks from decentralized data[C] / / Artificial intelligence and statistics.2017: 1273-1282. The other three are federated learning algorithms based on synthetic data and knowledge distillation. FedFTG is from the paper Zhang L, Shen L, Ding L, et al. Fine-tuning global modelvia data-free knowledge distillation for non-iid federated learning[C] / / Proceedings of the IEEE / CVF conference on computer vision and patternrecognition. 2022: 10174-10183, and DENSE is from the paper Zhang J, Chen C, Li B, et al.Dense: Data-free one-shot Federated learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 21414-21428. Co-Boosting is from the paper Dai R, Shen L, He F, et al. Dispfl: Towards communication-efficient personalized federated learning via decentralized sparse training[C] / / International conference on machine learning. 2022: 4587-4604. DeepInversion is a deep model inversion method from the paper Yin H, Molchanov P, Alvarez JM, et al.Dreaming to distill: Data-free knowledge transfer via deepinversion[C] / / Proceedings of the IEEE / CVFconference on computer vision and pattern recognition. 2020: 8715-8724. This approach can be applied to federated learning settings. The experimental results are shown in Tables 1 and 2.

[0140] Table 1 Different Dirichlet sampling distribution data heterogeneity levels (parameters The lower the value, the higher the heterogeneity level.) Experimental results of the method of the present invention and the five selected comparative methods on the CIFAR10 dataset:

[0141]

[0142] Table 2 Different Dirichlet sampling distribution data heterogeneity levels (parameters The lower the value, the higher the heterogeneity level.) Experimental results of the proposed method and the five selected comparative methods on the OfficeHome dataset:

[0143]

[0144] The experimental results show that under the same experimental environment settings, the performance of the method proposed in the present invention exceeds that of the other five methods, especially when the heterogeneity of the Dirichlet distribution is high, the performance is greatly improved, thus proving the feasibility of the method proposed in the present invention.

Claims

1. A federated learning image recognition method based on model inversion and distillation, characterized in that: is applied by In a system consisting of clients and a federated central server, Each client has a private local set of labeled image samples , and Respectively represent Client's local image samples and their corresponding true labels, , is the total number of clients, , For the The total number of local image samples of the client; the federated learning image recognition method is performed in the following steps: Step 1: Train the local image model on each client, obtain the trained local image model and upload it to the federation center server; Step 2: The federation center server receives The trained local image models uploaded by each client are weighted and aggregated to obtain the global model using formula (2). Parameters ; (2) In formula (2), Indicates the Client-trained local image model Parameters, express The weight of Step 3: The federation center server receives The trained local image model uploaded by the client is used for depth inversion to construct the overall loss function of the model inversion , synthetic dataset and sparse primitive datasets ; Step 3.1: Randomly sample a batch of noise samples with real labels. ,in, represents a set of noise samples, Indicates the noise samples, and conform to the Gaussian distribution, express The true label set of express The true label, , is the total number of samples in a batch; Step 3.2, define the current moment as , and initialize ,Will As The set of noise samples with true labels at all times is recorded as ;in, express The noise sample set at time , express The moment noise samples, and conform to the Gaussian distribution, express The true label set of express The true label of Step 3.3: The federated central server uses the local image model right Each noise sample in is processed to obtain Moment local prediction probability vector ;in, express right The local prediction probability of Step 3.4: Construct according to formula (3) Moment-local image model The classification loss ; (3) In formula (3), is a symbolic function, indicating The first time in a batch noise samples The real category information, if , indicating the noise samples Belong to categories, if , indicating the noise samples Does not belong to categories; Step 3.5: The federation center server uses the global model right Each noise sample in is processed to obtain The global prediction probability vector at time ;in, express right The global prediction probability of Step 3.6: Construct according to formula (4) The moment JS divergence loss ; (4) In formula (4), represents KL divergence; Step 3.7: Calculate according to formula (5) The L2 norm total variation regularization loss ; (5) In formula (5), Represent the number of channels, height and width of the noise sample respectively; express The difference between adjacent pixels along the width direction, express The difference between adjacent pixels along the height direction, express The difference between adjacent pixels along the diagonal direction from top to bottom and from left to right, express Differences between adjacent pixels along the diagonal direction from top to bottom and from right to left; Step 3.8, Perform different linear transformations and get Query vector at time 、 Key vector at time and The value vector at time , and input into the ViT model for processing, we get Attention weight at the moment ; and The first row vector in is denoted as ; Step 3.9: Use formula (6) to obtain High information density graphics at all times ; (6) In formula (6), express Attention weight at the moment The first row vector The mask matrix of Step 3.10: Using formula (7), we can get Low information density primitives at the moment ; (7) Step 3.11: According to formula (8), build a federation center server for the The first model inversion Overall loss function ; (8) In formula (8), and Represents the scaling factor hyperparameters of JS divergence and regularization loss; Step 3.12: Local image model of each client Using loss function right Training, get Synthetic dataset for sub-model inversion and sparse primitive datasets ; Step 3.13, Assign to , then return to step 3.3 and execute sequentially until the maximum number of iterations is reached So far, The synthetic data inverted by each client is aggregated to obtain the federation center server Synthetic dataset after sub-model inversion and sparse primitive datasets , and build a federation center server according to formula (9) for The overall loss function of the sub-model inversion : (9) In formula (9), represents the set of trained local image models, and , Indicates the A local image model trained on the client; Step 4: The federation center server uses and right Perform knowledge distillation and graph relabeling to construct the loss function for knowledge distillation , classification loss of sparse primitives and primitive relabeling loss , thus obtaining the loss function ; Step 5: The federation center server is based on and Constructing the total loss function , used for global model Perform training to obtain a trained global model And sent to each client for image classification and recognition.

2. The method for federated learning image recognition based on model inversion and distillation according to claim 1, characterized in that: The step 1 comprises: No. The client will Input local image model Processed in the local image samples Belong to The probability of the categories ; , Represents the total number of categories; thus constructing the local image model according to formula (1) The cross entropy loss function , and use the stochastic gradient descent algorithm to optimize the local image model The parameters are updated and When it becomes stable, the trained local image model is obtained And upload it to the federation center server; (1) In formula (1), is a symbolic function, indicating the Client's local image samples If the category , indicating the local image samples Belong to categories, if , indicating the local image samples Does not belong to categories.

3. The federated learning image recognition method based on model inversion and distillation according to claim 2, characterized in that: The step 4 comprises: Step 4.1: Build an integrated model according to formula (10) , and Process and obtain right The integrated prediction probability ; (10) In formula (10), represents a set of parameters, and ; Step 4.2, As a teacher model, As a student model, the loss function of knowledge distillation is constructed according to formula (11) ; (11) In formula (11), express right The integrated prediction probability of express right The global prediction probability of Step 4.3: The federation center server uses the global model Compute sparse primitive datasets Medium and high information density graphics elements The predicted probability vector ;in, Represents the global model right Middle High-information-density graphics elements The predicted probability of Step 4.4: Build a global model according to formula (12) The classification loss ; (12) In formula (12), is a symbolic function, indicating Middle High-information-density graphics elements If the category , indicating the High-information-density graphics elements Belong to category ,like , indicating the High-information-density graphics elements Not in category ; Step 4.5: The federation center server uses the global model Compute sparse primitive datasets Low and medium information density graphics elements The predicted probability vector ,in, Represents the global model right Middle Low-information-density primitives The global prediction probability of The federation center server uses an integrated model Compute sparse primitive datasets Low and medium information density graphics elements The predicted probability vector ,in, express right Middle Low information density primitives The integrated prediction probability of Step 4.6: Construct the primitive re-labeling loss through formula (13) ; (13) Step 4.7: Build a global model according to formula (14) The loss function : (14) In formula (14), and Represents the scaling factor hyperparameter for the supervision loss and the relabeling loss.

4. The method for federated learning image recognition based on model inversion and distillation according to claim 3, characterized in that: The step 5 comprises: Step 5.1: Construct the total loss function using formula (15) and formula (16) : (15) (16) In formula (15) and formula (16), Express satisfaction The optimal noise sample set with the minimum time; Step 5.2: The federal central server and Alternate optimization to optimize the global model Parameters Train so that the total loss function reaches to the minimum and obtain the optimal parameters Global model .

5. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports a processor to execute the method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are performed.

Images