Federal learning image recognition method based on model inversion and distillation

Abstract

The invention discloses a federal learning image recognition method based on model inversion and distillation. The method comprises the following steps: 1, training a local image model on each client and uploading the local image model to a federal center server; 2, the federal center server receives the local image model and performs weighted aggregation; 3, the federal center server uses a local image model to carry out depth inversion so as to construct a synthetic data set; 4, the federal center server performs primitive re-marking and knowledge distillation by using the synthesized data set; and 5, performing iterative optimization on the global image model parameters to obtain a global model with optimal parameters, and issuing the global model to each client for image recognition. According to the method, data are synthesized by adopting a depth inversion method, generated primitives with different information densities are screened through attention weights, and primitives with low information densities are re-marked by utilizing an integrated model in a subsequent knowledge distillation stage, so that the image recognition problem of single communication federal learning in a data heterogeneous scene is solved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and involves technologies such as deep learning, federated learning and knowledge distillation. Specifically, it is a federated learning image recognition method based on model inversion and distillation. Background Art

[0002] With the booming development of big data-driven artificial intelligence technology, deep learning technology has ushered in a further development climax in the fields of image recognition, video surveillance, personalized recommendation, etc. The key factor behind this series of progress lies in the collection and utilization of massive data. However, in actual application scenarios, due to factors such as data security, privacy protection and legal constraints, the collection and centralized storage of private data are usually faced with great limitations. The implementation of various privacy protection policies and regulations has brought significant challenges to traditional centralized data processing and storage of deep learning technology. Federated learning, as an emerging distributed machine learning framework, has emerged to help multiple institutions perform data utilization and machine learning modeling while meeting the requirements of user privacy protection, data security and government regulations. It allows multiple participating clients to train models across distributed data sources without accessing the original data. At present, federated learning has been widely explored in various industries. Compared with traditional centralized data processing methods, federated learning can fully tap the value of distributed data while protecting user privacy. However, in terms of communication costs and data heterogeneity across clients, federated learning faces major challenges.

[0003] Although traditional methods can improve the performance of server-side models, the quality of synthesized data is relatively poor, and using this low-quality data in federated distillation limits the extent of performance enhancement. On the one hand, previous work has focused on training generators and using relatively simple generator model structures to synthesize data; on the other hand, facing the highly heterogeneous data challenges of federated learning, many generated data labels may not match the original image data, which will cause the global model to learn a lot of wrong knowledge in the subsequent distillation process, thus limiting performance improvement. Summary of the invention

[0004] In order to address the shortcomings of the above-mentioned prior art, the present invention proposes a federated learning image recognition method based on model inversion and distillation, in order to train a global model for image classification with higher performance under the condition of a single communication between the server and the client, thereby realizing privacy-preserving image recognition based on federated learning.

[0005] The present invention adopts the following technical solutions to solve the technical problems:

[0006] The federated learning image recognition method based on model inversion and distillation is characterized in that it is applied to In a system consisting of clients and a federation central server, Each client has a private local set of labeled image samples. , and Respectively represent Client's local image samples and their corresponding true labels, , is the total number of clients, , For the The total number of local image samples of the client; the federated learning image recognition method is performed in the following steps:

[0007] Step 1: Train the local image model on each client, obtain the trained local image model and upload it to the federation center server;

[0008] Step 2: The federation center server receives The trained local image models uploaded by the clients are weighted and aggregated to obtain the global model using formula (2): Parameters ;

[0009] (2)

[0010] In formula (2), Indicates The local image model trained by the client Parameters, express The weight of

[0011] Step 3: The federation center server receives The trained local image model uploaded by the client is used for deep inversion to construct the overall loss function of the model inversion. , synthetic dataset and sparse primitive datasets ;

[0012] Step 4: The federation center server uses and right Perform knowledge distillation and graph re-labeling to construct the loss function of knowledge distillation , classification loss of sparse primitives and primitive relabeling loss , thus obtaining the loss function ;

[0013] Step 5: The federation center server is based on and Constructing the total loss function , used for the global model Perform training to obtain a trained global model And sent to each client to realize image classification and recognition.

[0014] The federated learning image recognition method based on model inversion and distillation described in the present invention is also characterized in that the step 1 comprises:

[0015] No. The client will Input local image model is processed and the Local image samples Belong to The probability of the categories ; , Represents the total number of categories; thus, the local image model is constructed according to formula (1) The cross entropy loss function , and uses the stochastic gradient descent algorithm to optimise the local image model The parameters are updated and When it becomes stable, the trained local image model is obtained And upload to the federation center server;

[0016] (1)

[0017] In formula (1), is a symbolic function, indicating the Client's Local image samples If , indicating the Local image samples Belong to categories, if , indicating the Local image samples Not in categories.

[0018] Furthermore, the step 3 comprises:

[0019] Step 3.1: Randomly sample a batch of noise samples with true labels. ,in, represents a set of noise samples, Indicates noise samples, and conform to Gaussian distribution, express The true label set, express The real label, , is the total number of samples in a batch;

[0020] Step 3.2: Define the current time as , and initialize ,Will As The set of noise samples with true labels at all times is denoted as ;in, express The noise sample set at time, express The moment noise samples, and conform to Gaussian distribution, express The true label set, express The true label of

[0021] Step 3.3: The federated central server uses the local image model right After processing each noise sample in, we get Moment local prediction probability vector ;in, express right The local prediction probability of

[0022] Step 3.4: Construct according to formula (3) Moment local image model The classification loss ;

[0023] (3)

[0024] In formula (3), is a symbolic function, indicating At the moment of a batch Noise samples If the real category information , indicating the Noise samples Belong to categories, if , indicating the Noise samples Not in categories;

[0025] Step 3.5: The federation center server uses the global image model right After processing each noise sample in, we get The global prediction probability vector at time ;in, express right The global prediction probability of

[0026] Step 3.6: Construct according to formula (4) The moment JS divergence loss ;

[0027] (4)

[0028] In formula (4), represents KL divergence;

[0029] Step 3.7: Calculate according to formula (5) The L2 norm total variation regularization loss ;

[0030] (5)

[0031] In formula (5), Respectively represent the number of channels, height and width of the noise sample; express The difference between adjacent pixels along the width direction, express The difference between adjacent pixels along the height direction, express The difference between adjacent pixels along the diagonal direction from top to bottom and from left to right, express Differences between adjacent pixels along the diagonal direction from top to bottom and from right to left;

[0032] Step 3.8: Perform different linear transformations and get The query vector at the moment , The key vector at the moment and The value vector at time , and input into the ViT model for processing, we get Attention weight at each moment ; and The first row vector in is denoted as ;

[0033] Step 3.9: Use formula (6) to obtain High information density graphics at all times ;

[0034] (6)

[0035] In formula (6), express Attention weight at each moment The first row vector The mask matrix of

[0036] Step 3.10: Using formula (7), we get Low information density primitives at the moment ;

[0037] (7)

[0038] Step 3.11: According to formula (8), build a federation central server for The first model inversion The overall loss function ;

[0039] (8)

[0040] In formula (8), and Represents the scaling factor hyperparameters of JS divergence and regularization loss;

[0041] Step 3.12: Local image model of each client Using loss function right Training, get Synthetic dataset for sub-model inversion and sparse primitive datasets ;

[0042] Step 3.13: Assign to , then return to step 3.3 and execute sequentially until the maximum number of iterations is reached So far, The synthetic data inverted by each client is aggregated to obtain the federation center server Synthetic dataset after sub-model inversion and sparse primitive datasets , and build a federation central server according to formula (9) for The overall loss function of the sub-model inversion :

[0043] (9)

[0044] In formula (9), represents the set of trained local image models, and . Indicates A local image model trained on a client.

[0045] Further, the step 4 comprises:

[0046] Step 4.1: Build an integrated model according to formula (10) , and Process and obtain right The ensemble prediction probability ;

[0047] (10)

[0048] In formula (10), represents a set of parameters, and ;

[0049] Step 4.2: As a teacher model, As the student model, the loss function of knowledge distillation is constructed according to formula (11) ;

[0050] (11)

[0051] In formula (11), express right The integrated prediction probability of express right The global prediction probability of

[0052] Step 4.3: The federation center server uses the global image model Compute sparse primitive datasets Medium and high information density graphics The predicted probability vector ;in, Represents the global image model right Middle High Information Density Element The predicted probability of

[0053] Step 4.4: Build a global model according to formula (12) The classification loss ;

[0054] (12)

[0055] In formula (12), is a symbolic function, indicating Middle High Information Density Element If , indicating the High Information Density Element Belongs to category ,like , indicating the High Information Density Element Not in category ;

[0056] Step 4.5: The federation center server uses the global model Compute sparse primitive datasets Low and medium information density primitives The predicted probability vector ,in, Represents the global image model right Middle Low-information-density primitives The global prediction probability of

[0057] The federation central server uses an integrated model Compute sparse primitive datasets Low and medium information density primitives The predicted probability vector ,in, express right Middle Low Information Density Element The integrated prediction probability of

[0058] Step 4.6: Construct the primitive re-labeling loss through formula (13) ;

[0059] (13)

[0060] Step 4.7: Construct a global model according to formula (14) The loss function :

[0061] (14)

[0062] In formula (14), and Represents the scaling factor hyperparameter for the supervision loss and the relabeling loss.

[0063] Further, the step 5 comprises:

[0064] Step 5.1: Construct the total loss function using equations (15) and (16) :

[0065] (15)

[0066] (16)

[0067] In formula (15) and formula (16), Express satisfaction The optimal noise sample set with the smallest amount of time;

[0068] Step 5.2: The federal central server and Alternate optimization to optimize the global image Parameters Train so that the total loss function reaches to the minimum and obtain the optimal parameters The global model .

[0069] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the method, and the processor is configured to execute the program stored in the memory.

[0070] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the method when executed by a processor.

[0071] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0072] 1. This paper introduces the Vision Transformer (ViT) model into the federated learning task of single communication for the first time, and proposes an image recognition method based on model inversion and distillation, that is, deep model inversion is performed on the local ViT models trained by multiple clients to obtain pseudo data, and then knowledge distillation from the integrated model to the global model is performed based on the obtained data information, instead of directly training the traditional generator and then synthesizing the data, thereby solving the problem of low quality of the synthetic image data set and improving the accuracy and robustness of the image classification and recognition model.

[0073] 2. The core of this invention is to use the attention value of the classification label to evaluate the quality of the generated image, thereby realizing weighted distillation. Many previous works have used server-side knowledge distillation to improve the global model, mainly focusing on designing loss functions and training generators on the server to generate data. For environments with heterogeneous labels, under the framework of model distillation using synthetic datasets, a method of model inversion and primitive re-labeling is proposed to solve the client offset problem in extreme label heterogeneity, while improving the label and data mismatch problem of synthetic image datasets.

[0074] 3. The present invention comprehensively considers the extreme data heterogeneity problem in federated learning. Each local client device independently trains a local model, and the model is uniformly processed on the server side. The integrated method can be effectively applied to capture global information across clients. This single-communication federated learning is more in line with the current model market environment and is more practical in actual image recognition scenarios where image data is more complex. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0076] In this embodiment, a federated learning image recognition method based on model inversion and distillation is applied to In a system consisting of clients and a federation central server, Each client has a private local set of labeled image samples. , and Respectively represent Client's local image samples and their corresponding true labels, , is the total number of clients, , For the The total number of local image samples of each client; In this embodiment, public image datasets such as CIFAR10 and OfficeHome are used. , the image dataset is segmented using a method based on Dirichlet distribution , for each label, the label category The data is distributed to the client The ratio is .parameter Control the degree of data imbalance, The smaller the value, the more unbalanced the label distribution among local clients. Figure 1 , the federated learning image recognition method is carried out in the following steps:

[0077] Step 1: Train the local image model on each client and upload it to the federation center server;

[0078] No. The client will Input local image model is processed and the Local image samples Belong to The probability of the categories ; , Represents the total number of categories; in this embodiment, for the dataset CIFAR10, , for the dataset OfficeHome, ; Thus, the local image model is constructed according to formula (1) The cross entropy loss function , and uses the stochastic gradient descent algorithm to optimise the local image model The parameters are updated and When it becomes stable, the trained local image model is obtained And upload it to the federation center server; in this embodiment, DeiT / 16-Tiny pre-trained on ImageNet-1K is used as the local image model of the client;

[0079] (1)

[0080] In formula (1), is a symbolic function, indicating the Client's Local image samples If , indicating the Local image samples Belong to categories, if , indicating the Local image samples Not in categories.

[0081] Step 2: The federation center server receives The trained local image models uploaded by the clients are weighted and aggregated to obtain the global model using formula (2): Parameters ;

[0082] (2)

[0083] in, Represents the trained local image model Parameters, express The weight of .

[0084] Step 3: The federation center server receives The trained local image model uploaded by each client is used for depth inversion to construct a synthetic dataset;

[0085] Step 3.1: Randomly sample a batch of noise samples with true labels. ,in, represents a set of noise samples, Indicates noise samples, and conform to Gaussian distribution, express The true label set, express The real label, , is the total number of samples in a batch; in this embodiment, .

[0086] Step 3.2: Define the current time as , and initialize ,Will As The set of noise samples with true labels at all times is denoted as ;in, express The noise sample set at time, express The moment noise samples, and conform to Gaussian distribution, express The true label set, express The real label.

[0087] Step 3.3: The federated central server uses the local image model right After processing each noise sample in, we get Moment local prediction probability vector ;in, express right The local prediction probability of .

[0088] Step 3.4: Construct according to formula (3) Moment local image model The classification loss ;

[0089] (3)

[0090] In formula (3), is a symbolic function, indicating At the moment of a batch Noise samples If the real category information , indicating the Noise samples Belong to categories, if , indicating the Noise samples Not in categories.

[0091] Step 3.5: The federated central server uses the global image model right After processing each noise sample in, we get The global prediction probability vector at time ;in, express right The global prediction probability of .

[0092] Step 3.6: Construct according to formula (4) The moment JS divergence loss ;

[0093] (4)

[0094] In formula (4), represents the KL divergence.

[0095] Step 3.7: Calculate according to formula (5) The L2 norm total variation regularization loss ;

[0096] (5)

[0097] In formula (5), Respectively represent the number of channels, height and width of the noise sample; express The difference between adjacent pixels along the width direction, express The difference between adjacent pixels along the height direction, express The difference between adjacent pixels along the diagonal direction from top to bottom and from left to right, express Differences between adjacent pixels along the diagonal direction from top to bottom and from right to left.

[0098] Step 3.8: Perform different linear transformations and get The query vector at the moment , The key vector at the moment and The value vector at time , and input into the ViT model for processing, we get Attention weight at each moment ; and The first row vector is denoted as .

[0099] Step 3.9: Use formula (6) to obtain High information density graphics at all times ;

[0100] (6)

[0101] In formula (6), express Attention weight at each moment The first row vector In this embodiment, the screening ratio of the mask matrix is ​​set to , The role of is to evaluate and sort the importance of sparse primitives, and then stop the inversion of low attention weight primitives in proportion, so as to divide the primitives into high information density primitives and low information density primitives for subsequent primitive re-labeling.

[0102] Step 3.10: Using formula (7), we get Low information density primitives at the moment ;

[0103] (7)

[0104] Step 3.11: According to formula (8), build a federation central server for The first model inversion The overall loss function ;

[0105] (8)

[0106] In formula (8), and represents the scaling factor hyperparameter of JS divergence and regularization loss; in this embodiment, the optimized loss function Using Adam optimizer, learning rate , hyperparameters Setting a negative value encourages synthetic images to cause divergence between the student model and the teacher model and enhance image diversity. , .

[0107] Step 3.12: Local image model of each client Using loss function right Training, get Synthetic dataset for sub-model inversion and sparse primitive datasets ;

[0108] Step 3.13: Assign to , then return to step 3.3 and execute sequentially until the maximum number of iterations is reached So far, in this embodiment, the number of iterations ;Will The synthetic data inverted by each client is aggregated to obtain the federation center server Synthetic dataset after sub-model inversion and sparse primitive datasets , and build a federation central server according to formula (9) for The overall loss function of the sub-model inversion ;

[0109] (9)

[0110] In formula (9), represents the set of trained local image models, and , Indicates A local image model trained on a client.

[0111] Step 4: The federated central server uses the data set and right Perform knowledge distillation and primitive re-labeling;

[0112] Step 4.1: Build an integrated model according to formula (10) , and Process and obtain right The ensemble prediction probability ;

[0113] (10)

[0114] In formula (10), represents a set of parameters, and .

[0115] Step 4.2: As a teacher model, As the student model, the loss function of knowledge distillation is constructed according to formula (11): ;

[0116] (11)

[0117] In formula (11), express right The integrated prediction probability of express right The global prediction probability of .

[0118] Step 4.3: The federated central server uses the global image model Compute sparse primitive datasets Medium and high information density graphics The predicted probability vector ;in, Represents the global image model right Middle High Information Density Element The predicted probability of .

[0119] Step 4.4: Build a global model according to formula (12) The classification loss ;

[0120] (12)

[0121] In formula (12), is a symbolic function, indicating Middle High Information Density Element If , indicating the High Information Density Element Belongs to category ,like , indicating the High Information Density Element Not in category ;

[0122] Step 4.5: The federation central server uses the global model Compute sparse primitive datasets Low and medium information density primitives The predicted probability vector ,in, Represents the global image model right Middle Low-information-density primitives The predicted probability of

[0123] Federated central server utilization integration model Compute sparse primitive datasets Low and medium information density primitives The predicted probability vector ,in, express right Middle Low Information Density Element The predicted probability of

[0124] Step 4.6: Construct the primitive re-labeling loss through formula (13) ;

[0125] (13)

[0126] Step 4.7: Construct a global model according to formula (14) The loss function :

[0127] (14)

[0128] In formula (14), and Represents the scaling factor hyperparameter of the supervision loss and the relabeling loss; in this example, the optimized loss function Using SGD optimizer, learning rate , hyperparameters , the distillation temperature is set to ,The purpose of this process optimization is to utilize the primitives with different information density generated in the ,model inversion stage while distilling knowledge, use synthetic labels to ,assist supervise the training of the global model for the primitives with high ,information density, and use the integrated model to re-label the primitives with

[0129] Step 5: The federation center server checks the global model Perform training to obtain a trained global model And sent to each client to realize image classification and recognition;

[0130] Step 5.1: Construct the total loss function using equations (15) and (16) :

[0131] (15)

[0132] (16)

[0133] In formula (15) and formula (16), Express satisfaction The optimal noise sample set with the smallest amount of time;

[0134] Step 5.2: The federation center server and Alternate optimization to optimize the global image Parameters Train so that the total loss function reaches to the minimum and obtain the optimal parameters The global model .

[0135] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0136] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0137] Example:

[0138] In order to verify the effectiveness of the method of the present invention, this embodiment selected commonly used public data sets such as CIFAR10 and OfficeHome, and adopted the test accuracy of the global model as the quantitative evaluation standard.

[0139] In this embodiment, five methods are selected to compare the effects with the method of the present invention. The selected methods are FedAvg, FedFTG, DENSE, Co-Boosting and DeepInversion. The first four are federated learning methods. FedAvg is the basic federated learning algorithm in the paper McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deepnetworks from decentralized data[C] / / Artificial intelligence and statistics.2017: 1273-1282. The other three are federated learning algorithms based on synthetic data and knowledge distillation. FedFTG comes from the paper Zhang L, Shen L, Ding L, et al. Fine-tuning global modelvia data-free knowledge distillation for non-iid federated learning[C] / / Proceedings of the IEEE / CVF conference on computer vision and patternrecognition. 2022: 10174-10183, and DENSE comes from the paper Zhang J, Chen C, Li B, et al.Dense: Data-free one-shot federated learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 21414-21428, Co-Boosting comes from the paper Dai R, Shen L, He F, et al. Dispfl: Towards communication-efficient personalized federated learning via decentralized sparse training[C] / / International conference on machine learning. 2022: 4587-4604, DeepInversion is a deep model inversion method, from the paper Yin H, Molchanov P, Alvarez JM, et al.Dreaming to distill: Data-free knowledge transfer via deepinversion[C] / / Proceedings of the IEEE / CVFconference on computer vision and pattern recognition. 2020: 8715-8724, can be applied to federated learning settings, and the experimental results are shown in Table 1 and Table 2:.

[0140] Table 1 Different Dirichlet sampling distribution data heterogeneity levels (parameters The lower the value, the higher the heterogeneity level.) Experimental results of the method of the present invention and the five selected comparative methods on the CIFAR10 dataset:

[0141]

[0142] Table 2 Different Dirichlet sampling distribution data heterogeneity levels (parameters The lower the value, the higher the heterogeneity level.) Experimental results of the proposed method and the five selected comparative methods on the OfficeHome dataset:

[0143]

[0144] The experimental results show that under the same experimental environment settings, the performance of the method of the present invention exceeds that of the other five methods, especially when the heterogeneity of the Dirichlet distribution is high, the performance is greatly improved, thereby proving the feasibility of the method proposed in the present invention.

Claims

1. A federated learning image recognition method based on model inversion and distillation, characterized in that: is applied to In a system consisting of clients and a federation central server, Each client has a private local set of labeled image samples. , and Respectively represent Client's local image samples and their corresponding true labels, , is the total number of clients, , For the The total number of local image samples of the client; the federated learning image recognition method is performed in the following steps: Step 1: Train the local image model on each client, obtain the trained local image model and upload it to the federation center server; Step 2: The federation center server receives The trained local image models uploaded by the clients are weighted and aggregated to obtain the global model using formula (2): Parameters ; (2) In formula (2), Indicates The local image model trained by the client Parameters, express The weight of Step 3: The federation center server receives The trained local image model uploaded by the client is used for deep inversion to construct the overall loss function of the model inversion. , synthetic dataset and sparse primitive datasets ; Step 4: The federation center server uses and right Perform knowledge distillation and graph re-labeling to construct the loss function of knowledge distillation , classification loss of sparse primitives and primitive relabeling loss , thus obtaining the loss function ; Step 5: The federation center server is based on and Constructing the total loss function , used for the global model Perform training to obtain a trained global model And sent to each client to realize image classification and recognition.

2. The method for federated learning image recognition based on model inversion and distillation according to claim 1, characterized in that: The step 1 comprises: No. The client will Input local image model is processed and the Local image samples Belong to The probability of the categories ; , Represents the total number of categories; thus, the local image model is constructed according to formula (1) The cross entropy loss function , and uses the stochastic gradient descent algorithm to optimise the local image model The parameters are updated and When it becomes stable, the trained local image model is obtained And upload to the federation center server; (1) In formula (1), is a symbolic function, indicating the Client's Local image samples If , indicating the Local image samples Belong to categories, if , indicating the Local image samples Not in categories.

3. The method for federated learning image recognition based on model inversion and distillation according to claim 2, characterized in that: The step 3 comprises: Step 3.1: Randomly sample a batch of noise samples with true labels. ,in, represents a set of noise samples, Indicates noise samples, and conform to Gaussian distribution, express The true label set, express The real label, , is the total number of samples in a batch; Step 3.2: Define the current time as , and initialize ,Will As The set of noise samples with true labels at all times is denoted as ;in, express The noise sample set at time, express The moment noise samples, and conform to Gaussian distribution, express The true label set, express The true label of Step 3.3: The federated central server uses the local image model right After processing each noise sample in, we get Moment local prediction probability vector ;in, express right The local prediction probability of Step 3.4: Construct according to formula (3) Moment local image model The classification loss ; (3) In formula (3), is a symbolic function, indicating At the moment of a batch Noise samples If the real category information , indicating the Noise samples Belong to categories, if , indicating the Noise samples Not in categories; Step 3.5: The federation center server uses the global image model right After processing each noise sample in, we get The global prediction probability vector at time ;in, express right The global prediction probability of Step 3.6: Construct according to formula (4) The moment JS divergence loss ; (4) In formula (4), represents KL divergence; Step 3.7: Calculate according to formula (5) The L2 norm total variation regularization loss ; (5) In formula (5), Respectively represent the number of channels, height and width of the noise sample; express The difference between adjacent pixels along the width direction, express The difference between adjacent pixels along the height direction, express The difference between adjacent pixels along the diagonal direction from top to bottom and from left to right, express Differences between adjacent pixels along the diagonal direction from top to bottom and from right to left; Step 3.8: Perform different linear transformations and get The query vector at the moment , The key vector at the moment and The value vector at time , and input into the ViT model for processing, we get Attention weight at each moment ; and The first row vector in is denoted as ; Step 3.9: Use formula (6) to obtain High information density graphics at all times ; (6) In formula (6), express Attention weight at each moment The first row vector The mask matrix of Step 3.10: Using formula (7), we get Low information density primitives at the moment ; (7) Step 3.11: According to formula (8), build a federation central server for The first model inversion The overall loss function ; (8) In formula (8), and Represents the scaling factor hyperparameters of JS divergence and regularization loss; Step 3.12: Local image model of each client Using loss function right Training, get Synthetic dataset for sub-model inversion and sparse primitive datasets ; Step 3.13: Assign to , then return to step 3.3 and execute sequentially until the maximum number of iterations is reached So far, The synthetic data inverted by each client is aggregated to obtain the federation center server Synthetic dataset after sub-model inversion and sparse primitive datasets , and build a federation central server according to formula (9) for The overall loss function of the sub-model inversion : (9) In formula (9), represents the set of trained local image models, and . Indicates A local image model trained on a client.

4. The method for federated learning image recognition based on model inversion and distillation according to claim 3, characterized in that: The step 4 comprises: Step 4.1: Build an integrated model according to formula (10) , and Process and obtain right The ensemble prediction probability ; (10) In formula (10), represents a set of parameters, and ; Step 4.2: As a teacher model, As the student model, the loss function of knowledge distillation is constructed according to formula (11) ; (11) In formula (11), express right The integrated prediction probability of express right The global prediction probability of Step 4.3: The federation center server uses the global image model Compute sparse primitive datasets Medium and high information density graphics The predicted probability vector ;in, Represents the global image model right Middle High Information Density Element The predicted probability of Step 4.4: Build a global model according to formula (12) The classification loss ; (12) In formula (12), is a symbolic function, indicating Middle High Information Density Element If , indicating the High Information Density Element Belongs to category ,like , indicating the High Information Density Element Not in category ; Step 4.5: The federation center server uses the global model Compute sparse primitive datasets Low and medium information density primitives The predicted probability vector ,in, Represents the global image model right Middle Low-information-density primitives The global prediction probability of The federation central server uses an integrated model Compute sparse primitive datasets Low and medium information density primitives The predicted probability vector ,in, express right Middle Low Information Density Element The integrated prediction probability of Step 4.6: Construct the primitive re-labeling loss through formula (13) ; (13) Step 4.7: Construct a global model according to formula (14) The loss function : (14) In formula (14), and Represents the scaling factor hyperparameter for the supervision loss and the relabeling loss.

5. The method for federated learning image recognition based on model inversion and distillation according to claim 4, characterized in that: The step 5 comprises: Step 5.1: Construct the total loss function using equations (15) and (16) : (15) (16) In formula (15) and formula (16), Express satisfaction The optimal noise sample set with the smallest amount of time; Step 5.2: The federal central server and Alternate optimization to optimize the global image Parameters Train so that the total loss function reaches to the minimum and obtain the optimal parameters The global model .

6. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports a processor to execute the method according to any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are performed.

Images