A multi-granularity federated learning approach for large models

By fine-tuning local model and distance-based personalized parameter aggregation on edge devices, combined with a cross-grain size guidance mechanism, the problem of high computational pressure and overfitting in traditional federated learning methods is solved, and the performance of large models on edge devices is improved. and accuracy.

CN117973501BActive Publication Date: 2024-11-15TIANJIN UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410110219.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-26
Publication Date
2024-11-15
Estimated Expiration
2044-01-26

AI Technical Summary

Technical Problem

Traditional federated learning methods cause high computing and communication pressure on edge devices, insufficient training data leads to overfitting, and cannot effectively measure the similarity between edge devices of the same granularity, and do not really fit the big model.

Method used

Using multi-grained federated learning methods, including local model fine-tuning, distance-based personalized parameter aggregation mechanism and cross-grained guidance mechanism, edge devices download pre-trained models for fine-tuning, use KL divergence to measure similarity, and interact through cross-grained guidance mechanism Prompt parameters to reduce calculation and communication pressure and improve model performance.

Benefits of technology

It reduces the computing and communication pressure of edge devices, alleviates the overfitting problem, improves the model performance in the non-independent and same-distribution environment of data, and improves the accuracy of edge device models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117973501B_ABST
    Figure CN117973501B_ABST
Patent Text Reader

Abstract

The present invention discloses a multi-granularity federated learning method for large models, which belongs to the field of edge computing and machine learning technology. The method includes the following steps: local model fine-tuning; distance-based personalized parameter aggregation mechanism; cross-granularity guidance mechanism; after the distance-based personalized parameter aggregation mechanism and the cross-granularity guidance mechanism, the personalized prompt parameters of other edge device knowledge are combined, and the edge device will download this personalized prompt parameter from the cloud server, and then start a new global round. The present invention reduces the computing and communication pressure of edge devices in federated learning, and can alleviate the overfitting problem caused by insufficient edge device data. It can also cope with edge scenarios with complex data, successfully solve the problem of non-independent and identically distributed data, and use the correlation of semantic knowledge of different granularities to improve the performance of edge device models.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of edge computing and machine learning technology, and in particular to a multi-granularity federated learning method for large models. Background Art

[0002] Recently, large models (LMs) have shown remarkable effectiveness in various fields. For example, the pre-trained visual language model CLIP has demonstrated great potential in various visual tasks. Leveraging its powerful representation capabilities, CLIP can be effectively adapted to downstream tasks. At the same time, an increasing number of edge devices, e.g., IoT devices, are providing most of the training data for LMs. In the traditional model training paradigm, the cloud server will first collect the raw data generated by the edge devices and then train the LM in a centralized strategy. However, this process may lead to significant communication pressure and potential user privacy leakage. Therefore, LM faces the risk of insufficient training data when adapting to downstream tasks in edge computing. Federated learning (FL) has emerged as a distributed training paradigm that can collaborate with edge devices to train shared models while leaving the raw data on the edge devices. However, in edge environments with thousands of edge devices, data heterogeneity and limited resources, such as computing resources and bandwidth, hinder the process of combining LM and FL. Specifically, data heterogeneity can be summarized as non-independent and identically distributed data and multi-granularity data. As Figure 1 As shown in Figure 1, the multi-granularity data on the edge device is organized into a hierarchical structure: the upper-level labels are considered coarse-grained labels, while the lower-level labels are considered fine-grained labels. Each coarse-grained label has at least one corresponding fine-grained label. In order to achieve effective FL and LM combination in large edge scenarios, we propose a new personalized and multi-granularity FL framework (PMG-FL), which provides personalized prompt parameters for each edge device while considering the interactions between edge devices of the same granularity and cross-granularity edge devices.

[0003] At present, the traditional federated learning method still has the following shortcomings: 1) In the traditional federated learning method, the edge device needs to fine-tune most of the parameters of the LM locally and upload and download these parameters to the cloud. This process will bring great computing and communication pressure, and insufficient training data on the edge device will lead to overfitting; 2) Personalized federated learning methods can provide ideas for solving the problem of non-independent and identically distributed data, but the existing personalized methods cannot measure the similarities between edge devices of the same granularity well; 3) Most of the current federated learning methods ignore the interaction between edge devices with data of different granularities. Some work has taken this into consideration, but it is only verified for ordinary convolutional neural networks and is not truly adapted to large models. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-granularity federated learning method for large models, which solves the problem that insufficient training data on edge devices can lead to overfitting, the similarity between edge devices of the same granularity cannot be well measured, and the problem that they are not truly adapted to large models.

[0005] To achieve the above object, the present invention provides a multi-granularity federated learning method for a large model, comprising the following steps:

[0006] S1, local model fine-tuning, the i-th edge device will first download the pre-trained visual language model CLIP from the server side, and use local data to fine-tune CLIP to adapt it to the corresponding task;

[0007] S2, personalized parameter aggregation mechanism based on distance. After completing the local model fine-tuning in step S1, the edge device will upload the trained prompt parameters to the cloud server to interact with the prompt parameters of other edge devices to form personalized prompt parameters;

[0008] S3, cross-granularity guidance mechanism, uses the sum of the corresponding fine-grained class prediction probabilities to guide the coarse-grained class prediction probability. In the guidance process from the coarse-grained edge device to the fine-grained edge device, the prediction probability of the coarse-grained class is used as the prior knowledge of the prediction probability of the corresponding fine-grained class;

[0009] S4. After the prompt parameters of the edge device pass through the distance-based personalized parameter aggregation mechanism and the cross-granularity guidance mechanism, a personalized prompt parameter combined with the knowledge of other edge devices will be formed on the cloud server. The edge device will download this personalized prompt parameter from the cloud server, and then start a new global round and re-execute steps S1, S2 and S3.

[0010] Preferably, in step S1, the local model fine-tuning adopts prompt training to freeze the text encoder and the image encoder during the training model, and only optimizes a small number of learnable prompt parameters, where the prompt parameters are composed of M continuous vectors, represented as M = 16, single vector The dimension is 512;

[0011] During training, the complete cue for the kth class can be expressed as where c k is the word embedding vector of the class name of the kth class, and given an image x i,,j , the predicted probability of its category is:

[0012]

[0013] Among them, cos[·|·] represents cosine similarity, V iIt is updated during the local back-propagation process, and the training loss function uses the cross entropy loss.

[0014] Preferably, in step S2, the distance-based personalized parameter aggregation mechanism is specifically as follows:

[0015] S21. For the i-th edge device, based on the predicted probability of the same data, KL divergence is used to measure the similarity between it and other devices holding the same granularity data, prompting parameter V i and V e In the shared sample The KL divergence on can be expressed as:

[0016]

[0017] Where KLD(·) is the KL divergence equation, is the sample of the ith edge device on the cloud shared dataset, is the predicted probability calculated using formula (1);

[0018] S22. Adjust the KL divergence equation to Calculate the KL divergence value on all samples and add it up as the similarity, which is expressed as and store it in the matrix , normalize the matrix row by row:

[0019]

[0020] To ensure that the similarity between the ith edge device and other edge devices is between 0 and 1, and to ensure that the sum of the similarities between the ith edge device and other edge devices is 1, the similarity matrix is ​​further operated, and the final similarity matrix is ​​as follows:

[0021] S23, cloud server based on similarity matrix A KL , aggregate the prompt parameters of devices with the same granularity data; for the i-th edge device, the personalized prompt parameters can be expressed as:

[0022]

[0023] Among them, ε is an optional set of edge devices. If the i-th edge device has coarse-grained data, this process only occurs between edge devices with coarse-grained data. If the i-th edge device has fine-grained data, this process only occurs between edge devices with fine-grained data.

[0024] Preferably, in step S3, the cross-granularity guidance mechanism is a bidirectional mechanism, including guidance from a fine-grained edge device to a coarse-grained edge device and from a coarse-grained edge device to a fine-grained edge device.

[0025] Preferably, in step S3, from fine-grained edge devices to coarse-grained edge devices, the overall loss function can be expressed as follows:

[0026]

[0027] in, ε c and ε f They represent the coarse-grained edge device set and the fine-grained edge device set respectively. KLD(·) is calculated by formula (2), is the predicted probability of the coarse-grained class, is the predicted probability of the corresponding fine-grained class after transformation;

[0028] Sum the probabilities of fine-grained classes and align the dimensions of the predicted probabilities of fine-grained classes and coarse-grained classes; select the best fine-grained edge device to guide the coarse-grained edge device based on the accuracy of all fine-grained edge devices on the shared dataset.

[0029] Preferably, in step S3, from the coarse-grained edge device to the fine-grained edge device, the loss function is the same as formula (5), where i∈ε f ,π∈ε c ,

[0030] Preferably, in step S3, The predicted probability of the coarse-grained class is used as the prior knowledge of the predicted probability of the corresponding fine-grained class, and the dimensions of the predicted probability of the fine-grained class and the predicted probability of the coarse-grained class are aligned. The process is as follows:

[0031]

[0032] in, It is fine-grained category, k is the kth coarse-grained category;

[0033] After the cross-granularity guidance mechanism, V i Further updates will be made on the cloud server:

[0034]

[0035] Therefore, the present invention adopts a multi-granularity federated learning method for a large model using the above structure, which has the following beneficial effects:

[0036] (1) The present invention reduces the computational and communication pressure of edge devices in federated learning, and applies prompt training during the model fine-tuning process of edge devices. This process only requires training a small number of learnable prompt parameters and can alleviate the overfitting problem caused by insufficient edge device data.

[0037] (2) The present invention can cope with edge scenarios with complex data. It proposes a distance-based personalized parameter aggregation mechanism between edge devices of the same granularity and a cross-granularity guidance mechanism between edge devices of different granularities. The former can successfully solve the problem of non-independent and identically distributed data, and the latter can utilize the correlation of semantic knowledge of different granularities to improve the performance of edge device models.

[0038] (3) For non-IID data, the test set accuracy of the present invention on CIFAR-100 and TinyImageNet exceeds that of other methods by 7.98% and 5.60% respectively.

[0039] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of a scenario architecture diagram in the specification of a multi-granularity federated learning method for a large model of the present invention;

[0041] Figure 2 This is a schematic diagram of an algorithm flow of a multi-granularity federated learning method for a large model according to the present invention;

[0042] Figure 3 A schematic diagram of a cross-granularity guidance mechanism flow of a multi-granularity federated learning method for a large model according to the present invention;

[0043] Figure 4 A schematic diagram of the coarse-grained edge device performance of CIFAR-100 for a multi-granularity federated learning method for a large model according to the present invention;

[0044] Figure 5 A schematic diagram of fine-grained edge device performance of CIFAR-100 for a multi-granularity federated learning method for a large model according to the present invention;

[0045] Figure 6 A schematic diagram of the coarse-grained edge device performance of TinyImageNet of a multi-granularity federated learning method for a large model according to the present invention;

[0046] Figure 7 A schematic diagram of fine-grained edge device performance of TinyImageNet for a multi-granularity federated learning method for a large model according to the present invention;

[0047] Figure 8 A schematic diagram of the convergence of coarse-grained edge devices of CIFAR-100 for a multi-granularity federated learning method for a large model according to the present invention;

[0048] Fig. 9 Schematic diagram of the coarse-grained edge device convergence of TinyImageNet of a multi-granularity federated learning method for large models in the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0051] Example

[0052] like Figure 2 As shown, the present invention provides a multi-granularity federated learning method for a large model, comprising the following steps:

[0053] S1, local model fine-tuning, the i-th edge device will first download the pre-trained visual language model CLIP from the server side, and use local data to fine-tune CLIP to adapt it to the corresponding task;

[0054] Local model fine-tuning uses prompt training to freeze the text encoder and image encoder during the training model, and only optimizes a small number of learnable prompt parameters. The prompt parameters are composed of M continuous vectors, expressed as M = 16, single vector The dimension is 512;

[0055] During training, the complete cue for the kth class can be expressed as where c k is the word embedding vector of the class name of the kth class (such as "dolphin"), and given an image x i,j , the predicted probability of its category is:

[0056]

[0057] Among them, cos[·|·] represents cosine similarity, V i It is updated during the local back-propagation process, and the training loss function uses the cross entropy loss.

[0058] Replacing the overall model parameters in traditional FL with a small number of learnable hint parameters can effectively reduce the computational and communication pressure. In addition, fast training can also solve the problem of insufficient data on edge devices and avoid overfitting. In short, the knowledge of edge devices is stored in hint parameters, and the interaction between edge devices at the same granularity and across granularities is based on these hint parameters.

[0059] S2, personalized parameter aggregation mechanism based on distance. After completing the local model fine-tuning in step S1, the edge device will upload the trained prompt parameters to the cloud server to interact with the prompt parameters of other edge devices to form personalized prompt parameters;

[0060] The specific mechanism of distance-based personalized parameter aggregation is as follows:

[0061] S21. For the i-th edge device, based on the predicted probability of the same data, KL divergence is used to measure the similarity between it and other devices holding the same granularity data, prompting parameter V i and V e In the shared sample The KL divergence on can be expressed as:

[0062]

[0063] Where KLD(·) is the KL divergence equation, is the sample of the ith edge device on the cloud shared dataset, is the predicted probability calculated using formula (1);

[0064] S22. Adjust the KL divergence equation to Calculate the KL divergence value on all samples and add it up as the similarity, which is expressed as and store it in the matrix , normalize the matrix row by row:

[0065]

[0066] To ensure that the similarity between the ith edge device and other edge devices is between 0 and 1, and to ensure that the sum of the similarities between the ith edge device and other edge devices is 1, the similarity matrix is ​​further operated, and the final similarity matrix is ​​as follows:

[0067] S23, cloud server based on similarity matrix A KL , aggregate the prompt parameters of devices with the same granularity data; for the i-th edge device, the personalized prompt parameters can be expressed as:

[0068]

[0069] Among them, ε is an optional set of edge devices. If the i-th edge device has coarse-grained data, this process only occurs between edge devices with coarse-grained data. If the i-th edge device has fine-grained data, this process only occurs between edge devices with fine-grained data.

[0070] S3, cross-granularity guidance mechanism, uses the sum of the corresponding fine-grained class prediction probabilities to guide the coarse-grained class prediction probability. In the guidance process from the coarse-grained edge device to the fine-grained edge device, the prediction probability of the coarse-grained class is used as the prior knowledge of the prediction probability of the corresponding fine-grained class;

[0071] like Figure 3 As shown, the cross-granularity guidance mechanism is a bidirectional mechanism, including guidance from fine-grained edge devices to coarse-grained edge devices and from coarse-grained edge devices to fine-grained edge devices;

[0072] From fine-grained edge devices to coarse-grained edge devices, the overall loss function can be expressed as follows:

[0073]

[0074] in, ε c and ε f They represent the coarse-grained edge device set and the fine-grained edge device set respectively. KLD(·) is calculated by formula (2), is the predicted probability of the coarse-grained class, is the predicted probability of the corresponding fine-grained class after transformation;

[0075] Sum the probabilities of fine-grained classes and align the dimensions of the predicted probabilities of fine-grained classes and coarse-grained classes; select the best fine-grained edge device to guide the coarse-grained edge device based on the accuracy of all fine-grained edge devices on the shared dataset.

[0076] From coarse-grained edge devices to fine-grained edge devices, the loss function is the same as formula (5), where i∈ε f ,π∈ε c ,

[0077] The predicted probability of the coarse-grained class is used as the prior knowledge of the predicted probability of the corresponding fine-grained class, and the dimensions of the predicted probability of the fine-grained class and the predicted probability of the coarse-grained class are aligned. The process is as follows:

[0078]

[0079] in, It is fine-grained category, k is the kth coarse-grained category;

[0080] After the cross-granularity guidance mechanism, V i Further updates will be made on the cloud server:

[0081]

[0082] S4. After the prompt parameters of the edge device pass through the distance-based personalized parameter aggregation mechanism and the cross-granularity guidance mechanism, a personalized prompt parameter combined with the knowledge of other edge devices will be formed on the cloud server. The edge device will download this personalized prompt parameter from the cloud server, and then start a new global round and re-execute steps S1, S2 and S3.

[0083] To demonstrate the effectiveness of the present invention, we verify it on two public datasets, CIFAR-100 and TinyImageNet. CIFAR-100 contains 50,000 training samples and 10,000 test samples, and for one sample, it has a finer-grained class name and a coarser-grained class name. Specifically, CIFAR-100 consists of 20 coarse-grained classes and 100 fine-grained classes, and each coarse-grained class contains an average of 5 fine-grained classes. TinyImageNet contains 120,000 samples of 200 classes, each with 500 training samples and 50 test samples. We take the original class as the fine-grained class and identify its corresponding coarse-grained class according to the category hierarchy in WordNet. In addition, we balance the entire dataset by selecting a subset of classes. The subset consists of 74 fine-grained classes and 25 coarse-grained classes. Each coarse-grained class corresponds to 2 to 5 fine-grained classes. During the training process, distance-based personalized cue aggregation and cross-granularity guidance mechanisms appear alternately.

[0084] Following the previous hint training method in FL, we sample 16 samples from each class. In the experimental setting, we have 5 edge devices with fine-grained data and 5 edge devices with coarse-grained data. Between edge devices with the same granularity data, two data distributions are considered: (1) Independent and Identically Distributed (IID). Each edge device randomly allocates 20% of the entire dataset, so the data distribution on each device is the same. (2) Non-Independent and Identically Distributed (non-IID). We first sort the data by label, and then divide the data into several shards. Each edge device randomly selects a specific shard, so each edge device holds a different class.

[0085] We compared three methods: Local, PromptFL, and PromptProx. Local means that each edge device uses its own dataset for training, PromptFL uses average aggregation, and PromptProx adds a regularization term during local training. Figure 4-Figure 7 The performance differences are illustrated. Our method outperforms other methods on CIFAR-100 and TinyImageNet in both IID and non-IID scenarios. On IID data distribution, the FL method shows significant improvement over Local trained on each edge device individually, indicating that collaboration between FL edge devices is beneficial for performance. In real-world edge environments, data distribution is often non-IID. Our method performs better than other methods. For CIFAR-100 with non-IID data, our method outperforms other methods by 7.98% and 2.12% on coarse-grained edge devices and 2.12% on fine-grained edge devices, respectively. For TinyImageNet with non-IID data, our method outperforms other methods by 5.60% and 2.43% on coarse-grained edge devices and 2.43% on fine-grained edge devices, respectively.

[0086] Figure 8-Figure 9 The training process results of this method are given, and it can be found that this patented method has rapid performance improvement and convergence in a short time. A noteworthy phenomenon is that although the performance of Local is very good, combined with Figure 6 and Figure 7 This is due to the overfitting phenomenon caused by a small amount of training data.

[0087] Therefore, the present invention adopts the above-mentioned multi-granularity federated learning method for large models, which reduces the computing and communication pressure of edge devices in federated learning, and applies prompt training in the model fine-tuning process of edge devices. This process only requires training a small number of learnable prompt parameters, and can alleviate the overfitting problem caused by insufficient edge device data. And it can cope with edge scenarios with complex data, and proposes a distance-based personalized parameter aggregation mechanism between edge devices of the same granularity, and proposes a cross-granularity guidance mechanism between cross-granularity edge devices. The former can successfully solve the problem of non-independent and identically distributed data, and the latter can utilize the correlation of semantic knowledge of different granularities to improve the performance of edge device models.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-granularity federated learning method for large models, characterized in that: The following steps are involved: S1, local model fine-tuning, the i-th edge device will first download the pre-trained visual language model CLIP from the server side, and use local data to fine-tune CLIP to adapt it to the corresponding task; S2, personalized parameter aggregation mechanism based on distance. After completing the local model fine-tuning in step S1, the edge device will upload the trained prompt parameters to the cloud server to interact with the prompt parameters of other edge devices to form personalized prompt parameters; In step S2, the distance-based personalized parameter aggregation mechanism is specifically as follows: S21. For the i-th edge device, based on the predicted probability of the same data, KL divergence is used to measure the similarity between it and other devices holding the same granularity data, prompting parameter V i and V e In the shared sample The KL divergence on can be expressed as: Where KLD(·) is the KL divergence equation, is the sample of the ith edge device on the cloud shared dataset, is the predicted probability calculated using formula (1); S22. Adjust the KL divergence equation to Calculate the KL divergence value on all samples and add it up as the similarity, which is expressed as and store it in the matrix , normalize the matrix row by row: To ensure that the similarity between the ith edge device and other edge devices is between 0 and 1, and to ensure that the sum of the similarities between the ith edge device and other edge devices is 1, the similarity matrix is ​​further operated, and the final similarity matrix is ​​as follows: S23, cloud server based on similarity matrix A KL , aggregate the prompt parameters of devices with the same granularity data; for the i-th edge device, the personalized prompt parameters can be expressed as: Where ε is an optional set of edge devices. If the i-th edge device has coarse-grained data, this process only occurs between edge devices with coarse-grained data. If the i-th edge device has fine-grained data, this process only occurs between edge devices with fine-grained data. S3, cross-granularity guidance mechanism, uses the sum of the corresponding fine-grained class prediction probabilities to guide the coarse-grained class prediction probability. In the guidance process from the coarse-grained edge device to the fine-grained edge device, the prediction probability of the coarse-grained class is used as the prior knowledge of the prediction probability of the corresponding fine-grained class; S4. After the prompt parameters of the edge device pass through the distance-based personalized parameter aggregation mechanism and the cross-granularity guidance mechanism, a personalized prompt parameter combined with the knowledge of other edge devices will be formed on the cloud server. The edge device will download this personalized prompt parameter from the cloud server, and then start a new global round and re-execute steps S1, S2 and S3.

2. The multi-granularity federated learning method for a large model according to claim 1, characterized in that: In step S1, the local model fine-tuning adopts prompt training to freeze the text encoder and image encoder during the training model, and only optimizes a small number of learnable prompt parameters. The prompt parameters are composed of M continuous vectors, represented as M = 16, single vector The dimension is 512; During training, the complete cue for the kth class can be expressed as where c k is the word embedding vector of the class name of the kth class, and given an image x i,j , the predicted probability of its category is: Among them, cos[·|·] represents cosine similarity, V i It is updated during the local back-propagation process, and the training loss function uses the cross entropy loss.

3. The multi-granularity federated learning method for a large model according to claim 2, characterized in that: In step S3 , the cross-granularity guidance mechanism is a bidirectional mechanism, including guidance from a fine-grained edge device to a coarse-grained edge device and from a coarse-grained edge device to a fine-grained edge device.

4. The multi-granularity federated learning method for a large model according to claim 3, characterized in that: In step S3, from fine-grained edge devices to coarse-grained edge devices, the overall loss function can be expressed as follows: in, π∈ε f , ε c and ε f They represent the coarse-grained edge device set and the fine-grained edge device set respectively. KLD(·) is calculated by formula (2). is the predicted probability of the coarse-grained class, is the predicted probability of the corresponding fine-grained class after transformation; Sum the probabilities of fine-grained classes and align the dimensions of the predicted probabilities of fine-grained classes and coarse-grained classes; select the best fine-grained edge device to guide the coarse-grained edge device based on the accuracy of all fine-grained edge devices on the shared dataset.

5. The multi-granularity federated learning method for a large model according to claim 4, characterized in that: In step S3, from the coarse-grained edge device to the fine-grained edge device, the loss function is the same as formula (5), where i∈ε f ,π∈ε c , 6. The multi-granularity federated learning method for a large model according to claim 5, characterized in that: In step S3, The predicted probability of the coarse-grained class is used as the prior knowledge of the predicted probability of the corresponding fine-grained class, and the dimensions of the predicted probability of the fine-grained class and the predicted probability of the coarse-grained class are aligned. The process is as follows: in, It is fine-grained category, k is the kth coarse-grained category; After the cross-granularity guidance mechanism, V i Further updates will be made on the cloud server:

Citation Information

Patent Citations

  • Method based on multi-granularity federated learning

    CN114154647A

  • Personalized federal learning method for double perception of metadata and image features

    CN115312158A

  • Federal learning optimization method and system based on reinforcement learning

    CN116523079A

  • Layered personalized federal learning method and device in edge computing network and medium

    CN116579417A

  • Federal learning method and device and computer readable storage medium

    CN116776969A