Federated learning methods, apparatuses, devices, and media

Abstract

The present disclosure provides a federated learning method, device, equipment and medium. It relates to the technical field of data processing, and particularly relates to the technical field of artificial intelligence, deep learning and the like. The specific implementation scheme is: obtaining performance parameters of a plurality of edge devices; performing cluster analysis on the plurality of edge devices based on the performance parameters of the plurality of edge devices to obtain a plurality of device clusters; performing knowledge distillation on a cloud global model for the plurality of device clusters to distill a plurality of first sub-models; and distributing the plurality of first sub-models to the corresponding device clusters for federated learning. The embodiment of the present disclosure can divide a plurality of device clusters through cluster analysis, distill adaptive models for different device clusters through the mode of knowledge distillation, and reduce the communication burden.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to the fields of artificial intelligence and deep learning. Background Technology

[0002] Artificial intelligence technology is developing rapidly, and deep learning technology, as one of the most important technologies, requires a huge amount of data as its foundation.

[0003] However, traditional machine learning involves collecting data and then centrally training it. This poses a significant threat to privacy. Federated learning emerged to address this issue. Unlike previous distributed machine learning techniques, federated learning does not collect user data. Instead, it keeps this data locally, allowing user devices to train machine learning models on their own and then upload the trained models to a server. In this way, data remains local, ensuring data security. However, how to implement federated learning still warrants further research. Summary of the Invention

[0004] This disclosure provides a federated learning method, apparatus, device, and storage medium.

[0005] According to one aspect of this disclosure, a federated learning method is provided, comprising:

[0006] Obtain performance parameters from multiple edge devices;

[0007] Cluster analysis is performed on multiple edge devices based on their performance parameters to obtain multiple device clusters;

[0008] Knowledge distillation is performed on the global model in the cloud for multiple device clusters to distill multiple first sub-models;

[0009] Multiple first sub-models are distributed to corresponding device clusters for federated learning.

[0010] According to another aspect of this disclosure, a federated learning method is provided, comprising:

[0011] The performance parameters of the current edge device are sent to the server, so that the server can perform cluster analysis on multiple edge devices based on the performance parameters of multiple edge devices to obtain multiple device clusters, and perform knowledge distillation on the global model in the cloud to obtain a first sub-model applicable to the target device cluster; where the target device cluster is the device cluster to which the current edge device belongs;

[0012] Federated learning is based on the first sub-model.

[0013] According to another aspect of this disclosure, a federated learning apparatus is provided, comprising:

[0014] The acquisition module is used to acquire performance parameters of multiple edge devices;

[0015] The clustering module is used to perform cluster analysis on multiple edge devices based on their performance parameters, resulting in multiple device clusters.

[0016] The distillation module is used to perform knowledge distillation on the global model in the cloud for multiple device clusters, so as to distill out multiple first sub-models;

[0017] The learning module is used to distribute multiple first sub-models to corresponding device clusters for federated learning.

[0018] According to another aspect of this disclosure, a federated learning apparatus is provided, comprising:

[0019] The sending module is used to send the performance parameters of the current edge device to the server, so that the server can perform cluster analysis on multiple edge devices based on the performance parameters of multiple edge devices to obtain multiple device clusters, and perform knowledge distillation on the global model in the cloud to obtain a first sub-model applicable to the target device cluster; wherein, the target device cluster is the device cluster to which the current edge device belongs;

[0020] The training module is used for federated learning based on the first sub-model.

[0021] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0022] At least one processor; and

[0023] The memory is communicatively connected to the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0025] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0026] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0027] This disclosure embodiment can divide heterogeneous edge models into multiple device clusters through cluster analysis, and distill suitable models for different device clusters for training through knowledge distillation, thereby reducing communication burden.

[0028] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0029] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0030] Figure 1 This is a flowchart illustrating a federated learning method according to an embodiment of the present disclosure;

[0031] Figure 2 This is a schematic diagram of the process of guiding each device cluster to generate an augmented dataset based on a global student model and a second sub-model of each device cluster according to an embodiment of the present disclosure.

[0032] Figure 3 This is a schematic diagram of the pseudo-sample update process according to an embodiment of this disclosure;

[0033] Figure 4 This is a flowchart illustrating another federated learning method according to an embodiment of the present disclosure;

[0034] Figure 5 This is a schematic diagram illustrating the generation of amplified samples according to a diffusion model of an embodiment of this disclosure;

[0035] Figure 6 This is a framework diagram of federated learning according to an embodiment of the present disclosure;

[0036] Figure 7 This is a schematic diagram of the structure of a federated learning device according to an embodiment of the present disclosure;

[0037] Figure 8 This is a schematic diagram of the structure of a federated learning device according to another embodiment of the present disclosure;

[0038] Figure 9 This is a block diagram of an electronic device used to implement the federated learning method of the embodiments of this disclosure. Detailed Implementation

[0039] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0040] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means two or more, unless otherwise explicitly specified.

[0041] The continuous development of science and technology has brought about increasingly powerful devices, such as smartphones, tablets, and smartwatches, which, while providing numerous conveniences, also collect vast amounts of data. On the other hand, artificial intelligence technology is also developing rapidly. Deep learning, as one of the most important technologies in this field, requires a massive amount of data as its foundation. The data from these smart devices is undoubtedly very attractive.

[0042] However, traditional machine learning involves collecting data and then centrally training it, posing a significant threat to the privacy of this information. Federated learning offers a solution. Federated learning is a distributed machine learning technology that, unlike previous methods, doesn't collect user data. Instead, it keeps this data locally, allowing user devices to train machine learning models on-premises and then upload the trained models to a server. This approach ensures data security by keeping the data on-premises and significantly reduces communication overhead by only transmitting model parameters.

[0043] However, in federated learning systems, the more devices involved in computation, the more resources are available. As computational and data resources increase, model size also expands, leading to the emergence of large models. Large models are driven by big data, possess massive parameters, and have broad application capabilities. They require processing large amounts of data, and the increasing model size gradually raises computational costs. Furthermore, the efficiency of large models on device hardware is limited. Due to their high computational complexity and resource requirements, deploying them on resource-constrained devices, such as mobile phones or IoT devices, can be challenging. This typically leads to a trade-off between model performance and device feasibility.

[0044] In view of this, in order to effectively utilize the computing resources and data on the device to train large models, embodiments of this disclosure provide a federated learning method. For example... Figure 1 The diagram shown is a flowchart of the method, including the following:

[0045] S101, obtain performance parameters of multiple edge devices.

[0046] Edge devices include smart terminals and computers.

[0047] Performance parameters of edge devices describe their data processing capabilities. These parameters may include computing resources (such as CPU (Central Processing Unit) information, GPU (Graphics Processing Unit) information, and memory information), storage resources (such as disk space), and network resources (such as bandwidth).

[0048] S102, based on the performance parameters of multiple edge devices, performs cluster analysis on multiple edge devices to obtain multiple device clusters.

[0049] This disclosure performs cluster analysis based on the performance parameters of edge devices, which can group edge devices with similar performance into the same device cluster. This disclosure does not limit the cluster analysis method; for example, k-means clustering can be used.

[0050] S103 performs knowledge distillation on the global model in the cloud for multiple device clusters to distill multiple first sub-models.

[0051] For example, the global model in the cloud is a large-scale model. Large models have high performance requirements for devices, and not all edge devices can support such a large model. Therefore, when multiple device clusters are obtained through clustering, a first sub-model suitable for each device cluster can be distilled based on the performance parameters of each device cluster.

[0052] S104 distributes multiple first sub-models to the corresponding device clusters for federated learning.

[0053] Different edge devices exhibit varying performance characteristics. For heterogeneous edge devices, embodiments of this disclosure use clustering analysis to group edge devices with similar performance into the same device cluster. Each device cluster can support the execution of the same first sub-model. Therefore, for cloud-based global models that cannot be directly run on edge devices, a first sub-model suitable for each device cluster can be distilled. Thus, the number of distilled first sub-models depends on the number of device clusters, rather than requiring a separate sub-model to be distilled for each edge device, thereby reducing the computational load of distillation operations. Embodiments of this disclosure use knowledge distillation to lightweight the cloud-based global model and distribute it to edge devices. The lightweight model has fewer parameters than the cloud-based global model, thereby reducing the system communication burden. Furthermore, embodiments of this disclosure employ a federated learning method to protect privacy information security. In summary, this disclosure provides a general and efficient federated learning scheme suitable for heterogeneous edge devices.

[0054] Federated learning requires training using data from edge devices. Regarding the data required for training the model disclosed herein, the acquisition, storage, and application of user personal information involved in the technical solution of this disclosure all comply with relevant laws and regulations and do not violate public order and good morals.

[0055] To facilitate understanding, the distillation of the first sub-model will be explained in detail below, including two processes: cluster analysis and distillation of the first sub-model.

[0056] 1) Cluster analysis

[0057] In some embodiments, multiple device clusters are obtained through cluster analysis, which can be implemented as follows:

[0058] Step A1: Determine the performance similarity between edge devices based on the performance parameters of multiple edge devices;

[0059] Step A2 involves performing cluster analysis based on the performance similarity between edge devices to obtain multiple device clusters.

[0060] For example, devices can be clustered based on their CPU performance, memory capacity, and disk space. In practice, the similarity between edge devices is calculated based on the numerical characteristics of these metrics, and similar devices are grouped into the same device cluster.

[0061] In practice, either the cosine similarity between different performance parameters can be calculated as the similarity between different edge devices, or the distance between different performance parameters can be calculated as the similarity between different edge devices.

[0062] For example, clustering all edge devices D based on predefined rules yields a set of device clusters. Where t represents the number of device clusters. In this embodiment, when an edge device connects to the network, it can send predefined information (i.e., performance parameters) to the server. The server randomly selects t samples from the collected performance parameters of the edge devices as cluster centers, where t is a positive integer greater than 0. Then, it calculates the distance from each performance parameter to the cluster center (this distance represents the similarity between edge devices). For each performance parameter, it is classified into the device cluster containing the nearest cluster center. Then, for each device cluster, the average value of all performance parameters in the cluster is recalculated, and this average value is used as the new cluster center. The above operation is repeated to classify each edge device into the nearest cluster until the cluster center no longer changes or the maximum number of iterations is reached, thereby obtaining multiple device clusters.

[0063] In this embodiment of the disclosure, performance similarity can be used to measure the performance differences between different edge devices. Thus, in the case of heterogeneous edge devices, edge devices with similar performance can be accurately classified into the same cluster, so as to facilitate federated learning in heterogeneous edge devices.

[0064] 2) Distill the first sub-model for each equipment cluster.

[0065] In some embodiments, the server can access a public dataset. The public dataset includes data already collected by the server, containing a large and rich set of samples. After training a global model in the cloud using the public dataset, the server can then distill corresponding first sub-models for each device cluster.

[0066] In implementation, the full amount of data in the public dataset can be used to distill multiple first sub-models from the global model in the cloud through knowledge distillation.

[0067] In another possible embodiment, to improve efficiency, the desired first sub-model can also be distilled for each of the multiple device clusters in the following manner:

[0068] Step B1: Sample a training sample set of performance parameters for the adapted device cluster from the public dataset.

[0069] The performance parameters of a device cluster can be represented by the edge device with the lowest performance parameters within the cluster. Alternatively, they can be represented by the average performance of the edge devices within the cluster. This disclosure does not limit the specific performance parameters in this way.

[0070] In some embodiments, when sampling a training sample set for a device cluster, the number of samples supported by the device cluster in one round of training can be determined based on the sample size and computing resources available to the device cluster. The required training sample set for the device cluster is then downsampled from a public dataset based on this sample size. For example, if device cluster 1 supports 5000 samples per training round, then 5000 samples are downsampled for training device cluster 1. If device cluster 2 supports 6000 samples per training round, then 6000 samples are downsampled for training device cluster 2.

[0071] Sampling can be done randomly or according to the categories included in the device cluster. Prioritizing sampling of categories not included in the device cluster allows each cluster to learn from a wider range of categories. For example, in image classification, if the local data for device cluster 1 is mostly composed of category 1 and category 2 samples, samples of category 3 or category 4 can be sampled from a public dataset to distill the first sub-model for that cluster. Thus, the first sub-model for each device cluster is trained based on samples from a wider range of categories, preventing any cluster from becoming overly focused on learning a single category.

[0072] Step B2: Create an initial model of the performance parameters for the adapted device cluster.

[0073] The model size can be determined based on the sample size supported by one round of training for each device cluster and the computational resources of the device cluster. Higher performance device clusters support larger model sizes, allowing for the design of larger initial models for high-performance device clusters. For example, an initial model with a 3-layer network can be designed for low-performance device cluster 1, while an initial model with a 6-layer network can be designed for high-performance device cluster 2. This allows for the design of heterogeneous first sub-models based on the differences between device clusters.

[0074] In practice, a correspondence between different performance parameters and key model indicators can be established, and an initial model can be created for different device clusters based on this correspondence.

[0075] Of course, in other embodiments, initial models of each device cluster can be created in response to user operations based on prior knowledge, and this disclosure does not limit this.

[0076] It should be noted that the execution order of steps B1 and B2 is not restricted in this embodiment.

[0077] Step B3: Using the global model in the cloud as the teacher model and the initial model as the student model, train the initial model based on the training sample set to distill the first sub-model.

[0078] In this embodiment of the disclosure, the server is based on the device clustering information. and the server's public dataset Knowledge distillation is performed on the global cloud model w0. In this embodiment, the server obtains the public dataset corresponding to each device cluster by randomly sampling the public dataset. in, This represents the training sample set for the first device cluster. Let represent the training sample set of the t-th device cluster.

[0079] This disclosure describes an embodiment that considers distilling a first sub-model based on response-based knowledge. The main idea is to directly simulate the final prediction of the teacher model. Generally, the output of the last layer of a neural network, followed by a softmax (activation function layer), can be converted into a probability distribution. A logit (log odds) is given as the output of this last layer (e.g., a fully connected layer). Response-based knowledge is called a soft target. The soft target is the probability distribution of the input samples, which can be estimated using the softmax function as follows:

[0080]

[0081] In equation (1), z i is the logit of the i-th probability value in the probability distribution; T is the temperature factor used to control the importance of each soft target. Therefore, for each device cluster, the difference between the logit of the first sub-model and the logit of the teacher model can be used to determine the loss. The training objective is to minimize the difference between the logit predicted by the first sub-model and the teacher model so that the first sub-model learns the knowledge of the teacher model. Its loss function can be expressed as shown in equation (2):

[0082] L R (z t ,z s ) = L R (p(z t ,T),p(z s ,T))(2)

[0083] In equation (2), L R (.) represents the loss between logit values, z t and z s Let these represent the logit values ​​for the teacher model and the student model, respectively.

[0084] Taking classification as an example, the teacher model classifies the training sample set to obtain the probability distribution, and the student model also classifies the training sample set to obtain the probability distribution of the input samples. With the probability distribution of the input samples obtained, the loss value can be determined based on formulas (1) and (2), thereby adjusting the parameters of the student model and distilling the first sub-model from the global model in the cloud.

[0085] Finally, assume that the total set of student models corresponding to each device cluster is w. s ={w1,w2...w t}, where w1 represents the model parameters of the first sub-model of the first device cluster. And so on, w t This represents the model parameters of the first sub-model of the t-th device cluster.

[0086] Knowledge distillation is a technique that transfers knowledge from a teacher model to a student model. The teacher model is a global model in the cloud. For each device cluster, the server trains a small student model to obtain the first sub-model for that device cluster. Because the training sample set may differ for each device cluster, the training results of each first sub-model will also vary.

[0087] In summary, based on the performance differences of different device clusters, the embodiments of this disclosure can construct different first sub-models for different device clusters, and the use of different training sample sets may also lead to differences in the training results of different first sub-models. Therefore, the embodiments of this disclosure can perform personalized knowledge distillation for different device clusters based on their differences.

[0088] After the first sub-model is distilled, it is distributed to the corresponding device cluster for federated learning. Each edge device within the cluster receives this first sub-model. Each edge device trains the first sub-model independently using its local private dataset. The device cluster then aggregates the first sub-models learned by each edge device within the cluster to obtain the second sub-model. This can be understood as each device cluster training its corresponding first sub-model to obtain the second sub-model. The specific training method within the device cluster will be explained in later embodiments. It is worth noting that after the server obtains the trained second sub-models from each device cluster, it can train the cloud-based global model based on these second sub-models, thereby updating the cloud-based global model. For example, using the prediction results of each second sub-model for the input sample as the soft objective, the cloud-based global model can be trained to update the cloud-based global model. This achieves the transfer of knowledge from each second sub-model to the cloud-based global model, using each second sub-model as the soft objective and the cloud-based global model as the student model.

[0089] In other embodiments, data augmentation can be used to enhance the training of the global cloud model. Specifically, this can be implemented as follows:

[0090] Step C1: Perform knowledge distillation on the second sub-model of each device cluster to obtain the global student model.

[0091] In order to learn the knowledge of each second sub-model, this embodiment of the disclosure proposes to use knowledge distillation to compress multiple second sub-models into a global student model.

[0092] In one possible implementation, since the parameter counts of each second sub-model may differ, each second sub-model can be used sequentially as a teacher model to guide the training of the global learning model. Assume we have three second sub-models: second sub-model A, second sub-model B, and second sub-model C. We can first use second sub-model A as the teacher model and the initial global model O as the student model, training the initial global model A using knowledge distillation. Then, using the initial global model A as the student model and second sub-model B as the teacher model, we perform knowledge distillation on second sub-model B to obtain the initial global model B. Similarly, using the initial global model B as the student model and second sub-model C as the teacher model, we perform knowledge distillation on second sub-model C to obtain the final global student model.

[0093] In another possible implementation, multiple second sub-models can be aggregated to obtain an aggregated learning objective, thereby enabling the global student model to be obtained through knowledge distillation. Specifically, knowledge distillation of the second sub-models of each device cluster to obtain the global student model can be implemented as follows:

[0094] Step C11: Input the distillation sample into the second sub-model of each equipment cluster to obtain the processing results of the distillation sample by the second sub-model of each equipment cluster.

[0095] The distillation samples can be selected from publicly available datasets. That is, during implementation, some samples can be selected from publicly available datasets as distillation samples.

[0096] The processing result can be a soft target as shown in formula (1). It can also be the prediction result of the model, such as the probability distribution output by the final sofmax. In classification scenarios, when the target class of the distillation sample is known, the processing result can also be the predicted probability value of the target class of the distillation sample, rather than the probability distribution. The processing result can be determined according to the actual situation, and this disclosure does not limit it.

[0097] Step C12: The processing results of each second sub-model on the distilled sample are fused to obtain the soft target.

[0098] One fusion method is to take the average, that is, to take the average of the processing results of multiple second sub-models on the same distillation sample. Considering the differences in training performance among different second sub-models, a weighted average can also be used for fusion. The weight allocation can primarily be based on model performance. For example, the weight of each second sub-model can be determined based on its accuracy on the validation set; where higher accuracy corresponds to higher weight. The processing results of each second sub-model on the distillation sample are then weighted and summed to obtain the soft objective.

[0099] For example, in one possible weighting strategy, if the accuracy of the second sub-model A is 80% and the accuracy of the second sub-model B is 70%, then when weighted summing, the weight of the second sub-model A is 0.8 and the weight of the second sub-model B is 0.7.

[0100] Of course, during implementation, the importance of each device cluster can be determined based on predefined rules, and the weights of each second sub-model can be allocated according to the importance of the device cluster. The importance of the device cluster can be determined by referring to the importance of the local dataset within the device cluster, the accuracy of the second sub-model learned by the device cluster, etc., and this disclosure does not limit this.

[0101] In this embodiment of the disclosure, during the knowledge distillation process, multiple second sub-models are aggregated using a weighted summation method. In particular, considering the accuracy of the models, the soft objective constructed thereby allows the global student model to focus on learning the teacher model with high accuracy, thus distilling out a highly usable global learning sub-model.

[0102] Step C13: Train a global student model based on the soft target and the distillation samples to minimize the loss between the global student model's processing results for the distillation samples and the soft target.

[0103] By minimizing the loss, the global student model's processing results for samples can be made to approximate the processing results of multiple second sub-models, thereby allowing the global student model to imitate multiple second sub-models and thus complete knowledge distillation.

[0104] This disclosed embodiment can fuse the processing results of multiple second sub-models on distilled samples, and use this to construct a soft-objective-guided distillation model to generate a global student model. This allows the global model to simultaneously learn the knowledge of multiple second sub-models, thereby compressing multiple heterogeneous second sub-models into a single global student model.

[0105] For example, the server collects the second sub-models trained from all device clusters. This embodiment of the disclosure compresses the knowledge of these trained second sub-models (as teacher models) into a smaller model (i.e., a global student model) through knowledge distillation. It mainly includes the following three parts: (a1)-(a3).

[0106] (a1) Softened output of teacher models: For each teacher model, calculate its output probability distribution for the distilled samples. These outputs are typically probability distributions transformed by the softmax function. During knowledge distillation, a higher temperature T can be used to soften the softmax function in order to capture the relative confidence differences between teacher models. The softened probability function is shown in Equation (3):

[0107]

[0108] In formula (3), z i P(z) is the probability value of the i-th class in the output probability distribution, T is the temperature parameter, and P(z) is the probability value of the i-th class. i ,T) represents the processing result of the teacher model on the distilled sample.

[0109] (a2) Fusion of Teacher Model Outputs: For each distilled sample, the softened output probability distributions of all teacher models are weighted and summed to obtain a fused target probability distribution (i.e., a soft target). The weights can be allocated according to the performance of the teacher models. The fused soft target is shown in Equation (4):

[0110] P target =∑ k (W k *P k (4)

[0111] In formula (4), P target For soft targets, W k It is the weight of the k-th teacher model, P k It is the output probability distribution of the k-th teacher model (i.e., the result of processing the distilled samples).

[0112] (a3) Training the student model: The global student model is trained using the fusion target probability distribution as a soft target. The loss function of the global student model can be defined as the cross-entropy loss between the fusion target probability distribution and the probability distribution output by the global student model itself. The loss function is shown in Equation (5):

[0113]

[0114] In formula (5), L KD This represents the cross-entropy loss value. It represents the probability of the i-th category in the probability distribution of the fused target. This is the score for the i-th category output by the global student model. Then, the updated model parameters are calculated to obtain the global student model. Where t represents the global student model obtained in the t-th round of aggregation.

[0115] It should be noted that using soft targets yields better results, and the decision to use soft targets should be made based on the specific circumstances. Furthermore, the softmax function makes the probability distribution of the model's output smoother, allowing observation of the probability distribution of the model's predictions for each category. However, the use of the softmax function in this embodiment is merely an example and is not intended to limit the scope of this embodiment.

[0116] In this embodiment of the disclosure, the global student model serves to guide the device clusters in generating augmented datasets. Specifically, in step C2, based on the global student model and / or the second sub-model of each device cluster, each device cluster is guided to generate augmented datasets that satisfy preset data distribution characteristics; wherein, for each device cluster, the preset data distribution characteristics are the data distribution characteristics of the local dataset within the device cluster.

[0117] For example, edge devices within a device cluster can generate new data as augmented samples based on the distribution characteristics of local data within the edge devices, guided by the second sub-model and / or the global student model, to construct an augmented dataset. The augmented samples differ from real privacy data but share similar distribution characteristics. In this way, the augmented samples can replace privacy data for transmission over the network, and in step C3, a global model in the cloud is trained based on the augmented datasets of each device cluster.

[0118] For example, the following formula (6) can be used based on the augmented dataset. Update the global model in the cloud:

[0119]

[0120] In formula (6), This represents the cloud-based global model at time step t, i.e., the cloud-based global model before the update, where η is the learning rate. It's about the model. The loss function in augmenting the dataset The gradient on the model. The specific calculation method for the loss function can be determined based on actual business needs. This formula represents the amount of data used in each update step, utilizing the augmented dataset. Calculate the gradient of the loss function. That is, after updating the global model in the cloud based on the augmented dataset, it is possible to continue distilling the corresponding sub-models for each device cluster, and then generate a new augmented dataset to iteratively optimize the global model in the cloud.

[0121] Therefore, in this embodiment, an augmented dataset similar to the local dataset of the device cluster can be used to update and train the cloud-based global model. The augmented dataset differs from the privacy data that needs protection within the device cluster, but its data distribution characteristics are similar. The augmented dataset constructed in this way can be transmitted over the network, protecting privacy data while enabling the cloud-based global model to learn from the local data of edge devices. More importantly, this embodiment uses a second sub-model and a global student model to guide the generation of the augmented dataset. This allows the use of the prior knowledge of the second sub-model and the global student model to help constrain the device cluster to generate a high-quality augmented dataset for optimizing the cloud-based global model.

[0122] Of course, in some embodiments, given the known distribution characteristics of the local dataset of the device cluster, the augmented dataset can also be generated based on a GAN (Generative Adversarial Network). For example, a GAN network includes a generator and a discriminator. The generator generates the augmented dataset, and the discriminator determines whether it conforms to the distribution characteristics of the local dataset of the device cluster. If it does not conform, the generator's model parameters can be optimized so that the generator generates a dataset that conforms to the distribution characteristics.

[0123] When using a second sub-model and / or a global learning model to guide the generation of augmented data, higher quality augmented datasets can be generated compared to using only the data distribution characteristics of the local dataset as a constraint.

[0124] In implementation, the guidance of the second sub-model and / or the global learning model can be such that the outputs of the augmented samples in the augmented dataset are all target results. This can be understood as the device cluster generating augmented samples based on local raw data. These augmented samples are pseudo-samples, but they should be similar to the raw data. In this case, the processing results of the global student model and / or the second sub-model on the augmented samples should be similar to the processing results on the raw data. Therefore, the second sub-model and / or the global student model can select pseudo-samples similar to the raw data from the multiple pseudo-samples generated by the device cluster as augmented samples in the augmented dataset. Thus, the resulting augmented dataset performs well in the second sub-model and / or the global student model, and the knowledge of the second sub-model can be distilled into the cloud-based global model through the augmented dataset.

[0125] In other possible embodiments, to better optimize the global cloud model, this disclosure embodiment guides each device cluster to generate an augmented dataset based on the global student model and the second sub-model of each device cluster. Specifically, this can be implemented as follows: Figure 2 As shown:

[0126] S201, For each device cluster, obtain pseudo samples generated by the device cluster based on the local data distribution characteristics of the device cluster.

[0127] S202, for spurious samples, perform the following operations in a loop:

[0128] S2021, based on the global student model, processes pseudo-samples to obtain the first comparison result of the pseudo-samples; and...

[0129] S2022, based on each second sub-model, the pseudo-samples are processed to obtain multiple processing results for the pseudo-samples;

[0130] During implementation, the execution order of steps S2021 and S2022 is not restricted.

[0131] It should be noted that the server can send the global student model to the edge device, which then inputs the pseudo-samples into the global student model to obtain the first comparison result. Similarly, the server can input the pseudo-samples into the second sub-model within the device cluster through the device cluster, thereby obtaining the processing result of the pseudo-samples in the second sub-model.

[0132] Since the second sub-model may be heterogeneous, multiple processing results can be fused in S2023 to obtain the second comparison result of the pseudo sample.

[0133] The fusion processing can be achieved by taking the average, or by using a weighted average. The weighting method can also be based on the accuracy of the second sub-model on the validation set, generating corresponding weights. That is, the higher the accuracy, the greater the weight. Alternatively, weights can be generated based on the importance of the device clusters. This disclosure does not limit the scope of the embodiments.

[0134] In practice, both the second comparison result and the second comparison result can be the soft objective mentioned above, and the solution method can be as shown in formulas (1) and (2), which will not be repeated here. Of course, it can also be the probability distribution predicted by the final model itself.

[0135] S2024, if the difference between the first and second alignment results meets the preset conditions, add the pseudo-sample to the amplified dataset.

[0136] S2025, if the difference between the first comparison result and the second comparison result does not meet the preset conditions, request the device cluster to update the pseudo sample until the preset conditions are met.

[0137] Therefore, guided by the second sub-model and the global student model, the pseudo-samples are dynamically updated, which ensures that the pseudo-samples have consistent performance in both the second sub-model and the global student model. This is beneficial for generating high-quality augmented datasets, thereby effectively distilling the information from the global student model into the cloud-based global model.

[0138] The preset conditions include at least one of the following:

[0139] Condition 1) The difference between the first and second alignment results of the pseudo-sample is less than a preset threshold.

[0140] In other words, this condition requires that the difference between the prediction results of the teacher model (i.e., the second sub-model) and the prediction results of the global student model be small enough, so that the generated pseudo-samples can have consistent performance in both the teacher model and the global student model. This is beneficial for distilling the information of the second sub-model or the global student model into the cloud global model by augmenting the dataset.

[0141] The preset threshold can be determined based on empirical values, and this embodiment does not limit it.

[0142] Condition 2) The trend of the difference between the first and second comparison results of the pseudo-sample is from continuously decreasing to decreasing at the end.

[0143] For example, such as Figure 3 As shown, pseudo-sample A is input into the global student model to obtain the first comparison result B11. This pseudo-sample A is then input into multiple second sub-models (i.e., teacher models) to obtain multiple processing results. These multiple processing results are aggregated to obtain the second comparison result B21. The difference between B11 and B21 is calculated to obtain C1. Pseudo-sample A is updated based on C1 to obtain pseudo-sample A', which is then input into the global student model to obtain the first comparison result B12. This pseudo-sample A' is then input into multiple second sub-models (i.e., teacher models) to obtain multiple processing results. These multiple processing results are aggregated to obtain the second comparison result B22. The difference between B12 and B22 is calculated to obtain C2. If C2 is less than C1, the pseudo-sample A' is updated to obtain pseudo-sample A'". This pseudo-sample A' is input into the global student model to obtain the first comparison result B13. This is then input into multiple second sub-models (i.e., teacher models) to obtain multiple processing results. These multiple processing results are aggregated to obtain the second comparison result B23. The difference between B13 and B23 is calculated to obtain C3. If C3 is less than C2, the above operation can be repeated to update pseudo-sample A'". If C3 is not less than C2, it indicates that the trend of decreasing difference has ended. Iteratively updating pseudo-samples may not achieve good performance in both the teacher and student models. Therefore, pseudo-sample A'" or A' with the smallest difference can be added to the augmented dataset as augmented data. The method of updating pseudo-samples will be explained later.

[0144] Therefore, this condition ensures that the difference between the prediction results of the teacher model (i.e., the second sub-model) and the prediction results of the global student model is small enough, so that the generated pseudo-samples can have consistent performance in both the teacher model and the global student model. This is beneficial for distilling the information of the second sub-model and / or the global student model into the cloud global model by augmenting the dataset.

[0145] In some embodiments, a generative model is a type of model capable of generating synthetic data. For example, a model trained to generate faces generates a face each time that has never been seen by the model or anyone else. The most well-known example of a generative model is a GAN network. It has a generator and a discriminator that compete against each other to generate augmented data. Because the model itself is adversarial, it is difficult to train. This makes it difficult to achieve an optimal balance. This problem can be solved using a diffusion model. Therefore, the augmented dataset in this embodiment can be generated by clusters of devices based on a diffusion model. The diffusion model includes a forward process and a backward process. In implementation, the backward process of the diffusion model can be constructed using a global student model. Thus, the global student model has learned the data distribution of each edge device. By using a global student model to construct the backward process of the diffusion model, the data distribution of the generated augmented samples can be made closer to the data distribution of the edge devices. This allows the global model in the cloud to learn the data distribution on the edge devices more accurately, thereby improving its predictive performance.

[0146] In summary, this disclosure focuses on the heterogeneity of edge devices, particularly the differences in their resources. For example, there are significant differences in resources such as CPU, GPU, memory, storage space, and network bandwidth among various devices. This heterogeneity presents a challenge: not all edge devices can support the deployment of a global model in the cloud. Therefore, during model training or inference, knowledge distillation methods are needed to transform the model into a suitable size adapted to the characteristics of the device resources. Furthermore, data augmentation techniques are used to expand the server dataset and dynamically update the global model in the cloud to improve accuracy.

[0147] Based on the same technical concept, this disclosure also provides a federated learning method applied to edge devices, such as... Figure 4 As shown, it includes:

[0148] S401, send the performance parameters of the current edge device to the server so that the server can perform cluster analysis on the multiple edge devices based on the performance parameters of multiple edge devices, obtain multiple device clusters, and perform knowledge distillation on the global model in the cloud to obtain a first sub-model applicable to the target device cluster; wherein, the target device cluster is the device cluster in which the current edge device is located.

[0149] Performance parameters of edge devices describe their data processing capabilities. These parameters may include computing resources (such as CPU, GPU, and memory), storage resources (such as disk space), and network resources (such as bandwidth).

[0150] S402, federated learning based on the first sub-model.

[0151] Because different edge devices exhibit performance differences, this disclosure employs clustering analysis to group edge devices with similar performance into the same device cluster. Each device cluster can support the execution of the same first sub-model. Therefore, for large cloud-based global models that cannot be directly run on edge devices, a first sub-model suitable for each cluster can be distilled. This disclosure uses knowledge distillation to lightweight the large cloud-based global model and distribute it to edge devices. The lightweight model has fewer parameters than the large cloud-based model, thereby reducing the system communication burden. Furthermore, this disclosure uses a federated learning method to protect privacy and information security. In summary, this disclosure provides a general and efficient federated learning scheme suitable for heterogeneous edge devices.

[0152] In some embodiments, federated learning based on the first sub-model can be implemented by training the first sub-model using the local dataset of the current edge device to obtain the sub-model to be aggregated; and generating a second sub-model based on the target device cluster and the sub-model to be aggregated.

[0153] The trained second sub-model can be used to update the global model in the cloud. These points have been explained earlier and will not be repeated here.

[0154] It should be noted that edge devices in the device cluster can train the first sub-model based on the local dataset, thereby enabling the optimization of the global model in the cloud through federated learning.

[0155] For example, the first sub-model of all device clusters {w1, w2... w t The data is distributed to each device cluster. For the i-th device cluster, the set of edge devices within that cluster can be represented as S. i ={x1,x2,...,x N}, where N represents the total number of edge devices in the i-th device cluster. The dataset of the i-th device cluster can be represented as D. i ={d1,d2,...,d N}, where, assuming d is used n Define the number of local datasets for a single edge device, which yields... D t Define the total local dataset size for a single device cluster, then define the total local dataset size for all device clusters.

[0156] Edge devices within a device cluster train models based on local datasets. Assume... This represents collaborative knowledge from device cluster n, while D represents the cluster-wide public dataset D that can be accessed by all edge devices within the device cluster n. iThe data samples within the cluster are the training dataset obtained by the server for sampling the device clusters. The loss for each device cluster can then be expressed as shown in equation (7):

[0157]

[0158] In formula (7), λ>0 is a hyperparameter, L KL Representing the Kullback-Leibler (KL, divergence function), it is primarily used for personalized knowledge transfer. n This represents a single edge device within a cluster on its local dataset d. n The loss can be determined based on business needs. mn is the knowledge coefficient matrix, used to estimate the contribution of the training cluster m to n. m The model parameters represent the first sub-model of the device cluster m; w n This represents the model parameters of the first sub-model of device cluster n. During implementation, different device clusters can exchange collaborative knowledge so that each device cluster can calculate the loss based on expression (7).

[0159] The knowledge coefficient matrix is ​​defined as follows:

[0160]

[0161] The objective of edge device training is shown in Equation (8):

[0162]

[0163] In formula (8), The connection vector used to represent weights. For example, suppose there are two device clusters with first sub-models w1 and w2. The model parameters of the two first sub-models can be represented as w1 = [a,b] and w2 = [c,d,e]. Then, the weights of the two first models concatenated together are [a,b,c,d,e].

[0164] In formula (8), d n It is the model parameter w n The dimension of the first sub-model w1 is 2, and the dimension of the second sub-model w2 is 3. ρ is a regularization parameter greater than 0. D represents the dataset of all device clusters. t Let represent the dataset of the t-th device cluster.

[0165] In some embodiments, to better learn from the local dataset, the first sub-model can be iteratively trained a preset number of times using the local dataset of the current edge device. By requiring the edge device to iterate and train multiple times based on the local dataset, it can fully learn from the local dataset, and the knowledge from the local dataset can be transferred to the model to be aggregated, so as to better optimize the global model in the cloud.

[0166] The device cluster contains multiple edge devices, each of which has a sub-model to be aggregated. These sub-models can be aggregated within the device cluster and then sent to the server, or the server can perform the aggregation itself.

[0167] When aggregation occurs within a device cluster, the cluster can be divided into master devices and slave devices. A high-performance edge device can be selected as the master device. If the current edge device is the master device of the target device cluster, multiple sub-models to be aggregated are obtained from the training of each edge device within the target cluster. These sub-models are then aggregated to obtain the second sub-model of the first sub-model.

[0168] In this embodiment of the disclosure, the device cluster completes the aggregation of the sub-models to be aggregated for each edge device, which can save network bandwidth and reduce the amount of data transmitted between systems.

[0169] For example, in some embodiments, the aggregation of multiple sub-models to be aggregated to obtain a second sub-model of the first sub-model can be implemented in any of the following three ways:

[0170] Aggregation Method 1) Perform a weighted average of the model parameters of multiple sub-models to be aggregated to obtain the second sub-model of the first sub-model.

[0171] The weights can be determined based on the accuracy of the sub-models to be aggregated on the validation set.

[0172] Aggregation Method 2) Calculate the average of the model parameters of multiple sub-models to be aggregated to obtain the second sub-model of the first sub-model.

[0173] In practice, if there is a significant deviation in the data distribution of the local datasets of various edge devices within a device cluster, a weighted average can be chosen. This involves summing the model parameters of each sub-model to be aggregated using weights and then dividing by the sum of the weights. This balances the differences and contributions between different edge devices.

[0174] The embodiments disclosed herein employ a weighted average or mean-based aggregation method, which is simple and easy to implement and applicable to all device clusters.

[0175] Aggregation Method 3) When the validation set includes multiple categories, for each category, determine the sub-model to be aggregated with the highest prediction accuracy for that category as an intermediate sub-model to obtain multiple intermediate sub-models; calculate the average of the model parameters of multiple intermediate sub-models to obtain the second sub-model.

[0176] For example, consider image classification. The sub-models trained on various edge devices predict the same category; a vote is taken based on the prediction results for that category; the model parameters of the sub-model with the most accurate prediction for that category are selected. For each category, corresponding model parameters are selected, and the average of the selected model parameters for all categories is calculated to obtain the model parameters of the second sub-model.

[0177] In this embodiment of the disclosure, each edge device votes based on its processing results of the input sample. The aggregated result is the value with the most votes.

[0178] Aggregation method 3) is suitable for situations where the processing results of edge devices differ greatly. In such cases, it is desirable to determine the final aggregation result through voting, thereby making the second sub-model obtained by aggregation more representative of the knowledge learned by the device cluster.

[0179] In summary, within each device cluster, after the training of each edge device is completed, the models will be aggregated. The model aggregation process is shown in the following formula (9):

[0180]

[0181] In formula (9), This represents the aggregation model obtained in the t-th round of training within device cluster i.

[0182] This represents the aggregation model trained on the nth edge device within the device cluster, where η is the learning rate. It is the average gradient of edge device n during the t-th training round. After aggregation, the second sub-model corresponding to this device cluster is obtained. Send the second sub-model to the server.

[0183] In some embodiments, a second sub-model is sent to the server to generate augmented samples based on the distribution characteristics of the local dataset of the current edge devices, guided by the global student model and / or the second sub-models of each device cluster, so that the server obtains the augmented dataset of the target device cluster. The global student model is obtained by the server through knowledge distillation of the second sub-models of each device cluster. This embodiment can guide the generation of high-quality augmented datasets.

[0184] In this embodiment, the method of generating augmented datasets based on the global student model and / or the second sub-model has been described above and will not be repeated here. This embodiment mainly introduces the method of generating augmented datasets using a diffusion model.

[0185] Specifically, due to the public availability of data datasets on the server side The sample size is limited, and data samples from edge devices are lacking; therefore, a diffusion model is needed to augment the dataset. In this embodiment, the global student model w obtained in the previous step is used. student As a guide, data augmentation is performed using a diffusion model. The specific process of generating augmented datasets involves two stages: a forward process and a backward process. The following explanation uses images as an example.

[0186] Taking an image as an example, the forward process: each time step t in the forward process depends only on time step t-1, so it can be regarded as a Markov process. The purpose of diffusion is to gradually map the original image x0 to a multidimensional normal distribution x through the Markov process. t (i.e., Gaussian noise). The stochastic process at each step is q(x) t |x t-1 The process of adding noise can be defined as shown in formula (10):

[0187] x t = α t x t-1 +β t ∈ t , ∈ t ~N(0,I) (10)

[0188] In formula (10), α t ,β t Satisfying the harmonic relation α t 2 +β t 2 =1, and β t It increases with the increase of t. Therefore, the forward process can be defined as shown in formula (11):

[0189]

[0190] Due to the properties of the Gaussian distribution, we can obtain expression (12):

[0191]

[0192] In the above expressions (10)-(12), I is the identity matrix. Due to α tIt gradually decreases and takes a value as close to 0 as possible, so when t approaches infinity, expression (12) can be expressed as expression (13):

[0193] q(x t |0)=N(x t ;0,I) (13)

[0194] That is, at this time x t It follows a standard normal distribution.

[0195] Reverse process: from x t The process to x0 is the reverse of diffusion; the image gradually transforms from Gaussian noise to a normal image. Each step of this random process is q(x... t-1 |x t The stochastic process is unknown, so what the diffusion model needs to do is define a learnable value for p. θ (x t-1 |x t The reverse process of ) . Using the global student model w obtained in the previous step student The optimization parameter θ is used to make the process as close as possible to the actual inverse process q(x). t-1 |x t This allows us to generate the desired normal image from a Gaussian noise. Using Bayes' theorem, with x0 known, we obtain the expression (14):

[0196]

[0197] Based on the definition of the probability density function of the Gaussian distribution and the above result (i.e., completing the square), the mean of the posterior distribution can be obtained. and variance As shown in expression (15):

[0198]

[0199]

[0200] The meaning of each parameter in expression (15) has been explained above and will not be repeated here.

[0201] Furthermore, by analyzing the stochastic process q(x) t Using the reparameter method, we can obtain x (x0). t The expression for x0 (16):

[0202]

[0203] Substitute the expression This yields expression (17):

[0204]

[0205] Here, random noise ∈ t It follows a standard normal distribution. To ensure accurate prediction of the noise value at each step of the reverse process, a global student model w can be used. student To predict this noise value, let the reverse process be as shown in expression (18):

[0206]

[0207] In expression (18), ∈ θ This represents the global student model w. student The predicted noise, θ represents the model parameters of the global student model, and the input of the global student model is x. t And t, then predict the noise ∈ at that time. t , through x t =α t x t-1 +β t ∈ t x is calculated t-1 That's all.

[0208] During the training process, the global student model should be able to perform the reverse process to obtain the inverse model.

[0209] During the reverse process, the output at each moment is fitted to the noise sampled at the corresponding moment of the forward process, which can be expressed as shown in equation (19):

[0210]

[0211] In expression (19), ∈ θ This represents the noise predicted by the inverse model. This represents the noise generated during the forward process at time t.

[0212] The results obtained through the forward and reverse processes described above As a new data sample, i.e., an amplified sample, the amplified dataset generated based on the inverse model can be represented as follows: Where n represents the sample size.

[0213] In short, the process of generating amplified samples using a diffusion model can be described as follows: Figure 5 As shown. The edge device learns the data feature distribution of its local dataset during the forward pass. Then, during the reverse pass, it learns how to generate new data based on this data feature distribution. For the original data x0 within the edge device, noise ∈ is superimposed through the forward pass. tThe original data x0 is used to obtain Gaussian noise. In the reverse process, the noise ∈ [the original data set] is gradually removed from the predicted data. θ This generates new data, thereby achieving data augmentation.

[0214] To improve the quality of the new data generated, this real-time example introduces a global student model and / or a second sub-model for each device cluster to guide data augmentation.

[0215] In some embodiments, under the guidance of the global student model and the second sub-model of each device cluster, augmented samples are generated based on the distribution characteristics of the local dataset of the current edge device. This can be implemented as follows: pseudo-samples are generated based on the diffusion model; wherein the diffusion model learns the data distribution characteristics of the local dataset of the current edge device; and the reverse process of the diffusion model is constructed based on the global student model; the pseudo-samples are updated based on the diffusion model under the guidance of the global student model and the second sub-model of each device cluster to obtain augmented samples.

[0216] The specific guidance methods have been explained above and will not be repeated here.

[0217] The embodiments disclosed herein can guide the generation of augmented samples based on the second sub-model and the global student model, ensuring that these augmented sample data can perform well in the global student model and the second sub-model, thereby effectively distilling the information of the global student model into the cloud global model on the server.

[0218] The diffusion model, built based on the global student model, can be executed separately by each edge device, and can be summarized as follows:

[0219] Step D1: Based on the forward process model of the diffusion model, determine the Gaussian noise of the training samples, time step t, and the forward superposition noise corresponding to time step t.

[0220] The training samples are samples from the local dataset of the edge device. This allows the forward process to learn the data distribution of the local dataset of the edge device.

[0221] Step D2: Input the Gaussian noise of the training samples and the time step t into the global student model to obtain the prediction noise ∈ of the global student model. θ .

[0222] Step D3: Determine the loss between the predicted noise and the forward superimposed noise corresponding to time step t (as shown in formula (19)). With minimizing this loss as the training objective, adjust the parameters of the global student model to obtain the inverse process model of the diffusion model.

[0223] In summary, due to the limited number of publicly available samples on the server side and the lack of data samples from edge devices, this disclosure introduces a diffusion model strategy. This strategy can generate new sample data that is highly similar to the data samples from edge devices. During the inverse process of the diffusion model, the termination stage of the inverse process is determined by judging the soft label differences of the generated pseudo-samples in the global student model and the second sub-models of each device cluster. This stage can be achieved through the following three steps:

[0224] Step E1, Generating Pseudo-Samples: First, a set of pseudo-samples has been generated using the diffusion model. Typically, the diffusion model learns the data distribution at the edge device, thus generating pseudo-samples that have similar distribution characteristics to the data at the edge device.

[0225] Step E2 evaluates the generated data using the global student model and each second sub-model: that is, the generated pseudo-samples are fed into the global student model and the second sub-models of each device cluster (collectively referred to as the teacher model), both of which will produce a soft label, i.e., the distribution of predicted probability values.

[0226] Step E3 calculates the difference in soft labels output by the global student model and the teacher model. This difference measures the performance of the generated sample data in these two models. By observing changes in this soft label difference, the end point of the reverse process is determined. If the difference is decreasing, it indicates that the quality of the generated data is improving, and the reverse process can continue; if the difference is increasing or remains unchanged, it indicates that the quality of the generated data is no longer improving, and the reverse process ends.

[0227] In summary, the training framework provided by the embodiments of this disclosure is as follows: Figure 6 As shown, the central server performs cluster analysis on multiple heterogeneous edge devices, grouping edge devices with similar performance into the same device cluster. For each device cluster, the central server distills a small, adapted first sub-model based on the cloud-based global model. These first sub-models are then distributed to each device cluster, allowing edge devices within the cluster to train their first sub-models using local privacy data. After training their first sub-models, each edge device uses them as models to be aggregated. Models to be aggregated within the same device cluster are aggregated into second sub-models and sent to the central server. The central server performs knowledge distillation on multiple second sub-models, compressing them into a global student model. Guided by the second sub-models and the global student model, each edge device generates an augmented dataset. The central server then optimizes and trains the cloud-based global model based on the augmented dataset.

[0228] Based on the same technological concept, such as Figure 7 As shown, this disclosure also provides a federated learning device 700, comprising:

[0229] The acquisition module 701 is used to acquire performance parameters of multiple edge devices;

[0230] Clustering module 702 is used to perform cluster analysis on multiple edge devices based on the performance parameters of multiple edge devices to obtain multiple device clusters;

[0231] Distillation module 703 is used to perform knowledge distillation on the global model in the cloud for multiple device clusters to distill out multiple first sub-models;

[0232] Learning module 704 is used to distribute multiple first sub-models to corresponding device clusters for federated learning.

[0233] In some embodiments, the distillation module is specifically used for:

[0234] For each of the multiple device clusters, execute the following separately:

[0235] A training sample set of performance parameters for the adapted device cluster was sampled from a publicly available dataset; and,

[0236] Create an initial model of the performance parameters for the adapted device cluster;

[0237] Using the global model in the cloud as the teacher model and the initial model as the student model, the initial model is trained based on the training sample set to distill the first sub-model.

[0238] In some embodiments, the clustering module is specifically used for:

[0239] Determine the performance similarity between edge devices based on the performance parameters of multiple edge devices;

[0240] Clustering analysis based on the performance similarity between edge devices yields multiple device clusters.

[0241] In some embodiments, each device cluster is used to train a corresponding first sub-model to obtain a second sub-model;

[0242] The device also includes a guidance module, which guides each device cluster to generate an augmented dataset that meets preset data distribution characteristics based on the second sub-model and / or global student model of each device cluster; wherein, for each device cluster, the preset data distribution characteristics are the data distribution characteristics of the local dataset within the device cluster; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster;

[0243] The learning module is also used to train a global cloud model based on the augmented datasets of each device cluster.

[0244] In some embodiments, the distillation module includes:

[0245] The input submodule is used to input the distillation sample into the second sub-model of each equipment cluster, and obtain the processing results of the distillation sample by the second sub-model of each equipment cluster;

[0246] The fusion submodule is used to fuse the processing results of the distilled samples from each second sub-model to obtain a soft target;

[0247] The distillation submodule is used to train a global student model based on the soft objective and distillation samples, in order to minimize the loss between the global student model's processing results of the distillation samples and the soft objective.

[0248] In some embodiments, the fusion submodule is specifically used for:

[0249] The weights of each second sub-model are determined based on its accuracy on the validation set; where higher accuracy corresponds to higher weight.

[0250] The soft objective is obtained by weighted summation of the processing results of each second sub-model on the distilled sample.

[0251] In some embodiments, the boot module includes:

[0252] The acquisition submodule is used to acquire pseudo samples generated by the device clusters based on the local data distribution characteristics of the device clusters for each device cluster.

[0253] The optimization submodule is used to perform the following operations in a loop for pseudo-samples:

[0254] The pseudo-samples are processed based on the global student model to obtain the first comparison result of the pseudo-samples; and...

[0255] Based on the processing of pseudo-samples by each second sub-model, multiple processing results of pseudo-samples are obtained;

[0256] The results of multiple processing steps are fused to obtain the second comparison result of the pseudo-sample;

[0257] If the difference between the first and second alignment results meets the preset conditions, the pseudo-sample is added to the amplified dataset.

[0258] If the difference between the first comparison result and the second comparison result does not meet the preset conditions, request the device cluster to update the pseudo sample until the preset conditions are met.

[0259] In some embodiments, the preset conditions include at least one of the following:

[0260] The difference between the first and second alignment results of the pseudo-sample is less than a preset threshold.

[0261] The trend of the difference between the first and second alignment results of the pseudo-samples is from continuously decreasing to decreasing at the end.

[0262] In some embodiments, the augmented dataset is generated by each device cluster based on the diffusion model, and the global student model is used to construct the reverse process of the above diffusion model.

[0263] Based on the same technological concept, such as Figure 8 As shown, this disclosure also provides a federated learning device 800, comprising:

[0264] The sending module 801 is used to send the performance parameters of the current edge device to the server, so that the server can perform cluster analysis on multiple edge devices based on the performance parameters of multiple edge devices to obtain multiple device clusters, and perform knowledge distillation on the global model in the cloud to obtain a first sub-model applicable to the target device cluster; wherein, the target device cluster is the device cluster to which the current edge device belongs;

[0265] Training module 802 is used for federated learning based on the first sub-model.

[0266] In some embodiments, the training module includes:

[0267] The model training submodule is used to train the first sub-model using the local dataset of the current edge device to obtain the sub-model to be aggregated.

[0268] The aggregation submodule is used to generate a second submodel of the first submodel based on the target device cluster and the submodel to be aggregated.

[0269] In some embodiments, the model training submodule is specifically used to iteratively train the first sub-model a preset number of times using the local dataset of the current edge device.

[0270] In some embodiments, the aggregation submodule is specifically used for:

[0271] When the current edge device is the master device of the target device cluster, obtain multiple sub-models to be aggregated obtained from the training of each edge device in the target device cluster;

[0272] Multiple sub-models to be aggregated are aggregated to obtain the second sub-model of the first sub-model.

[0273] In some embodiments, the aggregation submodule is specifically used for:

[0274] The model parameters of multiple sub-models to be aggregated are weighted and averaged to obtain the second sub-model of the first sub-model; or,

[0275] The average of the model parameters of multiple sub-models to be aggregated is used to obtain the second sub-model of the first sub-model.

[0276] In some embodiments, the aggregation submodule is specifically used for:

[0277] When the validation set includes multiple categories, the sub-model with the highest prediction accuracy for each category is selected as the intermediate sub-model to obtain multiple intermediate sub-models.

[0278] The second sub-model is obtained by averaging the model parameters of multiple intermediate sub-models.

[0279] In some embodiments, the sending module is further configured to send the second sub-model to the server;

[0280] The generation module is used to generate augmented samples based on the distribution characteristics of the local dataset of the current edge devices, under the guidance of the second sub-model and / or global student model of each device cluster, so that the server can obtain the augmented dataset of the target device cluster.

[0281] The global student model is obtained by the server through knowledge distillation of the second sub-model of each device cluster.

[0282] In some embodiments, the generation module includes:

[0283] The sample generation submodule is used to generate pseudo samples based on the diffusion model; the diffusion model learns the data distribution characteristics of the local dataset of the current edge device; and the inverse process of the diffusion model is built based on the global student model.

[0284] The update submodule is used to update pseudo-samples based on the diffusion model, guided by the global student model and the second sub-model of each device cluster, to obtain amplified samples.

[0285] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0286] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0287] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0288] Figure 9A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0289] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0290] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0291] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as federated learning methods. For example, in some embodiments, the federated learning method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the federated learning method described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform federated learning methods by any other suitable means (e.g., by means of firmware).

[0292] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0293] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0294] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0295] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0296] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0297] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0298] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0299] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A federated learning method, comprising: Obtain performance parameters from multiple edge devices; Based on the performance parameters of the multiple edge devices, cluster analysis is performed on the multiple edge devices to obtain multiple device clusters; Knowledge distillation is performed on the global model in the cloud for the multiple device clusters to distill multiple first sub-models; The sampled image is used as a sample to distill the first sub-model; The multiple first sub-models are distributed to corresponding device clusters for federated learning; Each device cluster is used to train the corresponding first sub-model to obtain the second sub-model; Also includes: Based on the second sub-model and global student model of each device cluster, each device cluster is guided to generate an augmented dataset that meets preset data distribution characteristics; wherein, for each device cluster, the preset data distribution characteristics are the data distribution characteristics of the local dataset within the device cluster; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster; including: For each device cluster, obtain pseudo samples generated by the device cluster based on the local data distribution characteristics of the device cluster; For the pseudo-sample, perform the following operations in a loop: The pseudo-samples are processed based on a global student model to obtain the first comparison result of the pseudo-samples; and... The pseudo-samples are processed based on each second sub-model to obtain multiple processing results for the pseudo-samples; The multiple processing results are fused to obtain the second comparison result of the pseudo sample; If the difference between the first comparison result and the second comparison result meets the preset conditions, the pseudo-sample is added to the amplified dataset; If the difference between the first comparison result and the second comparison result does not meet the preset condition, the device cluster is requested to update the pseudo sample until the preset condition is met. The cloud-based global model is trained using augmented datasets from each device cluster.

2. The method according to claim 1, wherein, The process of knowledge distillation for the cloud-based global model by the multiple device clusters includes: For each of the multiple device clusters, execute the following respectively: A training sample set of performance parameters adapted to the device cluster was sampled from a publicly available dataset; and, Create an initial model that adapts to the performance parameters of the device cluster; Using the cloud-based global model as the teacher model and the initial model as the student model, the initial model is trained based on the training sample set to distill the first sub-model.

3. The method according to claim 1 or 2, wherein, The method involves clustering the multiple edge devices based on their performance parameters to obtain multiple device clusters, including: Based on the performance parameters of the multiple edge devices, the performance similarity between the edge devices is determined; Clustering analysis is performed based on the performance similarity between edge devices to obtain the multiple device clusters.

4. The method according to claim 1, wherein, Knowledge distillation is performed on the second sub-model of each device cluster to obtain the global student model, including: The distilled samples are input into the second sub-model of each equipment cluster to obtain the processing results of the second sub-model of each equipment cluster on the distilled samples; The processing results of each second sub-model on the distilled sample are fused to obtain a soft target; The global student model is trained based on the soft objective and the distillation samples to minimize the loss between the global student model's processing result of the distillation samples and the soft objective.

5. The method according to claim 4, wherein, The process of fusing the processing results of each second sub-model on the distilled sample to obtain a soft target includes: The weights of each second sub-model are determined based on its accuracy on the validation set; where higher accuracy corresponds to higher weight. The soft target is obtained by weighted summation of the processing results of each second sub-model on the distilled sample.

6. The method according to claim 1, wherein the preset condition includes at least one of the following: The difference between the first comparison result and the second comparison result of the pseudo-sample is less than a preset threshold. The trend of the difference between the first comparison result and the second comparison result of the pseudo-sample is from continuously decreasing to decreasing at the end.

7. The method according to claim 1, wherein the augmented dataset is generated by each device cluster based on the diffusion model, and the global student model is used to construct the reverse process of the diffusion model.

8. A federated learning method, comprising: The performance parameters of the current edge device are sent to the server, so that the server can perform cluster analysis on the multiple edge devices based on the performance parameters of the multiple edge devices to obtain multiple device clusters, and perform knowledge distillation on the global model in the cloud to obtain a first sub-model applicable to the target device cluster; wherein, the target device cluster is the device cluster in which the current edge device is located; the sampled image is used as the sample for distilling the first sub-model; Federated learning is performed based on the first sub-model, including: The first sub-model is trained using the local dataset of the current edge device to obtain the sub-model to be aggregated; A second sub-model of the first sub-model is generated based on the target device cluster and the sub-model to be aggregated. Based on the second sub-model and the global student model of each device cluster, each device cluster is guided to generate an augmented dataset that meets preset data distribution characteristics; wherein, for each device cluster, the preset data distribution characteristics are the data distribution characteristics of the local dataset within the device cluster; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster, including: For each device cluster, obtain pseudo samples generated by the device cluster based on the local data distribution characteristics of the device cluster; For the pseudo-sample, perform the following operations in a loop: The pseudo-samples are processed based on a global student model to obtain the first comparison result of the pseudo-samples; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster; and... The pseudo-samples are processed based on each second sub-model to obtain multiple processing results for the pseudo-samples; The multiple processing results are fused to obtain the second comparison result of the pseudo sample; If the difference between the first comparison result and the second comparison result meets the preset conditions, the pseudo-sample is added to the augmented dataset; the augmented dataset is used to train the cloud global model. If the difference between the first comparison result and the second comparison result does not meet the preset condition, the device cluster is requested to update the pseudo sample until the preset condition is met.

9. The method according to claim 8, wherein, Training the first sub-model using the local dataset of the current edge device includes: The first sub-model is trained iteratively a preset number of times using the local dataset of the current edge device.

10. The method according to claim 8, wherein, The step of generating a second sub-model based on the target device cluster and the sub-model to be aggregated includes: When the current edge device is the master device of the target device cluster, multiple sub-models to be aggregated are obtained from the training of each edge device in the target device cluster. The multiple sub-models to be aggregated are aggregated to obtain the second sub-model of the first sub-model.

11. The method according to claim 10, wherein, The aggregation process of the plurality of sub-models to be aggregated to obtain the second sub-model of the first sub-model includes: A weighted average is taken of the model parameters of the multiple sub-models to be aggregated to obtain the second sub-model of the first sub-model; or, The average of the model parameters of the multiple sub-models to be aggregated is used to obtain the second sub-model of the first sub-model.

12. The method according to claim 10, wherein, The aggregation process of the plurality of sub-models to be aggregated to obtain the second sub-model of the first sub-model includes: When the validation set includes multiple categories, the sub-model with the highest prediction accuracy for each category is determined as an intermediate sub-model to obtain multiple intermediate sub-models. The average of the model parameters of the multiple intermediate sub-models is calculated to obtain the second sub-model.

13. The method according to any one of claims 8-12, further comprising: The second sub-model is sent to the server.

14. The method according to claim 13, wherein, Obtaining pseudo-samples generated by the device cluster based on the local data distribution characteristics of the device cluster includes: Pseudo-samples are generated based on a diffusion model; wherein the diffusion model learns the data distribution characteristics of the local dataset of the current edge device; and the reverse process of the diffusion model is constructed based on the global student model.

15. A federated learning device, comprising: The acquisition module is used to acquire performance parameters of multiple edge devices; The clustering module is used to perform cluster analysis on the multiple edge devices based on their performance parameters to obtain multiple device clusters; The distillation module is used to perform knowledge distillation on the cloud-based global model for the multiple device clusters, so as to distill out multiple first sub-models; The sampled image is used as a sample to distill the first sub-model; The learning module is used to distribute the multiple first sub-models to the corresponding device clusters for federated learning; Each device cluster is used to train the corresponding first sub-model to obtain the second sub-model; The device further includes a guidance module for guiding each device cluster to generate an augmented dataset that satisfies preset data distribution characteristics based on the second sub-model and the global student model of each device cluster; wherein, for each device cluster, the preset data distribution characteristics are the data distribution characteristics of the local dataset within the device cluster; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster; the guidance module includes: The acquisition submodule is used to acquire pseudo samples generated by the device clusters based on the local data distribution characteristics of the device clusters for each device cluster. The optimization submodule is used to perform the following operations in a loop for pseudo-samples: The pseudo-samples are processed based on the global student model to obtain the first comparison result of the pseudo-samples; and... Based on the processing of pseudo-samples by each second sub-model, multiple processing results of pseudo-samples are obtained; The results of multiple processing steps are fused to obtain the second comparison result of the pseudo sample; If the difference between the first and second alignment results meets the preset conditions, the pseudo-sample is added to the amplified dataset. If the difference between the first comparison result and the second comparison result does not meet the preset conditions, request the device cluster to update the pseudo sample until the preset conditions are met. The learning module is also used to train the cloud-based global model based on the augmented dataset of each device cluster.

16. The apparatus according to claim 15, wherein, The distillation module is specifically used for: For each of the multiple device clusters, execute the following respectively: A training sample set adapted to the performance parameters of the device cluster was sampled from a publicly available dataset; as well as, Create an initial model that adapts to the performance parameters of the device cluster; Using the cloud-based global model as the teacher model and the initial model as the student model, the initial model is trained based on the training sample set to distill the first sub-model.

17. The apparatus according to claim 15 or 16, wherein, The clustering module is specifically used for: Based on the performance parameters of the multiple edge devices, the performance similarity between the edge devices is determined; Clustering analysis is performed based on the performance similarity between edge devices to obtain the multiple device clusters.

18. The apparatus according to claim 15, wherein, The distillation module includes: The input submodule is used to input the distillation sample into the second sub-model of each equipment cluster, and obtain the processing results of the second sub-model of each equipment cluster on the distillation sample; The fusion submodule is used to fuse the processing results of each second sub-model on the distilled sample to obtain a soft target; The distillation submodule is used to train the global student model based on the soft target and the distillation samples to minimize the loss between the global student model's processing result of the distillation samples and the soft target.

19. The apparatus according to claim 18, wherein, The fusion submodule is specifically used for: The weights of each second sub-model are determined based on its accuracy on the validation set; where higher accuracy corresponds to higher weight. The soft target is obtained by weighted summation of the processing results of each second sub-model on the distilled sample.

20. The apparatus according to claim 15, wherein the preset conditions include at least one of the following: The difference between the first comparison result and the second comparison result of the pseudo-sample is less than a preset threshold. The trend of the difference between the first comparison result and the second comparison result of the pseudo-sample is from continuously decreasing to decreasing at the end.

21. The apparatus of claim 15, wherein the augmented dataset is generated by each device cluster based on a diffusion model, and the global student model is used to construct the reverse process of the diffusion model.

22. A federated learning device, comprising: The sending module is used to send the performance parameters of the current edge device to the server, so that the server can perform cluster analysis on the multiple edge devices based on the performance parameters of the multiple edge devices to obtain multiple device clusters, and perform knowledge distillation on the global model in the cloud to obtain a first sub-model applicable to the target device cluster; wherein, the target device cluster is the device cluster to which the current edge device belongs; the sampled image is used as the sample for distilling the first sub-model; The training module includes: The model training submodule is used to train the first sub-model using the local dataset of the current edge device to obtain the sub-model to be aggregated. The aggregation submodule is used to generate a second submodel of the first submodel based on the target device cluster and the submodel to be aggregated; It also includes a generation module for: Based on the second sub-model and the global student model of each device cluster, each device cluster is guided to generate an augmented dataset that meets preset data distribution characteristics; wherein, for each device cluster, the preset data distribution characteristics are the data distribution characteristics of the local dataset within the device cluster; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster, including: For each device cluster, obtain pseudo samples generated by the device cluster based on the local data distribution characteristics of the device cluster; For the pseudo-sample, perform the following operations in a loop: The pseudo-samples are processed based on a global student model to obtain the first comparison result of the pseudo-samples; the global student model is obtained by knowledge distillation of the second sub-model of each device cluster; and... The pseudo-samples are processed based on each second sub-model to obtain multiple processing results for the pseudo-samples; The multiple processing results are fused to obtain the second comparison result of the pseudo sample; If the difference between the first comparison result and the second comparison result meets the preset conditions, the pseudo-sample is added to the augmented dataset; the augmented dataset is used to train the cloud global model. If the difference between the first comparison result and the second comparison result does not meet the preset condition, the device cluster is requested to update the pseudo sample until the preset condition is met.

23. The apparatus according to claim 22, wherein, The model training submodule is specifically used to iteratively train the first sub-model a preset number of times using the local dataset of the current edge device.

24. The apparatus according to claim 22, wherein, The aggregation submodule is specifically used for: When the current edge device is the master device of the target device cluster, multiple sub-models to be aggregated are obtained from the training of each edge device in the target device cluster. The multiple sub-models to be aggregated are aggregated to obtain the second sub-model of the first sub-model.

25. The apparatus according to claim 24, wherein, The aggregation submodule is specifically used for: A weighted average is taken of the model parameters of the multiple sub-models to be aggregated to obtain the second sub-model of the first sub-model; or, The average of the model parameters of the multiple sub-models to be aggregated is used to obtain the second sub-model of the first sub-model.

26. The apparatus according to claim 24, wherein, The aggregation submodule is specifically used for: When the validation set includes multiple categories, the sub-model with the highest prediction accuracy for each category is determined as an intermediate sub-model to obtain multiple intermediate sub-models. The average of the model parameters of the multiple intermediate sub-models is calculated to obtain the second sub-model.

27. The apparatus according to any one of claims 22-26, The sending module is also used to send the second sub-model to the server.

28. The apparatus according to claim 22, wherein, The generation module includes: The sample generation submodule is used to generate pseudo-samples based on the diffusion model.

29. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-14.

30. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-14.

31. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-14.

Images