A prototype-based heterogeneous edge-end collaborative learning method and edge-end system

Abstract

The application discloses a kind of based on prototype's heterogeneous edge end collaborative learning method and edge end system, focus on solving multiple edge end collaborative scene under the challenge caused by end side data heterogeneity and edge end model heterogeneity.The method realizes the collaborative learning of heterogeneous edge end model by introducing global prototype aggregation training and global prototype-based data generation mechanism, effectively cope with the diversified edge end scene demand.This application considers the time and resource consumption required for device training in the scene of limited edge end resources.By introducing prototype as the only knowledge transfer method, the knowledge sharing burden in traditional edge end collaborative learning is greatly reduced.This not only breaks through the limitation of traditional methods that cannot train heterogeneous models based on prototypes, breaking the application barrier of heterogeneous models in edge end collaboration, but also significantly reduces the resource consumption required for training in resource-limited edge end scene.The application significantly improves the training performance of edge end heterogeneous model.

Description

Technical Field

[0001] This invention belongs to the field of edge computing, specifically relating to a prototype-based heterogeneous edge-end collaborative learning method and edge-end system. Background Technology

[0002] Cloud-edge-device collaborative learning is currently the mainstream training architecture for edge devices. It utilizes the resources of edge devices for local training, while edge servers perform small-scale knowledge aggregation, and finally, the cloud processes and distributes the data. While this architecture effectively utilizes the resources of edge devices, it still suffers from high communication latency and heavy transmission burden. Edge-device collaborative AI technology migrates cloud functions to edge servers, jointly training models through continuous knowledge sharing between edge servers and edge devices. Both edge servers and edge devices perform model training and inference; the larger the model on the edge server, the greater its potential to assist inference on the edge device. Communicating only with edge servers, rather than with the cloud, can reduce communication latency and improve user experience in many commercial and industrial scenarios.

[0003] In practice, the knowledge transfer burden of edge-device collaboration must be considered, as it directly impacts latency and energy consumption, further determining whether resource-constrained edge devices can complete the expected training task. Current methods train by transmitting gradients and models, intermediate results such as logits, and processed training data. However, since the model needs to be shared in each round, the transfer burden of gradient and model sharing is enormous, making it unsuitable for heterogeneous model training. Furthermore, while logits can improve the performance of heterogeneous models, their transfer cost is proportional to the amount of data, which is often unacceptable for terminal devices. Processing training data transfer to train models inevitably incurs high transfer costs and potential user privacy leaks. In contrast, prototypes are calculated as the average feature vector of data belonging to the same class, significantly reducing the transfer burden. Moreover, it is used for few-shot learning because it can handle data heterogeneity, demonstrating its great potential in edge-device collaborative heterogeneous learning. However, current prototype-based research only focuses on improving edge model performance, not edge-device heterogeneous models, ignoring resource-constrained scenarios with multiple edges and multiple terminal devices. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a prototype-based edge-to-edge collaborative learning method, which is suitable for edge-to-edge systems consisting of multiple end-side devices and multiple edge-side servers. The aim is to reduce the transmission burden of edge-to-edge collaborative learning and achieve collaborative learning of heterogeneous edge-to-edge models by transmitting only lightweight prototypes.

[0005] The present invention also provides a corresponding edge system.

[0006] To achieve the above-mentioned objectives, the technical solution of the present invention is as follows:

[0007] Firstly, a prototype-based heterogeneous edge-end collaborative learning method is provided, the method comprising:

[0008] (1) The terminal device and all edge servers confirm their affiliation, and all edge servers randomly select one server as the aggregated edge server;

[0009] (2) The terminal device initializes the local model, performs local training in combination with the global prototype issued in the previous round, and sends the locally trained terminal prototype to the corresponding edge server. The terminal prototype is the average value of the feature vectors of all samples divided by category and calculated by category after the local model training of the terminal device.

[0010] (3) The edge server aggregates the end prototypes of all terminal devices within its jurisdiction into an edge prototype, and sends the edge prototype to the aggregation edge server. The edge prototype is obtained by aggregating the end prototypes according to the category distribution of terminal device data.

[0011] (4) Aggregate edge servers according to their category distribution and aggregate the edge models of all edge servers into a global prototype;

[0012] (5) The aggregation edge server distributes the global prototype to the edge server, and the edge server distributes the global prototype to the terminal device.

[0013] (6) All edge servers generate data on the specified category prototype using the global prototype and pre-trained generative model to train the edge model;

[0014] (7) Determine whether the edge model meets the convergence condition. If it does not converge, return to step (2) to continue training; otherwise, end the training.

[0015] Furthermore, the terminal device initializes the local model and performs local training using the global prototype distributed in the previous round, including:

[0016] The terminal device initializes the local model, prepares training data, and interacts with the edge server to determine whether there is a global prototype. If there is, the global prototype is used as the target to be approximated during the training process, and training is carried out in combination with local data. If there is no global prototype, training is carried out using only local data.

[0017] Furthermore, each terminal device performs local training on its own local data to minimize its own loss function, which is calculated using the following formula: in It is the cross-entropy loss function; y is the output of the terminal device model to the sample data x, and y is the label corresponding to the sample data x; Represents the global prototype Harmony Prototype The distance between them; λ is the control parameter for loss.

[0018] Furthermore, the formula for calculating the terminal prototype generated by the terminal device is as follows: in It is the prototype of the j-th class sent by terminal device n to edge server e; the subscript ne indicates that terminal device n and edge server e are associated. and f ne (x) represent the feature vectors of the j-th type of data possessed by terminal device n and the feature vectors of data x after feature extraction, respectively.

[0019] Furthermore, the calculation formula for edge servers generating edge prototypes is as follows: in N represents the prototype vector of the j-th class corresponding to edge server e; e and Each represents the set of terminal devices under the jurisdiction of edge server e and the number of samples of the corresponding j-th class.

[0020] Furthermore, the aggregation formula for the global prototype by the aggregation edge server is as follows: in E represents the number of samples of the j-th class owned by edge server e, where E is the set of edge servers.

[0021] Furthermore, all edge servers generate data on specified category prototypes using the global prototype and pre-trained generative models, including:

[0022] All edge servers randomly generate several noise vectors using a Gaussian distribution. These noise vectors are combined with the global prototype to form new feature vectors. These new feature vectors are then used as input to a pre-trained generative model to generate data by category. The generated data will be used to train heterogeneous edge models on the edge servers.

[0023] Secondly, an edge system is provided, including a number of terminal devices and a number of edge servers, wherein the terminal devices and the edge servers cooperate to complete model training.

[0024] In one iteration, the terminal device is configured to: initialize a local model, perform local training in conjunction with the global prototype issued in the previous round, and send the locally trained terminal prototype to the corresponding edge server. The terminal prototype is the average value of the feature vectors of all samples divided by category and calculated by category after the local model training of the terminal device.

[0025] The edge server is configured to: aggregate end prototypes of all terminal devices within its jurisdiction into edge prototypes, send the edge prototypes to the aggregation edge server, wherein the edge prototypes are obtained by aggregating end prototypes according to the category distribution of terminal device data, and the aggregation edge server is a server randomly selected from a number of edge servers; receive global prototypes from the aggregation edge server and send global prototypes to terminal devices; and generate data on specified category prototypes using a pre-trained generative model based on the global prototypes, thereby training the edge model.

[0026] The aggregated edge server is configured to: aggregate the edge models of all edge servers into a global prototype according to the category distribution of edge servers; and distribute the global prototype to the edge servers.

[0027] Beneficial Effects: This invention proposes a method for heterogeneous edge-device collaborative learning applicable to multiple terminal devices and multiple edge server frameworks, relying solely on lightweight prototype transmission. The method first obtains end prototypes through terminal devices, then aggregates them to obtain edge prototypes, which are then aggregated into a global prototype. The global prototype can not only be used for training end devices but also serve as a central vector in a multi-dimensional space, mixed with random noise to obtain different vectors. Then, a generative model processes these vectors to generate samples, which are used to train the edge model. The method completes heterogeneous model training by relying on both aggregated prototypes and prototype-based data generation, offering the following advantages compared to existing technologies:

[0028] (1) The communication cost of this method is low, making it suitable for edge communication. The cost of transmitting the prototype is reduced to at least 1 / 20 of the cost of transmitting the model / gradient / sample, which greatly saves the communication overhead of edge collaborative learning and allows more clients and one edge to interact under limited bandwidth conditions.

[0029] (2) The prototype transmitted by this method is the average of the sum of features of multiple samples. After obtaining the prototype, the attacker cannot infer the features of a single sample from the aggregated prototype. Thanks to the transmission of only the prototype, attack methods based on models / gradients / samples will not cause privacy violations to the collaborative learning of this invention;

[0030] (3) This invention has good scalability. It not only improves the performance of heterogeneous edge models by transmitting lightweight prototypes, but can also be combined with other logits-based methods to further enhance performance. In addition, it can replace edge learning components in the current cloud architecture and support the collaboration of heterogeneous models in various industry sectors by sharing prototypes, bringing new ideas to the knowledge sharing method of collaboration between large and small models. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the following drawings are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of a prototype-based edge collaborative learning method.

[0033] Figure 2 This is an architecture diagram of a prototype-based edge collaborative learning method.

[0034] Figure 3 This is a comparison chart showing the accuracy of the prototype-based edge collaborative learning method on the comparative dataset with a Dirichlet distribution parameter of 0.1.

[0035] Figure 4 This is a comparison chart showing the edge-side model accuracy of the prototype-based edge collaborative learning method on the comparative dataset with a Dirichlet distribution parameter of 0.1.

[0036] Figure 5 This is a comparison of the communication burden during a single round of communication for a prototype-based edge collaborative learning method. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of this invention. After reading this invention, any modifications of this invention in various equivalent forms by those skilled in the art fall within the scope defined by the appended claims.

[0038] This invention belongs to the field of edge computing, focusing on designing a collaborative learning method applicable to multiple terminal devices and multiple edge server frameworks. Combining the lightweight and class representative characteristics of class prototypes, it utilizes global prototype aggregation to assist training and prototype-based data generation to complete the collaborative training of heterogeneous edge-device models. The scenario of heterogeneous edge-device models is assumed as follows: there are E := {1,...,E} edge servers and N := {1,...,N} terminal devices, and the models on the edge servers (edges) and the models on the terminal devices (devices) are heterogeneous. The model on the edge is defined as ω. e The model on the defined end is ω n Different endpoints and edges perform collaborative training. Because edge servers possess strong computing power, they can communicate with each other for prototype transmission. Each edge server covers only a subset of endpoint devices, and each edge server e has its own set of endpoint devices N.e There are |N e Each terminal device possesses a dataset of [number]. Where X i Let Y represent the dataset of the i-th client. i Let represent the labels corresponding to the dataset owned by the i-th client. The total number of samples from all terminal devices is... Each terminal device has a different number of classes, i.e., |C n The following section will demonstrate the implementation process of the prototype-based edge collaborative learning method through specific examples. The entire process is as follows: Figure 1 As shown, the specific implementation method is as follows:

[0039] Step 1: First, obtain the topology of the current network structure, in order to Figure 2 For example, six terminal devices correspond to two edge servers. The Dirichlet distribution is used to simulate data heterogeneity. A parameter of 1 is set to randomly generate a corresponding training dataset for each device. After random data generation, each terminal device has both a training dataset and a test dataset from the CIFAR-100 dataset. The first three terminal devices correspond to the first edge server, and the last three edge servers correspond to the second edge server. Each terminal device runs a ResNet-8 model. Each edge server runs a ResNet-34 model. The generative model on each edge server is a pre-trained StyleGAN. The rightmost edge server is randomly selected as the aggregation edge server (also referred to as the aggregation server in this invention). Besides fulfilling the responsibilities of a normal edge server, it also needs to collect and aggregate edge prototypes and distribute the aggregated global prototypes to other edge servers.

[0040] Step 2: All terminals randomly initialize their local models, with an initial global training epoch of 100 epochs, a batch size of 10, and a learning rate of 0.005, decaying by 10% after 20 epochs. Each terminal checks if it has received the global prototype from its respective edge server. If no global prototype is received, local training proceeds normally. If a global prototype is detected, a prototype loss is added to the local cross-entropy, ensuring that the prototypes corresponding to the categories in the local dataset are close to the global prototype during training. After training, all terminals partition the feature vectors of all samples by category and calculate the average value for each category to obtain the terminal prototype. Additionally, all terminals need to test the accuracy of the trained model on the test set and send the results to the corresponding edge server for recording.

[0041] Each terminal device performs local training on its own local data to minimize a loss function, the formula for which the loss function is calculated is: in It is the cross-entropy loss function. It is the output of the terminal device to the sample data x. The global prototype was measured. and terminal prototype (end prototype) The distance between them. λ is the control parameter for the loss. If the terminal device does not receive the global prototype, the loss function is... This represents the calculation of cross-entropy loss using an end-to-end model, where y is the label corresponding to the sample data x.

[0042] The formula for calculating the terminal prototype generated by the terminal device is as follows: in This is the prototype of the j-th class sent by terminal device n to edge server e. The subscript ne indicates that terminal device n and edge server e are associated. and f ne (x) represent the feature vectors of the j-th type of data possessed by terminal device n and the feature vectors of data x after feature extraction, respectively.

[0043] Step 3: After all terminal devices obtain the terminal prototype, they send their own terminal prototype to the corresponding edge server according to the topology determined in Step 1. For example, terminal device n will send its terminal prototype. This information is then sent to the corresponding edge server e. At this point, all end devices record the size of the end prototype, and this information will be used as a transmission burden metric to provide to the edge server for communication burden statistics.

[0044] Step 4: All edge servers receive the end prototypes from the terminal devices and aggregate them according to the category distribution of the data from the end devices under their jurisdiction to obtain the edge prototypes. Then, all edge servers send the edge prototypes to the random aggregation edge server determined in Step 1. The aggregation edge server calculates the category distribution of each edge server and aggregates the edge prototypes into a global prototype based on the category distribution information.

[0045] The calculation formula for edge servers to generate edge prototypes is as follows: in N represents the prototype vector of the j-th class corresponding to edge server e. e and Each represents the set of terminal devices under the jurisdiction of edge server e and the number of samples of the corresponding j-th class.

[0046] The formula for aggregating the global prototype using the aggregation edge server is as follows: in This represents the number of samples of the j-th class owned by edge server e.

[0047] Step 5: The global prototype is distributed from the aggregation edge server to other edge servers, which then distribute the global prototype to their respective terminal devices for the next training step. Similarly, all edge servers need to record the size of the global prototype; this information will be used as a transmission burden metric for communication burden statistics by the aggregation edge server.

[0048] Step 6: All edge servers use the received global prototype as the basis vector and randomly generate 1000 feature vectors of the same dimension as the global prototype using a normal distribution. These feature vectors correspond to the 100 classes of CIFAR-100. The generated random noise vector is then added to the global vector to represent different feature vectors in the sample space. All edge servers feed these generated feature vectors as input to the pre-trained generative model StyleGAN, generating 1000 samples for each of the 100 classes, with 10 samples per class. These samples are then used to train the edge model. Let the vector generated by the normal distribution be denoted as... The different feature vectors generated are Generative Models Generate a large number of samples X g The process is represented as Each edge server trains the heterogeneous model using generated samples. The number of training rounds and the number of generated samples can be determined based on the idle computing power of the edge server. In this example, the global number of training rounds for the edge server is set to 100, the batch size is 10, the learning rate is set to 0.005, and the learning rate decays at a rate of 10% after 20 rounds.

[0049] Step 7: Determine if the current terminal model and edge server model have converged. If the edge model does not converge, return to Step 2 and repeat the above process; otherwise, training ends, and the trained heterogeneous model is obtained. After training, test the model on the CIFAR-100 test dataset, obtain the test accuracy, and record it.

[0050] Furthermore, the performance of the prototype-based edge-end collaborative learning method described in this invention was verified through simulation experiments. The prototype-based edge-end collaborative learning method was implemented using Python and the PyTorch framework, and HierAVG, FedAgg, and FedGKT methods were selected for comparison. HierAVG uses a two-layer framework for edge-end collaboration to collaboratively train a homogeneous model through model and gradient transfer. FedGKT supports the training of heterogeneous edge-end models by sharing feature maps through knowledge distillation. FedAgg processes the raw data to generate bridge samples, which can then be used to train larger edge models using the edge model. The comparison criteria included the average test accuracy of the edge model and the highest test accuracy of the edge model after convergence or 100 rounds of communication. All programs were run on a standard Linux server equipped with an Intel(R) Xeon(R) Gold5218R CPU@2.10GHz, 64GB RAM, 1TB SSD, GeForce RTX 3060, and Ubuntu. Experimental results are as follows: Figures 3 to 5 As shown, where Figure 3 The invention (PBT) demonstrates that the end device achieves the highest test accuracy on the CIFAR-100 dataset, exceeding the highest accuracy among HierFAVG, FedAgg, and FedGKT by at least 35.08%, 25.02%, and 3.17%, respectively. Figure 4 The invention demonstrates that it achieves the highest accuracy of 78.12%, 53.16%, and 20.31% on the MNIST, CIFAR-10, and CIFAR-100 datasets, respectively, which is at least 7.96% higher than other methods. Figure 5 The study demonstrates the communication overhead of all methods in a single round when sharing knowledge at the edge. Thanks to the transmission of the global prototype vector, the results show that our method requires only 0.11M of transmission overhead, which is the lowest among all methods.

[0051] In summary, this prototype-based edge-to-edge collaborative learning method effectively achieves knowledge sharing on the edge side by using prototypes as a means of knowledge sharing. Utilizing prototypes for knowledge transfer significantly reduces the burden of knowledge transmission in single-round transfers, thereby greatly improving the overall speed of edge-to-edge collaborative learning. Furthermore, prototype sharing not only does not negatively impact the training of edge models but also enhances their training performance through data generation. Through prototype transfer, a rich source of knowledge can be continuously provided to the system throughout its entire lifecycle. This method can rapidly advance the model training process after the dataset is constructed on the edge side. More importantly, the training of the local model is not interfered with or delayed during prototype aggregation. Experimental results demonstrate that this method exhibits good adaptability on multiple different datasets and can effectively support the training of heterogeneous edge models. This indicates that the prototype-based edge-to-edge collaborative learning method has application potential and value in diverse application scenarios.

[0052] The present invention also provides an edge system, including a plurality of terminal devices and a plurality of edge servers, wherein the plurality of terminal devices and the plurality of edge servers complete model training through cooperation;

[0053] In one iteration, the terminal device is configured to: initialize a local model, perform local training in conjunction with the global prototype issued in the previous round, and send the locally trained terminal prototype to the corresponding edge server. The terminal prototype is the average value of the feature vectors of all samples divided by category and calculated by category after the local model training of the terminal device.

[0054] The edge server is configured to: aggregate end prototypes of all terminal devices within its jurisdiction into edge prototypes, send the edge prototypes to the aggregation edge server, wherein the edge prototypes are obtained by aggregating end prototypes according to the category distribution of terminal device data, and the aggregation edge server is a server randomly selected from a number of edge servers; receive global prototypes from the aggregation edge server and send global prototypes to terminal devices; and generate data on specified category prototypes using a pre-trained generative model based on the global prototypes, thereby training the edge model.

[0055] The aggregated edge server is configured to: aggregate the edge models of all edge servers into a global prototype according to the category distribution of edge servers; and distribute the global prototype to the edge servers.

[0056] The specific implementation methods of the training process of the terminal device and the edge server, the aggregation process of the edge server, and the aggregation process of the aggregated edge server can be referred to the description of the foregoing method embodiments, and will not be repeated here.

Claims

1. A prototype-based heterogeneous edge collaboration learning method, characterized in that, Includes the following steps: (1) The terminal device and all edge servers confirm their affiliation, and all edge servers randomly select one server as the aggregated edge server; (2) The terminal device initializes the local model, performs local training in combination with the global prototype issued in the previous round, and sends the local trained terminal prototype to the corresponding edge server. The terminal prototype is the average value of the feature vectors of all samples divided by category after the local model of the terminal device is trained. The process of the terminal device initializing the local model and performing local training in combination with the global prototype issued in the previous round includes: the terminal device initializes the local model, prepares training data, and determines whether there is a global prototype by interacting with the edge server. If there is a global prototype, the global prototype is used as the target to be approached during the training process and is used for training in combination with local data. If there is no global prototype, only local data is used for training. (3) The edge server aggregates the end prototypes of all terminal devices within its jurisdiction into an edge prototype and sends the edge prototype to the aggregation edge server. The edge prototype is obtained by aggregating the end prototypes according to the category distribution of terminal device data. (4) Aggregate edge servers according to their category distribution and aggregate the edge models of all edge servers into a global prototype; (5) The aggregation edge server distributes the global prototype to the edge server, and the edge server distributes the global prototype to the terminal device; (6) All edge servers generate data on the specified category prototype using the global prototype and the pre-trained generative model to train the edge model; wherein, all edge servers generate data on the specified category prototype using the global prototype and the pre-trained generative model includes: all edge servers randomly generate several noise vectors using Gaussian distribution, combine several noise vectors and the global prototype into new feature vectors, and provide these new feature vectors as input to the pre-trained generative model to generate data by category, and the generated data will be used for training the heterogeneous edge models on the edge servers; (7) Determine whether the edge model meets the convergence condition. If it does not converge, return to step (2) to continue training; otherwise, end the training.

2. The method according to claim 1, characterized in that, Each terminal device locally trains on its own local data to minimize its own loss function, which is calculated as wherein is a cross-entropy loss function; is an output of the terminal device model on the sample data , and y is a corresponding label of the sample data ; denotes a distance between the global prototype and the end prototype ; is a control parameter of the loss.

3. The method according to claim 1, characterized in that, The formula for calculating the terminal prototype generated by the terminal device is as follows: ,in Terminal equipment Send to edge server The The prototype of a class; subscript Represents terminal equipment and edge servers Related; and They represent terminal devices. The first Class data and data The feature vector after feature extraction.

4. The method according to claim 1, characterized in that, The calculation formula for edge servers to generate edge prototypes is as follows: ,in Represents edge server The corresponding number The prototype vector of each class; and Each represents a server belonging to the edge server. The set of terminal devices under jurisdiction and the corresponding third Number of samples in each class.

5. The method according to claim 4, characterized in that, The aggregation formula for the global prototype on the aggregation edge server is as follows: ,in Represents edge server The first The number of samples in each class This is a collection of edge servers.

6. An edge system, characterized in that, It includes several terminal devices and several edge servers, which collaborate to complete model training through the method described in any one of claims 1-5.

Images