A cloud edge end collaborative federated learning intelligent optimization method and device

Abstract

The application discloses a cloud-edge-end cooperative federated learning intelligent optimization method and device, and relates to the field of artificial intelligence.The method of the application reasonably constructs a federated learning framework topology based on a cloud edge end through a lightweight trainer, constructs a state of an environment of a round based on model parameters, training time, training power consumption and communication time, generates actions including edge aggregation frequency and terminal training round number through an agent decision model, and trains the edge and terminal devices according to the actions while collecting information to form a next round state.The federated learning framework and the decision model continuously interact to generate a large amount of decision trajectory information, which is used for updating the decision model until the model converges.The trained agent decision model can distribute different federated learning training round numbers according to the calculation speed, training power consumption and communication time of each device, thereby achieving the purposes of balancing calculation heterogeneity and data heterogeneity and reducing energy consumption overhead.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a cloud-edge-end collaborative federated learning intelligent optimization method. BACKGROUND

[0002] With the continuous improvement of chip computing power and the development of artificial intelligence technology, the current model is becoming more and more complex and large. Using traditional computing power equipment for calculation is time-consuming and resource-consuming, and for centralized learning, the user's data needs to be uploaded to the server, which violates the privacy agreement. Federated learning is a model training mechanism that arises with the development of artificial intelligence in the big data era. Using federated learning, users do not need to upload data, only the central server needs to issue the model to be trained to each client, and the client trains locally, and finally uploads the trained model to the central server for model aggregation. This method breaks down the data island and plays a role in privacy protection, while fully utilizing the computing power of edge devices, so it has been widely used and popularized in recent years.

[0003] However, federated learning also faces many problems, of which heterogeneity is a common problem in federated learning. Heterogeneity mainly includes two aspects: one is the heterogeneity of computing resources of devices, and the other is the heterogeneity of training data. In real life, the computing power of devices for federated learning training is different, which may be a computer, a mobile phone or other edge devices, so there is a significant difference in computing speed. And even if it is the same device, the programs running in the background at the same time may also be different, so each device has certain differences in resource occupation due to other programs, so different devices have different training times. The devices that train fast will be affected by the devices that train slowly, thus causing the "straggler effect". At the same time, terminal devices mostly have limited network bandwidth and battery power, and the power consumption of devices under different loads is different. If the training times of devices are uniformly set, it will not only cause additional time overhead, but also cause certain energy consumption overhead. The heterogeneity of training data includes the heterogeneity of data quantity and data distribution. Since different devices obtain different information, the number of data and the distribution of data under each terminal device are different, which easily causes the deviation of the trained model under the traditional federated learning algorithm, thus causing the problems of slowing down the training speed and reducing the convergence boundary.

[0004] To solve the problem of high communication latency caused by frequent aggregation of devices and cloud in traditional federated learning, a cloud-edge-based federated learning framework has been proposed and widely used. Unlike traditional federated learning, this framework introduces edge devices between the cloud and the devices. Multiple terminal devices and an edge device form a subnet for federated learning training, while multiple edge devices communicate with the cloud for model aggregation. The latency of communication between edge and terminal in the subnet is much lower than that of communication between edge and cloud. Therefore, using this framework can effectively solve the problem of frequent cloud communication overhead in traditional federated learning. Based on the above federated learning framework, how to correctly allocate the number of training rounds of edge and terminal devices is the key to solving the heterogeneity problem. Moreover, under the premise of ensuring the total accuracy of federated learning, it is also an important research direction of federated learning to minimize the budget overhead such as energy consumption.

[0005] To solve the above problem, the data-driven modeling method has high precision and training efficiency, and the data of edge and terminal devices is easy to collect. Applying the data-driven idea to the edge computing field can effectively analyze the accumulated training data and extract relevant information to guide the training process of federated learning. Deep reinforcement learning is one of the effective methods of data-driven modeling. It uses computers to automatically interact with the environment and can learn action strategies from interaction information, which is very suitable for scenarios where mathematical models are difficult to establish. In recent years, with the continuous improvement of computing power, deep reinforcement learning has been fully developed and has very outstanding performance in robot control, autonomous driving, computer games and other fields.

[0006] The optimization method based on cloud-edge collaboration is often multi-constrained and multi-objective. Currently, some works have successfully applied reinforcement learning to federated learning and other algorithms, and have shown good results. These works can be roughly divided into two categories. One is to optimize the heterogeneity of devices, using reinforcement learning to select appropriate devices to minimize training time to solve the problem of heterogeneous computing resources, or using reinforcement learning to select appropriate devices to improve the problem of non-independent and identically distributed data, but the above methods will cause a large overhead. The second category is to consider resource consumption from the perspective of reducing overhead by using reinforcement learning to select appropriate devices, but often ignores the heterogeneity of devices. At present, only a small number of works have considered the computing heterogeneity and energy consumption of devices, but there is still room for exploration in terms of combining data heterogeneity and computing resource utilization. And the common edge optimization algorithm based on reinforcement learning is based on the traditional federated learning, and has not combined more advanced cloud-edge frameworks, so there is room for improvement in communication latency. Therefore, it is of great significance to design an intelligent algorithm device that takes into account computing heterogeneity, data heterogeneity, and energy consumption overhead based on the cloud-edge framework. SUMMARY

[0007] In view of the problems in the background art, the present application provides a cloud-edge-terminal collaborative federated learning intelligent optimization method to take into account computing heterogeneity, data heterogeneity and energy consumption overhead, fully utilize the computing power of terminal devices and thus improve the performance of federated learning.

[0008] To achieve the above object, the present application provides the following solutions.

[0009] In one aspect, the present application provides a cloud-edge-terminal collaborative federated learning intelligent optimization method, comprising:

[0010] A federated learning framework based on cloud-edge-terminal is constructed; the federated learning framework comprises a central server, an edge device in communication with the central server and a terminal device in communication with the edge device; a lightweight trainer is implanted in the central server and a central model and an agent decision model are cached in the central server; an edge model is cached in the edge device; a local model is implanted in the terminal device;

[0011] The edge device downloads the edge model parameters cached by the edge device to the terminal devices connected to the edge device as the parameters of the local model implanted in the terminal devices;

[0012] The terminal devices train the local model using local data, and after a number of terminal training rounds, submit the trained local model to the corresponding edge device for aggregation to obtain an edge aggregated model;

[0013] The edge device collects the training time and training power consumption of the terminal devices in the training process as training data, and updates the cached edge model parameters using the edge aggregated model parameters, and when the edge aggregation frequency is reached, submits the aggregated edge model to the central server for aggregation to obtain a central aggregated model;

[0014] The central server updates the cached central model parameters using the central aggregated model parameters, and calculates the accuracy of the central model;

[0015] The return value is calculated according to the training data of the terminal devices in the training process and the accuracy of the central model;

[0016] The central server generates the state of the agent decision model according to the central model, the edge model and the training data of the terminal devices in the training process;

[0017] The agent decision model takes the state as input and outputs the corresponding action; the action includes the edge aggregation frequency and the number of terminal training rounds;

[0018] The central server generates decision trajectory information according to the state, the action and the return value, and updates the agent decision model using the decision trajectory information until the updated agent decision model converges to an optimal solution, obtaining a trained agent decision model;

[0019] The edge aggregation frequency of the edge device and the terminal training round number of the terminal device in the federated learning process are coordinated by using the trained agent decision model.

[0020] Optionally, the cloud-edge-end based federated learning framework is constructed, specifically comprising:

[0021] The lightweight trainer is implanted in the central server;

[0022] The central server sends the lightweight trainer to the plurality of terminal devices;

[0023] The plurality of terminal devices are pre-trained by using the lightweight trainer to obtain the computing resource prior parameters of the plurality of terminal devices; the computing resource prior parameters include the unit power consumption and the unit training time of the terminal device;

[0024] The plurality of terminal devices are clustered and connected to the corresponding edge devices for communication by using the computing resource prior parameters, and the cloud-edge-end based federated learning framework is constructed.

[0025] Optionally, the terminal device trains the local model by using the local data, specifically comprising:

[0026] The terminal device trains the local model by using the local data based on the formula ; wherein represents the round number of the local update; represents the round number of the edge aggregation; represents the round number of the cloud aggregation; represents the local model of the terminal device in the first edge aggregation in the first round of cloud aggregation; in the first edge aggregation in the first round of cloud aggregation; ; wherein is the learning rate; represents the risk function of the terminal device ; wherein represents the local model of the terminal device in the first edge aggregation in the first round of cloud aggregation.

[0027] Optionally, after the terminal training round number is reached, the trained local model is submitted to the corresponding edge device for aggregation to obtain an edge aggregation model, specifically comprising:

[0028] After the terminal training round number is reached, the trained local model is submitted to the corresponding edge device, based on the formula ; wherein represents the edge aggregation model.​​​The edge aggregation model is obtained by performing aggregation; wherein represents the edge aggregation model of the edge device in the terminal device in the edge aggregation in the terminal device connected to the edge device ; is the local data sample of the terminal device ; is the local data sample set of all terminal devices connected to the edge device ; represents the edge aggregation model of the edge device in the edge aggregation in the round of cloud-side aggregation.

[0029] Optionally, when the edge aggregation frequency is reached, the edge model after aggregation is submitted to the central server for aggregation to obtain a central aggregation model, and specifically includes:

[0030] When the edge aggregation frequency is reached , the edge model is submitted to the central server, and aggregation is performed based on the formula to obtain a central aggregation model; wherein represents the edge model of the edge device after aggregation in the round of cloud-side aggregation. represents the local data sample set of all terminal devices represents the set of all edge devices ; represents the central aggregation model obtained by the round of cloud-side aggregation.

[0031] Optionally, the accuracy of the central model is calculated, specifically including:

[0032] Obtain a test set of the central model

[0033] Test the accuracy of the central model using the test set.

[0034] Optionally, the reward value is calculated according to the training data of the terminal device in the training process and the accuracy of the central model, specifically including:

[0035] According to the training data of the terminal device in the training process and the accuracy of the central model, the formula is used to calculate the reward value; wherein is the reward value of the agent decision model in the round of cloud-side aggregation. and The first Wheel and First The accuracy of the central model obtained through cloud aggregation; For the first Cloud-based aggregation of terminal devices Training power consumption; and These are the first weighting coefficient and the second weighting coefficient, respectively.

[0036] Optionally, the central server generates the state of the agent decision-making model based on the central model, the edge model, and the training data from the terminal devices during the training process, specifically including:

[0037] The central server calculates the first [number] based on the training data from the terminal devices during the training process. Training time aggregated in the cloud and total power consumption ;

[0038] Central server according to the Central model of cloud aggregation , No. Edge model of cloud-based aggregation , No. Training time aggregated in the cloud and the Total power consumption of cloud-based aggregation Based on the formula Generate the first The state of the agent decision-making model in the cloud aggregation ;in For the first The communication time of cloud aggregation; This indicates that the matrix is ​​concatenated.

[0039] Optionally, the agent decision-making model takes the state as input and outputs the corresponding action, specifically including:

[0040] The agent decision model will use the state As input, output the first... Actions corresponding to cloud aggregation ;action Including the Edge aggregation frequency used in cloud aggregation Number of training rounds at the terminal .

[0041] In another aspect, the present application also provides a cloud-edge-terminal collaborative federated learning intelligent optimization device, comprising: a central server, an edge device in communication with the central server, and a terminal device in communication with the edge device; a lightweight trainer is implanted in the central server, and a central model and an agent decision model are cached in the central server; an edge model is cached in the edge device; a local model is implanted in the terminal device;

[0042] The edge device downloads the edge model parameters cached by the edge device to the terminal device connected to the edge device as the parameters of the local model implanted in the terminal device;

[0043] The terminal device trains the local model using local data, and after a number of terminal training rounds, submits the trained local model to the corresponding edge device for aggregation to obtain an edge aggregation model;

[0044] The edge device collects the training time and training power consumption of the terminal device in the training process as training data, and updates the cached edge model parameters using the edge aggregation model parameters, and when the edge aggregation frequency is reached, submits the aggregated edge model to the central server for aggregation to obtain a central aggregation model;

[0045] The central server updates the cached central model parameters using the central aggregation model parameters, and calculates the accuracy of the central model;

[0046] The central server calculates a reward value according to the training data of the terminal device in the training process and the accuracy of the central model;

[0047] The central server generates a state of an agent decision model according to the central model, the edge model, and the training data of the terminal device in the training process;

[0048] The central server takes the state as an input of the agent decision model to output a corresponding action; the action includes the edge aggregation frequency and the number of terminal training rounds;

[0049] The central server generates decision trajectory information according to the state, the action, and the reward value, and updates the agent decision model using the decision trajectory information until the updated agent decision model converges to an optimal solution to obtain a trained agent decision model;

[0050] The central server coordinates the edge aggregation frequency of the edge device and the number of terminal training rounds of the terminal device in the federated learning process using the trained agent decision model.

[0051] According to the specific embodiments of the present application, the following technical effects are disclosed:

[0052] The cloud edge end collaborative federated learning intelligent optimization method provided by the application constructs a federated learning framework topology structure of edge devices and terminal devices through a lightweight trainer, constructs the state of the current round of environment based on model parameters, training time, training power consumption and communication time, generates actions including edge aggregation frequency and terminal training round number through an agent decision model in the central server, and the edge devices and terminal devices train according to the actions, collect edge model parameters, training time, training power consumption and communication time information to form the state of the next round of environment, continuously interact based on the federated learning framework of the cloud edge end and the agent decision model to generate a large amount of decision trajectory information for updating the decision model until the decision model converges, and the trained agent decision model can allocate different federated learning training round numbers according to the calculation speed, training power consumption and communication time of each device, so as to balance calculation heterogeneity and data heterogeneity and reduce energy consumption overhead, and fully utilize the computing power of terminal devices to improve the performance of federated learning. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 A flowchart of the cloud edge end collaborative federated learning intelligent optimization method provided by the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0056] The purpose of the present application is to provide a cloud edge end collaborative federated learning intelligent optimization method to balance calculation heterogeneity, data heterogeneity and energy consumption overhead, fully utilize the computing power of terminal devices and improve the performance of federated learning.

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0058] Figure 1 A flowchart of the cloud edge end collaborative federated learning intelligent optimization method provided by the present application, see Figure 1A cloud edge end collaborative federated learning intelligent optimization method, comprising:

[0059] Step 1: Construct a cloud edge end based federated learning framework.

[0060] A lightweight trainer is implanted in the central server; the central server sends the lightweight trainer to a plurality of terminal devices; the plurality of terminal devices are pre-trained by using the lightweight trainer to obtain computing resource prior parameters of the plurality of terminal devices; the computing resource prior parameters include unit power consumption and unit training time of the terminal devices; the plurality of terminal devices are clustered by using the computing resource prior parameters and connected and communicated with corresponding edge devices to construct the cloud edge end based federated learning framework. The "cloud" in the cloud edge end based federated learning framework is the central server, the "edge" is the edge device in communication with the central server, and the "end" is the terminal device in communication with the edge device.

[0061] The basic architecture of the cloud edge end based federated learning framework of the application is that a lightweight trainer is deployed in the central server, the computing power and data information of each terminal device can be obtained through a small amount of training tasks, and a fixed topology cloud edge end architecture can be reasonably clustered through the information, and a deep reinforcement learning based intelligent agent decision model is deployed in the central server, the overall cloud edge end based federated learning framework is an environment, and through the continuous interaction between the model and the environment, the optimal global training parameter distribution scheme is learned, the global training parameters include edge aggregation frequency And the number of terminal training rounds The edge aggregation frequency refers to the frequency of the edge device submitting the edge model to the cloud; the number of terminal training rounds refers to the number of rounds of local model updating of the terminal device locally.

[0062] Therefore, in the cloud edge terminal federated learning framework of the application, a lightweight trainer is implanted in the central server and the central model and the agent decision model are cached. The edge model is cached in the edge device. The local model is implanted in the terminal device. The central model and the edge model are of the same type, and the input and output thereof are determined by the machine learning task, such as inputting picture pixels and outputting picture labels, or inputting voice fragments and outputting voice labels. The agent decision model is used to guide the aggregation frequency of the edge device and the terminal device in each round, the input thereof is the current "state" (including the compressed central model and edge model, the resource information parameters such as the training power consumption and training time of each device), and the output thereof is the "action" of the current round (the aggregation frequency of each edge device and terminal device in the current round). After the action is performed, the agent also obtains a "reward" to evaluate the good or bad of the action. The state-action-reward is a set of information in the reinforcement learning training process, and multiple state-action-rewards constitute a "trajectory". The agent updates the agent decision model by using the information of the trajectory, so that the "action" becomes better and better.

[0063] In practical applications, the central server can use cloud service devices such as Huawei Cloud and Ali Cloud; the edge and terminal devices can use Raspberry Pi, smart phones or personal computers.

[0064] The central server can obtain the prior parameters of the terminal device based on the computing resources by issuing a small amount of task training. The central server is also used to aggregate the edge model cached in the edge device, and is used to determine the terminal training round number and the edge aggregation frequency of the terminal device based on the terminal training round number and the edge aggregation frequency of the terminal device based on the terminal training round number

[0065] Step 2: The edge device issues the edge model parameters cached by the edge device to the terminal devices connected to the edge device as the parameters of the local model implanted in the terminal devices.

[0066] Step 3: The terminal device trains the local model by using the local data, and after the terminal training round number is reached, submits the trained local model to the corresponding edge device for aggregation to obtain the edge aggregation model.

[0067] The terminal device obtains local data samples such as picture sets and voice sets according to the training task, and trains the local model by using the local data, which can be represented as:

[0068] (1)

[0069] Equation (1) is the process of gradient descent update, wherein denotes the round number of local update; denotes the round number of edge aggregation; denotes the round number of cloud aggregation; is the label of the terminal device; then denotes the local model of the terminal device updated in the th round of edge aggregation in the th round of cloud aggregation in the th round of local update. is the learning rate; denotes the risk function of the terminal device . denotes the local model of the terminal device updated in the th round of edge aggregation in the th round of cloud aggregation in the th round of local update.

[0070] After the local model training is completed, the edge device collects the training data such as training time and training power consumption of the terminal device in the training process, and submits the training data to the central server in the subsequent edge aggregation process as part of the state of the next round and for the calculation of the return value.

[0071] The present application uses to represent the upper limit of the round of local training of the terminal device, which is referred to as the number of terminal training rounds. After the number of terminal training rounds is reached, the terminal device submits the trained local model to the corresponding edge device for aggregation to obtain an edge aggregation model, which is specifically represented as:

[0072] (2)

[0073] wherein denotes the trained local model of the terminal device updated in the th round of edge aggregation in the th round of cloud aggregation; is the label of the edge device, denotes the set of all terminal devices connected to the edge device ; is the local data sample of the terminal device ; is the local data sample set of all terminal devices connected to the edge device ; denotes the The first round of cloud aggregation Edge devices in sub-edge aggregation Edge aggregation model.

[0074] Step 4: The edge device collects the training time and power consumption of the terminal device during the training process as training data, and updates the cached edge model parameters using the edge aggregation model parameters. When the edge aggregation frequency is reached, the aggregated edge model is submitted to the central server for aggregation to obtain the central aggregation model.

[0075] The edge model of the edge device reaches a certain number of aggregations (i.e., edge aggregation frequency). After that, the edge model is submitted to the central server for aggregation, resulting in the central aggregated model, which is specifically represented as follows:

[0076] (3)

[0077] in Indicates the first Cloud-based aggregation of edge devices The edge model after aggregation; This represents the set of local data samples from all terminal devices; Represents all edge devices A set; Indicates the first The central aggregation model obtained from cloud aggregation.

[0078] Step 5: The central server updates the cached central model parameters using the central aggregated model parameters and calculates the accuracy of the central model.

[0079] First, a test set for the central model is retrieved from the central server. This test set is then used to test the accuracy of the central model. The test set is specific to the machine learning (federated learning) task, and its acquisition depends on the task. Each machine learning task typically includes a training set and a test set. The training set is used to train the model, and the test set is used to test the model's accuracy. The test set is fed into the model, and the accuracy of the model's predictions is its precision. For example, in an image classification task, the test set images are input into the model, and the output predicted labels are compared with the true labels to obtain the prediction accuracy.

[0080] Step 6: Calculate the reward value based on the training data from the terminal device during the training process and the accuracy of the central model.

[0081] The central server can determine the reward value of the agent's decision-making model in the current round based on the test accuracy of the central model and the training data obtained from edge devices. This reward value is used as a bonus for subsequent updates to the agent's decision-making model.

[0082] The reward value is calculated according to the training data of the terminal device in the training process and the accuracy of the central model, and is specifically represented as:

[0083] (4)

[0084] wherein is the reward value of the agent decision model in the i-th round of cloud aggregation, that is, the reward; and are the accuracies of the central model obtained by the i-th round and the j-th round of cloud aggregation, respectively; is the training power consumption of the terminal device in the i-th round of cloud aggregation; and are the first weight coefficient and the second weight coefficient, respectively. Because the goal of the agent is to maximize the reward value, according to the designed formula (4), if the accuracy is greater and the power consumption is smaller, the reward obtained is greater, so the action taken by the agent in the updating process will gradually be executed in the direction of achieving high accuracy and low power consumption.

[0085]

[0086] Step 7: The central server generates the state of the agent decision model according to the central model, the edge model and the training data of the terminal device in the training process.

[0087] The central server can collect information through the edge device, and the collected information includes edge model parameters, device computing power under the edge subnetwork, device power consumption under the edge subnetwork and communication time. The central server generates the state of the environment according to the collected information, and processes the state by using the agent decision model to obtain the round number output, which is further used to obtain the aggregated central model.

[0088] Specifically, the central server calculates the training time and the total power consumption of the i-th round of cloud aggregation according to the training data of the terminal device in the training process; in addition, the central server also obtains the communication time of the i-th round of cloud aggregation.

[0089] The central server generates the state of the agent decision model according to the central model of the i-th round of cloud aggregation , the edge model of the i-th round of cloud aggregation , the training time of the i-th round of cloud aggregation , the total power consumption of the i-th round of cloud aggregation , the communication time of the i-th round of cloud aggregation , and the accuracy of the central model of the i-th round of cloud aggregation .​​​​​​​ Cloud-aggregated communication time and the Total power consumption of cloud-based aggregation Based on the formula Generate the first The state of the agent decision-making model in the cloud aggregation . This indicates that the matrix is ​​concatenated.

[0090] It is important to note that due to the large number of parameters in the model, this invention uses principal component analysis to compress the parameters of multiple models, obtaining the final compressed model parameters to generate the state of the agent decision-making model. Experiments have confirmed that the compressed model parameters can accurately reflect the data distribution information of the device without infringing on privacy. Regarding the compressed model and other prior parameters of computing resources (including...) , , The final state input is constructed by concatenating the parameters. The model parameter compression process involves: collecting the central and edge models from the central server and edge devices; flattening each model parameter into a one-dimensional matrix and concatenating them; the rows of the concatenated matrix represent the number of devices, and the columns represent all parameters in the model; applying Principal Component Analysis (PCA) to reduce the dimensionality of the model parameters, resulting in a two-dimensional matrix composed of compressed model parameters. The number of rows in this two-dimensional matrix remains unchanged, still representing the number of devices, but the number of columns is significantly reduced due to PCA compression. In addition to the compressed model parameters, the state also includes resource information parameters such as training power consumption, training time, and communication time for each device. The parameters of each edge device are then concatenated to the compressed model parameters. The final state remains a two-dimensional matrix, but with several additional columns added after the basic compressed model parameters.

[0091] Step 8: The agent decision model takes the state as input and outputs the corresponding action.

[0092] State The corresponding normal distribution is generated by inputting it into the agent's decision-making model, and then sampling from the normal distribution can yield the first... Actions corresponding to cloud aggregation Specifically, the two values ​​output by the agent's decision-making model are taken as the mean and variance of a normal distribution. After obtaining the normal distribution, sampling and rounding are performed to obtain the action. ;action Including the Edge aggregation frequency used in cloud aggregation Number of training rounds at the terminal When edge devices or terminal equipment When the value is 0, it means that the edge device or the terminal device under the edge device does not participate in the current round of training. or When the value is not 0, the edge and the terminal device train according to the number of decision rounds.

[0093] Step 9: The central server generates decision trajectory information according to the state, action and reward value, and updates the agent decision model using the decision trajectory information until the updated agent decision model converges to an optimal solution, obtaining a trained agent decision model.

[0094] The central server is configured to generate a normal distribution for each edge device according to the state of the current round of the updated environment , and sample the normal distribution to obtain the edge aggregation frequency and the number of training rounds of the terminal device under the edge ; within a specified threshold time, the terminal device trains the local model using local data until the threshold time is exceeded, and the decision trajectory information is collected; the decision trajectory information includes: the state of all input agent decision models in the current round of training , the reward value of the agent decision model , the edge aggregation frequency and the number of terminal training rounds . The central server updates the agent decision model, and continues to make new decisions using the updated model until the updated agent decision model converges to an optimal solution, obtaining a trained agent decision model in the central server.

[0095] Step 10: Use the trained agent decision model to coordinate the edge aggregation frequency of the edge device and the number of terminal training rounds of the terminal device in the federated learning process.

[0096] In real life, federated learning has been widely used in privacy computing, for example, training a model with local data distributed in mobile phones, and collecting data on each user's mobile phone for centralized training is an invasion of privacy. Federated learning does not obtain data, but issues and receives models, which well protects the privacy of users. However, due to the different data distribution and data volume of each user, and the different training operation and network communication levels of different devices, there is strong heterogeneity. The trained agent decision model can be applied to actual cloud-edge-end-based federated learning training tasks, and the edge aggregation frequency and the number of terminal training rounds are determined in each round of federated learning training task to guide federated learning training.

[0097] In practical applications, if the action output by the agent decision model includes the edge aggregation frequency is 0, which means that the edge and its connected terminal device do not participate in this training, and if the terminal training round number of the terminal device is 0, which means that the terminal device does not participate in this training. In summary, the participating device set of this round of training can be obtained, and only the federal learning training model of the device participating in the training is issued during the training, thereby reducing unnecessary communication.

[0098] In summary, the cloud-edge-end collaborative federal learning intelligent optimization method provided by the application utilizes the federal learning framework based on cloud-edge-end, the terminal device and the edge device under the high-speed local area network form a subnetwork for frequent communication, the edge device is responsible for communication with the cloud, and the communication overhead of the federal learning training of the traditional cloud architecture is fully relieved. Meanwhile, the reinforcement learning is applied to train the agent for guiding the edge and terminal aggregation frequency of each round of federal learning training, and the agent decision model ensures that even in a very strong heterogeneous federal learning system, the method can very well optimize the training process of the federal learning, compared with the traditional federal learning, the accuracy of the federal learning training task is improved, and the overhead of the terminal device in the training process is greatly reduced.

[0099] Based on the method provided by the application, the application also provides a cloud-edge-end collaborative federal learning intelligent optimization device, which comprises a central server, an edge device in communication with the central server, and a terminal device in communication with the edge device; the central server is implanted with a lightweight trainer and caches a central model and an agent decision model; the edge device caches an edge model; the terminal device is implanted with a local model;

[0100] The edge device issues the edge model parameters cached by the edge device to the terminal devices connected thereto as the parameters of the local model implanted in the terminal devices;

[0101] The terminal device trains the local model by using the local data, and after the terminal training round number is reached, the trained local model is submitted to the corresponding edge device for aggregation to obtain an edge aggregation model;

[0102] The edge device collects the training time and training power consumption of the terminal device in the training process as training data, and updates the cached edge model parameters by using the edge aggregation model parameters, and when the edge aggregation frequency is reached, the aggregated edge model is submitted to the central server for aggregation to obtain a central aggregation model;

[0103] The central server updates the cached central model parameters by using the central aggregation model parameters, and calculates the accuracy of the central model;

[0104] The central server calculates the return value according to the training data of the terminal device in the training process and the accuracy of the central model;

[0105] The central server generates the state of the agent's decision-making model based on the training data from the central model, edge models, and terminal devices during the training process.

[0106] The central server uses the state as input to the agent's decision-making model to output the corresponding action; the action includes the edge aggregation frequency and the number of terminal training rounds.

[0107] The central server generates decision trajectory information based on the state, action, and reward value, and uses the decision trajectory information to update the agent decision model until the updated agent decision model converges to the optimal solution, thus obtaining the trained agent decision model.

[0108] The central server uses a trained agent decision-making model to coordinate the edge aggregation frequency of edge devices and the number of terminal training rounds of terminal devices during the federated learning process.

[0109] The following uses a Raspberry Pi, a personal computer, and a Huawei Cloud federated learning framework as an example to illustrate the implementation process of the cloud-edge-device collaborative federated learning intelligent optimization method and device of the present invention. In this embodiment, the lightweight trainer is deployed on a central server, i.e., a server located in Huawei Cloud; the edge devices are multiple desktop computers; and the terminal devices are composed of Raspberry Pis. The meanings of the various symbols in this embodiment are as follows: For the number of edge devices, For edge devices Number of Raspberry Pis and Edge Aggregation Frequency Number of terminal training rounds for each edge device connected to the terminal device It can be represented in vector form, that is ,in and They represent edge devices respectively. The corresponding terminal device Number of training rounds on the terminal and edge devices The edge aggregation frequency of the aggregation model itself towards the cloud. This indicates the training time for the terminal device. This indicates the maximum training time required for all terminal devices under the current edge device to train synchronously. This refers to the communication time between edge devices and the central server. Power consumption information for the set training threshold time. Indicates the first Power consumption of all terminal devices during round-robin communication The sum; For the first The accuracy of the central model aggregated in the cloud.

[0110] The method provided by the embodiment has the following implementation process:

[0111] S1: training all terminal devices by a lightweight trainer to obtain the corresponding computing resource of the terminal devices, clustering the terminal devices according to the computing power, placing the devices with similar computing power under the same edge device, and fixing the topology structure of the cloud-edge-terminal federated learning framework.

[0112] S2: initializing a central model and an agent decision model , and deploying the central model to edge devices and terminal devices as edge models and terminal models. The central server can obtain the state of the current environment including the parameters of the central model and the edge model , the training time , the communication time and the total power consumption according to the result of the lightweight trainer.

[0113] S3: inputting the state into the decision model to obtain the corresponding normal distribution, sampling according to the distribution can obtain an action , and assigning the corresponding global training parameters and to each edge device.

[0114] S4: after the edge device obtains the global training parameters, the edge device starts to guide the connected terminal devices to perform local training according to the corresponding parameters, wherein the local model is updated using formula (1). Meanwhile, after the training round of the terminal device reaches , the terminal device uploads the local model to the edge device for aggregation, and the aggregation formula is formula (2).

[0115] S5: after the edge aggregation number reaches the upper limit , the edge model is submitted to the central server for aggregation operation, and the aggregation formula is formula (3).

[0116] In addition to providing the edge model to the central server, the edge device also provides the corresponding collection information parameters, including the training time , the communication time and the total power consumption of the current round, which together with the central model and the edge model of the current round constitute the state of the next round.

[0117] Meanwhile, the reward value of the first round The calculation is performed by formula (4).

[0118] S6: Repeat S3-S5 multiple times until the total training time exceeds the threshold Then, a decision trajectory information is obtained .

[0119] Specifically, the personal computer edge provides the last round of central model , the last round of edge model , the last round of training time , the last round of communication time and the last round of total power consumption as the current state of the environment , that is, the state input of the agent decision model is , and the action of the edge device and the terminal device is obtained after the agent decision model . The accuracy and training power consumption of the central model in the adjacent two rounds are used as the value function of the agent decision model, that is, the reward value of the current round . The agent decision model of the central server takes the state information as input and outputs the action , which contains the edge aggregation frequency and the number of terminal training rounds . Based on the cloud edge federated learning framework, the parameters are trained to obtain the new round of central model , the new round of edge model , the new round of training time and communication time and the new round of total power consumption constitute the new state of the environment , so that the agent and the environment interact continuously to generate a large amount of trajectory information for updating the decision model , until the decision model converges.

[0120] S7: Repeat steps S2-S6 multiple times to obtain multiple decision trajectory information, and when the number of trajectories in the memory pool reaches a certain number, update the agent decision model using the multiple decision trajectory information, which is specifically implemented as:

[0121] (5)

[0122] (6)

[0123] (7)

[0124] wherein, denotes the parameters of the decision model , denotes the updated parameters; and denote the length and the number of decision trajectories, respectively; denotes the discount factor, generally a value between 0 and 1; , and denote the state, action and cumulative discounted return value of the i-th state on the j-th trajectory, respectively; the baseline denotes the average discounted return value of the trajectory; denotes the return value of the i-th state on the j-th trajectory. is the operation of calculating the gradient of the decision model . denotes the cumulative multiplication of the discount factor in the trajectory, which gives more weight to the recent actions by gradually reducing the value of future rewards. Until the cloud edge device finally converges to the optimal solution, the trained intelligent agent decision model is obtained, and the optimization of the federated training is completed. Based on the above description, compared with the current common technology, the method provided by the application has the following outstanding advantages:

[0125] 1. The application is based on the constructed cloud edge-based federated learning framework, uses advanced hierarchical federated learning algorithm, provides a set of intelligent optimization method idea, solves the balance problem of calculation heterogeneity and data heterogeneity by deploying the topology structure of edge devices and terminal devices, and allocating different training round numbers to different edge devices and terminal devices, and the final effect is greatly improved compared with the traditional federated learning;

[0126] 2. The application mainly uses deep reinforcement learning to solve the multi-objective and multi-constraint problem, and through the continuous interaction between the agent and the environment, the optimal scheme can be automatically learned without complex mathematical modeling process, which provides a new idea for optimizing the federated learning training process;

[0127] 3. While focusing on optimizing the accuracy of federated learning, the method of the application also optimizes the energy consumption of terminal devices, improves the training speed and accuracy of the model on the basis of as small energy consumption as possible, and guarantees the sustainability of federated training, thereby further meeting the needs of edge intelligent multi-objective optimization.

[0128] 3. While focusing on optimizing the accuracy of federated learning, the method of the application also optimizes the energy consumption of terminal devices, improves the training speed and accuracy of the model on the basis of as small energy consumption as possible, and guarantees the sustainability of federated training, thereby further meeting the needs of edge intelligent multi-objective optimization.

[0129] 3. While focusing on optimizing the accuracy of federated learning, the method of the application also optimizes the energy consumption of terminal devices, improves the training speed and accuracy of the model on the basis of as small energy consumption as possible, and guarantees the sustainability of federated training, thereby further meeting the needs of edge intelligent multi-objective optimization.

[0130] ​​The various embodiments described in this specification are presented for the purpose of illustrating the principles of the present application and its best mode of operation. Each of the embodiments described in this specification has been provided for the purpose of illustration and is not intended to limit the application.

[0131] The principles and implementations of the present application have been described above with the specific examples. The above description of the embodiments is only for the purpose of helping to understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation and application range of the present application can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A cloud-edge-device collaborative federated learning intelligent optimization method, characterized in that, include: Construct a federated learning framework based on cloud, edge, and device; The federated learning framework includes a central server, edge devices communicating with the central server, and terminal devices communicating with the edge devices; a lightweight trainer is embedded in the central server and the central model and the agent decision model are cached; the edge devices cache the edge models; and the terminal devices embed local models. The edge device sends the cached edge model parameters to the connected terminal device as parameters of the local model embedded in the terminal device; The terminal device uses local data to train the local model. After reaching the required number of training rounds, the trained local model is submitted to the corresponding edge device for aggregation to obtain the edge aggregated model. Edge devices collect training time and power consumption data of terminal devices during the training process as training data, and update the cached edge model parameters using edge aggregation model parameters. When the edge aggregation frequency is reached, the aggregated edge model is submitted to the central server for aggregation to obtain the central aggregation model. The central server updates the cached central model parameters using the central aggregated model parameters and calculates the accuracy of the central model. The reward value is calculated based on the training data from the terminal device during the training process and the accuracy of the central model. The central server generates the state of the agent's decision-making model based on the training data from the central model, edge models, and terminal devices during the training process. The agent decision model takes the state as input and outputs the corresponding action; the action includes the edge aggregation frequency and the number of terminal training rounds. The central server generates decision trajectory information based on the state, action, and reward value, and uses the decision trajectory information to update the agent decision model until the updated agent decision model converges to the optimal solution, thus obtaining the trained agent decision model. The trained agent decision-making model is used to coordinate the edge aggregation frequency of edge devices and the number of terminal training rounds of terminal devices during the federated learning process.

2. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 1, characterized in that, The construction of the cloud-edge-device-based federated learning framework specifically includes: A lightweight trainer is embedded in the central server; The central server sends the lightweight trainer to multiple terminal devices; A lightweight trainer is used to pre-train multiple terminal devices to obtain prior parameters of computing resources for the multiple terminal devices; the prior parameters of computing resources include the unit power consumption and unit training time of the terminal devices. By using prior parameters of computing resources, multiple terminal devices are clustered and connected to corresponding edge devices for communication, thus constructing a cloud-edge-device-based federated learning framework.

3. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 2, characterized in that, The terminal device trains a local model using local data, specifically including: Terminal devices utilize local data, based on formulas Train the local model; where Indicates the round number of the local update; Indicates the round number of edge aggregation; Indicates the round number of cloud aggregation; Indicates the first The first round of cloud aggregation In the second edge aggregation Rotating updated terminal equipment The local model; The learning rate; Indicates terminal device The risk function; Indicates the first The first round of cloud aggregation In the second edge aggregation Rotating updated terminal equipment The local model.

4. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 3, characterized in that, After reaching the required number of training rounds on the terminal, the trained local model is submitted to the corresponding edge device for aggregation to obtain an edge-aggregated model, specifically including: Reach the terminal training rounds Afterwards, the trained local model is submitted to the corresponding edge device, based on the formula. Aggregation is performed to obtain an edge aggregation model; where Indicates the first The first round of cloud aggregation Terminal devices in secondary edge aggregation The trained local model; Indicates edge device All connected terminal devices A set; For terminal devices Local data samples; For edge devices The local data sample set of all connected terminal devices; Indicates the first The first round of cloud aggregation Edge devices in sub-edge aggregation Edge aggregation model.

5. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 4, characterized in that, Once the edge aggregation frequency is reached, the aggregated edge models are submitted to the central server for further aggregation to obtain the central aggregated model. This process specifically includes: When the edge aggregation frequency is reached Then, the edge model is submitted to the central server, based on the formula. Aggregation is performed to obtain a central aggregation model; where Indicates the first Cloud-based aggregation of edge devices The edge model after aggregation; This represents the set of local data samples from all terminal devices; Represents all edge devices A set; Indicates the first The central aggregation model obtained from cloud aggregation.

6. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 5, characterized in that, The accuracy of the central model for calculation specifically includes: Obtain the test set for the central model; The accuracy of the central model is tested using a test set.

7. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 6, characterized in that, The calculation of the reward value based on the training data from the terminal device during the training process and the accuracy of the central model specifically includes: Based on the training data from the terminal devices during the training process and the accuracy of the central model, the formula is used. Calculate the return value; where For the first The return value of the intelligent agent decision-making model in the cloud aggregation; and The first Wheel and the first The accuracy of the central model obtained through cloud aggregation; For the first Cloud-based aggregation of terminal devices Training power consumption; and These are the first weighting coefficient and the second weighting coefficient, respectively.

8. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 7, characterized in that, The central server generates the state of the agent decision-making model based on the central model, edge models, and training data from terminal devices during the training process, specifically including: The central server calculates the first [number] based on the training data from the terminal devices during the training process. Training time aggregated in the cloud and total power consumption ; Central server according to the Central model of cloud aggregation , No. Edge model of cloud-based aggregation , No. Training time aggregated in the cloud and the Total power consumption of cloud-based aggregation Based on the formula Generate the first The state of the agent decision-making model in the cloud aggregation ;in For the first The communication time of cloud aggregation; This indicates that the matrix is ​​concatenated.

9. The cloud-edge-device collaborative federated learning intelligent optimization method according to claim 8, characterized in that, The agent decision-making model takes the state as input and outputs the corresponding action, specifically including: The agent decision model will use the state As input, output the first... Actions corresponding to cloud aggregation ;action Including the Edge aggregation frequency used in cloud aggregation Number of training rounds at the terminal .

10. A cloud-edge-device collaborative federated learning intelligent optimization device, characterized in that, include: A central server, edge devices communicating with the central server, and terminal devices communicating with the edge devices; a lightweight trainer is embedded in the central server and a central model and an agent decision model are cached; an edge model is cached in the edge devices; and a local model is embedded in the terminal devices. The edge device sends the cached edge model parameters to the connected terminal device as parameters of the local model embedded in the terminal device; The terminal device uses local data to train the local model. After reaching the required number of training rounds, the trained local model is submitted to the corresponding edge device for aggregation to obtain the edge aggregated model. Edge devices collect training time and power consumption data of terminal devices during the training process as training data, and update the cached edge model parameters using edge aggregation model parameters. When the edge aggregation frequency is reached, the aggregated edge model is submitted to the central server for aggregation to obtain the central aggregation model. The central server updates the cached central model parameters using the central aggregated model parameters and calculates the accuracy of the central model. The central server calculates the reward value based on the training data from the terminal devices during the training process and the accuracy of the central model; The central server generates the state of the agent's decision-making model based on the training data from the central model, edge models, and terminal devices during the training process. The central server uses the state as input to the agent's decision-making model to output the corresponding action; the action includes the edge aggregation frequency and the number of terminal training rounds. The central server generates decision trajectory information based on the state, action, and reward value, and uses the decision trajectory information to update the agent decision model until the updated agent decision model converges to the optimal solution, thus obtaining the trained agent decision model. The central server uses a trained agent decision-making model to coordinate the edge aggregation frequency of edge devices and the number of terminal training rounds of terminal devices during the federated learning process.

Images