Federated learning methods and related devices

Abstract

The application provides a federated learning method and related equipment. By comprehensively considering the delay information, energy consumption information and data label distribution information of the edge server, a target edge server is determined from a plurality of edge servers to participate in the training of a global model, thereby reducing the energy consumption of the federated learning process and improving the convergence speed of the global model, and further improving the learning accuracy of the federated learning. The scheme of the application solves the problem of low classification accuracy caused by uneven distribution of data labels due to the heterogeneity of edge servers.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, and particularly relates to a federated learning method and related equipment. BACKGROUND

[0002] With the development of Internet of Things technology, federated learning as a distributed machine learning paradigm has played a role in more and more fields, such as financial risk control, smart city and telemedicine, etc. In federated learning, edge servers use their local data to train and update the machine learning model required by the center server, and the edge servers only interact with the center server with model update information, effectively reducing many privacy risks brought by traditional machine learning source data aggregation, and greatly reducing the latency.

[0003] However, due to the heterogeneity of edge servers, the data label distribution is uneven, resulting in low classification accuracy, thereby causing the quality of the final global model aggregation to be low and the global model precision to be poor. SUMMARY

[0004] Therefore, the present application aims to provide a federated learning method and related equipment to solve or partially solve the above problems.

[0005] In a first aspect, the present application provides a federated learning method implemented by a center server, comprising:

[0006] obtaining time delay information, energy consumption information and data label distribution information of a plurality of edge servers; wherein the data label distribution information comprises the number of data samples, the number of data label types and the proportion of each data label type;

[0007] determining a target edge server from the plurality of edge servers according to the time delay information, the energy consumption information and the data label distribution information;

[0008] sending parameters of a global model to the target edge server;

[0009] obtaining parameters of a trained global model sent by the target edge server; wherein the parameters of the trained global model are obtained by the target edge server based on local data;

[0010] aggregating the parameters of the trained global model to obtain parameters of an updated global model;

[0011] sending the parameters of the updated global model to the target edge server.

[0012] In a second aspect, the present application provides a federated learning method implemented by an edge server, comprising:

[0013] sending the time delay information, the energy consumption information and the data label distribution information to a center server; wherein the data label distribution information comprises a number of data samples, a number of data label types and a proportion of each data label type;

[0014] in response to being the target edge server, receiving parameters of a global model sent by the center server; wherein the target edge server is determined by the center server according to the time delay information, the energy consumption information and the data label distribution information;

[0015] training the parameters of the global model based on local data to obtain trained parameters of the global model, and sending the trained parameters of the global model to the center server;

[0016] receiving updated parameters of the global model sent by the center server; wherein the updated parameters of the global model are obtained by the center server aggregating the trained parameters of the global model.

[0017] In a third aspect of the present application, a center server is provided, comprising:

[0018] a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the processor implements the method of the first aspect when executing the computer program.

[0019] In a fourth aspect of the present application, an edge server is provided, comprising:

[0020] a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the processor implements the method of the second aspect when executing the computer program.

[0021] In a fifth aspect of the present application, a federated learning system is provided, comprising:

[0022] the center server as described in the third aspect and the edge server as described in the fourth aspect.

[0023] As can be seen from the above, the federated learning method and related equipment provided by the present application determine a target edge server from a plurality of edge servers to participate in the training of a global model of a center server by comprehensively considering the time delay information, the energy consumption information and the data label distribution information of the edge servers, thereby reducing the energy consumption of the federated learning process and improving the convergence speed of the global model, and further improving the learning accuracy of the federated learning. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 A flowchart of a federated learning method implemented by a center server according to an embodiment of the application;

[0026] Figure 2 A flowchart of a federated learning method implemented by an edge server according to an embodiment of the application;

[0027] Figure 3 A structural diagram of a center server according to an embodiment of the application. DETAILED DESCRIPTION

[0028] In order to make the purposes, technical solutions and advantages of the application clearer, the embodiments of the application will be described in detail below with reference to the drawings.

[0029] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the application should be understood as the usual meanings understood by those skilled in the art to which the embodiments of the application belong. The terms "first", "second" and the like used in the embodiments of the application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and the like do not mean physical or mechanical connection, but can include electrical connection, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships may also change accordingly.

[0030] With the development of Internet of Things technology, federated learning as a distributed machine learning paradigm has played a role in more and more fields, such as financial risk control, smart city and telemedicine, etc. In federated learning, edge servers use their local data to train and update the machine learning model required by the center server, and the edge servers only interact with the center server with model update information, effectively reducing many privacy risks brought by traditional machine learning source data aggregation, and greatly reducing the latency.

[0031] However, the uneven distribution of data labels caused by the heterogeneity of edge servers is not considered in the federated learning scheme in the related art, resulting in poor accuracy of the final global model. In addition, due to different focuses of actual application requirements, the energy consumption of edge servers and the learning accuracy of federated learning need to be balanced.

[0032] Therefore, the embodiments of the present application provide a federated learning method and related equipment, which determine a target edge server from a plurality of edge servers to participate in the training of a global model by comprehensively considering the delay information, energy consumption information and data label distribution information of the edge servers, thereby reducing the energy consumption of the federated learning process and improving the convergence speed of the global model, and further improving the learning accuracy of the federated learning.

[0033] It should be noted that the center server in the embodiments of the present application can be a base station, and the edge server can be a user equipment (such as a computer, a mobile phone, an Internet of Things device, etc.), which is not specifically limited.

[0034] In the embodiments of the present application, each edge server does not transmit any data information to each other in order to ensure privacy; the center server and the edge server trust each other and can transmit correct model parameters and are not maliciously attacked.

[0035] In the embodiments of the present application, the center server and the edge server train the global model, wherein the edge server is responsible for training and updating the global model of the center server using its local data, and the center server is responsible for weighting and aggregating the parameters of the trained global model uploaded by the edge server, and finally obtains a model close to the centralized machine learning result after multiple global iterations.

[0036] In the embodiments of the present application, the local data of the edge server and the final global model obtained based on the center server and the edge server after multiple global iterations can be applied to various machine learning tasks, such as image classification, which is not specifically limited.

[0037] Reference Figure 1 is a flowchart of the federated learning method 100 implemented by the center server in the embodiments of the present application. As shown in Figure 1 , the method 100 can include the following steps.

[0038] Step S101, obtain the delay information, energy consumption (energy loss) information and data label distribution information of a plurality of edge servers; wherein the data label distribution information includes the number of data samples, the number of data label types and the proportion of each data label type.

[0039] In this embodiment, the latency information refers to the time consumption between the edge server submitting a request and receiving a response, and the latency information of the edge server includes a training latency and a transmission latency. Specifically, the training latency is the time for the edge server to train and update the global model of the central server using its local data, and the transmission latency is the time for uploading the parameters of the trained global model to the central server. It can be understood that, since the downlink bandwidth is much larger than the uplink bandwidth, the time for the central server to issue the global model to the edge server can be ignored. In this way, through reasonable optimization strategies, the latency of the edge server can be effectively reduced, thereby improving the overall performance.

[0040] In specific implementation, the method can be applied to a federated learning system, which can include one central server and N edge servers, and the set of edge servers is defined as N={1,..., N}, and n∈N; the total bandwidth is B, the total number of global iterations is T, and t∈T; let be the bandwidth allocation ratio of the nth edge server in the tth global iteration, then the allocated bandwidth of the nth edge server is

[0041] In specific implementation, the training latency of the nth edge server in the tth global iteration is

[0042]

[0043] where t represents the current global iteration round, tth round; U t，n represents the local training round of the nth edge server in the tth global iteration; c n represents the number of CPU cycles of the nth edge server for executing one data sample; D n represents the number of data samples of the nth edge server; f n represents the CPU cycle frequency of the nth edge server.

[0044] It can be understood that, in this embodiment, the calculation of the latency of the edge server is based on the CPU, without considering the use of GPU and the interference of physical factors such as heat during data transmission; and all data samples have the same data size (i.e., bit number).

[0045] In specific implementation, in response to the completion of the training of the global model, the edge server uploads the parameters of the trained global model to the central server based on orthogonal frequency division multiple access (OFDMA) to update the parameters.

[0046] ​According to Shannon theorem, the reachable transmission rate of the nth edge server is defined as:

[0047]

[0048] wherein, denotes the transmission rate, denotes the transmission power of the nth edge server in the tth global iteration round; N0 denotes the power spectral density of Gaussian white noise; G n denotes the channel gain between the nth edge server and the center server, and it is assumed that G n is constant in each global iteration round.

[0049] Further, the data size of the model parameters and the gradient is set as S n , and S n is constant, the transmission delay of the nth edge server in the tth global iteration round is:

[0050]

[0051] The total delay T n，t of the nth edge server in the tth global iteration round is:

[0052]

[0053] In some embodiments, the energy consumption information includes training energy consumption information and transmission energy consumption information. Specifically, the number of data samples of the edge server, the local training rounds, the CPU cycle frequency and the number of CPU cycles for executing one data sample are obtained, and the training energy consumption information is calculated according to the number of data samples, the local training rounds, the CPU cycle frequency and the number of CPU cycles for executing one data sample; the bandwidth allocation ratio and the transmission power of the edge server are obtained, and the transmission energy consumption information is calculated according to the bandwidth allocation ratio and the transmission power.

[0054] Further, the training energy consumption is calculated by the following formula:

[0055]

[0056] wherein, α n denotes the effective capacitance coefficient of the computing chip of the nth edge server.

[0057] According to the above transmission delay the transmission energy consumption is calculated by the following formula:

[0058]

[0059] then the total energy consumption E of the nth edge server in the tth global iteration n，t is:

[0060]

[0061] Further, the data label distribution information can be defined as is a binary variable, and in response to the nth edge server participating in training in the tth global iteration, then in response to the nth edge server not participating in training in the tth global iteration, then

[0062] then the total energy consumption E of all edge servers participating in training in the tth global iteration g is:

[0063]

[0064] It can be understood that, since the edge servers are different physical entities, it is necessary to consider consuming as low total energy consumption as possible while ensuring the training effect on the global model of the center server.

[0065] In the related art, due to the biased data distribution between edge servers, federated learning in a non-independent and identically distributed scenario can cause low classification accuracy problems. Non-independent and identically distributed has always been a key problem in federated learning, which refers to the case that sample data does not conform to independent and identically distributed. Because of the bias, the model effect will be significantly affected. Due to the great heterogeneity of edge servers, there are few same sample labels and same data distributions, such as the cifar-10 data set, 5% of the devices have 3 types of samples, 10% of the devices have 5 types of samples, and 30% of the devices have 10 types of samples. Therefore, in the embodiments of the present application, the data label distribution information is taken as one of the factors to be considered to analyze and select the edge servers participating in the training of the global model of the center server.

[0066] In step S102, the target edge server is determined from the plurality of edge servers according to the latency information, the energy consumption information, and the data label distribution information.

[0067] In the embodiments, in response to the latency information, the energy consumption information, and the data label distribution information of the edge server satisfying the pre-set constraint condition, the corresponding edge server is determined as a plurality of candidate edge servers, and the target edge server is determined from the plurality of candidate edge servers.

[0068] In some embodiments, the constraint condition comprises: a time delay limit of the edge server, a bandwidth allocation ratio limit, a CPU cycle frequency limit, a transmission power limit. The time delay limit is that the time delay of the edge server is less than or equal to a preset deadline. The bandwidth allocation ratio limit is that the bandwidth allocation ratio of the edge server is greater than or equal to a preset bandwidth allocation ratio threshold and less than or equal to 1. The CPU cycle frequency limit is that the CPU cycle frequency of the edge server is greater than or equal to a preset CPU cycle frequency first threshold and less than or equal to a CPU cycle frequency second threshold. The transmission power limit is that the transmission power of the edge server is greater than or equal to a working power first threshold of the edge server and less than or equal to a working power second threshold of the edge server.

[0069] Specifically, in addition to minimizing the time delay, the deadline of the task can represent the urgency of the task, especially for some tasks with high time delay sensitivity, which will have serious consequences if not completed before the deadline. Therefore, the deadline needs to be set to constrain the time delay of the edge server to be less than or equal to the deadline to ensure the effectiveness of the task.

[0070] It can be understood that, in response to the nth edge server not participating in training in the tth global iteration, no bandwidth needs to be allocated to the nth edge server, i.e., In response to the nth edge server participating in training in the tth global iteration, at least a preset bandwidth allocation ratio threshold b min , i.e., It should be understood that in actual application, due to the limited size of the resource block, the system cannot allocate any small bandwidth to a single edge server. In addition, a bandwidth allocation close to zero will require extremely high transmission power, which will result in extremely high energy consumption to achieve the target transmission rate. Therefore, the bandwidth allocation ratio threshold b min .

[0071] In some embodiments, a plurality of combinations are generated according to the plurality of alternative edge servers; wherein the number of the alternative edge servers in the combination is less than or equal to a preset number threshold.

[0072] In some embodiments, due to the limited wireless resources, it is necessary to preset the proportion of edge servers participating in training in each global iteration, and this proportion remains unchanged throughout the entire global iteration process, i.e., it is necessary to preset the number threshold of edge servers participating in training in each global iteration. Let C be the proportion of edge servers participating in training in each global iteration, then the number threshold of edge servers participating in training in each global iteration is C*N.

[0073] In addition, the data heterogeneity of the edge server can be understood as class imbalance. The respective edge servers do not follow a common data distribution, i.e., the data distribution in the edge servers is not independent and identically distributed. And this data heterogeneity will weaken the model aggregation quality, leading to low classification accuracy. This difference is called skewness of data distribution. Since the test accuracy is determined by the trained weights, the difference between the weights of FedAvg and SGD is used to measure this skewness, which is called weight divergence, which can be calculated by the following formula:

[0074]

[0075] where w SGD is the model weight after updating all the data on the center server;

[0076] is the result of the aggregation average of the updated weights trained by the data at the respective edge servers.

[0077] It can be understood that the weight divergence will increase with the increase of the degree of non-independent and identically distributed data, therefore, it is necessary to select edge servers with uniform data distribution to participate in training as much as possible in each global iteration. However, in order to ensure fairness and ensure that each edge server has the opportunity to participate in training, the overall data label distribution needs to be considered.

[0078] In some embodiments, according to the data label distribution information of the edge servers in the combination, a global distance of the combination is calculated; wherein the global distance represents the skewness of the data label distribution between the edge servers in the combination.

[0079] Specifically, assuming that the center server can obtain the number information of each type of label from each edge server, the global distance GD can be calculated by the following formula:

[0080]

[0081] where S represents the number of data label types, and i∈S; D n represents the number of data samples of the nth edge server; y = i represents the data label type of the ith type; represents the proportion of the ith data label type in the nth edge server; P y＝i is the global proportion of the ith data label type.

[0082] It can be understood that, compared with the sum of the label distribution difference of a single edge server and the overall data, the distance between the overall aggregated distribution of the labels of all selected edge servers and the global distribution can better reflect the deviation of the edge servers, that is, the global distance GD can better reflect the deviation of the edge servers. And, in response to the GD being small, it indicates that the overall training data of all edge servers participating in training tends to be independent and identically distributed, that is, the number of each data label is relatively average.

[0083] In some embodiments, in response to the global distance of the combination being less than or equal to a pre-set global distance threshold, a weighted sum of the energy consumption information of the edge servers in the combination and the global distance is calculated according to a pre-determined allocation weight, the combination with the smallest weighted sum is taken as a target combination, and the alternative edge server in the target combination is determined as the target edge server.

[0084] Specifically, the allocation weight of the energy consumption information of the edge servers in the combination and the global distance can be set according to actual application requirements. For example, in a resource-scarce task scenario, the energy consumption needs to be considered as a priority, and the allocation weight corresponding to the energy consumption is higher. In this way, in the case of device shortage and insufficient total energy of the system, the reduction of energy consumption can be given priority, for example, the power or frequency is set to be lower. In addition, since the units of energy consumption and global distance are different, the allocation weight can play a role of normalization and standardization to a certain extent.

[0085] In some optional embodiments, a scoring mechanism can be established to determine the allocation weight of the energy consumption information and the allocation weight of the global distance according to application requirements.

[0086] Specifically, for the allocation weight of the energy consumption information, the scoring formula is as follows:

[0087]

[0088] Wherein, Pi represents the score of the i-th factor (that is, in this formula, the factor is the energy consumption information), which can be dynamically changed according to application requirements to meet different focuses under different requirements.

[0089] It can be understood that, for the scoring mechanism in this embodiment, there are two factors, namely the energy consumption information and the global distance.

[0090] Further, for the allocation weight set according to different actual application requirements, it can be as follows: in response to the allocation weight corresponding to the energy consumption being very high, the purpose of minimizing the energy consumption is completely pursued, in which case, the edge servers are all at a very low CPU cycle frequency (close to idle state), and the global distance GD is also very small. In response to the allocation weight corresponding to the energy consumption being very low, the purpose of minimizing the global distance GD is completely pursued, in which case, the edge servers are all at a high CPU cycle frequency (close to full load state), and the energy consumption is also very high. ) is higher, the global training energy consumption is minimized at the cost of partial training accuracy; in response to the allocation weight of energy consumption being proportional to the allocation weight of global distance, the balance of the two can be achieved; in response to the allocation weight of energy consumption being lower, the minimization of global distance is mainly considered; in response to the allocation weight of energy consumption being very low, the statistical heterogeneity of the data of the edge server is mainly considered to select an edge server with small non-independent and identically distributed degree of overall data, so as to realize faster convergence of the model and lower overall loss.

[0091] In another embodiment, an edge server determination model can be constructed, the several candidate edge servers are generated into several combinations, and the time delay information, energy consumption information and data label distribution information of the candidate edge servers in the combinations are input into the edge server determination model to obtain a target edge server output by the edge server determination model.

[0092] Specifically, the edge server determination model P1 is:

[0093]

[0094] Wherein, α1 and α2 respectively represent the allocation weights of total energy consumption E g and global distance GD, and satisfy α1+α2=1; the global distance is less than or equal to a pre-set global distance threshold.

[0095] It should be understood that the total bandwidth allocation of all participating training edge servers in the tthglobal iteration needs to be less than or equal to 1. Specifically, in response to not contributing the total bandwidth to the federated learning task in the embodiment, the total bandwidth allocation of all participating training edge servers in the tthglobal iteration needs to be less than 1.

[0096] In addition, other constraint conditions of the edge server determination model P1 have the same meaning as the pre-set constraint conditions for determining the several candidate edge servers from the several edge servers, which will not be described here.

[0097] Further, since the edge server determination model P1 has complex constraints and multi-dimensional decision variables, it is an NP-complete problem, and the problem can only be solved by enumerating all edge server combinations, which has high time complexity and cannot be solved in polynomial time. Therefore, a heuristic algorithm based on the greedy strategy can be used to calculate the edge server determination model P1.

[0098] Thus, based on the edge server determination model, the target edge server is determined to participate in the training of the global model, which can balance the energy consumption and data label distribution, thereby balancing the energy consumption of the federated learning process and the learning accuracy of the federated learning.

[0099] In some optional embodiments, let wherein a t represents a vector composed of edge servers, indicating the selection decision of the edge servers in the tthglobal iteration. A greedy algorithm is used to find the target edge server that can minimize in each global iteration.

[0100] Specifically, the algorithm flow is as follows:

[0101] Input:

[0102] First, initialization is performed;

[0103] for (each edge server) do

[0104] if the constraints of the delay limit and the bandwidth allocation ratio limit are satisfied then

[0105] Calculate E n,t ;

[0106] Calculate the global distance GD;

[0107] According to the pre-allocated weight, calculate α1E n,t +α2GD, and store them in η t in turn, and then arrange η t in descending order; / / find the edge servers (alternative edge servers) that meet the predetermined constraints

[0108] else

[0109] Store the edge server n in a' t , and set the values to 0; / / abandoned edge server

[0110] end

[0111] end

[0112] Let m = 1; / / start searching from the first edge server

[0113] while (m≤|η t |, m≤C×N)

[0114] if (the edge server m meets the GD constraint) then

[0115]

[0116]

[0117] if(a 1,t ,...,a m-1,t ,a m,t If ≡1, m≤CN) satisfies the bandwidth allocation ratio limit and the global distance constraint, then

[0118] / / Select the edge server combination that minimizes the optimization objective.

[0119] else

[0120] break;

[0121] end

[0122] / / Take the minimum of the two

[0123]

[0124] / / Merge with edge servers that were not selected

[0125] else

[0126]

[0127] else

[0128] m = m + 1; / / Iterative search

[0129] end

[0130] output: / / Output selection results (target edge server)

[0131] end

[0132] It is understandable that this embodiment needs to select edge servers with uniform data distribution to participate in training in each round of global iteration as much as possible. However, to ensure fairness, it is necessary to ensure that each edge server has the opportunity to participate in training. When the number of data samples on each edge server is the same, the more uniform the overall data label distribution of the edge servers participating in training in each round of global iteration, the smaller the loss function of the final training result. However, this does not mean that the data distribution of each edge server to be selected tends to be independent and identically distributed, but rather that the data of the selected edge servers are complementary to each other.

[0133] Therefore, by sequentially storing the above data of each edge server and the weighted sum thereof and arranging them in descending order, subsequent search based on a greedy algorithm is facilitated to combine a plurality of candidate edge servers satisfying the constraint condition to further determine the target edge server. In this way, each candidate edge server can be allowed and qualified to participate in global training based on the descending arrangement, rather than only selecting edge servers with data distribution tending to be independent and identically distributed, thereby ensuring fairness between edge servers.

[0134] Step S103: sending the parameters of the global model to the target edge server.

[0135] In this embodiment, the center server initializes the global model (parameters) and sends it to the target edge server to train the global model. Specifically, the global model can be sent to all target edge servers through broadcasting.

[0136] Step S104: obtaining the parameters of the trained global model sent by the target edge server; wherein the parameters of the trained global model are obtained by the target edge server based on local data.

[0137] In this embodiment, the center server only interacts with the target edge server with the update information (parameters) of the global model, without the need to obtain the local data in the target edge server, effectively reducing the privacy risk and reducing the time delay.

[0138] Step S105: aggregating the parameters of the trained global model to obtain the parameters of the updated global model.

[0139] In this embodiment, the center server updates the global model by aggregating the parameters of the trained global model uploaded by the target edge server. Specifically, the center server aggregates the parameters of the trained global model from the target edge server to obtain the updated global model and its parameters in a way of minimizing the global loss function. Wherein, the method of average value can be used when aggregating the model parameters and weights. It can be understood that the global loss function is inversely proportional to the accuracy of the global model, so the global model can be made to reach the required accuracy by making the global loss function reach the minimum value.

[0140] Specifically, the optimization target can be:

[0141]

[0142] Wherein, F(w, D) is the global loss function using cross-entropy loss function.

[0143] Step S106, sending the parameters of the updated global model to the target edge server.

[0144] In this embodiment, the center server broadcasts the parameters of the updated global model to all target edge servers for the next round of training.

[0145] It should be noted that after multiple global iterations (the total number of global iterations is T), a model close to the centralized machine learning result is finally obtained. That is, the federated learning method 100 of the embodiments of the present application is cycled until the global loss function finally converges. In this way, before each global training, according to the availability and available resource state of the edge servers, it is determined which edge servers (i.e., it is determined which edge servers are target edge servers) participate in this round of training and upload updated parameters, and resource allocation is performed to maximize learning performance.

[0146] Figure 2 The flowchart of the federated learning method implemented by the edge server according to the embodiments of the present application is shown. As shown in the figure, the method can include the following steps. Figure 2

[0147] Step S201, sending the delay information, energy consumption information and data label distribution information to the center server; wherein the data label distribution information includes the number of data samples, the number of data label types and the proportion of each data label type.

[0148] In this embodiment, based on the request of the center server, the edge server sends the delay information, energy consumption information and data label distribution information to the center server.

[0149] In some embodiments, the delay information includes training delay and transmission delay. Specifically, the training delay is the time for training and updating the global model of the center server using local data, and the transmission delay is the time for uploading the parameters of the trained global model to the center server.

[0150] In some embodiments, the energy consumption information includes training energy consumption information and transmission energy consumption information. Specifically, the training energy consumption information is calculated according to the number of data samples, the number of local training rounds, the CPU cycle frequency, and the number of CPU cycles for executing one data sample; the transmission energy consumption information is calculated according to the bandwidth allocation ratio and the transmission power.

[0151] Step S202, in response to being a target edge server, receiving the parameters of the global model sent by the center server; wherein the target edge server is determined by the center server according to the delay information, the energy consumption information and the data label distribution information.

[0152] ​In this embodiment, whether the edge server is the target edge server is determined by the center server. The specific determination method has been described in the federated learning method 100, and will not be repeated here.

[0153] In this embodiment, based on the determination that the center server is the target edge server, the parameters of the global model sent by the center server are received to train the global model.

[0154] Step S203, training the parameters of the trained global model based on the local data, and sending the parameters of the trained global model to the center server.

[0155] In this embodiment, the global model (parameters) sent by the center server is trained using the local data to obtain the parameters of the trained global model, and the parameters of the trained global model are sent to the center server. In this way, the center server and the target edge server only interact with the updated parameters of the global model, without the need to upload the local data to the center server, effectively reducing the privacy risk and reducing the time delay.

[0156] Specifically, the update method of the edge server is:

[0157] w n,t (j+1)=w n,t (j)-η▽F n (w n,t (j));

[0158] And because the local data can be large, and the processing capacity of the edge server is limited, the stochastic gradient descent (SGD) method can be used for updating, and the batch size (the number of samples selected before adjusting the parameters each time) is set before training.

[0159] Step S204, receiving the updated global model parameters sent by the center server; wherein the updated global model parameters are obtained by the center server aggregating the trained global model parameters.

[0160] In this way, by comprehensively considering the time delay information, energy consumption information and data label distribution information of the edge server, the target edge server is determined from a plurality of edge servers to participate in the training of the global model, thereby reducing the energy consumption of the federated learning process and improving the learning accuracy of the federated learning.

[0161] It is to be understood that the above-described embodiments of this application are illustrative of only a few of the many embodiments which can be made within the scope and spirit of the application. Embodiments have been provided within the scope of the claims and other embodiments can be apparent to those skilled in the art from consideration of the specification and can be made without departing from the spirit or scope of the application. It is intended that the specification and illustrated embodiments be considered as exemplary only with the true scope and spirit of the present application being indicated by the following claims.

[0162] It can be understood that the method of the embodiment can be applied to a distributed scenario, which is completed by multiple devices cooperating with each other. One of the multiple devices can only perform one or more steps in the method of the embodiment, and the multiple devices can interact with each other to complete the method.

[0163] Based on the same technical concept, the application further provides a center server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the federated learning method described in the corresponding embodiments described above when executing the program. And the center server has the beneficial effects of the corresponding method embodiments, which are not described here.

[0164] Figure 3 A hardware structure schematic diagram of a more specific center server provided by the embodiment is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0165] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit, central processor), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.

[0166] The memory 1020 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are saved in the memory 1020 and are called and executed by the processor 1010.

[0167] The input / output interface 1030 is configured to connect an input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0168] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0169] The bus 1050 includes a channel for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0170] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the embodiments of the present specification, and does not have to contain all the components shown in the figure.

[0171] Further, the present application also provides an edge server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to realize the federated learning method described in the above corresponding embodiments. And the edge server has the beneficial effects of the corresponding method embodiments, which are not described here.

[0172] In addition, the application further provides a federated learning system comprising the center server and the edge server. The federated learning system utilizes the center server and the edge server to determine a target edge server from a plurality of edge servers to participate in training of a global model by comprehensively considering time delay information, energy consumption information and data label distribution information of the edge servers, so as to reduce energy consumption of the federated learning process and improve convergence speed of the global model, thereby improving learning accuracy of the federated learning.

[0173] Those skilled in the art will understand that the above discussion of any of the embodiments is merely exemplary and is not intended to be limiting of the scope of the application (including claims) as it is intended to be interpreted. The above embodiments or technical features among different embodiments can also be combined, and steps can be implemented in any order, and there are many other changes to the aspects of the embodiments of the application as described above. In order to be brief, they are not provided in detail.

[0174] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the application difficult to understand, the known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, the devices can be shown in the form of block diagrams in order to avoid making the embodiments of the application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented to implement the embodiments of the application (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an illustrative embodiment of the application, it will be apparent to those skilled in the art that the embodiments of the application can be practiced without these specific details or with variations on these specific details. Therefore, these descriptions should be considered as illustrative rather than limiting.

[0175] Although the application has been described in conjunction with specific embodiments thereof, numerous alternatives, modifications, and variations will be readily apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0176] The embodiments of the application are intended to cover all such alternatives, modifications, and variations as falling within the broad scope of the appended claims. Accordingly, any one of the steps of the embodiments of the application can be carried out in any order or simultaneously, and the application should not be limited to the specific order of steps as set forth above.

Claims

1. A federated learning method, characterized in that, Implemented through a central server, the method includes: The system acquires latency information, energy consumption information, and data tag distribution information of several edge servers; wherein the data tag distribution information includes the number of data samples, the number of data tag types, and the proportion of each data tag type. In response to the fact that the latency information, energy consumption information and data tag distribution information of the edge server meet the preset constraints, the corresponding edge server is determined as a number of candidate edge servers; a number of combinations are generated based on the number of candidate edge servers; wherein the number of candidate edge servers in the combination is less than or equal to a preset number threshold. The global distance of the combination is calculated based on the data label distribution information of the edge servers in the combination; wherein the global distance characterizes the skewness of the data label distribution among the edge servers in the combination. In response to the global distance of the combination being less than or equal to a preset global distance threshold, the energy consumption information of the edge servers in the combination and the weighted sum of the global distance are calculated according to the preset allocation weights. The combination with the smallest weighted sum is taken as the target combination, and the candidate edge servers in the target combination are determined as the target edge servers. The global distance is calculated using the following formula: GD ; Where GD represents global distance; the number of edge servers is... N One, and n C represents the number of data label types, and ;D n This represents the number of data samples on the nth edge server; This indicates that the data label type is the first. i kind; Indicates the nth edge server i The percentage of each data tag type; For the first i Global proportion of various data label types; The parameters of the global model are sent to the target edge server; Obtain the parameters of the trained global model sent by the target edge server; wherein the parameters of the trained global model are obtained by the target edge server based on local data. The parameters of the trained global model are aggregated to obtain the parameters of the updated global model; The updated global model parameters are sent to the target edge server.

2. The method according to claim 1, characterized in that, The energy consumption information includes training energy consumption information and transmission energy consumption information; The method further includes: The number of data samples, local training rounds, CPU cycle frequency, and number of CPU cycles to execute one data sample are obtained from the edge server. The training energy consumption information is calculated based on the number of data samples, the local training rounds, the CPU cycle frequency, and the number of CPU cycles to execute one data sample. Obtain the bandwidth allocation ratio and transmission power of the edge server, and calculate the transmission energy consumption information based on the bandwidth allocation ratio and the transmission power.

3. The method according to claim 2, characterized in that, The constraints include: latency limits, bandwidth allocation ratio limits, CPU cycle frequency limits, and transmission power limits for the edge server; The latency limitation is that the latency of the edge server is less than or equal to a preset deadline. The bandwidth allocation ratio is limited to the following: the bandwidth allocation ratio of the edge server is greater than or equal to a preset bandwidth allocation ratio threshold and less than or equal to 1; The CPU cycle frequency is limited as follows: the CPU cycle frequency of the edge server is greater than or equal to a preset first threshold for CPU cycle frequency, and less than or equal to a second threshold for CPU cycle frequency. The transmission power limit is: the transmission power of the edge server is greater than or equal to the first threshold of the operating power of the edge server, and less than or equal to the second threshold of the operating power of the edge server.

4. A federated learning method, characterized in that, Implemented via an edge server, the method includes: The latency information, energy consumption information, and data tag distribution information are sent to the central server; wherein, the data tag distribution information includes the number of data samples, the number of data tag types, and the proportion of each data tag type; In response to the fact that the latency information, energy consumption information and data tag distribution information of the edge server meet the preset constraints, the corresponding edge server is determined as a number of candidate edge servers; a number of combinations are generated based on the number of candidate edge servers; wherein the number of candidate edge servers in the combination is less than or equal to a preset number threshold. The global distance of the combination is calculated based on the data label distribution information of the edge servers in the combination; wherein the global distance characterizes the skewness of the data label distribution among the edge servers in the combination. In response to the global distance of the combination being less than or equal to a preset global distance threshold, the energy consumption information of the edge servers in the combination and the weighted sum of the global distance are calculated according to the preset allocation weights. The combination with the smallest weighted sum is taken as the target combination, and the candidate edge servers in the target combination are determined as the target edge servers. The global distance is calculated using the following formula: GD ; Where GD represents global distance; the number of edge servers is... N One, and n C represents the number of data label types, and ;D n This represents the number of data samples on the nth edge server; This indicates that the data label type is the first. i kind; Indicates the nth edge server i The percentage of each data tag type; For the first i Global proportion of various data label types; In response to being a target edge server, it receives parameters of the global model sent by the central server; wherein, the target edge server is determined by the central server based on the latency information, the energy consumption information, and the data tag distribution information; The parameters of the trained global model are obtained based on local data, and the parameters of the trained global model are sent to the central server. The system receives updated global model parameters sent by the central server; wherein the updated global model parameters are obtained by the central server aggregating the parameters of the trained global model.

5. A central server, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 3.

6. An edge server, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, characterized in that, When the processor executes the computer program, it implements the method as described in claim 4.

7. A federated learning system, characterized in that, include: The central server as described in claim 5 and the edge server as described in claim 6.

Images