Federated learning method, storage medium, terminal, server, and federated learning system

Abstract

The present invention discloses a federated learning method, a storage medium, a server, a terminal and a federated learning system. The federated learning method comprises: the server obtains a constraint time according to the training information of the terminal, and the constraint time is used to enable the terminal to complete the model training within the constraint time; the terminal obtains training parameters according to the constraint time; the terminal completes the model training according to the training parameters, and the training parameters are used to enable the terminal to consume the minimum energy consumption when completing the model training. On the one hand, the federated learning method of the present application controls the completion time of each round of terminal training according to the real-time status of the terminal, thereby reducing the overall training time; on the other hand, the terminal calculates the training speed with the lowest energy consumption according to the training completion time predicted by the server, thereby reducing the power consumption of each round of training.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and specifically, relates to a federated learning method, a storage medium, a terminal, a server, and a federated learning system. Background Art

[0002] In recent years, mobile devices have developed rapidly. Not only have they rapidly improved their computing power, but they are also equipped with a variety of sensors that collect information from different dimensions and record people's lives. Effective use of this data for training machine learning models will greatly improve the intelligent development of mobile devices and will also greatly improve user experience. The traditional training method will first centralize the user's initial data on the mobile device to the server, and then train the relevant deep learning model. When the application has a speculative demand, it uploads the initial data to the server, performs the inference process on the server, and then the server sends the inference results back to the mobile device. This training and inference method will cause serious data privacy issues.

[0003] In order to solve the privacy problem of distributed training, Google proposed federated learning in 2017. Federated learning first trains the data directly on the mobile device, then uploads the relevant gradients, fuses the gradients on the server, and sends the updated model to the mobile device. In this way, the model is iterated and updated until the model converges. However, model training is usually a computationally intensive task, which usually causes a lot of power consumption on mobile devices. To solve this problem, Google's existing solution is to train the model only when the mobile device is charging and there is no interaction with the mobile device. However, this seriously violates the original intention of using mobile devices for data analysis and cannot timely and extensively analyze mobile data. That is, the existing solution cannot effectively take into account the power consumption of the terminal and the overall training time. Summary of the invention

[0004] (I) Technical Problems to be Solved by the Present Invention

[0005] The technical problem solved by the present invention is: how to reduce the overall training time and reduce the training energy consumption of the terminal.

[0006] (II) Technical solution adopted by the present invention

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A federated learning method, comprising:

[0009] The server receives the training information sent from the terminal;

[0010] The server obtains a constraint time according to the training information, where the constraint time is used to enable the terminal to complete the model training within the constraint time;

[0011] The server sends the constraint time to the terminal.

[0012] Preferably, the training information includes hardware information and a preset amount of training data, wherein the method by which the server obtains the constraint time according to the training information includes:

[0013] The server obtains a preset training completion time according to the hardware information and the preset training data volume;

[0014] The server obtains the constraint time according to the preset training completion time and the preset gradient update ratio.

[0015] Preferably, the federated learning method further includes:

[0016] The server receives the actual speed of this round of training sent from the terminal;

[0017] The server obtains the constraint time of the next round of training according to the actual speed of the current round of training;

[0018] The server sends the constraint time of the next round of training to the terminal.

[0019] Preferably, the method by which the server obtains the constraint time of the next round of training according to the actual speed of the current round of training includes:

[0020] The server obtains the preset completion time of the next round of training according to the actual speed of the current round of training;

[0021] The server obtains the constraint time of the next round of training according to the preset completion time of the next round of training and the preset gradient update ratio.

[0022] Preferably, the federated learning method further includes:

[0023] The server obtains the progress value of the current round of training, and determines whether the progress value of the current round of training is greater than a preset value;

[0024] If so, the server sends a first instruction to the terminal to start the next round of training;

[0025] If not, the server sends a second instruction to the terminal to restart the current round of training.

[0026] Preferably, the method for the server to obtain the progress value of this round of training is:

[0027] The server obtains the current round training gradient value sent from each terminal, wherein the current round training progress value is the sum of the current round training gradient values ​​of each terminal.

[0028] The present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a federated learning control program, and the federated learning control program implements the above-mentioned federated learning method when executed by a processor.

[0029] The present invention discloses a server, which includes a computer-readable storage medium, a processor, and a federated learning control program stored on the computer-readable storage medium. When the federated learning control program is executed by the processor, the above-mentioned federated learning method is implemented.

[0030] The present invention discloses another federated learning method, comprising:

[0031] The terminal receives a constraint time sent from the server, wherein the constraint time is used to enable the terminal to complete the model training within the constraint time;

[0032] The terminal obtains training parameters according to the constraint time;

[0033] The terminal completes the model training according to the training parameters, and the training parameters are used to consume minimum energy when the terminal completes the model training.

[0034] Preferably, the method for the terminal to obtain the training parameters according to the constraint time includes:

[0035] The terminal generates training parameters according to the hardware information, the preset training data volume and the constraint time, and the training parameters include a training speed and a frequency of a processor of the terminal matching the training speed.

[0036] Preferably, the federated learning method further includes:

[0037] The terminal obtains the actual speed of this round of training;

[0038] The terminal sends the acquired actual speed of the current round of training to the server, wherein the actual speed of the current round of training is used to enable the server to acquire the constraint time of the next round of training.

[0039] Preferably, the federated learning method further includes:

[0040] The terminal sends the current round training gradient value to the server, where the current round training gradient value is used to enable the server to obtain the current round training progress value, where the current round training progress value is the sum of the current round training gradient values ​​of each terminal.

[0041] Preferably, the federated learning method further includes:

[0042] The terminal starts the next round of training according to the first instruction sent from the server, wherein the first instruction is an instruction generated by the server when the acquired progress value of the current round of training is greater than a preset value.

[0043] Preferably, the federated learning method further includes:

[0044] The terminal restarts the current round of training according to the second instruction sent from the server, wherein the second instruction is an instruction generated by the server when the acquired progress value of the current round of training is less than or equal to a preset value.

[0045] The present invention also discloses another computer-readable storage medium, which stores a federated learning control program. When the federated learning control program is executed by a processor, the above-mentioned federated learning method is implemented.

[0046] The present invention also discloses a terminal, which includes a computer-readable storage medium, a processor, and a federated learning control program stored on the computer-readable storage medium. When the federated learning control program is executed by the processor, the above-mentioned federated learning method is implemented.

[0047] The present invention discloses a federated learning system, which includes a server and a terminal;

[0048] The server is used to receive training information sent from the terminal; the server is used to obtain a constraint time according to the training information, and the constraint time is used to enable the terminal to complete the model training within the constraint time;

[0049] The terminal is used to receive the constraint time sent from the server; the terminal is used to obtain training parameters according to the constraint time; the terminal is also used to complete model training according to the training parameters, and the training parameters are used to consume minimum energy when the terminal completes the model training.

[0050] The present invention also discloses a federated learning method of a federated learning system, the federated learning method comprising:

[0051] The server obtains a constraint time according to the training information of the terminal, where the constraint time is used to enable the terminal to complete the model training within the constraint time;

[0052] The terminal obtains training parameters according to the constraint time;

[0053] The terminal completes the model training according to the training parameters, and the training parameters are used to consume minimum energy when the terminal completes the model training.

[0054] (III) Beneficial effects

[0055] The present invention discloses a federated learning method, a server, a terminal and a federated learning system, which have the following advantages and beneficial effects compared with the prior art:

[0056] (1) Reduced overall training time. The server controls the completion time of each round of terminal training according to the real-time status of each terminal, and the terminal can perform corresponding training in different states, which reduces the overall training time and avoids the drawback of the existing technical solution that the terminal can only be trained when it is in the charging state and not interacting with the outside world. It effectively improves the training efficiency and enhances the real-time nature of the training.

[0057] (2) Reduced energy consumption. The terminal calculates the training speed with the lowest energy consumption based on the training completion time predicted by the server, and dynamically adjusts the CPU frequency during the training process so that the training process is maintained at this training speed as much as possible, thereby reducing the power consumption of each round of training. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is an architecture diagram of a federated learning system according to Embodiment 1 of the present invention;

[0059] Figure 2 is a flowchart of a federated learning method according to a second embodiment of the present invention;

[0060] Figure 3 is a structural block diagram of a server according to a fourth embodiment of the present invention;

[0061] Figure 4 is a flowchart of a federated learning method according to a fifth embodiment of the present invention;

[0062] Figure 5 is a schematic diagram of the structure of a terminal according to Embodiment 7 of the present invention;

[0063] Figure 6 It is a flowchart of the federated learning method of embodiment 8 of the present invention. DETAILED DESCRIPTION

[0064] Below, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention can be implemented in many different forms, and the present invention should not be construed as being limited to the specific embodiments set forth herein. On the contrary, these embodiments are provided to explain the principles of the present invention and their practical applications, so that other persons skilled in the art can understand the various embodiments of the present invention and the various modifications suitable for specific intended applications. The same reference numerals may be used to represent the same elements throughout the specification and the accompanying drawings.

[0065] Embodiment 1

[0066] like Figure 1As shown, the federated learning system of the first embodiment of the present invention includes a server 10 and several terminals 20, and the server 10 and the terminals 20 can communicate with each other. The basic working process of the federated learning system is: each terminal 20 sends its own training information to the server 10, the server 10 obtains the constraint time according to the training information of each terminal 20, the terminal 20 obtains the training parameters according to the constraint time, and the terminal 20 completes the model training according to the training parameters. The constraint time is used to enable the terminal to complete the model training within the constraint time, and the training parameters are used to enable the terminal to consume the minimum energy when completing the model training. In this way, the federated learning system can reduce the overall training time and reduce energy consumption.

[0067] The federated learning system of this embodiment is described in detail below from two aspects: the server 10 and the terminal 20.

[0068] Embodiment 2

[0069] like Figure 2 As shown, the federated learning method of the second embodiment includes the following steps:

[0070] Step S10: The server 10 receives the training information sent from the terminal.

[0071] Before each training, the server 10 needs to collect the training information of each terminal 20 in advance. The training information includes hardware information and preset training data volume, wherein the hardware information includes information such as the processor information of the terminal, mainly refers to information such as CPU frequency and CPU cycle, and the preset training data volume refers to the amount of training data contained in each terminal, for example, for image recognition tasks, the amount of training data refers to how many pictures are contained in the terminal.

[0072] Step S20: the server 10 obtains a constraint time according to the training information, and the constraint time is used to enable the terminal to complete the model training within the constraint time.

[0073] Specifically, step S20 includes the following steps:

[0074] Step S21: The server obtains a preset training completion time according to the hardware information and the preset training data volume.

[0075] The preset training completion time refers to the time required for each terminal to complete local training. The specific calculation formula for the preset training completion time is: where c i represents the CPU cycles required to process a training sample, D i represents the number of training samples contained in the terminal device, f i Represents the CPU frequency used during training.

[0076] Step S22: The server obtains the constraint time according to the preset training completion time and the preset gradient update ratio.

[0077] Since the hardware information of each terminal is different, such as CPU frequency and CPU cycle, and the amount of training data of each terminal is also different, the preset training completion time calculated is also different, resulting in some terminals completing training earlier than other terminals in each round of training. According to the requirements of the federated learning system, the terminal that completes the training first needs to wait for other terminals to complete the training before it is considered that a round of training is completed. In addition, after the federated learning system starts training, not every terminal can complete the local training according to the preset instructions. For example, some terminals will terminate the training due to sudden shutdown or sudden power failure. At this time, the training results of the terminal cannot be fed back to the server. In the federated learning system, considering such unexpected situations, it is not required that every terminal must complete the training and upload the training results. Generally, a corresponding ratio is set, that is, the preset gradient update ratio. For example, the preset gradient update ratio is 90%, which means that when 90% of the terminals in the system complete the local training and upload the training results to the server, the training is successfully completed.

[0078] Furthermore, since the preset training time of each terminal is different, the server needs to set an appropriate constraint time to ensure that enough mobile devices can complete local training and upload training results. The setting of the constraint time should ensure that a predetermined proportion of terminals can successfully complete the training. On the other hand, the constraint time should not be too long to avoid excessive energy consumption caused by some terminals undergoing training for too long. For example, there are 100 terminals in the federated learning system that can complete local training, of which 80 terminals complete local training within 1 hour, and the preset training completion time for the other 20 terminals to complete local training is more than 1.5 hours. If the preset gradient update ratio is 80%, then the constraint time at this time can be set to a value between 1 hour and 1.5 hours, such as 1.2 hours. This ensures that the server 10 receives enough gradient updates without wasting excessive energy.

[0079] It should be noted that the setting of the constraint time needs to be actually determined according to the preset training completion time and preset gradient update ratio of each terminal in each federated learning system, and is not limited to the above example.

[0080] Step S30 : the server 10 sends the constraint time to the terminal 20 .

[0081] After the server 10 determines the constraint time, the server sends the constraint time to each terminal 20, and each terminal 20 optimizes the local training process according to the constraint time so that the terminal completes the training with minimum power consumption.

[0082] The federated learning method of this application also includes:

[0083] Step S40: the server 10 receives the actual speed of the current round of training sent from the terminal, obtains the constraint time of the next round of training according to the actual speed of the current round of training, and sends the constraint time of the next round of training to the terminal.

[0084] Specifically, since one training of the federated learning system includes multiple rounds of training, the constraint time of each round of training is different. Step S10 and step S20 calculate the constraint time corresponding to the first round of training, and the constraint time of each subsequent round of training, such as the second round, the third round, the fourth round, etc., needs to be recalculated. The method of obtaining the constraint time of each subsequent round of training in this embodiment is different from the method of obtaining the constraint time of the first round of training, but the constraint time required for the next round of training is predicted based on the actual speed of the current round of training.

[0085] Furthermore, the method for the server to obtain the constraint time of the next round of training according to the actual speed of the current round of training includes the following steps:

[0086] Step S41: the server obtains the preset completion time of the next round of training according to the actual speed of the current round of training.

[0087] Since there are some uncontrollable situations in the actual training process of the terminal, for example, the terminal is performing other tasks at the same time, which will occupy more computing resources, so that the training speed of the mobile terminal is slowed down and cannot reach the expected speed, the terminal will monitor the actual training speed of each round of training in real time during the training process, and feed back the actual training speed to the server 20. The server 20 obtains the preset training completion time of the next round according to the actual training speed of this round. As a preferred embodiment, we use the exponential moving weighted average method to predict the training completion speed of the next round according to the actual training speed of this round. Since the total training amount can be determined in advance, the training completion time of the next round can be calculated according to the predicted training completion speed and the total training amount. The exponential moving weighted average algorithm used here belongs to the prior art, and those skilled in the art are familiar with this algorithm and will not be repeated here.

[0088] Step S42: the server obtains the constraint time of the next round of training according to the preset completion time of the next round of training and the preset gradient update ratio.

[0089] Specifically, the constraint time of the next round of training is obtained according to the preset training completion time and the preset gradient update ratio of each terminal, and finally the constraint time of the next round of training is sent to each terminal, and each terminal 20 optimizes the local training process according to the constraint time so that the terminal completes the training with the minimum power consumption. It should be noted that the preset gradient update ratios of each round of training in a training in the federated learning system are the same value, and the method for setting the constraint time is the same as the method in step S20, which will not be repeated here.

[0090] Furthermore, the federated learning method of the present application also includes:

[0091] Step S50: the server obtains the progress value of this round of training, and determines whether the progress value of this round of training is greater than a preset value; if so, the server sends a first instruction to start the next round of training to the terminal; if not, the server sends a second instruction to restart the current round of training to the terminal.

[0092] As a preferred embodiment, the progress value of this round of training is selected as the total gradient update value, and the total gradient update value is the sum of the gradient values ​​of this round of training uploaded by each terminal. The preset value is set according to demand.

[0093] Specifically, during the actual training process, after each terminal 20 completes a round of training within the constraint time, it uploads its gradient value of this round of training to the server 10. Due to the difference in the completion time of each terminal 20, within the time limit of the constraint time, some terminals complete this round of training, and some terminals fail to complete this round of training. If the sum of the gradient values ​​of this round of training of each terminal within the time limit is greater than the preset value, for the federated learning system, it means that this round of training has been successfully completed. At this time, the server 10 sends a first instruction to start the next round of training to each terminal, and each terminal starts the next round of training according to the first instruction. If the sum of the gradient values ​​of this round of training of each terminal is less than or equal to the preset value, it means that this round of training has not been completed. At this time, the server sends a second instruction to restart this round of training to the terminal, and each terminal restarts this round of training according to the second instruction.

[0094] It should be noted that the server 10 in this embodiment is installed with a global controller, and the global controller is configured to execute the above steps.

[0095] The federated learning method of this embodiment obtains training information such as the hardware information of the terminal. The server calculates the corresponding constraint time based on the training information to control the training process of the terminal in real time. In addition, the constraint time is adjusted during different rounds of training to help reduce the overall training time as much as possible while ensuring the training accuracy.

[0096] Embodiment 3

[0097] The third embodiment discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a federated learning program. When the federated learning program is executed by a processor, the federated learning method described in the second embodiment is implemented.

[0098] Embodiment 4

[0099] like Figure 3 As shown, this fourth embodiment discloses a server. At the hardware level, the terminal includes a processor 12, an internal bus 13, a network interface 14, and a computer-readable storage medium 11. The processor 12 reads the corresponding computer program from the computer-readable storage medium 11 and then runs it, forming a request processing device at the logical level. Of course, in addition to software implementations, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logical unit, but can also be hardware or logic devices. The computer-readable storage medium 11 stores a federated learning program, and when the federated learning program is executed by the processor 12, it implements the federated learning method described in Example 2.

[0100] Embodiment 5

[0101] like Figure 4 As shown, this embodiment discloses a federated learning method, which includes the following steps:

[0102] Step S01: The terminal receives a constraint time sent from a server, wherein the constraint time is used to enable the terminal to complete model training within the constraint time.

[0103] Before each model training of the federated learning system is started, each terminal 20 uploads its own training information to the server 10 in advance. The training information includes hardware information and preset training data volume, wherein the hardware information includes information such as the processor information of the terminal, mainly referring to information such as CPU frequency and CPU cycle, and the preset training data volume refers to the number of training data contained in each terminal. For example, for image recognition tasks, the number of training data refers to the number of pictures contained in the terminal. The server calculates the constraint time based on the training information of each terminal and sends the constraint time to the terminal, so that each terminal can receive the constraint time for each round of training. The constraint time is used to enable the terminal 10 to complete the model training within the constraint time. The calculation method of the constraint time has been described in step S20 in Example 2, and will not be repeated here.

[0104] Step S02: The terminal obtains training parameters according to the constraint time.

[0105] Specifically, the method for the terminal to obtain training parameters according to the constraint time includes: the terminal generates training parameters according to hardware information, a preset training data volume and the constraint time, and the training parameters include a training speed and a frequency of a processor of the terminal matching the training speed.

[0106] After receiving the constraint time, the terminal optimizes the system configuration to complete the local training with the minimum power consumption within the constraint time. Specifically, the terminal calculates the preset training speed and CPU frequency with the best energy consumption based on its own hardware information and the preset training data volume. The specific calculation method is as follows:

[0107] We model the energy consumption required for a terminal device to complete a round of training as follows:

[0108] E=p train *t train +p idle *t idle

[0109] Among them, E represents the energy consumption required for one round of training, p train represents the power consumption during model training, t train Indicates the training completion time, p idle Indicates the power consumption when the system is idle, t idle It should be noted that for different terminals, the system idle time and training completion time are different, but the sum of the system idle time and training completion time is the constraint time. For example, when the server establishes the constraint time, if the terminal completes the training within the constraint time, the system idle time t idle is zero, and the training is completed at time t train is equal to the constraint time; if the terminal completes the training before the constraint time, the terminal needs to wait for the other terminals to complete the training until the constraint time. The waiting time in this process is the system idle time t idle .

[0110] The training power consumption of the system can be modeled as follows:

[0111] p train =β*f 3

[0112] Where β represents the resistance coefficient of the chip, and f represents the CPU frequency used during training.

[0113] We then refine the problem of determining the training parameters into a constrained optimization problem as follows:

[0114]

[0115]

[0116] where d k represents the constraint time, f i min Indicates the minimum frequency of the terminal's CPU, f i max Indicates the maximum frequency of the terminal's CPU, argmin indicates the minimum value, and the subscript i indicates different terminals. By solving this optimization problem, the optimal training completion time with the minimum energy consumption E is obtained. and the corresponding processor frequency f i At this time, the total training amount of each terminal has been predetermined, and the total training amount and the optimal training completion time are The training speed with optimal energy consumption can be determined. The specific solution process is an existing technology that can be known to those skilled in the art and will not be described in detail here.

[0117] Step S03: The terminal completes model training according to the training parameters, where the training parameters are used to enable the terminal to consume minimum energy when completing the model training.

[0118] Specifically, the model training is performed under the above-mentioned training parameters, that is, the model training is performed according to the optimal training speed and the frequency of the terminal processor that matches the training speed. Since the terminal may perform multiple tasks at the same time in the actual process, the processor frequency of the terminal cannot be maintained at the optimal frequency. At this time, the resource configurator of the terminal dynamically adjusts the processor frequency of the terminal so that the training speed is maintained at the optimal training speed as much as possible. At this optimal training speed, on the one hand, the terminal can complete the current round of model training within the constraint time, and on the other hand, the terminal can reduce energy consumption.

[0119] The federated learning method in this embodiment further includes:

[0120] Step S04: the terminal obtains the actual speed of this round of training; the terminal sends the obtained actual speed of this round of training to the server, wherein the actual speed of this round of training is used to enable the server to obtain the constraint time of the next round of training.

[0121] Since there are some uncontrollable situations in the actual training process of the terminal, for example, the terminal is performing other tasks at the same time, which will occupy more computing resources, slowing down the training speed of the mobile terminal and failing to reach the expected speed, the speed detector of the terminal will monitor the actual speed of this round of training in real time during the training process, and feed back the actual speed to the server 10, so that the server 10 can predict the preset completion time of the next round of training based on the actual speed of this round of training, and finally obtain the constraint time of the next round of training. The calculation method of the constraint time has been described in step S20 in Example 2 and will not be repeated here.

[0122] Furthermore, the federated learning method in this embodiment also includes:

[0123] Step S05: The terminal sends the current round training gradient value to the server, and the current round training gradient value is used to enable the server to obtain the current round training progress value. Among them, the current round training gradient value and the current round training progress value related content have been described in step S50 of embodiment 2, and will not be repeated here.

[0124] Further, the terminal starts the next round of training according to a first instruction sent from the server, wherein the first instruction is an instruction generated by the server when the obtained progress value of the current round of training is greater than a preset value. The terminal restarts the current round of training according to a second instruction sent from the server, wherein the second instruction is an instruction generated by the server when the obtained progress value of the current round of training is less than or equal to a preset value.

[0125] Specifically, during the actual training process, after each terminal 20 completes a round of training within the constraint time, it uploads its gradient value of this round of training to the server 10. Due to the difference in the completion time of each terminal 20, within the time limit of the constraint time, some terminals complete this round of training, and some terminals fail to complete this round of training. If the sum of the gradient values ​​of this round of training of each terminal within the time limit is greater than the preset value, for the federated learning system, it means that this round of training has been successfully completed. At this time, the server 10 generates a first instruction to start the next round of training and sends the first instruction to each terminal, and each terminal starts the next round of training according to the first instruction. If the sum of the gradient values ​​of this round of training of each terminal is less than or equal to the preset value, it means that this round of training has not been completed. At this time, the server generates a second instruction to restart this round of training and sends the second instruction to the terminal, and each terminal restarts this round of training according to the second instruction.

[0126] The federated learning method of this embodiment calculates the optimal training speed and processor frequency according to the constraint time, and performs model training according to the training speed and processor frequency, so that the terminal completes the model training with minimal energy consumption. In addition, the federated learning method is adaptable to different real-time states of the terminal, and model training can be performed in different states of the terminal, avoiding the disadvantage of the prior art solution that training can only be performed when the terminal is charging and not interacting with the outside world, thereby improving the effective utilization rate of the terminal.

[0127] It should be noted that each terminal 20 is installed with a local controller, and each local controller is configured to complete the above-mentioned federated learning method.

[0128] It should be pointed out that the above federated learning system has been tested on a simulation platform and a small-scale real test environment. The test results show that compared with the existing technical solutions, the federated learning system of this application can effectively reduce the system energy consumption by 32.8% and accelerate the training time by 2.27 times.

[0129] Combined with the above description, it can be seen that in the federated learning system provided by the present application, the server can calculate the constraint time of each round of training in real time, so that the server can reduce the overall time by effectively controlling the completion time of each round of training. The terminal can calculate the optimal training parameters for each round according to the constraint time, and complete each round of model training according to the training parameters, which reduces the energy consumption of each terminal when completing the model training, thereby effectively balancing the system energy consumption, training time and model accuracy.

[0130] Embodiment 6

[0131] The sixth embodiment discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a federated learning program. When the federated learning program is executed by a processor, the federated learning method described in the fifth embodiment is implemented.

[0132] Embodiment 7

[0133] like Figure 5 As shown, this embodiment seven discloses a terminal. At the hardware level, the terminal includes a processor 21, an internal bus 22, a network interface 23, and a computer-readable storage medium 24. Of course, it may also include hardware required for other services, such as a touch screen. The processor 21 reads the corresponding computer program from the computer-readable storage medium 24 and then runs it, forming a request processing device at the logical level. Of course, in addition to software implementations, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logical unit, but can also be hardware or logic devices. The computer-readable storage medium 24 stores a federated learning program, and when the federated learning program is executed by the processor 21, it implements the federated learning method described in Example 5.

[0134] Embodiment 8

[0135] like Figure 6 As shown, the federated learning method of the federated learning system of this embodiment includes the following steps:

[0136] Step S100: the server obtains a constraint time according to the training information of the terminal, and the constraint time is used to enable the terminal to complete the model training within the constraint time.

[0137] Step S200: The terminal obtains training parameters according to the constraint time.

[0138] Step S300: The terminal completes model training according to the training parameters, where the training parameters are used to enable the terminal to consume minimum energy when completing the model training.

[0139] Among them, the federated learning methods of the server 10 and the terminal 20 have been described in the second embodiment and the fifth embodiment respectively, and will not be repeated here.

[0140] The terminal may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device or a combination of any several of these devices.

[0141] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0142] The specific implementation methods of the present invention are described in detail above. Although some embodiments have been shown and described, those skilled in the art should understand that these embodiments can be modified and improved without departing from the principles and spirit of the present invention, the scope of which is defined by the claims and their equivalents. These modifications and improvements should also be within the scope of protection of the present invention.

Claims

1. A federated learning method, It is characterized in that include: The server receives the training information sent from the terminal; The server obtains a constraint time according to the training information, where the constraint time is used to enable the terminal to complete the model training within the constraint time; The server sends the constraint time to the terminal; The training information includes hardware information and a preset amount of training data, wherein the method for the server to obtain the constraint time according to the training information includes: the server obtains the preset training completion time according to the hardware information and the preset amount of training data; The server obtains the constraint time according to the preset training completion time and the preset gradient update ratio.

2. The federated learning method according to claim 1, It is characterized in that The federated learning method further includes: The server receives the actual speed of this round of training sent from the terminal; The server obtains the constraint time of the next round of training according to the actual speed of the current round of training; The server sends the constraint time of the next round of training to the terminal.

3. The federated learning method according to claim 2, It is characterized in that The method for the server to obtain the constraint time of the next round of training according to the actual speed of the current round of training includes: The server obtains the preset completion time of the next round of training according to the actual speed of the current round of training; The server obtains the constraint time of the next round of training according to the preset completion time of the next round of training and the preset gradient update ratio.

4. The federated learning method according to claim 1, It is characterized in that Federated learning methods also include: The server obtains the progress value of the current round of training, and determines whether the progress value of the current round of training is greater than a preset value; If so, the server sends a first instruction to the terminal to start the next round of training; If not, the server sends a second instruction to the terminal to restart the current round of training.

5. The federated learning method according to claim 4, It is characterized in that The method for the server to obtain the progress value of this round of training is: The server obtains the current round training gradient value sent from each terminal, wherein the current round training progress value is the sum of the current round training gradient values ​​of each terminal.

6. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a federated learning control program, and when the federated learning control program is executed by a processor, the federated learning method according to any one of claims 1 to 5 is implemented.

7. A server, It is characterized in that The server includes a computer-readable storage medium, a processor, and a federated learning control program stored on the computer-readable storage medium. When the federated learning control program is executed by the processor, the federated learning method according to any one of claims 1 to 5 is implemented.

8. A federated learning method, It is characterized in that include: The terminal receives a constraint time sent from the server, wherein the constraint time is used to enable the terminal to complete the model training within the constraint time; The terminal obtains training parameters according to the constraint time; The terminal completes the model training according to the training parameters, and the training parameters are used to consume minimum energy when the terminal completes the model training; Among them, the constraint time is obtained by the server based on training information, and the training information includes hardware information and a preset amount of training data, including: the server obtains a preset training completion time based on the hardware information and the preset amount of training data; the server obtains the constraint time based on the preset training completion time and a preset gradient update ratio.

9. The federated learning method according to claim 8, It is characterized in that The method for the terminal to obtain the training parameters according to the constraint time includes: The terminal generates training parameters according to the hardware information, the preset training data volume and the constraint time, and the training parameters include a training speed and a frequency of a processor of the terminal matching the training speed.

10. The federated learning method according to claim 8, It is characterized in that The federated learning method further includes: The terminal obtains the actual speed of this round of training; The terminal sends the acquired actual speed of the current round of training to the server, wherein the actual speed of the current round of training is used to enable the server to acquire the constraint time of the next round of training.

11. The federated learning method according to claim 8, It is characterized in that The federated learning method further includes: The terminal sends the current round training gradient value to the server, where the current round training gradient value is used to enable the server to obtain the current round training progress value, where the current round training progress value is the sum of the current round training gradient values ​​of each terminal.

12. The federated learning method according to claim 11, It is characterized in that The federated learning method further includes: The terminal starts the next round of training according to the first instruction sent from the server, wherein the first instruction is an instruction generated by the server when the acquired progress value of the current round of training is greater than a preset value.

13. The federated learning method according to claim 11, It is characterized in that The federated learning method further includes: The terminal restarts the current round of training according to the second instruction sent from the server, wherein the second instruction is an instruction generated by the server when the acquired progress value of the current round of training is less than or equal to a preset value.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a federated learning control program, and when the federated learning control program is executed by a processor, the federated learning method according to any one of claims 8 to 13 is implemented.

15. A terminal, It is characterized in that The terminal includes a computer-readable storage medium, a processor, and a federated learning control program stored on the computer-readable storage medium. When the federated learning control program is executed by the processor, it implements the federated learning method according to any one of claims 8 to 13.

16. A federated learning system, It is characterized in that The federated learning system includes a server and a terminal; The server is used to receive training information sent from the terminal; the server is used to obtain a constraint time according to the training information, and the constraint time is used to enable the terminal to complete the model training within the constraint time; The terminal is used to receive the constraint time sent from the server; The terminal is used to obtain training parameters according to the constraint time; the terminal is also used to complete model training according to the training parameters, and the training parameters are used to consume minimum energy when the terminal completes the model training; The training information includes hardware information and a preset amount of training data, and the method for the server to obtain the constraint time based on the training information includes: the server obtains the preset training completion time based on the hardware information and the preset amount of training data; the server obtains the constraint time based on the preset training completion time and the preset gradient update ratio.

17. A federated learning method of the federated learning system according to claim 16, It is characterized in that The federated learning method includes: The server obtains a constraint time according to the training information of the terminal, where the constraint time is used to enable the terminal to complete the model training within the constraint time; The terminal obtains training parameters according to the constraint time; The terminal completes the model training according to the training parameters, and the training parameters are used to consume minimum energy when the terminal completes the model training.

Images