Model training method and device based on federated learning

Abstract

The application discloses a model training method and device based on federated learning, wherein the method comprises the following steps: in each round of communication, a server acquires model parameters and training loss uploaded by a user, and updates a user historical gradient list, wherein the user historical gradient list comprises the latest gradient of all users and the round number to which the gradient belongs; the server calculates the average gradient of the user sampled in the current round; the server calculates the minimum value in the cosine angle values between the average gradient and the historical gradient of the user not sampled in the current round, and obtains the global gradient updated in the round through a constrained nonlinear programming objective function optimization, so as to obtain the model after final aggregation; thus, the global model is optimized without damaging the model of the user not sampled, the representativeness of the model for the user not sampled in the current round is improved, the negative influence of the sampling deviation when the server selects the user is reduced, and the accuracy and fairness of the federated learning model are improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a model training method based on federated learning, a computer-readable storage medium, a computer device, and a model training apparatus based on federated learning. Background Technology

[0002] In today's era of rapid information technology development, models trained on massive amounts of data using machine learning algorithms are applied across various industries. However, with increasing awareness of privacy and the implementation of relevant laws, directly obtaining user data for model training may no longer be feasible. Federated learning, as a distributed machine learning paradigm with privacy protection capabilities, has emerged to address this need. It no longer requires users to upload data, but only exchanges encrypted model parameters. This allows for the collaborative training of a model using user data and computing power while protecting the privacy of user data.

[0003] In practical applications, users' specific habits and preferences can lead to a non-independent and identically distributed distribution of user data. Secondly, due to limited communication resources and network bottlenecks, the users participating in training in each round of communication are obtained through sampling. The aggregated model has weak representativeness for unsampled users, and sampling bias is inevitable. This not only burdens the early communication but also leads to a poor user experience because the model ignores the effects of certain users. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. To this end, one objective of this invention is to propose a model training method based on federated learning, which optimizes the global model without compromising the model of unsampled users, improves the model's representativeness of unsampled users in the current round, reduces the negative impact of sampling bias when the server selects users, and thus improves the accuracy and fairness of the federated learning model.

[0005] A second objective of this invention is to provide a computer-readable storage medium.

[0006] The third objective of this invention is to provide a computer device.

[0007] The fourth objective of this invention is to propose a model training device based on federated learning.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a model training method based on federated learning, comprising the following steps: In each round of communication, the server randomly samples a subset of users from the user set and sends a global model to each client corresponding to the user subset; each client trains the global model according to its corresponding dataset to obtain model parameters corresponding to each client, and sends the model parameters to the server; the server updates the user historical gradient set according to the model parameters and calculates the average gradient of each client in the current round, wherein the user historical gradient set includes the gradient and the corresponding round number; the server obtains the user historical gradient array of clients not sampled in the user set in the current round and the relative update gradient array of the clients not sampled in the user set in the current round, and updates the gradient weights according to the user historical gradient array, the relative update gradient array of the previous round, and the average gradient, so as to update the gradient according to the updated gradient weights; the server obtains the updated global model according to the updated gradients corresponding to each client, so as to perform the next round of model update, until a specified number of communication rounds is reached or the average training loss reaches the required level to complete the training.

[0009] According to the federated learning-based model training method of this invention, in each round of communication, the server first randomly samples a subset of users from the user set and sends the global model to each client corresponding to the user subset; then, each client trains the global model according to the corresponding dataset to obtain the model parameters corresponding to each client, and sends the model parameters to the server; then, the server updates the user historical gradient set according to the model parameters and calculates the average gradient of each client in the current round, wherein the user historical gradient set includes the gradient and the corresponding round number; then, the server obtains the user historical gradient array of clients not sampled in the current round's user set and the user historical gradient array of clients not sampled in the current round's user set. The sampled client's relative gradient array from the previous round is updated, and the gradient weights are updated based on the user's historical gradient array, the relative gradient array from the previous round, and the average gradient, so that the gradient is updated according to the updated gradient weights. Finally, the server obtains the updated global model based on the updated gradients corresponding to each client, so as to perform the next round of model updates, until the specified number of communication rounds is reached or the average training loss reaches the required level to complete the training. In this way, the global model is optimized without harming the model of unsampled users, the representativeness of the model in the current round of unsampled users is improved, the negative impact of sampling bias when the server selects users is reduced, and the accuracy and fairness of the federated learning model are improved.

[0010] In addition, the federated learning-based model training method proposed in the above embodiments of the present invention may also have the following additional technical features:

[0011] Optionally, before each round of communication, the deep learning model parameters are initialized on the server side, the user's historical gradient set is saved, the number of communication rounds is set to zero, and the number of rounds of historical gradients to be recorded is set.

[0012] Optionally, the server updates the gradient weights according to the following objective function:

[0013]

[0014] Where ω is the gradient weight; W is the probabilistic simplex; and T represents the transpose. is the update gradient to be optimized; t is the current round number; G is the array of relative update gradients from the previous round for clients not sampled in the user set of the current round; δ is the array of user history gradients for clients not sampled in the user set of the current round. It is the average gradient for each client in the current round.

[0015] Optionally, the gradient is updated according to the following formula:

[0016]

[0017] in, It is the gradient after the current round of updates, ω * These are the optimized and updated gradient weights.

[0018] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium storing a federated learning-based model training program thereon, which, when executed by a processor, implements the federated learning-based model training method as described above.

[0019] According to embodiments of the present invention, a computer-readable storage medium stores a federated learning-based model training program, enabling a processor to implement the federated learning-based model training method described above when executing the federated learning-based model training program. This optimizes the global model without compromising the model of unsampled users, improves the model's representativeness for unsampled users in the current round, reduces the negative impact of sampling bias when the server selects users, and thus improves the accuracy and fairness of the federated learning model.

[0020] To achieve the above objectives, a third aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the federated learning-based model training method as described above.

[0021] According to the computer device of the present invention, a federated learning-based model training program is stored in a memory, so that when the processor executes the federated learning-based model training program, the federated learning-based model training method described above is implemented. As a result, the global model is optimized without compromising the model of unsampled users, the representativeness of the model in the current round of unsampled users is improved, the negative impact of sampling bias when the server selects users is reduced, and the accuracy and fairness of the federated learning model are improved.

[0022] To achieve the above objectives, a fourth aspect of the present invention proposes a model training apparatus based on federated learning, comprising: a broadcast module, which randomly samples a subset of users from the user set in each round of communication and sends a global model to each client corresponding to the user subset; a local training module, which trains the global model according to the corresponding dataset to obtain model parameters corresponding to each client and sends the model parameters to the server; and a gradient update module, which updates the user's historical gradient set according to the model parameters and calculates the average gradient of each client in the current round. The user historical gradient set includes gradients and corresponding round numbers; it obtains the user historical gradient array of clients not sampled in the current round and the relative update gradient array of the previous round for clients not sampled in the current round, and updates the gradient weights according to the user historical gradient array, the relative update gradient array of the previous round, and the average gradient, so as to update the gradients according to the updated gradient weights; the model aggregation module is used to obtain the updated global model according to the updated gradients corresponding to each client, so as to perform the next round of model update, until the specified number of communication rounds is reached or the average training loss reaches the requirement to complete the training.

[0023] According to an embodiment of the present invention, a federated learning-based model training device, in each round of communication, uses a broadcast module to randomly sample a subset of users from the user set and send the global model to each client corresponding to the user subset; a local training module trains the global model according to the corresponding dataset to obtain the model parameters corresponding to each client and sends the model parameters to the server; a gradient update module updates the user historical gradient set according to the model parameters and calculates the average gradient of each client in the current round, wherein the user historical gradient set includes the gradient and the corresponding round number; and obtains the user historical gradient array of clients not sampled in the current round's user set and the user historical gradient array of clients not sampled in the current round's user set. The federated learning module updates the gradient array of the previous round relative to the sampled client, and updates the gradient weights based on the user's historical gradient array, the previous round relative update gradient array, and the average gradient, so that the gradient is updated according to the updated gradient weights. The model aggregation module obtains the updated global model based on the updated gradients corresponding to each client, so as to perform the next round of model updates, until the specified number of communication rounds is reached or the average training loss reaches the required level to complete the training. In this way, the global model is optimized without harming the model of unsampled users, the representativeness of the model in the current round of unsampled users is improved, the negative impact of sampling bias when the server selects users is reduced, and the accuracy and fairness of the federated learning model are improved. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a model training method based on federated learning according to an embodiment of the present invention.

[0025] Figure 2 This is a flowchart illustrating a model training method based on federated learning according to an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram showing the comparison of the gradients of unsampled users before and after the current round, considering a federated learning-based model training method according to an embodiment of the present invention.

[0027] Figure 4 This is a block diagram of a model training apparatus based on federated learning according to an embodiment of the present invention. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0029] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0030] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0031] Figure 1 This is a flowchart illustrating a federated learning-based model training method according to an embodiment of the present invention, as shown below. Figure 1 As shown, this federated learning-based model training method includes the following steps:

[0032] S101, in each round of communication, the server randomly samples a subset of users from the user set and sends the global model to each client corresponding to the user subset.

[0033] In other words, the server from the user set A random subset S of users of size K is sampled. (t) ={c k}, k∈{1,2,...,K}, and the model parameters θ for this round (t) Broadcast to S (t) Users in the middle.

[0034] As an example, before each round of communication, the deep learning model parameters are initialized on the server side, the user's historical gradient set is saved, the number of communication rounds is set to zero, and the number of rounds of historical gradients to be recorded is set.

[0035] In other words, the server initializes the deep learning model parameters θ. (0) At the same time, save the user's historical gradient set. The number of communication rounds t = 0, and the number of rounds τ for recording historical gradients is set.

[0036] S102, each client trains the global model based on the corresponding dataset to obtain the model parameters corresponding to each client, and then sends the model parameters to the server.

[0037] As an example, user c is selected in round t. i Accepts global model θ from the server. (t) User C k Using local datasets The model θ is repeatedly trained according to the learning rate η and the batch size B.(t) There are a total of E local rounds; calculate the new model parameters. and training loss And send it to the server.

[0038] S103, the server updates the user's historical gradient set according to the model parameters and calculates the average gradient of each client in the current round, where the user's historical gradient set includes the gradient and the corresponding round number.

[0039] In other words, the server receives data from S. (t) From the data uploaded by the user, the updated model parameters are obtained. The server calculates the gradient for each user. Obtain the user gradient set And update the user's historical gradient set. The server calculates the average gradient for each user.

[0040] S104, the server obtains the user historical gradient array of the clients not sampled in the current round user set and the relative update gradient array of the clients not sampled in the current round user set in the previous round, and updates the gradient weights according to the user historical gradient array, the relative update gradient array of the previous round and the average gradient, so as to update the gradient according to the updated gradient weights.

[0041] It should be noted that the historical gradient array of users not sampled in the current round and the relative update gradient array of the previous round for users who were not sampled Where C is the number of unsampled users.

[0042] As an example, the server optimizes and updates the gradient weights according to the following objective function to obtain the updated gradient weights ω. * :

[0043]

[0044] Where ω is the gradient weight; W is the probabilistic simplex; and T represents the transpose. is the update gradient to be optimized; t is the current round number; G is the array of relative update gradients from the previous round for clients not sampled in the user set of the current round; δ is the array of user history gradients for clients not sampled in the user set of the current round. It is the average gradient for each client in the current round.

[0045] As an example, the gradient is updated according to the following formula:

[0046]

[0047] in, ω* is the gradient after the current round of updates, and ω* is the gradient weight after the optimized update.

[0048] It should be noted that the server-side adjusts the update gradient. This is to eliminate the impact of inconsistent step sizes on the optimized gradient.

[0049] S105, the server obtains the updated global model based on the updated gradient corresponding to each client, so as to carry out the next round of model update, until the specified number of communication rounds or the average training loss reaches the required level to complete the training.

[0050] In other words, the server updates the model parameters. The model is then distributed to all clients for the next round of updates. Training stops if the number of training rounds reaches the specified number of rounds T or the average training loss meets the requirements.

[0051] In summary, such as Figure 2-3 As shown, in each round of communication, the server obtains the model parameters and training loss uploaded by the user and updates the user's historical gradient list. This list records the latest gradient of all users and the round number to which that gradient belongs. The server calculates the average gradient of the users sampled in the current round. The server then calculates the minimum cosine angle between each pair of historical gradients of unsampled users and the average gradient, and optimizes this angle using a constrained nonlinear programming objective function to obtain the updated global gradient for that round, thus yielding the final aggregated model. Therefore, by employing a federated learning aggregation optimization method that considers server sampling bias, the aggregated model considers not only the sampled user models but also the unsampled user models in the current round. This optimizes the global model without compromising the representation of unsampled user models in the current round, improving the model's representativeness for unsampled users and reducing the negative impact of server sampling bias in user selection, thereby enhancing the accuracy and fairness of the federated learning model.

[0052] To implement the above embodiments, this invention proposes a computer-readable storage medium storing a federated learning-based model training program, which, when executed by a processor, implements the federated learning-based model training method as described above.

[0053] According to embodiments of the present invention, a computer-readable storage medium stores a federated learning-based model training program, enabling a processor to implement the federated learning-based model training method described above when executing the federated learning-based model training program. This optimizes the global model without compromising the model of unsampled users, improves the model's representativeness for unsampled users in the current round, reduces the negative impact of sampling bias when the server selects users, and thus improves the accuracy and fairness of the federated learning model.

[0054] To implement the above embodiments, this invention proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the federated learning-based model training method described above.

[0055] According to the computer device of the present invention, a federated learning-based model training program is stored in a memory, so that when the processor executes the federated learning-based model training program, the federated learning-based model training method described above is implemented. As a result, the global model is optimized without compromising the model of unsampled users, the representativeness of the model in the current round of unsampled users is improved, the negative impact of sampling bias when the server selects users is reduced, and the accuracy and fairness of the federated learning model are improved.

[0056] To implement the above embodiments, this invention also proposes a model training device based on federated learning, such as... Figure 4 As shown, the federated learning-based model training device includes: a broadcast module 10, a local training module 20, a gradient update module 30, and a model aggregation module 40.

[0057] The broadcast module 10 is used to randomly sample a subset of users from the user set in each round of communication and send the global model to each client corresponding to the user subset; the local training module 20 is used to train the global model according to the corresponding dataset to obtain the model parameters corresponding to each client and send the model parameters to the server; the gradient update module 30 is used to update the user historical gradient set according to the model parameters and calculate the average gradient of each client in the current round, wherein the user historical gradient set includes the gradient and the corresponding round number; obtain the user historical gradient array of clients not sampled in the current round and the relative update gradient array of clients not sampled in the current round in the previous round, and update the gradient weights according to the user historical gradient array, the relative update gradient array of the previous round and the average gradient, so as to update the gradient according to the updated gradient weights; the model aggregation module 40 is used to obtain the updated global model according to the updated gradient corresponding to each client, so as to perform the next round of model update, until the specified number of communication rounds is reached or the average training loss reaches the requirement to complete the training.

[0058] As an example, the broadcast module 10 is also used to initialize the deep learning model parameters on the server side before each round of communication, save the user's historical gradient set, set the number of communication rounds to zero, and set the number of rounds of historical gradients to be recorded.

[0059] As an example, the server optimizes and updates the gradient weights according to the following objective function:

[0060]

[0061] Where ω is the gradient weight; W is the probabilistic simplex; and T represents the transpose. is the update gradient to be optimized; t is the current round number; G is the array of relative update gradients from the previous round for clients not sampled in the current round's user set; δ is the array of user history gradients for clients not sampled in the current round's user set; It is the average gradient for each client in the current round.

[0062] As an example, the gradient is updated according to the following formula:

[0063]

[0064] in, It is the gradient after the current round of updates, ω * These are the optimized and updated gradient weights.

[0065] In summary, according to an embodiment of the present invention, a federated learning-based model training device, through a broadcast module, randomly samples a subset of users from the user set in each round of communication and sends the global model to each client corresponding to the user subset; a local training module trains the global model according to the corresponding dataset to obtain the model parameters corresponding to each client and sends the model parameters to the server; a gradient update module updates the user historical gradient set according to the model parameters and calculates the average gradient of each client in the current round, wherein the user historical gradient set includes the gradient and the corresponding round number; and obtains the user historical gradient array of clients not sampled in the current round's user set and the user historical gradient array of the current round's user set. The algorithm updates the gradient array of unsampled clients relative to the previous round, and updates the gradient weights based on the user's historical gradient array, the gradient array updated relative to the previous round, and the average gradient, so that the gradients can be updated according to the updated gradient weights. The model aggregation module obtains the updated global model based on the updated gradients corresponding to each client, so as to perform the next round of model updates, until the specified number of communication rounds is reached or the average training loss reaches the required level to complete the training. In this way, the global model is optimized without harming the model of unsampled users, the representativeness of the model in the current round of unsampled users is improved, the negative impact of sampling bias when the server selects users is reduced, and the accuracy and fairness of the federated learning model are improved.

[0066] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0071] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0072] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0073] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0074] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0075] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0076] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0077] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A model training method based on federated learning, characterized in that, The method comprises the following steps: In each round of communication, the server randomly samples a user subset from the user set and sends a global model to each client corresponding in the user subset; Each client trains the global model according to the corresponding data set to obtain the model parameters corresponding to each client and sends the model parameters to the server; The server updates a user historical gradient set according to the model parameters and calculates the average gradient of each client in the current round, wherein the user historical gradient set comprises a gradient and a corresponding round number; The server obtains a user historical gradient array of the clients not sampled in the current round in the user set and a last round relative update gradient array of the clients not sampled in the current round in the user set and updates a gradient weight according to the user historical gradient array, the last round relative update gradient array and the average gradient so as to update the gradient according to the updated gradient weight; The server obtains an updated global model according to the updated gradient of each client to perform the next round of model updating until a specified communication round number is reached or the average training loss reaches a requirement to complete the training; The server updates the gradient weight according to the following target function: where ω is the gradient weight; W is the probability simplex; T represents the transpose; is the update gradient to be optimized; t is the current round; G is the relative update gradient array of the last round of the clients that are not sampled in the current round of the user set; δ is the user historical gradient array of the clients that are not sampled in the current round of the user set; is the average gradient of each client in the current round; The gradient is updated according to the following formula: wherein, is the current round updated gradient, ω * is the optimized updated gradient weight. 2.The federated learning based model training method of claim 1, wherein, Before each round of communication, the server also initializes deep learning model parameters and saves a user historical gradient set, sets the communication round number to zero and sets the round number of the historical gradient to be recorded.

3. A computer-readable storage medium, characterized in that, A processor executes a model training program based on federated learning stored thereon, and the model training program based on federated learning implements the model training method based on federated learning according to any one of claims 1-2 when executed by the processor.

4. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program, and the model training method based on federated learning according to any one of claims 1-2 is implemented. 5.A model training apparatus based on federated learning, characterized in that, The method comprises: A broadcast module is arranged on the server and is used to randomly sample a user subset from the user set in each round of communication and send a global model to each client corresponding in the user subset; A local training module is used to train the global model according to the corresponding data set to obtain the model parameters corresponding to each client and send the model parameters to the server; A gradient updating module is used to update a user historical gradient set according to the model parameters and calculate the average gradient of each client in the current round, wherein the user historical gradient set comprises a gradient and a corresponding round number; obtain a user historical gradient array of the clients not sampled in the current round in the user set and a last round relative update gradient array of the clients not sampled in the current round in the user set and update a gradient weight according to the user historical gradient array, the last round relative update gradient array and the average gradient so as to update the gradient according to the updated gradient weight; a model aggregation module configured to obtain an updated global model according to the updated gradient corresponding to each client, so as to perform a next round of model updating, until a specified communication round number is reached or average training loss reaches a requirement after which training is completed; wherein the server side optimizes the gradient weight according to a following target function: where ω is the gradient weight; W is the probability simplex; T represents the transpose; is the update gradient to be optimized; t is the current round; G is the relative update gradient array of the last round of the clients that are not sampled in the current round of the user set; δ is the user historical gradient array of the clients that are not sampled in the current round of the user set; is the average gradient of each client in the current round; wherein the gradient is updated according to a following formula: wherein, is the current round updated gradient, ω * is the optimized updated gradient weight. 6.The federated learning based model training apparatus of claim 5, wherein, The broadcast module is further configured to initialize deep learning model parameters at the server side before each round of communication, save a user historical gradient set, set a communication round number to zero, and set a round number of historical gradients to be recorded.

Images