Distributed training method and system, storage medium and electronic equipment

By determining the target device end in federated learning, optimizing the global model using adaptive learning rate algorithm and momentum cache, the problems of client drift and communication overhead are solved, model performance and training efficiency are improved, and computation and memory consumption are reduced.

CN118917383APending Publication Date: 2024-11-08HANGZHOU HEXIN SOFTWARE TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410962387.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In federated learning, due to the different distribution of data sets of each device, client drift problems occur during model training, which affects model performance and accuracy, and it becomes complicated to summarize the scattered data to the central server, increasing communication overhead.

Method used

During each round of training, the target device side is determined, the server issues a global model and updates the local model using the adaptive learning rate algorithm, uploads the updated local model changes, the server aggregates parameters and optimizes according to the exponential weighted average of the historical gradient , reset the first and second order momentum caches to reduce additional communication, maintain control variables to avoid customer drift.

Benefits of technology

It effectively reduces the performance reduction caused by model fine-tuning, reduces communication overhead, improves training efficiency and accuracy, and reduces computational volume and memory consumption, avoids customer drift problems caused by Non-IID.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917383A_ABST
    Figure CN118917383A_ABST
Patent Text Reader

Abstract

The invention provides a distributed training method, which comprises the following steps of: in each round of training, determining that a target equipment end server participating in the round of training in equipment ends issues a global model to a target equipment end; the server receives the local model change quantity of each target equipment end, aggregates to obtain a global parameter update quantity, and determines a gradient according to an exponential weighted average value of historical gradients; and optimizing the global model by applying the global parameter update quantity and the gradient, wherein the optimized global model is used for executing the next round of iterative training. According to the method, the calculation amount and memory consumption can be reduced as much as possible while the training effect is ensured. A control variable is maintained for each device on a server, so that the problem of customer drift caused by Non-IID is avoided. The invention also provides a distributed training system, a computer readable storage medium and electronic equipment, which have the above beneficial effects, which are not repeated here.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a distributed training method, system, storage medium and electronic device. Background Art

[0002] Federated Learning (FL), as a promising paradigm for processing decentralized data, makes it possible to conduct collaborative model training among a large number of devices without transmitting the original data to a central server or data center. However, in practical applications, it becomes complicated or even impossible to aggregate decentralized data to a central server or data center. FL usually manages distributed model updates in devices by using a parameter server module, which only exchanges the parameters of the updated model between the parameter server and the device without involving the transmission of the original data. Since the datasets of each participating device are usually not independent and identically distributed (Non-IID), this will lead to client drift problems during model training. The client drift problem refers to the fact that in federated learning, due to the different data distributions of each client, the global model differs between the local models on these clients, thereby affecting the performance and accuracy of the overall model. Summary of the invention

[0003] The purpose of this application is to provide a distributed training method, system, computer-readable storage medium and electronic device, which can avoid the performance degradation caused by model fine-tuning during federated learning and effectively reduce communication overhead.

[0004] In order to solve the above technical problems, this application provides a distributed training method, and the specific technical solution is as follows:

[0005] In each round of training, the target device end participating in the current round of training is determined;

[0006] The server sends the global model to the target device; wherein the target device applies the received global model using an adaptive learning rate algorithm to update the local model, and uploads the updated local model change to the server; wherein when the target device starts to update the local model, the first-order momentum cache and the second-order momentum cache are both set to 0;

[0007] The server receives the local model change amount of each target device, aggregates it to obtain the global parameter update amount, and determines the gradient according to the exponentially weighted average of the historical gradients;

[0008] The global model is optimized by applying the global parameter update amount and the gradient, and the optimized global model is used to perform the next round of iterative training.

[0009] Optionally, the aggregating to obtain a global parameter update amount and determining a gradient according to an exponentially weighted average of historical gradients includes:

[0010] The global parameter update amount is obtained by aggregation according to the aggregation formula, and the gradient is determined according to the exponentially weighted average of the historical gradients;

[0011] The aggregation formula is:

[0012]

[0013] Among them, Δw r is the global parameter update amount, r is the number of iterations, is the amount of local model changes uploaded by each target device, N is the number of target devices, n i is the sample size.

[0014] Optionally, after the server receives the local model change amount of each target device end, aggregates to obtain the global parameter update amount, and determines the gradient according to the exponentially weighted average of the historical gradients, it also includes:

[0015] The control variable change amount of each target device end is calculated according to the local update step size of each target device end and the global parameter update amount.

[0016] Optionally, after calculating the control variable change amount of each target device end according to the local update step size of each target device end and the global parameter update amount, the method further includes:

[0017] The global control variable of each target device end is calculated according to the control variable change amount, the number of the device ends and the number of the target device ends.

[0018] Optionally, the calculation formula for the control variable change is:

[0019]

[0020] Among them, α is the local update step size, which is used to obtain the change of the control variable of each device; c i is the change in the control variable of each target end, T i Represents the local epoch of the i-th round of training.

[0021] Optionally, the calculation formula of the global control variable is:

[0022]

[0023] Where Δc r Depend on To obtain, is the control variable of the i-th device during the r-th round of training, is the control variable of the i-th device during the r-1th round of training; is the global control variable after the update in round r, is the global control variable after the update in the r-1th round, |M| is the total number of devices under distributed machine learning, and |S| is the target device for the current round of training.

[0024] Optionally, applying the global parameter update amount and the gradient to optimize the global model includes:

[0025] Determining a global gradient and a momentum at each of the target devices;

[0026] Determine the global momentum according to the momentum of each of the target devices;

[0027] The global model is optimized according to the global gradient, the global momentum and the global parameter update amount.

[0028] The present application also provides a distributed training system, including:

[0029] A device determination module is used to determine the target device end participating in the current round of training in each round of training;

[0030] A model sending module, used to send the global model to the target device end; wherein the target device end applies the received global model using an adaptive learning rate algorithm to update the local model, and uploads the updated local model change amount to the server; wherein, when the target device end starts to update the local model, the first-order momentum cache and the second-order momentum cache are both set to 0;

[0031] A parameter aggregation module, used for receiving the local model change amount of each target device end, aggregating to obtain the global parameter update amount, and determining the gradient according to the exponentially weighted average of the historical gradient;

[0032] A global model optimization module is used to optimize the global model by applying the global parameter update amount and the gradient; the optimized global model is used to perform the next round of iterative training.

[0033] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above-mentioned method when executed by a processor.

[0034] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.

[0035] The present application provides a distributed training method, comprising: in each round of training, determining the target device server participating in the current round of training in the device end and sending the global model to the target device end; wherein the target device end applies the received global model and uses an adaptive learning rate algorithm to update the local model, and uploads the updated local model change amount to the server; wherein, when the target device end starts to update the local model, the first-order momentum cache and the second-order momentum cache are set to 0; the server receives the local model change amount of each target device end, aggregates it to obtain the global parameter update amount, and determines the gradient according to the exponentially weighted average of the historical gradient; the global parameter update amount and the gradient are applied to optimize the global model, and the optimized global model is used to perform the next round of iterative training.

[0036] During the distributed training process, this application resets the first-order and second-order momentum caches to 0 at the beginning of the local model update on each target device to avoid additional communication of momentum variables between the server and the device. The learning process is accelerated on the server side by accumulating previous gradient information, saving time costs. On the target device side, the optimization algorithm of adaptive learning rate is suitable for scenarios with limited computing resources on the device side. It can minimize the amount of calculation and memory consumption while ensuring the training effect. This application can also avoid the problem of client drift caused by Non-IID by maintaining a control variable for each device on the server.

[0037] The present application also provides a distributed training system, a computer-readable storage medium and an electronic device, which have the above-mentioned beneficial effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A flow chart of a distributed training system method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the structure of a distributed training system provided in an embodiment of the present application;

[0041] Figure 3 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] The technical problem to be solved in this application is to optimize large language models in distributed machine learning. Consider a distributed machine learning setting with a parameter server and M devices. Assume that the data used to adjust the global model w is distributed among multiple devices.

[0044] On each device i, there is a dataset where s i represents the sample, m i Indicates i The corresponding label, n i Indicates D i The number of samples in . N represents the total number of samples on all devices, S represents the set of all samples, and M represents the set of labels. The problem to be solved in this application can be transformed into how to efficiently optimize the global model w to minimize the global loss:

[0045]

[0046] Among them, F(w) represents the global loss, represents the loss function on device i, where f(w i ,s i ,m i ) Compute a large language model w i In the dataset {s i ,m i} local losses on.

[0047] Consider a parameter server and multiple devices for the tuning process. Assume that a large language model is deployed on each device. During the distributed machine learning tuning process, the parameters of each device will go through multiple rounds of updates. First, the server sends the global model to the selected devices. Second, these devices use the received model to update their local models and upload the local model changes to the server. Finally, the server optimizes based on the model changes uploaded by the device.

[0048] This application provides a new communication-efficient adaptive optimization method to solve the Non-IID problem without introducing additional communication overhead. In order to achieve superior performance, a server-side adaptive optimization method based on Momentum and a device-side adaptive optimization method based on Adam are proposed. At the beginning of the local update, the first-order and second-order momentum caches are reset to 0 to avoid additional communication of momentum variables between the server and the device. In addition, a control variable is maintained for each device on the server to avoid the client drift problem caused by Non-IID.

[0049] See also Figure 1 , Figure 1 This is a flow chart of a distributed training system method provided in an embodiment of the present application, the method comprising:

[0050] S101: In each round of training, determining a target device end among the device ends that participates in the current round of training;

[0051] S102: The server sends the global model to the target device; wherein the target device applies the received global model using an adaptive learning rate algorithm to update the local model, and uploads the updated local model change to the server; wherein when the target device starts to update the local model, the first-order momentum cache and the second-order momentum cache are both set to 0;

[0052] S103: The server receives the local model change amount of each target device, aggregates it to obtain the global parameter update amount, and determines the gradient according to the exponential weighted average of the historical gradient;

[0053] S104: Apply the global parameter update amount and the gradient to optimize the global model, and the optimized global model is used to perform the next round of iterative training.

[0054] This application proposes a new communication-efficient adaptive optimization method to solve the Non-IID problem without introducing additional communication overhead. In order to achieve superior performance, a server-side adaptive optimization method based on Momentum and a device-side adaptive optimization method based on Adam are proposed. At the beginning of the local update, the first-order and second-order momentum caches are reset to 0 to avoid additional communication of momentum variables between the server and the device. In addition, a control variable is maintained for each device on the server to avoid the client drift problem caused by Non-IID. The first-order momentum refers to the gradient, and the second-order momentum refers to the momentum, that is, the change of the gradient. By caching both the first-order momentum and the second-order momentum to 0 as the initial state, inconsistent values ​​can be avoided. By setting the second-order momentum, an inertia term is introduced in the gradient descent process to accelerate convergence, so that the optimization process can move faster towards the optimal solution while reducing oscillation. The transmission of these additional state information between the server and the device can be avoided, thereby reducing communication overhead.

[0055] At the beginning of the distributed training process, the global model w is initialized 0 w, and set the device-side control variable and global control variables and global momentum Set to 0. Global control variables refer to variables used to synchronize and control the training status of each device in distributed training. This variable can be a learning rate, number of iterations, cumulative gradient value, etc. They are kept consistent globally to ensure that the training process of all devices can be coordinated and carried out in a consistent manner. The global momentum is the summary of the gradient and momentum calculated by each device when performing training.

[0056] At the beginning of the rth round of training, the client S is first sampled from all devices M. The server sends the global model to the target device, for example, the model parameters w of the previous round are r-1 It is sent to each device in the client S. When the device i is updated, the first-order momentum of the device is first and second-order momentum Set to 0, and then use the adaptive learning rate algorithm locally for T i Update the local round and upload the updated parameters To the server, the global parameters are updated by aggregation at the server:

[0057]

[0058] Among them, N is the number of samples of all currently selected devices, n iis the number of samples at device end i, and this aggregation formula can also be used for aggregation of other parameters. The control variable change amount of each target device end is calculated according to the local update step size of each target device end and the global parameter update amount. The specific formula is as follows:

[0059]

[0060] Where α is the local update step size, and the lower right subscript g represents the global meaning, that is, represents the global control variable in the previous round of training, represents the local control variable of the device in this round of training, and Indicates the local control variable of the device in the last round of training. The change in the control variable of each device can be obtained T i Represents the local epoch of the i-th round of training. A local epoch refers to a set of iterations that use a subset of the dataset for training when training a neural network. Each local epoch usually includes one or more complete training iterations, and the iterations use the same subset of the dataset. The global parameter update formula can aggregate the control variable update amount Δc r . Afterwards, update the global control variables:

[0061]

[0062] Where |M| is the total number of devices in distributed machine learning, and |S| is the number of devices selected in the current round. At the same time, the global gradient is obtained:

[0063]

[0064] Based on the global gradient, the momentum of each device can be obtained and the momentum of the device can be updated to update the model:

[0065]

[0066] Among them, β is the global update step size. Finally, the server receives the local model changes of each target device, aggregates them to obtain the global parameter update, and determines the gradient according to the exponential weighted average of the historical gradients. That is, the global model w is updated by global parameter update aggregation. r and the global momentum m r .

[0067] During the distributed training process, this application resets the first-order and second-order momentum caches to 0 at the beginning of the local model update on each target device to avoid additional communication of momentum variables between the server and the device. The learning process is accelerated on the server side by accumulating previous gradient information, saving time costs. On the target device side, the optimization algorithm with adaptive learning rate is suitable for scenarios with limited computing resources on the device side. It can minimize the amount of calculation and memory consumption while ensuring the training effect. By maintaining a control variable for each device on the server, the problem of client drift caused by Non-IID can be avoided.

[0068] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a distributed training system provided in an embodiment of the present application, the system comprising:

[0069] A device determination module is used to determine the target device end participating in the current round of training in each round of training;

[0070] A model sending module, used to send the global model to the target device end; wherein the target device end applies the received global model using an adaptive learning rate algorithm to update the local model, and uploads the updated local model change amount to the server; wherein, when the target device end starts to update the local model, the first-order momentum cache and the second-order momentum cache are both set to 0;

[0071] A parameter aggregation module, used for receiving the local model change amount of each target device end, aggregating to obtain the global parameter update amount, and determining the gradient according to the exponentially weighted average of the historical gradient;

[0072] A global model optimization module is used to optimize the global model by applying the global parameter update amount and the gradient; the optimized global model is used to perform the next round of iterative training.

[0073] Based on the above embodiment, as an optional embodiment, the parameter aggregation module includes:

[0074] A determination unit is used to obtain the global parameter update amount according to the aggregation formula and determine the gradient according to the exponentially weighted average of the historical gradients;

[0075] The aggregation formula is:

[0076]

[0077] Among them, Δw r is the global parameter update amount, r is the number of iterations, is the amount of local model changes uploaded by each target device, N is the number of target devices, n i is the sample size.

[0078] Based on the above embodiment, as an optional embodiment, it also includes:

[0079] The first calculation unit is used to calculate the control variable change amount of each target device end according to the local update step size of each target device end and the global parameter update amount.

[0080] Based on the above embodiment, as an optional embodiment, it also includes:

[0081] The second calculation unit is used to calculate the global control variable of each target device end according to the control variable change amount, the number of the device ends and the number of the target device ends.

[0082] Based on the above embodiment, as an optional embodiment, the calculation formula of the control variable change is:

[0083]

[0084] Among them, α is the local update step size, which is used to obtain the change of the control variable of each device; c i The control variable change amount for each of the target terminals.

[0085] Based on the above embodiment, as an optional embodiment, the calculation formula of the global control variable is:

[0086]

[0087] Where Δc r Depend on To obtain, is the control variable of the i-th device during the r-th round of training, is the control variable of the i-th device during the r-1th round of training; is the global control variable after the update in round r, is the global control variable after the update in the r-1th round, |M| is the total number of devices under distributed machine learning, and |S| is the target device for the current round of training.

[0088] Based on the above embodiment, as an optional embodiment, the global model optimization module includes:

[0089] A first determining unit, used to determine a global gradient and a momentum of each of the target device ends;

[0090] A second determining unit, configured to determine a global momentum according to the momentum of each of the target device ends;

[0091] An optimization unit is used to optimize the global model according to the global gradient, the global momentum and the global parameter update amount.

[0092] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0093] The present application also provides an electronic device, see Figure 3 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 3 As shown, a processor 1410 and a memory 1420 may be included.

[0094] Among them, the processor 1410 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1410 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1410 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0095] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 1420 is at least used to store the following computer program 1421, wherein, after the computer program is loaded and executed by the processor 1410, it can implement the relevant steps in the method performed by the electronic device side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 1422 may include Windows, Linux, Android, etc.

[0096] In some embodiments, the electronic device may further include a display screen 1430 , an input / output interface 1440 , a communication interface 1450 , a sensor 1460 , a power source 1470 , and a communication bus 1480 .

[0097] certainly, Figure 3 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 3 More or fewer components than shown, or combinations of certain components.

[0098] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0099] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

[0100] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A distributed training method, characterized in that: include: In each round of training, the target device end participating in the current round of training is determined; The server sends the global model to the target device; wherein the target device applies the received global model using an adaptive learning rate algorithm to update the local model, and uploads the updated local model change to the server; wherein when the target device starts to update the local model, the first-order momentum cache and the second-order momentum cache are both set to 0; The server receives the local model change amount of each target device, aggregates it to obtain the global parameter update amount, and determines the gradient according to the exponentially weighted average of the historical gradients; The global model is optimized by applying the global parameter update amount and the gradient, and the optimized global model is used to perform the next round of iterative training.

2. The distributed training method according to claim 1, characterized in that: The aggregation obtains the global parameter update amount, and determines the gradient according to the exponentially weighted average of the historical gradients, including: The global parameter update amount is obtained by aggregation according to the aggregation formula, and the gradient is determined according to the exponentially weighted average of the historical gradients; The aggregation formula is: Among them, Δw r is the global parameter update amount, r is the number of iterations, is the amount of local model changes uploaded by each target device, N is the number of target devices, n i is the sample size.

3. The distributed training method according to claim 2, characterized in that: The server receives the local model change amount of each target device end, aggregates to obtain the global parameter update amount, and determines the gradient according to the exponential weighted average of the historical gradient, and further includes: The control variable change amount of each target device end is calculated according to the local update step size of each target device end and the global parameter update amount.

4. The distributed training method according to claim 3, characterized in that: After calculating the control variable change amount of each target device end according to the local update step size of each target device end and the global parameter update amount, the method further includes: The global control variable of each target device end is calculated according to the control variable change amount, the number of the device ends and the number of the target device ends.

5. The distributed training method according to claim 4, characterized in that: The calculation formula of the control variable change is: Among them, α is the local update step size, which is used to obtain the change of the control variable of each device; c i is the change in the control variable of each target end, T i Represents the local epoch of the i-th round of training.

6. The distributed training method according to claim 5, characterized in that: The calculation formula of the global control variable is: Where Δc r Depend on To obtain, is the control variable of the i-th device during the r-th round of training, is the control variable of the i-th device during the r-1th round of training; is the global control variable after the update in round r, is the global control variable after the update in the r-1th round, |M| is the total number of devices under distributed machine learning, and |S| is the target device for the current round of training.

7. The distributed training method according to claim 6, characterized in that: Applying the global parameter update amount and the gradient to optimize the global model includes: Determining a global gradient and a momentum at each of the target devices; Determine the global momentum according to the momentum of each of the target devices; The global model is optimized according to the global gradient, the global momentum and the global parameter update amount.

8. A distributed training system, characterized in that: include: The device determination module is used to determine the target device end participating in the current round of training in each round of training; A model sending module, used to send the global model to the target device end; wherein the target device end applies the received global model using an adaptive learning rate algorithm to update the local model, and uploads the updated local model change amount to the server; wherein, when the target device end starts to update the local model, the first-order momentum cache and the second-order momentum cache are both set to 0; A parameter aggregation module, used for receiving the local model change amount of each target device end, aggregating to obtain the global parameter update amount, and determining the gradient according to the exponentially weighted average of the historical gradient; A global model optimization module is used to optimize the global model by applying the global parameter update amount and the gradient; the optimized global model is used to perform the next round of iterative training.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the distributed training method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the steps of the distributed training method as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Federal learning optimization method and device, electronic equipment and storage medium

    CN118246567A

  • Federated learning model training method and apparatus, device, and storage medium

    WO2024099109A1