Communication method and apparatus, and storage medium

Abstract

Embodiments of the present application provide a communication method and apparatus, and a storage medium. The method comprises: receiving first information from multiple terminals, wherein the first information indicates a first local parameter, a second local parameter, and a third local parameter; on the basis of the total power of gradient noise and the first information from the multiple terminals, obtaining an update step size; and sending an updated global model or the update step size to the multiple terminals, respectively, wherein the updated global model is obtained by processing the global model on the basis of the update step size. The step size is dynamically adjusted on the basis of the gradient noise and a training state indicated by each local parameter, thus accelerating the gradient descent of neural network learning, and solving the problem of slow neural network training or even non-convergence in the presence of gradient noise. By accelerating the neural network training process, "Green AI" can be achieved, and the communication overhead can be saved. The present solution can be compatible with digital transmission of federated learning, analog transmission, and over-the-air aggregation.

Description

Communication method, device and storage medium Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a communication method, device, and storage medium. Background Art

[0002] In distributed learning scenarios (such as federated learning), local devices usually calculate and upload gradients, which are then used to update the global model on the server. However, there is a potential problem: the dimensionality of the gradients is very high, reaching tens of millions or even hundreds of millions, and uploading gradients will incur severe communication overhead and energy consumption.

[0003] One potential solution to these problems is to accelerate neural network training. A branch of research in this area focuses on finding an appropriate gradient and corresponding step size. However, current neural network updates do not consider the impact of gradient noise. When gradient noise is present, it increases as the update progresses, resulting in slow convergence of training / updates, or even non-convergence.

[0004] Summary of the Invention

[0005] The present application discloses a communication method, device and storage medium, which can accelerate the training process and achieve the purpose of reducing communication overhead.

[0006] In a first aspect, an embodiment of the present application provides a communication method. The method may be executed by a communication device, or by a component of a communication device (e.g., a chip (system)). The method may include: receiving first information from multiple terminals, the first information indicating a first local parameter, a second local parameter, and a third local parameter, wherein the first local parameter is used to characterize the stochastic gradient of the terminal during the current model training, the second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model corresponding to the current model training, and the third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training; obtaining an update step size according to the total power of the gradient noise and the first information from the multiple terminals; and sending the updated global model or the update step size to the multiple terminals respectively, wherein the updated global model is obtained by processing the global model according to the update step size.

[0007] In an embodiment of the present application, the terminal reports the first information so that the base station side calculates the update step size based on the total power of the gradient noise and the local parameters indicated by the first information, thereby obtaining an updated global model. With this solution, the step size is dynamically adjusted according to the gradient noise and the training state indicated by each local parameter, thereby accelerating the gradient descent of the neural network learning, successfully solving the problem of slow neural network training or even non-convergence in the presence of gradient noise; on the other hand, by accelerating the neural network training process, not only can Green Artificial Intelligence (Green AI) be achieved, but also communication overhead can be saved; and this solution is compatible with digital transmission, analog transmission, and air aggregation of federated learning.

[0008] In a possible implementation manner, obtaining an update step size according to the total gradient noise power and the first information from the multiple terminals includes:

[0009] A first global parameter, a second global parameter, and a third global parameter are obtained according to the first information from the multiple terminals; the first global parameter is used to represent the inner product of the global gradient of the current model training and a first value, the first value being the difference between the weight of the global model and the weight of the preset model; the second global parameter is used to represent the inner product of the global gradient of the current model training and the global gradient of the previous round of model training; and the third global parameter is used to represent the modulus of the global gradient of the current model training;

[0010] An update step size is obtained according to the total gradient noise power and the first global parameter, the second global parameter, and the third global parameter.

[0011] This example calculates global parameters based on the local parameters indicated by the first information. These global parameters indicate training state information, and then determines the update step size based on these global parameters and the total power of the gradient noise. This solution dynamically adjusts the step size based on the gradient noise and training state, thereby accelerating the gradient descent of neural network learning and successfully resolving the issue of slow or even non-convergence neural network training in the presence of gradient noise.

[0012] In a possible implementation, the base station further obtains the global parameter according to the inner product of the gradient used to update the model weight in the previous round and the first value. It can be expressed as: Optional, It can be estimated to be in, express Then, the base station calculates the estimated value of The total power of the gradient noise and the first global parameter, the second global parameter, and the third global parameter are updated with a step size.

[0013] In a possible implementation, the first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize the modulus of the stochastic gradient, and the third global parameter is obtained according to the fourth local parameter from the multiple terminals.

[0014] In a possible implementation, obtaining the first global parameter, the second global parameter, and the third global parameter according to the first information from the multiple terminals includes:

[0015] Obtaining a global gradient during current model training according to the first local parameters respectively sent by the multiple terminals;

[0016] Obtaining the third global parameter according to the global gradient during the current model training;

[0017] Obtaining the first global parameter according to the second local parameters respectively sent by the multiple terminals;

[0018] The second global parameter is obtained according to the third local parameters respectively sent by the multiple terminals.

[0019] In a possible implementation, the first information further indicates a fifth local parameter, where the fifth local parameter is used to characterize a sampling noise vector included in the random gradient, and the total gradient noise power is obtained based on the fifth local parameter from multiple terminals.

[0020] In a second aspect, an embodiment of the present application provides a communication method. The method can be executed by a communication device, or by a component of the communication device (such as a chip (system)). The method may include: receiving an updated global model or an update step; obtaining a first local parameter, a second local parameter, and a third local parameter according to the updated global model or the update step; wherein the first local parameter is used to characterize the stochastic gradient of the terminal during the current model training, the second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model, and the third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training; sending first information, the first information indicating the first local parameter, the second local parameter, and the third local parameter.

[0021] The terminal obtains various local parameters based on the received updated global model or update step size, and then reports them to the base station side, so that the base station side can calculate the step size or global model based on the local parameters indicated by the first information. Using this solution, the step size is dynamically adjusted according to the gradient noise and training status, thereby accelerating the gradient descent of neural network learning, successfully solving the problem of slow neural network training or even non-convergence in the presence of gradient noise. On the other hand, by accelerating the neural network training process, not only can "Green AI" be achieved, but communication overhead can also be saved. In addition, this solution is compatible with digital transmission, analog transmission, and over-the-air aggregation of federated learning.

[0022] In a possible implementation, obtaining the first local parameter, the second local parameter, and the third local parameter according to the updated global model includes:

[0023] Obtaining a first local parameter according to the updated global model and the local data set;

[0024] Obtaining a second local parameter according to the first local parameter and the weight of the updated global model;

[0025] A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

[0026] In a possible implementation, obtaining the first local parameter, the second local parameter, and the third local parameter according to the update step size includes:

[0027] Obtaining an updated global model according to the update step size;

[0028] Obtaining a first local parameter according to the updated global model and the local data set;

[0029] Obtaining a second local parameter according to the first local parameter and the weight of the updated global model;

[0030] A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

[0031] In this example, an updated global model is obtained based on the received step size, and then local parameters are calculated.

[0032] In a possible implementation, the first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize a modulus of the stochastic gradient.

[0033] In a possible implementation, the first information further indicates a fifth local parameter, and the method further includes:

[0034] A fifth local parameter is acquired according to the local data set, where the fifth local parameter is used to characterize a sampling noise vector included in a random gradient of the terminal during current model training.

[0035] This makes it easier for the base station to obtain the total power of the gradient noise.

[0036] In a third aspect, an embodiment of the present application provides a communication device. The device includes: a communication module for receiving first information from multiple terminals, the first information indicating a first local parameter, a second local parameter, and a third local parameter, wherein the first local parameter is used to characterize the stochastic gradient of the terminal during the current model training, the second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model corresponding to the current model training, and the third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training; a processing module for obtaining an update step size based on the total power of the gradient noise and the first information from the multiple terminals; the communication module is also used to send the updated global model or the update step size to the multiple terminals respectively, and the updated global model is obtained by processing the global model according to the update step size.

[0037] In one possible implementation, the processing module is used to: obtain a first global parameter, a second global parameter, and a third global parameter based on the first information from multiple terminals; the first global parameter is used to characterize the inner product of the global gradient of the current model training and a first value, the first value being the difference between the weight of the global model and the weight of the preset model; the second global parameter is used to characterize the inner product of the global gradient of the current model training and the global gradient of the previous round of model training; the third global parameter is used to characterize the modulus of the global gradient of the current model training; and obtain an update step size based on the total gradient noise power and the first global parameter, the second global parameter, and the third global parameter.

[0038] In one possible implementation, the processing module is further used to: obtain the global gradient during the current model training based on the first local parameters respectively sent by the multiple terminals; obtain the third global parameter based on the global gradient during the current model training; obtain the first global parameter based on the second local parameters respectively sent by the multiple terminals; and obtain the second global parameter based on the third local parameters respectively sent by the multiple terminals.

[0039] In a possible implementation, the first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize the modulus of the stochastic gradient, and the third global parameter is obtained according to the fourth local parameter from the multiple terminals.

[0040] In a possible implementation, the first information further indicates a fifth local parameter, where the fifth local parameter is used to characterize a sampling noise vector included in the random gradient, and the total gradient noise power is obtained based on the fifth local parameter from multiple terminals.

[0041] In a fourth aspect, an embodiment of the present application provides a communication device. The device includes: a communication module for receiving an updated global model or an update step; a processing module for obtaining a first local parameter, a second local parameter, and a third local parameter based on the updated global model or the update step; wherein the first local parameter is used to characterize the stochastic gradient of the terminal during the current model training, the second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model, and the third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training; the communication module is further used to send first information, wherein the first information indicates the first local parameter, the second local parameter, and the third local parameter.

[0042] In one possible implementation, the processing module is used to: obtain a first local parameter based on the updated global model and the local data set; obtain a second local parameter based on the first local parameter and the weight of the updated global model; and obtain a third local parameter based on the first local parameter and the global gradient of the previous round of model training.

[0043] In one possible implementation, the processing module is used to: obtain an updated global model based on the update step size; obtain a first local parameter based on the updated global model and the local data set; obtain a second local parameter based on the first local parameter and the weight of the updated global model; and obtain a third local parameter based on the first local parameter and the global gradient of the previous round of model training.

[0044] In a possible implementation, the first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize a modulus of the stochastic gradient.

[0045] In one possible implementation, the first information also indicates a fifth local parameter, and the processing module is further used to: obtain the fifth local parameter based on the local data set, and the fifth local parameter is used to characterize the sampling noise vector contained in the random gradient of the terminal during the current model training.

[0046] With respect to the third or fourth aspect above, in one possible implementation, the processing module may be a processor, and the communication module may be a transceiver module, a transceiver, or a communication interface. It is understood that the communication module may be a transceiver in the device, for example, implemented by an antenna, a feeder, and a codec in the device. Alternatively, if the communication device is a chip provided in the device, the communication module may be an input / output interface of the chip, such as an input / output circuit, a pin, etc.

[0047] In a fifth aspect, an embodiment of the present application provides a communication device, which includes one or more processors; wherein the one or more processors are used to execute computer programs stored in one or more memories, so that the communication device implements the method as described in any one of the first aspects, or implements the method as described in any one of the second aspects.

[0048] In a possible implementation manner, the communication device further includes the one or more memories.

[0049] In a possible implementation, the communication device is a chip or a chip system.

[0050] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed by a processor, the method described in any one of the first aspects is implemented, or the method described in any one of the second aspects is implemented.

[0051] In a seventh aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed, implements the method as described in any one of the first aspects, or implements the method as described in any one of the second aspects.

[0052] In an eighth aspect, an embodiment of the present application provides a communication system, comprising an apparatus as described in any one of the third aspects and an apparatus as described in any one of the fourth aspects.

[0053] It is understandable that the apparatus described in the third aspect, the apparatus described in the fourth aspect, the apparatus described in the fifth aspect, the computer storage medium described in the sixth aspect, the computer program product described in the seventh aspect, or the communication system described in the eighth aspect are all used to perform any of the methods provided in the first aspect or any of the methods provided in the second aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The following is an introduction to the drawings used in the embodiments of this application.

[0055] FIG1 is a schematic diagram of an application scenario of a communication system provided in an embodiment of the present application;

[0056] FIG2 is a flow chart of a communication method provided in an embodiment of the present application;

[0057] FIG3 is a flow chart of another communication method provided in an embodiment of the present application;

[0058] FIG4 is a schematic structural diagram of a communication device provided in an embodiment of the present application;

[0059] FIG5 is a schematic diagram of the hardware structure of a communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the embodiments of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0061] The system architecture of the embodiment of the present application will be described in detail below with reference to the accompanying drawings. FIG1 is a schematic diagram of an application scenario of a communication system provided by an embodiment of the present application. The communication system may include a terminal 101 and a network device 102.

[0062] Among them, the terminal 101 can also be called a terminal device, user equipment (UE), mobile station, mobile terminal, etc. The terminal can be widely used in various scenarios, such as device-to-device (D2D), vehicle-to-everything (V2X) communication, machine-type communication (MTC), Internet of Things (IOT), virtual reality, augmented reality, industrial control, autonomous driving, telemedicine, smart grid, smart furniture, smart office, smart wearable, smart transportation, smart city, etc. The terminal can be a mobile phone, tablet computer, computer with wireless transceiver function, wearable device, vehicle, drone, helicopter, airplane, ship, robot, robotic arm, smart home device, etc. The embodiments of this application do not limit the specific technology and specific device form adopted by the terminal.

[0063] The network device 102 can be a base station, an evolved NodeB (eNodeB), a transmission reception point (TRP), a next-generation NodeB (gNB) in a fifth-generation (5G) mobile communication system, a next-generation base station in a sixth-generation (6G) mobile communication system, a base station in a future mobile communication system, or an access node in a WiFi system, etc.; it can also be a module or unit that performs some of the functions of a base station, for example, a centralized unit (CU) or a distributed unit (DU). The network device 102 can be a macro base station, a micro base station or an indoor station, a relay node or a donor node, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the network device 102. For ease of description, the following description uses a base station as an example of a network device.

[0064] Base stations and terminals can be fixed or mobile. They can be deployed on land, indoors or outdoors, handheld or vehicle-mounted; on water; or in the air on aircraft, balloons, and satellites. The embodiments of this application do not limit the application scenarios of base stations and terminals.

[0065] Communication between base stations and terminals, between base stations, and between terminals can be carried out through authorized spectrum, unauthorized spectrum, or both; communication can be carried out through spectrum below 6 gigahertz (GHz), spectrum above 6 GHz, or spectrum below 6 GHz and spectrum above 6 GHz. The embodiments of the present application do not limit the spectrum resources used for wireless communication.

[0066] In the embodiments of the present application, the functions of the base station may also be performed by a module (such as a chip) in the base station, or by a control subsystem that includes the base station functions. The control subsystem that includes the base station functions here may be a control center in the above-mentioned application scenarios such as smart grid, industrial control, smart transportation, and smart city. The functions of the terminal may also be performed by a module (such as a chip or modem) in the terminal, or by a device that includes the terminal functions.

[0067] The above describes the architecture of the embodiment of the present application. The following describes the method of the embodiment of the present application in detail.

[0068] Referring to Figure 2, it is a flow chart of a communication method provided by an embodiment of the present application. Optionally, the method can be applied to the aforementioned communication system, such as the communication system shown in Figure 1. The communication method shown in Figure 2 may include steps 201-207. It should be understood that this application is described in the order of 201-207 for the convenience of description, and is not intended to limit execution to the above order. The embodiment of the present application does not limit the order of execution, execution time, number of executions, etc. of the above one or more steps. The following description is based on the example that the execution subject of steps 201-203 of the communication method is a terminal and the execution subject of steps 204-207 is a base station. This application is also applicable to other execution subjects. Steps 201-207 are as follows:

[0069] 201. The terminal receives a global model sent by a base station.

[0070] It can be understood that the global model is a global model obtained by the base station through, for example, federated learning, etc. The parameters of the global model are finally received, for example, the weights of the global model.

[0071] 202. The terminal obtains a first local parameter, a second local parameter, and a third local parameter according to the global model.

[0072] The first local parameter is used to characterize the stochastic gradient g of the terminal k during the current model training. k (w t ). t is the current training round, w t is the parameter of the global model. For example, the terminal samples the local training data set and then obtains the stochastic gradient based on the global model. Optionally, the stochastic gradient g k (w t ) is the gradient corresponding to the mini-batch. The mini-batch can be understood as a subset of the local dataset.

[0073] The second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model. For example, the second local parameter It can be expressed as:

[0074] The third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training. For example, the third local parameter It can be expressed as: w t-1 are the parameters of the global model in the previous round.

[0075] In a possible implementation manner, the terminal obtains a first local parameter, a second local parameter, a third local parameter, and a fourth local parameter according to the updated global model.

[0076] The fourth local parameter is used to characterize the modulus of the stochastic gradient. For example, the fourth local parameter It can be expressed as:

[0077] In a possible implementation, the terminal further obtains a fifth local parameter based on the updated global model. The fifth local parameter is used to characterize the sampling noise vector included in the stochastic gradient.

[0078] 203. The terminal sends first information to the base station, where the first information indicates the first local parameter, the second local parameter, and the third local parameter.

[0079] After obtaining the above local parameters, the terminal reports them to the base station.

[0080] In a possible implementation manner, the first information further indicates the fourth local parameter.

[0081] In a possible implementation manner, the first information further indicates the fifth local parameter.

[0082] 204. The base station receives first information from multiple terminals.

[0083] 205. The base station obtains an update step size according to the total gradient noise power and the first information from the multiple terminals.

[0084] In one possible implementation, the base station obtains a first global parameter, a second global parameter, and a third global parameter based on first information from multiple terminals, and further obtains an update step size based on the total gradient noise power and the first global parameter, the second global parameter, and the third global parameter.

[0085] Among them, the first global parameter is used to represent the inner product of the global gradient of the current model training and a first value, and the first value is the difference between the weight of the global model and the weight of the preset model.

[0086] The global gradient g(w t ) can be obtained by summing and averaging the first local parameters (i.e., stochastic gradients) sent by multiple terminals. The preset model can be, for example, an optimal model after potential convergence. For example, the value of the optimal model after potential convergence can be preset to 0, etc. Of course, other methods can also be used, and this solution does not limit this.

[0087] For example, the first global parameter It can be expressed as: in, To find the derivative symbol, Represents the global model w t The loss function under * The weight of the preset model. Optional, the first global parameter The second local parameter can be estimated as in, Indicates the first global parameter The estimated value of N T The total number of devices (such as terminals) selected to upload data.

[0088] The second global parameter is used to represent the inner product of the global gradient of the current model training and the global gradient of the previous round of model training. For example, the second global parameter It can be expressed as: Optional, second global parameter The third local parameter can be estimated as in, Indicates the second global parameter estimated value.

[0089] The third global parameter is used to characterize the modulus of the global gradient of the current model training. For example, the third global parameter It can be expressed as: The base station may obtain a corresponding modulus value based on the received random gradient of each device (such as a terminal), thereby further obtaining the third global parameter.

[0090] In one possible implementation, the third global parameter The fourth local parameter can be estimated as

[0091] Among them, the total power of gradient noise It can be expressed as: Where ∑ is the covariance matrix of the gradient noise vector, and Trace is the trace of the matrix. The total gradient noise power can be calculated locally by the base station based on the channel noise power.

[0092] In a possible implementation, the first information further indicates a fifth local parameter, and the fifth local parameter is used to characterize the sampling noise vector included in the stochastic gradient. Then, the base station can obtain the above-mentioned total gradient noise power according to the fifth local parameter and the channel noise power. It should be noted that, since the sampling noise power is much smaller than the channel noise power, the device (such as a terminal) may upload the fifth local parameter or not, and this solution does not impose any restrictions on this.

[0093] In one possible implementation, the base station obtains the global gradient during the current model training based on the first local parameters respectively sent by the multiple terminals. The base station then obtains the third global parameter based on the global gradient during the current model training, obtains the first global parameter based on the second local parameters respectively sent by the multiple terminals, and obtains the second global parameter based on the third local parameters respectively sent by the multiple terminals.

[0094] In another possible implementation, the first information indicates a first local parameter, a second local parameter, a third local parameter, and a fourth local parameter. In this case, the base station calculates the third global parameter based on the fourth local parameter. The first global parameter and the second global parameter are obtained based on the first local parameter, the second local parameter, and the third local parameter respectively transmitted by multiple terminals. For an introduction to this aspect, please refer to the aforementioned description and will not be repeated here.

[0095] The base station obtains an update step size according to the first global parameter, the second global parameter, the third global parameter, and the total power of gradient noise.

[0096] In one possible implementation, β t and It can be solved based on the following method:

[0097] When w t -w t-1 ≠0,

[0098] When w t -w t-1 =0, β t =0.

[0099] Among them, the above Represents the inner product of the gradient used to update the model weights in the previous round and the first value. It can be expressed as: Optional, It can be estimated to be in, express Optionally, before solving the update step size, the base station can solve for this parameter

[0100] Based on the above and βt That is, the update step size is obtained.

[0101] 206. The base station processes the global model according to the update step size to obtain an updated global model.

[0102] In one possible implementation, the neural network model updates the parameter w t+1 The following methods can be used:

[0103] Where η is a constant.

[0104] Based on the update step size, the model parameters are continuously adjusted, and multiple iterative training is performed until the model converges or the training is stopped after the preset number of iterations, to obtain the updated global model.

[0105] 207. The base station sends the updated global model to the multiple terminals respectively.

[0106] It is understood that the global model in step 201 is the global model corresponding to the current model training. The updated global model in step 207 is the global model corresponding to the next round of model training. That is, after step 207, steps 201-207 can be repeated until the model converges or the maximum number of rounds specified in advance is reached.

[0107] In an embodiment of the present application, the terminal side reports the first information so that the base station side calculates the update step size based on the total power of the gradient noise and the local parameters indicated by the first information, thereby obtaining an updated global model. With this solution, the step size is dynamically adjusted according to the gradient noise and the training state indicated by each local parameter, thereby accelerating the gradient descent of the neural network learning, successfully solving the problem of slow neural network training or even non-convergence in the presence of gradient noise. On the other hand, by accelerating the neural network training process, not only can "Green AI" be achieved, but communication overhead can also be saved. Moreover, this solution is compatible with digital transmission, analog transmission, and air aggregation of federated learning.

[0108] On the basis of the foregoing embodiment, as shown in FIG3 , there is a flow chart of another communication method provided by an embodiment of the present application. Optionally, the method can be applied to the aforementioned communication system, such as the communication system shown in FIG1 . The communication method shown in FIG3 may include steps 301-308. It should be understood that this application is described in the order of 301-308 for the convenience of description, and is not intended to limit execution to the above order. The embodiment of the present application does not limit the order of execution, execution time, number of executions, etc. of the above one or more steps. The following description is based on the example that the execution subject of steps 301-303 of the communication method is a terminal, and the execution subject of steps 305-308 is a base station. This application is also applicable to other execution subjects. Steps 301-308 are as follows:

[0109] 301. The terminal receives a step size sent by the base station.

[0110] In this example, the base station directly sends the step size to each terminal, and the terminal calculates the updated global model.

[0111] It will be appreciated that, alternatively, the terminal may receive second information sent by the base station, where the second information indicates the first global parameter, the second global parameter, the third global parameter, and the total gradient noise power. Furthermore, the terminal may determine the step size based on the second information. For an introduction to the first global parameter, the second global parameter, the third global parameter, and the total gradient noise power, please refer to the description of the embodiment shown in FIG2 , and will not be repeated here.

[0112] 302. The terminal obtains a global model according to the step size.

[0113] The terminal adjusts the parameters of the global model based on the above step size to obtain an updated global model.

[0114] 303. The terminal obtains a first local parameter, a second local parameter, and a third local parameter according to the global model.

[0115] The first local parameter is used to represent the stochastic gradient of the terminal during the current model training. The second local parameter is used to represent the inner product of the stochastic gradient and the weight of the global model. The third local parameter is used to represent the inner product of the stochastic gradient and the global gradient of the previous round of model training.

[0116] For the introduction of this part, please refer to the description of step 202 in the embodiment shown in FIG. 2 , which will not be repeated here.

[0117] 304. The terminal sends first information to the base station, where the first information indicates the first local parameter, the second local parameter, and the third local parameter.

[0118] For the introduction of this part, please refer to the description of step 203 in the embodiment shown in FIG. 2 , which will not be repeated here.

[0119] In a possible implementation, the first information further indicates a fourth local parameter. The fourth local parameter is used to characterize the modulus of the stochastic gradient.

[0120] In a possible implementation manner, the first information further indicates a fifth local parameter, where the fifth local parameter is used to characterize a sampling noise vector included in the stochastic gradient.

[0121] 305. The base station receives first information from multiple terminals.

[0122] 306. The base station obtains a first global parameter, a second global parameter, and a third global parameter according to the first information from multiple terminals.

[0123] The first global parameter is used to represent the inner product of the global gradient of the current model training and a first value, where the first value is the difference between the weight of the global model and the weight of the preset model. The second global parameter is used to represent the inner product of the global gradient of the current model training and the global gradient of the previous round of model training. The third global parameter is used to represent the modulus of the global gradient of the current model training.

[0124] For the introduction of this part, please refer to the description of step 205 in the embodiment shown in FIG. 2 , which will not be repeated here.

[0125] 307. The base station obtains an update step size according to the first global parameter, the second global parameter, the third global parameter, and the total gradient noise power.

[0126] For the introduction of this part, please refer to the description of step 206 in the embodiment shown in FIG. 2 , which will not be repeated here.

[0127] 308. The base station sends the update step size to each of the multiple terminals.

[0128] It should be noted that when step 301 is the terminal receiving the current first global parameter, second global parameter, third global parameter and total gradient noise power sent by the base station, accordingly, the first global parameter, second global parameter, third global parameter and total gradient noise power of the next round can be directly sent after step 306.

[0129] In an embodiment of the present application, the terminal side obtains a global model based on the step size sent by the base station, and then reports the first information so that the base station side calculates the global parameters based on the local parameters indicated by the first information. The global parameters indicate the training state information, and then the update step size is obtained based on the global parameters and the total power of the gradient noise. With this solution, the step size is dynamically adjusted according to the gradient noise and the training state, thereby accelerating the gradient descent of the neural network learning, successfully solving the problem of slow neural network training or even non-convergence in the presence of gradient noise. On the other hand, by accelerating the neural network training process, not only can "Green AI" be achieved, but also communication overhead can be saved. Moreover, this solution is compatible with digital transmission, analog transmission, and air aggregation of federated learning.

[0130] It should be noted that the above embodiments illustrate the example of a terminal uploading the first, second, and third local parameters. Alternatively, the terminal may upload a stochastic gradient parameter, etc., so that the base station can calculate the first, second, and third local parameters corresponding to each terminal based on the uploaded parameters, and further calculate the update step size. This solution does not impose any limitation on this.

[0131] It should be noted that, in the various embodiments of this application, unless otherwise specified or logically conflicting, the terms and / or descriptions between the various embodiments are consistent and can be referenced from each other. The technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. This solution does not impose any restrictions on this.

[0132] The above describes in detail the method of the embodiment of the present application, and the following provides the device of the embodiment of the present application. It will be understood that in the various device embodiments of the present application, the division of multiple units or modules is only a logical division based on function, and is not intended to limit the specific structure of the device. In a specific implementation, some functional modules may be subdivided into more small functional modules, and some functional modules may be combined into one functional module, but no matter whether these functional modules are subdivided or combined, the general process performed by the device is the same. For example, some devices include a receiving unit and a sending unit. In some designs, the sending unit and the receiving unit can also be integrated into a communication unit, which can implement the functions implemented by the receiving unit and the sending unit. Typically, each unit corresponds to its own program code (or program instructions), and when the program code corresponding to each of these units runs on the processor, the unit is controlled by the processing unit to execute the corresponding process to implement the corresponding function.

[0133] The embodiments of the present application also provide an apparatus for implementing any of the above methods. For example, a communication apparatus is provided that includes a module (or means) for implementing each step performed by a terminal in any of the above methods. For another example, another communication apparatus is provided that includes a module (or means) for implementing each step performed by a base station in any of the above methods.

[0134] For example, referring to Figure 4, which is a schematic diagram of the structure of a communication device provided in an embodiment of the present application, the communication device is used to implement the aforementioned communication method, such as the steps performed by the base station in the communication method shown in Figures 2 and 3.

[0135] As shown in FIG4 , the apparatus may include a communication module 401 and a processing module 402 , specifically as follows:

[0136] Communication module 401 is configured to receive first information from multiple terminals, where the first information indicates a first local parameter, a second local parameter, and a third local parameter, wherein the first local parameter is used to represent a stochastic gradient of the terminal during current model training, the second local parameter is used to represent an inner product of the stochastic gradient and a weight of a global model corresponding to the current model training, and the third local parameter is used to represent an inner product of the stochastic gradient and a global gradient from a previous round of model training;

[0137] A processing module 402 is configured to obtain an update step size according to the total power of the gradient noise and the first information from the multiple terminals;

[0138] The communication module 401 is further configured to send the updated global model or the update step to each of the multiple terminals. The updated global model is obtained by processing the global model according to the update step.

[0139] In a possible implementation, the processing module 402 is configured to:

[0140] A first global parameter, a second global parameter, and a third global parameter are obtained according to the first information from the multiple terminals; the first global parameter is used to represent the inner product of the global gradient of the current model training and a first value, the first value being the difference between the weight of the global model and the weight of the preset model; the second global parameter is used to represent the inner product of the global gradient of the current model training and the global gradient of the previous round of model training; and the third global parameter is used to represent the modulus of the global gradient of the current model training;

[0141] An update step size is obtained according to the total gradient noise power and the first global parameter, the second global parameter, and the third global parameter.

[0142] In a possible implementation, the first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize the modulus of the stochastic gradient, and the third global parameter is obtained according to the fourth local parameter from the multiple terminals.

[0143] In a possible implementation, the processing module 402 is further configured to:

[0144] Obtaining a global gradient during current model training according to the first local parameters respectively sent by the multiple terminals;

[0145] Obtaining the third global parameter according to the global gradient during the current model training;

[0146] Obtaining the first global parameter according to the second local parameters respectively sent by the multiple terminals;

[0147] The second global parameter is obtained according to the third local parameters respectively sent by the multiple terminals.

[0148] In a possible implementation, the first information further indicates a fifth local parameter, where the fifth local parameter is used to characterize a sampling noise vector included in the random gradient, and the total gradient noise power is obtained based on the fifth local parameter from multiple terminals.

[0149] For the introduction of the above modules, please refer to the description of the above embodiments, which will not be repeated here.

[0150] For example, referring to Figure 4 , an embodiment of the present application further provides a communication device for implementing the aforementioned communication method, such as the steps executed by the terminal in the communication method shown in Figures 2 and 3 .

[0151] As shown in FIG4 , the apparatus may include a communication module 401 and a processing module 402 , specifically as follows:

[0152] Communication module 401, used to receive the updated global model or update step size;

[0153] Processing module 402 is configured to obtain a first local parameter, a second local parameter, and a third local parameter based on the updated global model or the update step size; wherein the first local parameter is used to represent the stochastic gradient of the terminal during the current model training, the second local parameter is used to represent the inner product of the stochastic gradient and the weight of the global model, and the third local parameter is used to represent the inner product of the stochastic gradient and the global gradient of the previous round of model training;

[0154] The communication module 401 is further configured to send first information, where the first information indicates the first local parameter, the second local parameter, and the third local parameter.

[0155] In a possible implementation, the processing module 402 is configured to:

[0156] Obtaining a first local parameter according to the updated global model and the local data set;

[0157] Obtaining a second local parameter according to the first local parameter and the weight of the updated global model;

[0158] A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

[0159] In a possible implementation, the first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize the modulus of the stochastic gradient, and the third global parameter is obtained according to the fourth local parameter from the multiple terminals.

[0160] In a possible implementation, the processing module 402 is further configured to:

[0161] Obtaining an updated global model according to the update step size;

[0162] Obtaining a first local parameter according to the updated global model and the local data set;

[0163] Obtaining a second local parameter according to the first local parameter and the weight of the updated global model;

[0164] A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

[0165] In a possible implementation, the first information further indicates a fifth local parameter, and the processing module 402 is further configured to:

[0166] A fifth local parameter is acquired according to the local data set, where the fifth local parameter is used to characterize a sampling noise vector included in a random gradient of the terminal during current model training.

[0167] For the introduction of the above modules, please refer to the description of the above embodiments, which will not be repeated here.

[0168] It should be understood that the division of the modules in the above-mentioned devices is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. In addition, the modules in the communication device may be implemented in the form of a processor calling software; for example, the communication device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or the functions of the modules of the device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the modules in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units by designing the logical relationship of the components in the circuit. For another example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units. All modules of the above devices can be implemented in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0169] 5 is a schematic diagram of the hardware structure of another communication device provided in an embodiment of the present application. The communication device 500 shown in FIG5 (the device 500 may be a computer device) includes a memory 501, a processor 502, a communication interface 503, and a bus 504. The memory 501, the processor 502, and the communication interface 503 are connected to each other via the bus 504.

[0170] The memory 501 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0171] The memory 501 can store programs. When the program stored in the memory 501 is executed by the processor 502, the processor 502 and the communication interface 503 are used to perform the various steps of the communication method of the embodiment of the present application.

[0172] The processor 502 is a circuit with signal processing capabilities. In one implementation, the processor 502 can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor 502 can implement certain functions through the logical relationship of a hardware circuit. The logical relationship of the hardware circuit is fixed or reconfigurable. For example, the processor 502 is a hardware circuit implemented by an ASIC or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration file and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. The processor 502 is used to execute relevant programs to implement the functions required to be performed by the units in the communication device of the embodiment of the present application, or to execute the communication method of the method embodiment of the present application.

[0173] It can be seen that each module in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0174] In addition, the modules in the above device can be fully or partially integrated together, or can be implemented independently. In one implementation, these modules are integrated together and implemented in the form of a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the modules of the device. The type of the at least one processor can be different, for example, including a CPU and FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0175] The communication interface 503 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the apparatus 500 and other devices or a communication network. For example, data can be obtained through the communication interface 503 .

[0176] The bus 504 may include a path for transmitting information between various components of the device 500 (eg, the memory 501 , the processor 502 , and the communication interface 503 ).

[0177] It should be noted that although the device 500 shown in FIG5 only shows a memory, a processor, and a communication interface, during the specific implementation process, those skilled in the art will understand that the device 500 also includes other components necessary for normal operation. At the same time, according to specific needs, those skilled in the art will understand that the device 500 may also include hardware components that implement other additional functions. In addition, those skilled in the art will understand that the device 500 may also include only the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in FIG5.

[0178] An embodiment of the present application also provides a communication device, which includes one or more processors; wherein the one or more processors are used to execute computer programs stored in one or more memories, so that the communication device implements the method as described in any one of the first aspects, or implements the method as described in any one of the second aspects.

[0179] In a possible implementation manner, the communication device further includes the one or more memories.

[0180] In a possible implementation, the communication device is a chip or a chip system.

[0181] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or processor executes one or more steps in any of the above methods.

[0182] The present application also provides a computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute one or more steps in any of the above methods.

[0183] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the specific descriptions of the corresponding steps in the aforementioned method embodiments and will not be repeated here.

[0184] It should be understood that in the description of this application, unless otherwise specified, " / " indicates that the objects associated with each other are in an "or" relationship. For example, A / B can mean A or B; where A and B can be singular or plural. Also, in the description of this application, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural. In addition, to facilitate the clear description of the technical solutions of the embodiments of this application, in the embodiments of this application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or execution order, and words such as "first" and "second" do not necessarily mean different. At the same time, in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner to facilitate understanding.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling, direct coupling, or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms.

[0186] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic medium such as a floppy disk, a hard disk, a tape, a magnetic disk, or an optical medium such as a digital versatile disc (DVD), or a semiconductor medium such as a solid state disk (SSD).

[0188] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A communication method, characterized in that: include: Receive first information from multiple terminals, where the first information indicates a first local parameter, a second local parameter, and a third local parameter, wherein the first local parameter is used to characterize a stochastic gradient of a terminal during current model training, the second local parameter is used to characterize an inner product of the stochastic gradient and a weight of a global model corresponding to the current model training, and the third local parameter is used to characterize an inner product of the stochastic gradient and a global gradient of a previous round of model training; Obtaining an update step size according to the total power of the gradient noise and the first information from the plurality of terminals; The updated global model or the update step is sent to the multiple terminals respectively, where the updated global model is obtained by processing the global model according to the update step.

2. The method according to claim 1, characterized in that The step of obtaining an update step size according to the total power of the gradient noise and the first information from the plurality of terminals comprises: A first global parameter, a second global parameter, and a third global parameter are obtained according to the first information from the multiple terminals; the first global parameter is used to characterize the inner product of the global gradient of the current model training and a first value, the first value is the difference between the weight of the global model and the weight of the preset model, the second global parameter is used to characterize the inner product of the global gradient of the current model training and the global gradient of the previous round of model training, and the third global parameter is used to characterize the modulus of the global gradient of the current model training; An update step size is obtained according to the total gradient noise power and the first global parameter, the second global parameter, and the third global parameter.

3. The method according to claim 2, characterized in that The first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize the modulus of the stochastic gradient, and the third global parameter is obtained according to the fourth local parameter from the multiple terminals.

4. The method according to claim 2, characterized in that: The obtaining the first global parameter, the second global parameter and the third global parameter according to the first information from the multiple terminals includes: Obtaining a global gradient during the current model training according to the first local parameters respectively sent by the multiple terminals; Obtaining the third global parameter according to the global gradient during the current model training; Obtaining the first global parameter according to the second local parameters respectively sent by the multiple terminals; The second global parameter is obtained according to the third local parameters respectively sent by the multiple terminals.

5. The method according to any one of claims 1 to 4, characterized in that: The first information further indicates a fifth local parameter, where the fifth local parameter is used to characterize a sampling noise vector included in the random gradient, and the total gradient noise power is obtained according to the fifth local parameter from multiple terminals.

6. A communication method, characterized in that: include: Receive an updated global model or update step size; Obtaining a first local parameter, a second local parameter, and a third local parameter according to the updated global model or the update step size; wherein the first local parameter is used to characterize the stochastic gradient of the terminal in the current model training, the second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model, and the third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training; First information is sent, where the first information indicates the first local parameter, the second local parameter, and the third local parameter.

7. The method according to claim 6, characterized in that The obtaining of the first local parameter, the second local parameter and the third local parameter according to the updated global model comprises: Obtaining a first local parameter according to the updated global model and the local data set; Obtaining a second local parameter according to the first local parameter and the weight of the updated global model; A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

8. The method according to claim 6, characterized in that The obtaining of the first local parameter, the second local parameter and the third local parameter according to the update step size comprises: Obtaining an updated global model according to the update step size; Obtaining a first local parameter according to the updated global model and the local data set; Obtaining a second local parameter according to the first local parameter and the weight of the updated global model; A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

9. The method according to any one of claims 6 to 8, characterized in that: The first information further indicates a fourth local parameter, and the fourth local parameter is used to characterize the modulus value of the stochastic gradient.

10. The method according to any one of claims 6 to 9, characterized in that: The first information further indicates a fifth local parameter, and the method further includes: A fifth local parameter is acquired according to the local data set, and the fifth local parameter is used to characterize a sampling noise vector included in the random gradient of the terminal during the current model training.

11. A communication device, characterized in that: include: A communication module, configured to receive first information from a plurality of terminals, wherein the first information indicates a first local parameter, a second local parameter, and a third local parameter, wherein the first local parameter is used to characterize a stochastic gradient of a terminal during current model training, the second local parameter is used to characterize an inner product of the stochastic gradient and a weight of a global model corresponding to the current model training, and the third local parameter is used to characterize an inner product of the stochastic gradient and a global gradient of a previous round of model training; A processing module, configured to obtain an update step size according to the total power of the gradient noise and the first information from the plurality of terminals; The communication module is further used to send the updated global model or the update step to the multiple terminals respectively, and the updated global model is obtained by processing the global model according to the update step.

12. The device according to claim 11, characterized in that The processing module is used for: A first global parameter, a second global parameter, and a third global parameter are obtained according to the first information from the multiple terminals; the first global parameter is used to characterize the inner product of the global gradient of the current model training and a first value, the first value is the difference between the weight of the global model and the weight of the preset model, the second global parameter is used to characterize the inner product of the global gradient of the current model training and the global gradient of the previous round of model training, and the third global parameter is used to characterize the modulus of the global gradient of the current model training; An update step size is obtained according to the total gradient noise power and the first global parameter, the second global parameter, and the third global parameter.

13. The device according to claim 12, characterized in that The first information further indicates a fourth local parameter, where the fourth local parameter is used to characterize the modulus of the stochastic gradient, and the third global parameter is obtained according to the fourth local parameter from the multiple terminals.

14. The device according to claim 12, characterized in that The processing module is further used for: Obtaining a global gradient during the current model training according to the first local parameters respectively sent by the multiple terminals; Obtaining the third global parameter according to the global gradient during the current model training; Obtaining the first global parameter according to the second local parameters respectively sent by the multiple terminals; The second global parameter is obtained according to the third local parameters respectively sent by the multiple terminals.

15. The device according to any one of claims 11 to 14, characterized in that The first information further indicates a fifth local parameter, where the fifth local parameter is used to characterize a sampling noise vector included in the random gradient, and the total gradient noise power is obtained according to the fifth local parameter from multiple terminals.

16. A communication device, characterized in that: include: A communication module, used to receive an updated global model or an update step size; A processing module, configured to obtain a first local parameter, a second local parameter, and a third local parameter according to the updated global model or the update step size; wherein the first local parameter is used to characterize the stochastic gradient of the terminal in the current model training, the second local parameter is used to characterize the inner product of the stochastic gradient and the weight of the global model, and the third local parameter is used to characterize the inner product of the stochastic gradient and the global gradient of the previous round of model training; The communication module is further used to send first information, where the first information indicates the first local parameter, the second local parameter and the third local parameter.

17. The device according to claim 16, characterized in that The processing module is used for: Obtaining a first local parameter according to the updated global model and the local data set; Obtaining a second local parameter according to the first local parameter and the weight of the updated global model; A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

18. The device according to claim 16, characterized in that The processing module is used for: Obtaining an updated global model according to the update step size; Obtaining a first local parameter according to the updated global model and the local data set; Obtaining a second local parameter according to the first local parameter and the weight of the updated global model; A third local parameter is obtained according to the first local parameter and the global gradient of the previous round of model training.

19. The device according to any one of claims 16 to 18, characterized in that The first information further indicates a fourth local parameter, and the fourth local parameter is used to characterize the modulus value of the stochastic gradient.

20. The device according to any one of claims 16 to 19, characterized in that The first information further indicates a fifth local parameter, and the processing module is further configured to: A fifth local parameter is acquired according to the local data set, and the fifth local parameter is used to characterize a sampling noise vector included in the random gradient of the terminal during the current model training.

21. A communication device, characterized in that: The communication device comprises one or more processors; wherein the one or more processors are used to execute computer programs stored in one or more memories, so that the communication device implements the method according to any one of claims 1 to 5, or implements the method according to any one of claims 6 to 10.

22. The communication device according to claim 21, characterized in that The communication device also includes the one or more memories.

23. The communication device according to claim 21 or 22, characterized in that: The communication device is a chip or a chip system.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented, or the method according to any one of claims 6 to 10 is implemented.

25. A computer program product, characterized in that The method comprises a computer program, which, when executed, implements the method according to any one of claims 1 to 5, or implements the method according to any one of claims 6 to 10.

26. A communication system, characterized in that: It comprises the device as described in any one of claims 11 to 15, and the device as described in any one of claims 16 to 20.

Images