Model training method and apparatus, communication device, storage medium, and program product

Abstract

The present application relates to a model training method and apparatus, a communication device, a storage medium, and a program product. The method comprises: receiving training report data sent by data holders, wherein the training report data comprises at least the number of actual trainings completed by iteratively training a local model by the data holders in the current communication round; on the basis of the training report data, determining the number of target trainings of the data holders in the next communication round; and sending the number of target trainings in the next communication round to the data holders.

Description

Model training method, device, communication equipment, storage medium and program product

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 2023108683053, filed on July 14, 2023, entitled “Model training method, device, communication equipment, storage medium and program product,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, communication equipment, storage medium, and program product. Background Art

[0004] Federated Learning (FL) is a type of machine learning that allows data holders to combine their data to build a shared machine learning model, namely a federated model, without sharing private data.

[0005] Currently, during the training process of the federated model, the central server sends the initialized federated model to each data holder, and each data holder trains the federated model locally according to the same and fixed number of training tasks.

[0006] However, the above model training method has the problems of low model training efficiency and low model accuracy.

[0007] Summary of the Invention

[0008] In a first aspect, an embodiment of the present application provides a model training method. The method is applied to a server and includes:

[0009] Receive training report data sent by the data holder, where the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0010] Determining a target number of training times for the data holder in the next communication round based on the training reported data;

[0011] The target number of training times in the next communication round is sent to the data holder.

[0012] In one embodiment, the training report data further includes a loss value of the local model of the data holder in the current communication round; and determining a target number of training times for the data holder in the next communication round based on the training report data includes:

[0013] Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0014] Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in the previous communication round;

[0015] The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result, and the model loss change value.

[0016] In one embodiment, obtaining the target number of training times in the next communication round according to the actual number of training times, the comparison result, and the model loss change value includes:

[0017] If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change in the model loss;

[0018] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0019] In one embodiment, the loss value is a cross entropy loss value.

[0020] In one embodiment, the method further comprises:

[0021] Sending the time length corresponding to the next communication round to the data holder;

[0022] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0023] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method further includes:

[0024] Update the parameters of the global model according to the model parameters to obtain updated model parameters;

[0025] The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and the target number of training times in the next communication round.

[0026] In one embodiment, there are multiple data holders, and updating the global model parameters according to the model parameters to obtain updated model parameters includes:

[0027] Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0028] In one embodiment, before receiving the training report data sent by the data holder, the method further includes:

[0029] The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0030] In a second aspect, embodiments of the present application further provide a model training method. The method is applied to a data holder and includes:

[0031] Sending training report data to the server, where the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0032] receiving a target training number for the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0033] In the next communication round, the local model is iteratively trained according to a target number of training times in the next communication round.

[0034] In one embodiment, before iteratively training the local model according to the target number of training times in the next communication round, the method further includes:

[0035] receiving a time length corresponding to the next communication round sent by the server;

[0036] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0037] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method further includes:

[0038] receiving updated model parameters sent by the server, where the updated model parameters are obtained by the server updating the parameters of the global model according to the model parameters;

[0039] The iteratively training the local model according to the target number of training times in the next communication round includes:

[0040] The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

[0041] In one embodiment, before sending the training report data to the server, the method further includes:

[0042] Receiving the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder sent by the server;

[0043] In the initial communication round, the initialized model is used as a local model, and the local model is iteratively trained according to the initial training number of times.

[0044] In one embodiment, the training reporting data also includes the loss value of the local model of the data holder in the current communication round.

[0045] In a third aspect, an embodiment of the present application further provides a model training device. The device is provided on a server and includes:

[0046] A receiving module, configured to receive training report data sent by a data holder, wherein the training report data includes at least an actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0047] a determination module, configured to determine a target number of training times for the data holder in the next communication round based on the training reporting data;

[0048] A sending module is used to send the target number of training times in the next communication round to the data holder.

[0049] In a fourth aspect, an embodiment of the present application further provides a model training device. The device is provided at the data holder and includes:

[0050] A sending module, configured to send training report data to a server, wherein the training report data includes at least an actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0051] a receiving module, configured to receive a target number of training times for the data holder in a next communication round sent by the server, where the target number of training times in the next communication round is determined by the server according to the training reporting data;

[0052] The training module is used to iteratively train the local model according to the target number of training times in the next communication round.

[0053] In a fifth aspect, an embodiment of the present application further provides a communication device. The communication device includes: a transmitter, a processor, and a receiver;

[0054] The receiver is configured to receive training report data sent by the data holder, where the training report data includes at least an actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0055] The processor is configured to determine a target number of training times for the data holder in a next communication round based on the training reporting data;

[0056] The transmitter is configured to send the target number of training times in the next communication round to the data holder.

[0057] In a sixth aspect, an embodiment of the present application further provides a communication device. The communication device includes: a transmitter, a processor, and a receiver;

[0058] The transmitter is configured to send training report data to the server, wherein the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0059] The receiver is configured to receive a target number of training times for the data holder in a next communication round sent by the server, where the target number of training times in the next communication round is determined by the server according to the training reporting data;

[0060] The processor is configured to iteratively train the local model in the next communication round according to a target number of training times in the next communication round.

[0061] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first or second aspect above.

[0062] In an eighth aspect, an embodiment of the present application provides a chip, wherein the chip includes a programmable logic circuit and / or program instructions, and when the chip is executed, the steps of the method described in the first aspect or the second aspect are implemented.

[0063] In a ninth aspect, an embodiment of the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first or second aspect are implemented.

[0064] The details of one or more embodiments of the present application are set forth in the following drawings and description. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0066] FIG1 is a diagram illustrating an application environment of a model training method according to an embodiment;

[0067] FIG2 is a schematic diagram of a flow chart of a model training method in one embodiment;

[0068] Figure 3 is a schematic diagram of data interaction during an exemplary model training process

[0069] FIG4 is a schematic diagram of a flow chart of step 202 in one embodiment;

[0070] FIG5 is a schematic diagram of a flow chart of a model training method in another embodiment;

[0071] FIG6 is a schematic diagram of a flow chart of a model training method in another embodiment;

[0072] FIG7 is a schematic diagram of a flow chart of a model training method in another embodiment;

[0073] FIG8 is a schematic diagram of a flow chart of a model training method in another embodiment;

[0074] FIG9 is a schematic diagram of a flow chart of a model training method in another embodiment;

[0075] FIG10 is a block diagram of a model training device according to an embodiment;

[0076] FIG11 is a block diagram of a model training device according to another embodiment;

[0077] FIG12 is a schematic diagram of the internal structure of a communication device according to an embodiment;

[0078] FIG13 is a schematic diagram of the structure of a chip in one embodiment. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0080] Figure 1 is a schematic diagram of an implementation environment for a model training method provided in an embodiment of the present application. As shown in Figure 1, the implementation environment includes a server 102 and multiple data holders (Figure 1 only shows two data holders: data holder 104 and data holder 106). Data holders, also known as data holding nodes, can be servers that hold sample data required for model training.

[0081] In which, data is transmitted between the server 102 and the data holder through the network; the server 102 can be a single server or a server cluster composed of multiple servers; the data holder can be a single server or a server cluster composed of multiple servers.

[0082] Among them, server 102 can be an edge server, and of course it can also be any other server in the model training process, and no specific limitation is made here.

[0083] Take federated learning (FL) as an example. This is a type of machine learning that allows data holders to combine their data to build a shared machine learning model, known as a federated model, without sharing private data. In federated learning, each data holder (also known as an edge device) is typically responsible for model training, while the server aggregates the model to create a global model.

[0084] Currently, during federated model training, a central server distributes the initialized federated model to each data holder. Each data holder then uses the federated model as a local model and trains the local model locally according to the same fixed number of tasks until the model converges. However, because different data holders participating in local model training often have different training conditions, such as varying amounts of computing resources, sample data volumes, and network conditions, the number of tasks that different data holders can complete within a given timeframe varies significantly. Traditional techniques assign the same fixed number of tasks to different data holders. Consequently, data holders with poor training conditions may not be able to complete their training tasks in a timely manner, while data holders with better training conditions may not fully utilize their system resources. This results in inefficient model training and low model accuracy.

[0085] In view of this, an embodiment of the present application provides a model training method that can improve model training efficiency and increase model accuracy.

[0086] It should be noted that the beneficial effects or technical problems solved by the embodiments of the present application are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0087] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0088] In one embodiment, as shown in FIG2 , a model training method is provided, which is described by taking the method applied to the server 102 in FIG1 as an example, and includes the following steps:

[0089] Step 201: The server receives training report data sent by the data holder.

[0090] In the embodiment of the present application, the server acts as the "central server" in the federated model training process, and is responsible for issuing training tasks, aggregating and updating federated model parameters, etc. Data holders can act as edge devices to perform edge computing tasks such as local model training. Edge computing (EC) refers to a computing method that deploys workloads at the edge, aiming to migrate data processing to servers closer to users, effectively alleviating network congestion.

[0091] As an implementation method, the embodiment of the present application can train the federated model in communication rounds, that is, the server can set up communication rounds, one communication round corresponds to a time period, and assign a target number of training times corresponding to the communication round to each data holder, wherein the target number of training times refers to the number of training tasks corresponding to the communication round, that is, the task volume of the training times of the communication round, and the target number of training times represents the target number of training times for the data holder to iteratively train the local model within the communication round.

[0092] For the first training process, the server sends the parameter-initialized federated model to each data holder. The data holder uses the parameter-initialized federated model as the local model and iteratively trains the local model according to the communication rounds and number of tasks set by the server until the communication round ends.

[0093] It is understandable that the data holder may be able to complete the task within the communication round, that is, the number of training times actually completed by the data holder within the communication round can reach or exceed the set target number of training times. Of course, it may also be possible to fail to complete the task in time.

[0094] After the communication round ends, the data holder sends training report data to the server. The training report data includes at least the actual number of training times completed by the data holder during the iterative training of the local model in the current communication round. If the data holder completes the number of training times set by the server after the communication round ends, the actual number of training times is the target number of training times set by the server for the data holder in the current communication round. If the data holder does not complete the number of training times set by the server, the actual number of training times is the number of iterative training times actually completed by the data holder in the current communication round.

[0095] In an embodiment of the present application, the training reporting data also includes the model parameters obtained by the data holder after iteratively training the local model in the current communication round. The server updates the parameters of the global model based on the model parameters reported by each data holder after the end of the current communication round to obtain updated model parameters.

[0096] Then, during the non-initial training process, the server reassigns each data holder the target number of training cycles for the next communication round and sends the updated model parameters to each data holder. The data holder then continues to iteratively train the local model based on the target number of training cycles assigned by the server and the updated model parameters in the next time period.

[0097] After the next communication round, the data holder continues to send training report data to the server, and so on, until the local model converges.

[0098] It should be noted that in step 201, the server receives the training report data sent by the data holder. The training report data can be the training report data sent by the data holder to the server after any communication round is completed during the local model training process.

[0099] Step 202: The server determines the target number of training times for the data holder in the next communication round based on the training report data.

[0100] The actual number of training times completed by the data holder in iteratively training the local model in the current communication round can reflect the training conditions of the data holder, such as the storage space, computing power, network conditions, etc. of the data holder. It can be understood that the larger the value of the actual number of training times, the better the training conditions of the data holder, and the smaller the value of the actual number of training times, the worse the training conditions of the data holder.

[0101] In an embodiment of the present application, the server can set the target number of training times for the data holder in the next communication round based on the actual number of training times completed by the data holder in the current communication round, that is, the server sets the target number of training times for the next communication round based on the training conditions of the data holder reflected in the current communication round.

[0102] As an implementation method, if the data holder completes fewer rounds of training in the current communication round, it means that its training conditions are poor, and its target training times are reduced accordingly in the next communication round to ensure that it successfully completes the training task. If the data holder completes more rounds of training in the current communication round, it means that its training conditions are good, and its target training times are increased accordingly in the next communication round to fully utilize the system resources of the data holder.

[0103] In a possible implementation of step 202, the server may compare the actual number of training sessions with the target number of training sessions set for the current communication round. If the actual number of training sessions is greater than or equal to the target number of training sessions for the current communication round, the server increases the actual number of training sessions to obtain the target number of training sessions for the next communication round. If the actual number of training sessions is less than the target number of training sessions for the current communication round, the server decreases the actual number of training sessions to obtain the target number of training sessions for the next communication round. Alternatively, the server may directly use the actual number of training sessions as the target number of training sessions for the next communication round, and so on.

[0104] Other possible implementations of step 202 will be described in the following examples.

[0105] Step 203: The server sends the target number of training times in the next communication round to the data holder.

[0106] The target number of training times in the next communication round is the target number of training times that the data holder iteratively trains the local model in the next communication round.

[0107] In this way, after the data holder receives the target number of training times of the data holder in the next communication round sent by the server, the data holder iteratively trains the local model according to the target number of training times in the next communication round.

[0108] After the next communication round ends, the data holder continues to send training report data to the server, and so on, until the local model converges.

[0109] In order to more clearly understand the data interaction procedures between the server and the data holder in the embodiment of the present application, for example, refer to Figure 3, which is a schematic diagram of data interaction in an exemplary model training process.

[0110] After the current communication round of training is completed, the data holder uploads the training report data to the server. The training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round. The server receives the training report data and then adjusts the target number of training times of the data holder in the next communication round based on the training report data. The server sends the target number of training times in the next communication round to the data holder.

[0111] The above embodiment receives training report data sent by the data holder, which includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round. Then, based on the training report data, the target number of training times for the data holder in the next communication round is determined, and the target number of training times in the next communication round is sent to the data holder. In this way, since different data holders participating in local model training often have different training conditions, such as different amounts of computing resources, sample data volumes, network conditions, etc., there are large differences in the target number of training times that different data holders can complete within a given time. Related technologies assign the same and fixed number of tasks to different data holders. In this case, data holders with poor training conditions may not be able to complete the training tasks in time, and data holders with good training conditions may have the problem of underutilization of system resources, which leads to low model training efficiency and low model accuracy. In an embodiment of the present application, the target number of training times of the data holder in the next communication round is dynamically determined based on the training reported data actually completed by the data holder in the previous training time period (i.e., the current communication round). For example, if the data holder has completed fewer rounds of training in the current communication round, indicating that its training conditions are poor, then its target number of training times will be reduced accordingly in the next communication round to ensure that it successfully completes the training task. If the data holder has completed more rounds of training in the current communication round, indicating that its training conditions are good, then its target number of training times will be increased accordingly in the next communication round to fully utilize the system resources of the data holder, thereby improving the model training efficiency and model accuracy.

[0112] In one embodiment, based on the embodiment shown in FIG2 , referring to FIG4 , the training report data in this embodiment also includes the loss value of the data holder's local model in the current communication round. This embodiment relates to how the server determines the target number of training times for the data holder in the next communication round based on the training report data. As shown in FIG4 , step 202 includes steps 301, 302, and 303 shown in FIG4 :

[0113] Step 301: The server compares the actual number of training times with the target number of training times in the current communication round to obtain a comparison result.

[0114] Step 302: The server determines the model loss change value obtained by the data holder after the current communication round ends.

[0115] As described above, the target number of training times in the current communication round is the target number of training times for the data holder to iteratively train the local model in the current communication round. When the server determines the target number of training times for the data holder in the next communication round, it first compares the actual number of training times with the target number of training times in the current communication round to obtain a comparison result. The comparison result may include: the actual number of training times is less than the target number of training times in the current communication round, or the actual number of training times is greater than or equal to the target number of training times in the current communication round.

[0116] It can be understood that if the actual number of training times is less than the target number of training times in the current communication round, it means that the data holder has not completed the training task assigned by the server in the current time period and has fallen behind. Therefore, the server will reduce the target number of training times for the data holder in the next communication round, that is, it will reduce the training task of the data holder in the next communication round; and if the actual number of training times is greater than or equal to the target number of training times in the current communication round, it means that the data holder has completed the training task assigned by the server in the current time period. Therefore, the server will increase the target number of training times for the data holder in the next communication round.

[0117] Next, under each comparison result (the actual number of training sessions is less than the target number of training sessions in the current communication round, or the actual number of training sessions is greater than or equal to the target number of training sessions in the current communication round), the server needs to determine the corresponding number of training sessions that needs to be reduced or increased.

[0118] In this embodiment of the present application, the number of training times that needs to be reduced or increased is determined based on the change in the data holder's loss value after the end of the current communication round. Therefore, the server first determines the model loss change value obtained by the data holder after the end of the current communication round. The model loss change value is the difference between the loss value of the data holder's local model in the current communication round and the loss value of the data holder's local model in the previous communication round.

[0119] As an implementation method, the loss function of the local model can be a cross-entropy loss function, and the data holder can obtain a cross-entropy loss value each time the local model is trained. In this embodiment of the present application, the loss value obtained by the data holder when iteratively training the local model in the current communication round can refer to the cross-entropy loss value obtained by the data holder when training the local model for the last time in the current communication round; the loss value obtained by the data holder when iteratively training the local model in the previous communication round can refer to the cross-entropy loss value obtained by the data holder when training the local model for the last time in the previous communication round.

[0120] The server uses the cross entropy loss value obtained in the current communication round to subtract the cross entropy loss value obtained in the previous communication round to obtain the model loss change value.

[0121] In step 303, the server obtains the target number of training times in the next communication round based on the actual number of training times, the comparison result, and the model loss change value.

[0122] After the server obtains the comparison results between the actual number of training sessions, the target number of training sessions in the current communication round corresponding to the current communication round, and the model loss change value, it can determine the corresponding number of training sessions that need to be reduced or the number of training sessions that need to be increased under each comparison result. Then, the server subtracts the number of training sessions that need to be reduced from the actual number of training sessions or adds the number of training sessions that need to be increased to the actual number of training sessions to obtain the target number of training sessions in the next communication round.

[0123] In a possible implementation of step 303, the server may perform the following steps A1 and A2 to implement the process of step 303:

[0124] In step A1, if the comparison result shows that the actual number of training times is less than the target number of training times in the current communication round, the server reduces the actual number of training times to obtain the target number of training times in the next communication round. The amount of reduction in the number of training times is positively correlated with the change in model loss.

[0125] The reduction in the number of training times mentioned above refers to the number of training times reduced by the server from the actual number of training times.

[0126] In step A2, if the comparison result is that the actual number of training times is greater than or equal to the number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round. The increase in the number of training times is negatively correlated with the change in model loss.

[0127] The increase in the number of training times mentioned above refers to the number of training times increased by the server to the actual number of training times.

[0128] See the following formula for calculating the target number of training sessions:

[0129] in, is the target number of training times for data holder k in the next communication round (the next communication round corresponds to t+1 rounds of training), is the actual number of training times of data holder k in the current communication round (the current communication round corresponds to t rounds of training), is the target number of training times corresponding to the data holder k in the current communication round, is the loss value obtained by iteratively training the local model in the current communication round, The loss value obtained by iteratively training the local model in the last communication round (the last communication round corresponds to t-1 round of training).

[0130] In this way, when the actual number of training times is less than the target number of training times in the current communication round, the server uses the first formula in the above calculation formula, substitutes the model loss change value and the actual number of training times into the first formula, and obtains the target number of training times in the next communication round. It can be understood that in the first formula, the reduction in the number of training times is:

[0131] It can be seen from the first formula that when the actual number of training times is less than the target number of training times in the current communication round:

[0132] 1) If Data holder k did not complete the target number of training times in the current communication round, indicating that compared with the previous communication round, the loss value obtained by data holder k in training the local model in the current communication round has increased, which means that the training effect of the local model has deteriorated. The larger the value of , that is, the greater the change in model loss, the worse the training effect of the local model, and the greater the number of training times that need to be reduced (that is, the reduction in the number of training times); and The smaller the value of , that is, the smaller the change in model loss, the smaller the number of training times that need to be reduced (that is, the reduction in the number of training times) is.

[0133] 2) If Data holder k did not complete the target number of training times in the current communication round, indicating that compared with the previous communication round, the loss value obtained by data holder k in training the local model in the current communication round has become smaller. This means that although data holder k did not complete the target number of training times in the current communication round, the training effect of the local model has improved. The smaller the value of , that is, the smaller the change in model loss, the better the training effect of the local model, and the smaller the number of training times that need to be reduced (that is, the reduction in the number of training times). Conversely, The larger the value of , that is, the greater the change in model loss, the more the number of training times needs to be reduced.

[0134] To sum up, when the actual number of training times is less than the target number of training times in the current communication round, the greater the change in model loss, the greater the number of training times that need to be reduced, and the smaller the change in model loss, the smaller the number of training times that need to be reduced, that is, the reduction in the number of training times is positively correlated with the change in model loss.

[0135] When the actual number of training sessions is greater than or equal to the target number of training sessions in the current communication round, the server uses the second formula in the above calculation formula, substitutes the model loss change value and the actual number of training sessions into the second formula, and obtains the target number of training sessions in the next communication round. It can be understood that in the second formula, the increase in the number of training sessions is:

[0136] It can be seen from the second formula that when the actual number of training times is greater than or equal to the target number of training times in the current communication round:

[0137] 1) If Although data holder k has completed the target number of training times in the current communication round, the loss value obtained by data holder k in training the local model in the current communication round is larger than that in the previous communication round, which means that the training effect of the local model has deteriorated. The larger the value (i.e. The smaller the value of ), that is, the greater the change in model loss, the worse the training effect of the local model, and the smaller the number of training times that need to be increased (that is, the increase in the number of training times); and The smaller the value (i.e. The larger the value of ), that is, the smaller the change in model loss, the greater the number of training times that need to be increased (that is, the increase in the number of training times).

[0138] 2) If Data holder k has completed the target number of training times in the current communication round, and the loss value obtained by data holder k in training the local model in the current communication round is smaller than that in the previous communication round, which means that the training effect of the local model has improved. The smaller the value (i.e. The larger the value of , that is, the smaller the change in model loss, the better the training effect of the local model, and the greater the number of training times that need to be increased (that is, the increase in the number of training times). Conversely, The larger the value (i.e. The smaller the value of ), that is, the greater the change in model loss, the smaller the number of training times that need to be increased.

[0139] To sum up, when the actual number of training times is greater than or equal to the target number of training times in the current communication round, the greater the change in model loss, the smaller the number of training times that need to be increased, and the smaller the change in model loss, the greater the number of training times that need to be reduced, that is, the increase in the number of training times is negatively correlated with the change in model loss.

[0140] In this way, when the server determines the target number of training times for the data holder in the next communication round, in addition to the actual number of training times completed by the data holder in the current time period, it also takes into account the change in the loss value of the data holder's training of the local model: if the data holder does not complete the target number of training times in the current time period and the model loss change value increases, then the number of training times reduced will be greater; if the data holder does not complete the target number of training times in the current time period, but the model loss change value decreases, indicating that the training effect is still optimistic, then the number of training times reduced will be less; if the data holder can complete the target number of training times in the current time period, but the model loss change value increases, then the number of training times increased will be less; if the data holder can complete the target number of training times in the current time period and the model loss change value decreases, indicating that the training effect is very good, then the number of training times increased will be more, thereby improving the flexibility and rationality of setting the target number of training times, which is conducive to fully utilizing the training capabilities of each data holder and improving model training efficiency.

[0141] In one embodiment, based on the embodiment shown in FIG2 , referring to FIG5 , the model training method of this embodiment may further include step 401:

[0142] Step 401: The server sends the time length corresponding to the next communication round to the data holder.

[0143] In an embodiment of the present application, the server can also set different time lengths for each communication round, wherein the setting of the time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0144] The amount of computing resources represents the computing power of the data holder. The larger the amount of computing resources, the shorter the communication round. The amount of sample data corresponding to the local model refers to the amount of samples owned by the data holder for participating in federated training. The larger the amount of sample data, the longer the communication round. The amount of model parameters of the local model can refer to the number of model parameters of the local model. The larger the amount of model parameters, the more complex the local model, and the longer the communication round. The better the communication quality, the shorter the communication round.

[0145] The server combines the computing resources of each data holder, the amount of sample data corresponding to the local model, the model parameters of the local model, and the communication quality, and sets the time length corresponding to the communication round of each data holder in a targeted manner. It then allocates the target number of training times based on the set time length, further improving the resource utilization of each data holder and helping to improve the training efficiency of the federated model.

[0146] The server sends the time length corresponding to the next communication round to the data holder. After the data holder receives the time length corresponding to the next communication round sent by the server, it iteratively trains the local model according to the target number of training times in the next communication round.

[0147] In other possible embodiments, the server may also set a fixed time length for each data holder during the first training process, and directly reuse the time length in subsequent model training processes, without specific limitation here.

[0148] In one embodiment, based on the embodiment shown in FIG2 , referring to FIG6 , the training reporting data of this embodiment also includes model parameters obtained after the data holder iteratively trains the local model in the current communication round. The model training method of this embodiment further includes steps 501 and 502:

[0149] Step 501: The server updates the parameters of the global model according to the model parameters to obtain updated model parameters.

[0150] After the data holder iteratively trains the local model in the current communication round, the data holder obtains the model parameters, and sends the model parameters and the actual number of training times completed by iteratively training the local model in the current communication round as training report data to the server.

[0151] After receiving the training report data, the server not only determines the target number of training times for the data holder in the next communication round based on the actual number of training times, but also needs to update the parameters of the global model based on the model parameters to obtain the updated model parameters. That is, the server aggregates the local global model based on the received model parameters to complete the update of the global model.

[0152] As an implementation method, there are multiple data holders, that is, there are multiple data holders in the current communication round who train local models locally. In this way, after the current communication round ends, the server can receive the training report data sent by each data holder. In this way, the server can perform statistical processing on the model parameters sent by each data holder to obtain updated model parameters, thereby realizing the process of step 501.

[0153] Exemplarily, the server may average the model parameters sent by each data holder to obtain updated model parameters.

[0154] Step 502: The server sends the updated model parameters to the data holder.

[0155] The updated model parameters are used to instruct the data holder to train the local model in the next communication round according to the updated model parameters and the target number of training times in the next communication round.

[0156] The server will also send the target number of training times for the data holder in the next communication round determined based on the actual number of training times to the data holder. After the data holder receives the updated model parameters and the target number of training times in the next communication round sent by the server, it will train the local model according to the updated model parameters and the target number of training times in the next communication round.

[0157] After the next communication round, the data holder sends the trained model parameters and the actual number of training times completed by the data holder in iteratively training the local model in the next communication round to the server, and so on until the model converges, realizing flexible allocation of training tasks.

[0158] In one embodiment, based on the embodiment shown in FIG2 , referring to FIG7 , the model training method of this embodiment further includes step 601 before step 201:

[0159] In step 601, the server sends the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder.

[0160] In this way, the data holder uses the initialized model as the local model in the initial communication round and iteratively trains the local model according to the initial training number.

[0161] As mentioned above, the initialization model is a federated model with parameter initialization; the time length corresponding to the initial communication round is the first training time set by the server for the data holder, and the setting of the time length is related to at least one of the computing resources of the data holder, the amount of sample data corresponding to the local model, the model parameter amount of the local model, and the communication quality corresponding to the data holder.

[0162] The initial training number is the target number of training times set by the server for each data holder during the first training session, i.e., the first communication round. During non-initial training, the target number of training times for each communication round is determined based on the actual number of training times in the previous round. However, the initial training number can be determined based on factors such as the amount of sample data and computing resources of each data holder, ensuring that all data holders have the opportunity to effectively train their local models.

[0163] After the data holder receives the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder from the server, the data holder iteratively trains the initialization model according to the initial training times within the initial communication round.

[0164] After the initial communication round, the data holder sends the trained model parameters and the actual number of training times completed by the data holder in iteratively training the local model in the initial communication round to the server, and so on until the model converges, realizing flexible allocation of training tasks.

[0165] In one embodiment, referring to FIG8 , a model training method is provided for a data holder, the method comprising:

[0166] Step 701: The data holder sends training report data to the server.

[0167] The training reporting data includes the actual number of training times completed by the data holder during the iterative training of the local model in the current communication round.

[0168] Step 702: The data holder receives the target number of training times for the data holder in the next communication round sent by the server.

[0169] The target number of training times in the next communication round is determined by the server based on the training report data.

[0170] Step 703: The data holder iteratively trains the local model in the next communication round according to the target number of training times in the next communication round.

[0171] In one embodiment, before step 703, the model training method of this embodiment further includes:

[0172] Step B1: The data holder receives the time length corresponding to the next communication round sent by the server;

[0173] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0174] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. The model training method of this embodiment also includes:

[0175] Step B2: The data holder receives the updated model parameters sent by the server. The updated model parameters are obtained by the server updating the global model according to the model parameters.

[0176] Furthermore, the data holder may train the local model according to the updated model parameters and the target number of training times in the next communication round to implement the process of step 703 .

[0177] In one embodiment, before sending the training report data to the server, the model training method of this embodiment further includes:

[0178] Step B3: The data holder receives the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder from the server;

[0179] In step B4, the data holder iteratively trains the initialization model according to the initial training times in the initial communication round.

[0180] The specific implementation methods and beneficial effects of the model training method applied to the data holder are similar to the specific implementation methods and beneficial effects of the model training method of the server applied above. Please refer to the relevant records of the model training method of the server applied above, and no further details will be given here.

[0181] In one embodiment, as shown in FIG9 , a model training method is provided, including:

[0182] Step 801: The data holder sends training report data to the server.

[0183] Among them, the training reporting data includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round, the loss value of the local model of the data holder in the current communication round, and the model parameters obtained after the data holder iteratively trains the local model in the current communication round.

[0184] Before step 801, the server may also send the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder; the data holder receives the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder sent by the server; the data holder uses the initialization model as the local model in the initial communication round, and iteratively trains the initialization model according to the initial training times.

[0185] Step 802: The server receives training report data sent by the data holder.

[0186] In step 803 , the server compares the actual number of training times with the target number of training times in the current communication round, obtains a comparison result, and determines a model loss change value of the data holder.

[0187] Among them, the target number of training times in the current communication round is the target number of training times for the data holder to iteratively train the local model in the current communication round, and the model loss change value is the difference between the loss value of the local model of the data holder in the current communication round and the loss value of the local model of the data holder in the previous communication round.

[0188] In step 804, if the comparison result shows that the actual number of training times is less than the target number of training times in the current communication round, the actual number of training times is reduced to obtain the target number of training times in the next communication round. The amount of reduction in the number of training times is positively correlated with the change in the model loss.

[0189] Step 805: If the comparison result shows that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round. The increase in the number of training times is negatively correlated with the change in the model loss.

[0190] The target number of training times in the next communication round is the target number of training times for the data holder to iteratively train the local model in the next communication round.

[0191] In step 806, the server performs statistical processing on the model parameters sent by each data holder to obtain updated model parameters.

[0192] In step 807 , the server sends the target number of training times in the next communication round and the updated model parameters to the data holder.

[0193] In which, the server can also send the time length corresponding to the next communication round to the data holder; the time length is related to at least one of the computing resources of the data holder, the sample data amount corresponding to the local model, the model parameter amount of the local model and the communication quality corresponding to the data holder.

[0194] Step 808: The data holder receives the target number of training times for the data holder in the next communication round and the updated model parameters from the server.

[0195] In step 809 , the data holder trains the local model according to the updated model parameters and the target number of training times in the next communication round.

[0196] In this way, since different data holders participating in local model training often have different training conditions, such as different amounts of computing resources, sample data volumes, network conditions, etc., there are large differences in the target number of training times that different data holders can complete within a given time. The related technology assigns the same and fixed number of tasks to different data holders. Then, data holders with poor training conditions may not be able to complete the training tasks in time, and data holders with good training conditions may have the problem of not fully utilizing system resources, which leads to low model training efficiency. In the embodiment of the present application, the data holders are assigned the same number of tasks in the next communication. The target number of training rounds is dynamically determined based on the training reported data actually completed by the data holder in the previous training period (i.e., the current communication round). For example, if the data holder completes fewer rounds of training in the current communication round, it means that its training conditions are poor. In this case, the target number of training rounds will be reduced accordingly in the next communication round to ensure that it successfully completes the training task. If the data holder completes more rounds of training in the current communication round, it means that its training conditions are good. In this case, the target number of training rounds will be increased accordingly in the next communication round to make full use of the data holder's system resources, thereby improving the model training efficiency.

[0197] It should be understood that, although the various steps in the above flow chart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above flow chart may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0198] In one embodiment, as shown in FIG10 , a model training device is provided, which is disposed on a server and includes:

[0199] A receiving module 901 is configured to receive training report data sent by a data holder, wherein the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0200] A determination module 902 is configured to determine a target number of training times for the data holder in the next communication round based on the training reporting data;

[0201] The sending module 903 is configured to send the target number of training times in the next communication round to the data holder.

[0202] In one embodiment, the training report data further includes a loss value of the local model of the data holder in the current communication round; the determination module 902 includes:

[0203] a comparing unit, configured to compare the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0204] a determining unit, configured to determine a model loss change value of the data holder, the model loss change value being a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in a previous communication round;

[0205] An acquisition unit is used to obtain the target number of training times in the next communication round based on the actual number of training times, the comparison result and the model loss change value.

[0206] In one embodiment, the acquisition unit is specifically used to, if the comparison result is that the actual training number is less than the target training number in the current communication round, reduce the actual training number to obtain the target training number in the next communication round, and the reduction in the training number is positively correlated with the change value of the model loss; if the comparison result is that the actual training number is greater than or equal to the target training number in the current communication round, increase the actual training number to obtain the target training number in the next communication round, and the increase in the training number is negatively correlated with the change value of the model loss.

[0207] In one embodiment, the sending module 903 is further configured to send the time length corresponding to the next communication round to the data holder;

[0208] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0209] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the apparatus further includes:

[0210] A parameter updating module, configured to update the parameters of the global model according to the model parameters to obtain updated model parameters;

[0211] The sending module 903 is further configured to send the updated model parameters to the data holder, and the updated model parameters are used to instruct the data holder to train the local model in the next communication round according to the updated model parameters and the target number of training times in the next communication round.

[0212] In one embodiment, there are multiple data holders, and the parameter updating module is specifically configured to perform statistical processing on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0213] In one embodiment, the sending module 903 is further used to send the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder, so that the data holder iteratively trains the initialization model according to the initial training times within the initial communication round.

[0214] For the specific definition of the model training device provided on the server, please refer to the definition of the model training method for the server above, which will not be repeated here. Each module in the above-mentioned model training device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the server in hardware form, or can be stored in the memory in the server in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0215] In one embodiment, as shown in FIG11 , a model training device is provided, which is provided at a data holder, and includes:

[0216] A sending module 1001 is configured to send training report data to a server, wherein the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0217] a receiving module 1002, configured to receive a target number of training times for the data holder in a next communication round sent by the server, where the target number of training times in the next communication round is determined by the server based on the training reporting data;

[0218] The training module 1003 is configured to iteratively train the local model in the next communication round according to a target number of training times in the next communication round.

[0219] In one embodiment, before iteratively training the local model according to the target number of training times in the next communication round, the receiving module 1002 is further configured to receive a time length corresponding to the next communication round sent by the server;

[0220] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0221] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the receiving module 1002 is further used to receive updated model parameters sent by the server, where the updated model parameters are obtained by the server by updating the parameters of the global model according to the model parameters;

[0222] The training module 1003 is specifically configured to train the local model according to the updated model parameters and the target number of training times in the next communication round.

[0223] In one embodiment, before sending the training report data to the server, the receiving module 1002 is further configured to receive an initialization model, a time length corresponding to an initial communication round, and an initial training number corresponding to the data holder, sent by the server;

[0224] The training module 1003 is further configured to use the initialization model as a local model in the initial communication round, and iteratively train the local model according to the initial training times.

[0225] For the specific definition of the model training device provided in the data holder, please refer to the definition of the model training method for the data holder above, which will not be repeated here. The various modules in the above-mentioned model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the data holder in the form of hardware, or can be stored in the memory in the data holder in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0226] Figure 12 is a schematic diagram of the structure of a communication device provided in an embodiment of the present application. The communication device 1100 shown in Figure 12 includes: at least one processor 1101, a memory 1102, and at least one network interface 1104. The various components in the communication device 1100 are coupled together through a bus system 1105. It can be understood that the bus system 1105 is used to implement connection and communication between these components. In addition to the data bus, the bus system 1105 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus systems 1105 in Figure 12. In addition, in an embodiment of the present application, a transceiver 1106 is also included. The transceiver 1106 can be multiple elements, that is, it includes a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium.

[0227] It is understood that the memory 1102 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1102 of the systems and methods described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable types of memory.

[0228] In some embodiments, the memory 1102 stores the following elements, executable modules or data structures, or subsets thereof, or extended sets thereof: an operating system 1103. The operating system 1103 includes various system programs, such as a framework layer, a core library layer, and a driver layer, for implementing various basic services and processing hardware-based tasks.

[0229] In an embodiment of the present application, by calling the program or instructions stored in the memory 1102, the receiver is used to receive the training reporting data sent by the data holder, and the training reporting data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; the processor is used to determine the target number of training times of the data holder in the next communication round based on the training reporting data; and the transmitter is used to send the target number of training times in the next communication round to the data holder.

[0230] Some or all of the methods disclosed in the above embodiments of the present application can also be applied to the processor 1101, or implemented by the processor 1101, or implemented by the processor 1101 in conjunction with other components (such as a transceiver). The processor 1101 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 1101 or by instructions in the form of software. The above-mentioned processor 1101 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1102, and processor 1101 reads information in memory 1102 and, in conjunction with its hardware, completes the steps of the above method.

[0231] It is understood that the embodiments described in the embodiments of the present application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP devices, DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0232] For software implementation, the techniques described in the embodiments of the present application can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described in the embodiments of the present application. The software code can be stored in a memory and executed by the processor 1101. The memory can be implemented in the processor 1101 or external to the processor 1101.

[0233] In one embodiment, the training reporting data also includes the loss value of the local model of the data holder in the current communication round; the processor is specifically used to compare the actual number of training times with the target number of training times in the current communication round, obtain a comparison result, and determine the model loss change value of the data holder, the model loss change value being the difference between the loss value of the local model of the data holder in the current communication round and the loss value of the local model of the data holder in the previous communication round; according to the actual number of training times, the comparison result and the model loss change value, the target number of training times in the next communication round is obtained.

[0234] In one embodiment, the processor is specifically configured to, if the comparison result shows that the actual number of training times is less than the target number of training times in the current communication round, reduce the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change value of the model loss;

[0235] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0236] In one implementation, the transmitter is further configured to send a time length corresponding to the next communication round to the data holder;

[0237] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0238] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the processor is further configured to update the parameters of the global model according to the model parameters to obtain updated model parameters;

[0239] The transmitter is further used to send the updated model parameters to the data holder, and the updated model parameters are used to instruct the data holder to train the local model in the next communication round according to the updated model parameters and the target number of training times in the next communication round.

[0240] In one embodiment, there are multiple data holders, and the processor is specifically configured to perform statistical processing on the model parameters sent by each data holder to obtain the updated model parameters.

[0241] In one embodiment, the transmitter is further used to send the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0242] Alternatively, in an embodiment of the present application, by calling the program or instructions stored in the memory 1102, the transmitter is used to send training reporting data to the server, and the training reporting data includes the actual number of training times completed by the data holder in the iterative training of the local model in the current communication round; the receiver is used to receive the target training times of the data holder in the next communication round sent by the server, and the target training times in the next communication round is determined by the server based on the training reporting data; the processor is used to iteratively train the local model in the next communication round according to the target training times in the next communication round.

[0243] In one embodiment, the receiver is further configured to receive a time length corresponding to the next communication round sent by the server;

[0244] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0245] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. The receiver is further configured to receive updated model parameters sent by the server, where the updated model parameters are obtained by the server by updating the global model according to the model parameters.

[0246] The processor is specifically configured to train the local model according to the updated model parameters and the target number of training times in the next communication round.

[0247] In one embodiment, the receiver is further configured to receive the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder, sent by the server;

[0248] The processor is further configured to use the initialized model as a local model in the initial communication round, and iteratively train the local model according to the initial training times.

[0249] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0250] Receive training report data sent by the data holder, where the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0251] Determining a target number of training times for the data holder in the next communication round based on the training reported data;

[0252] The target number of training times in the next communication round is sent to the data holder.

[0253] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0254] Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0255] Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in the previous communication round;

[0256] The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result, and the model loss change value.

[0257] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0258] If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change in the model loss;

[0259] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0260] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0261] Sending the time length corresponding to the next communication round to the data holder;

[0262] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0263] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. When the computer program is executed by the processor, the following steps are further implemented:

[0264] Update the parameters of the global model according to the model parameters to obtain updated model parameters;

[0265] The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and the target number of training times in the next communication round.

[0266] In one embodiment, there are multiple data holders, and when the computer program is executed by a processor, the following steps are further implemented:

[0267] Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0268] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0269] The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0270] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0271] Sending training report data to the server, where the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0272] receiving a target training number for the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0273] In the next communication round, the local model is iteratively trained according to a target number of training times in the next communication round.

[0274] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0275] receiving a time length corresponding to the next communication round sent by the server;

[0276] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0277] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. When the computer program is executed by the processor, the following steps are further implemented:

[0278] receiving updated model parameters sent by the server, where the updated model parameters are obtained by the server updating the parameters of the global model according to the model parameters;

[0279] The iteratively training the local model according to the target number of training times in the next communication round includes:

[0280] The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

[0281] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0282] Receiving the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder sent by the server;

[0283] In the initial communication round, the initialized model is used as a local model, and the local model is iteratively trained according to the initial training number of times.

[0284] Figure 13 is a schematic structural diagram of a chip according to an embodiment of the present application. The chip 1200 shown in Figure 13 includes a processor 1210, which can call and run a computer program from a memory to implement the method according to the embodiment of the present application.

[0285] Optionally, as shown in FIG13 , the chip 1200 may further include a memory 1220. The processor 1210 may call and execute a computer program from the memory 1220 to implement the method in the embodiment of the present application.

[0286] The memory 1220 may be a separate device independent of the processor 1210 , or may be integrated into the processor 1210 .

[0287] Optionally, the chip 1200 may further include an input interface 1230. The processor 1210 may control the input interface 1230 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0288] Optionally, the chip 1200 may further include an output interface 1240. The processor 1210 may control the output interface 1240 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0289] Optionally, the chip 1200 can be applied to the communication device in the embodiment of the present application, and the chip 1200 can implement the corresponding processes implemented in the various methods in the embodiment of the present application. For the sake of brevity, they are not repeated here.

[0290] It should be understood that the chip 1200 mentioned in the embodiment of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0291] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0292] Receive training report data sent by the data holder, where the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0293] Determining a target number of training times for the data holder in the next communication round based on the training reported data;

[0294] The target number of training times in the next communication round is sent to the data holder.

[0295] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0296] Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0297] Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in the previous communication round;

[0298] The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result, and the model loss change value.

[0299] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0300] If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change in the model loss;

[0301] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0302] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0303] Sending the time length corresponding to the next communication round to the data holder;

[0304] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0305] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. When the computer program is executed by the processor, the following steps are further implemented:

[0306] Update the parameters of the global model according to the model parameters to obtain updated model parameters;

[0307] The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and the target number of training times in the next communication round.

[0308] In one embodiment, there are multiple data holders, and when the computer program is executed by a processor, the following steps are further implemented:

[0309] Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0310] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0311] The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0312] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0313] Sending training report data to the server, where the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0314] receiving a target training number for the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0315] In the next communication round, the local model is iteratively trained according to a target number of training times in the next communication round.

[0316] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0317] receiving a time length corresponding to the next communication round sent by the server;

[0318] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0319] In one embodiment, the training reporting data further includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. When the computer program is executed by the processor, the following steps are further implemented:

[0320] receiving updated model parameters sent by the server, where the updated model parameters are obtained by the server updating the parameters of the global model according to the model parameters;

[0321] The iteratively training the local model according to the target number of training times in the next communication round includes:

[0322] The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

[0323] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0324] Receiving the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder sent by the server;

[0325] In the initial communication round, the initialized model is used as a local model, and the local model is iteratively trained according to the initial training number of times.

[0326] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0327] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0328] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A model training method, applied to a server, comprising: Receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; Determining a target number of training times for the data holder in the next communication round according to the training reporting data; The target number of training times in the next communication round is sent to the data holder.

2. The method according to claim 1, wherein: The training report data also includes a loss value of the local model of the data holder in the current communication round; and determining a target number of training times of the data holder in the next communication round according to the training report data includes: Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result; Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in a previous communication round; The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result and the model loss change value.

3. The method according to claim 2, wherein: The step of obtaining the target number of training times in the next communication round according to the actual number of training times, the comparison result, and the model loss change value includes: If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change value of the model loss; If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: Sending the time length corresponding to the next communication round to the data holder; The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

5. The method according to any one of claims 1 to 4, wherein: The training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method also includes: Update the parameters of the global model according to the model parameters to obtain updated model parameters; The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and a target number of training times in the next communication round.

6. The method according to claim 5, wherein: The number of the data holders is multiple, and the updating of the global model parameters according to the model parameters to obtain the updated model parameters includes: Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

7. The method according to any one of claims 1 to 6, wherein: Before receiving the training report data sent by the data holder, the method further includes: The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

8. The method according to claim 3, wherein: The loss value is a cross entropy loss value.

9. A model training method, applied to a data holder, the method comprising: Sending training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; receiving a target training number of the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data; In the next communication round, the local model is iteratively trained according to the target training number in the next communication round.

10. The method according to claim 9, wherein: Before iteratively training the local model according to the target number of training times in the next communication round, the method further includes: Receive the time length corresponding to the next communication round sent by the server; The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

11. The method according to claim 9 or 10, wherein: The training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method also includes: Receiving updated model parameters sent by the server, wherein the updated model parameters are obtained by the server updating parameters of the global model according to the model parameters; The iterative training of the local model according to the target training times in the next communication round includes: The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

12. The method according to any one of claims 9 to 11, wherein: Before sending the training report data to the server, the method further includes: Receiving the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder; In the initial communication round, the initialization model is used as the local model, and the local model is iteratively trained according to the initial training number.

13. The method according to any one of claims 9 to 12, wherein: The training reporting data also includes the loss value of the local model of the data holder in the current communication round.

14. A model training device, arranged on a server, comprising: A receiving module, configured to receive training report data sent by a data holder, wherein the training report data includes at least an actual number of training times completed by the data holder in iteratively training the local model in the current communication round; A determination module, used to determine a target number of training times of the data holder in the next communication round according to the training reporting data; A sending module is used to send the target number of training times in the next communication round to the data holder.

15. A model training device, provided at a data holder, comprising: A sending module, used to send training report data to a server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; A receiving module, used to receive a target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data; The training module is used to iteratively train the local model in the next communication round according to the target number of training times in the next communication round.

16. A communication device comprising: transmitter, processor, and receiver; The receiver is used to receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; The processor is used to determine a target number of training times of the data holder in a next communication round according to the training reporting data; The transmitter is used to send the target training times in the next communication round to the data holder.

17. A communication device comprising: transmitter, processor, and receiver; The transmitter is used to send training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; The receiver is used to receive the target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data; The processor is used to iteratively train the local model in the next communication round according to the target training number in the next communication round.

18. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

19. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

20. A chip comprising a programmable logic circuit and / or program instructions, wherein the chip implements the steps of the method according to any one of claims 1 to 12 when the chip is running.

Images