Model training method and device, communication equipment, storage medium and program product

Abstract

The invention relates to a model training method and device, communication equipment, a storage medium and a program product. The method comprises the following steps: receiving training report data sent by a data holder, wherein the training report data at least comprises actual training times completed by the data holder iteratively training a local model in a current communication round; according to the training report data, determining a target training frequency of the data holder in a next communication round; and sending the target training frequency in the next communication round to the data holder. By adopting the method, the model training efficiency can be improved, and the model accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, communication equipment, storage medium and program product. Background Art

[0002] Federated Learning (FL) is a type of machine learning that allows data holders to combine their data to build a shared machine learning model, namely a federated model, without sharing private data.

[0003] Currently, during the training process of the federated model, the central server sends the initialized federated model to each data holder, and each data holder trains the federated model locally according to the same and fixed number of training tasks.

[0004] However, the above model training method has the problems of low model training efficiency and low model accuracy. Summary of the invention

[0005] The embodiments of the present application provide a model training method, apparatus, communication equipment, storage medium and program product, which can improve model training efficiency and model accuracy.

[0006] In a first aspect, an embodiment of the present application provides a model training method. The method is applied to a server, and the method includes:

[0007] Receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0008] Determining a target number of training times for the data holder in the next communication round according to the training reporting data;

[0009] The target number of training times in the next communication round is sent to the data holder.

[0010] In one embodiment, the training report data also includes a loss value of a local model of the data holder in the current communication round; and determining a target number of training times of the data holder in the next communication round according to the training report data includes:

[0011] Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0012] Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in a previous communication round;

[0013] The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result and the model loss change value.

[0014] In one embodiment, obtaining the target number of training times in the next communication round according to the actual number of training times, the comparison result, and the model loss change value includes:

[0015] If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change value of the model loss;

[0016] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0017] In one embodiment, the method further comprises:

[0018] Sending the time length corresponding to the next communication round to the data holder;

[0019] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0020] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method further includes:

[0021] Update the parameters of the global model according to the model parameters to obtain updated model parameters;

[0022] The updated model parameters are sent to the data holder, and the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and a target number of training times in the next communication round.

[0023] In one embodiment, the number of the data holders is multiple, and the updating of the global model parameters according to the model parameters to obtain updated model parameters includes:

[0024] Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0025] In one of the embodiments, before receiving the training report data sent by the data holder, the method further includes:

[0026] The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0027] In a second aspect, the present application also provides a model training method. The method is applied to a data holder, and the method includes:

[0028] Sending training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0029] receiving a target training number of the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0030] In the next communication round, the local model is iteratively trained according to the target training number in the next communication round.

[0031] In one of the embodiments, before iteratively training the local model according to the target number of training times in the next communication round, the method further includes:

[0032] Receive the time length corresponding to the next communication round sent by the server;

[0033] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0034] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method further includes:

[0035] Receiving updated model parameters sent by the server, wherein the updated model parameters are obtained by the server updating parameters of the global model according to the model parameters;

[0036] The iterative training of the local model according to the target training times in the next communication round includes:

[0037] The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

[0038] In one embodiment, before sending the training report data to the server, the method further includes:

[0039] Receiving the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder;

[0040] In the initial communication round, the initialization model is used as the local model, and the local model is iteratively trained according to the initial training number.

[0041] In a third aspect, the embodiment of the present application further provides a model training device. The device is arranged on a server, and the device comprises:

[0042] A receiving module, configured to receive training report data sent by a data holder, wherein the training report data includes at least an actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0043] A determination module, used to determine a target number of training times of the data holder in the next communication round according to the training reporting data;

[0044] A sending module is used to send the target number of training times in the next communication round to the data holder.

[0045] In a fourth aspect, the present application also provides a model training device. The device is provided at the data holder, and the device includes:

[0046] A sending module, used to send training report data to a server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0047] A receiving module, used to receive a target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0048] The training module is used to iteratively train the local model in the next communication round according to the target number of training times in the next communication round.

[0049] In a fifth aspect, an embodiment of the present application further provides a communication device. The communication device includes: a transmitter, a processor, and a receiver;

[0050] The receiver is used to receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0051] The processor is used to determine a target number of training times of the data holder in a next communication round according to the training reporting data;

[0052] The transmitter is used to send the target training times in the next communication round to the data holder.

[0053] In a sixth aspect, an embodiment of the present application further provides a communication device. The communication device includes: a transmitter, a processor, and a receiver;

[0054] The transmitter is used to send training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0055] The receiver is used to receive the target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0056] The processor is used to iteratively train the local model in the next communication round according to the target training number in the next communication round.

[0057] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0058] In an eighth aspect, an embodiment of the present application provides a chip, wherein the chip includes a programmable logic circuit and / or program instructions, and when the chip is running, the steps of the method described in the first aspect or the second aspect are implemented.

[0059] In a ninth aspect, an embodiment of the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0060] The above-mentioned model training method, apparatus, communication equipment, storage medium and program product receive training report data sent by the data holder, and the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round, and then determine the target number of training times of the data holder in the next communication round according to the training report data, and then send the target number of training times in the next communication round to the data holder, wherein the target number of training times refers to the number of training times, that is, the task amount of the training times. In this way, since different data holders participating in the local model training often have different training conditions, such as different amounts of computing resources, sample data amounts, network conditions, etc., there are large differences in the target number of training times that different data holders can complete within a given time. The related technology assigns the same and fixed number of tasks to different data holders. Then, the data holders with poor training conditions will have poor training conditions. The data holder may not be able to complete the training task in time, and the data holder with better training conditions may have the problem of under-utilization of system resources, which leads to low model training efficiency and low model accuracy. In the embodiment of the present application, the target number of training for the data holder in the next communication round is dynamically determined based on the training reported data actually completed by the data holder in the previous training time period (i.e., the current communication round). For example, if the data holder completes fewer rounds of training in the current communication round, it means that its training conditions are poor, and its target number of training is reduced accordingly in the next communication round to ensure that it successfully completes the training task. If the data holder completes more rounds of training in the current communication round, it means that its training conditions are better, and its target number of training is increased accordingly in the next communication round to make full use of the system resources of the data holder, thereby improving the model training efficiency and model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 An application environment diagram of a model training method in an embodiment;

[0062] Figure 2 A schematic diagram of a flow chart of a model training method in one embodiment;

[0063] Figure 3 A schematic diagram of data interaction during an exemplary model training process

[0064] Figure 4 is a flow chart of step 202 in one embodiment;

[0065] Figure 5 A schematic diagram of a flow chart of a model training method in another embodiment;

[0066] Figure 6A schematic diagram of a flow chart of a model training method in another embodiment;

[0067] Figure 7 A schematic diagram of a flow chart of a model training method in another embodiment;

[0068] Figure 8 A schematic diagram of a flow chart of a model training method in another embodiment;

[0069] Fig. 9 A schematic diagram of a flow chart of a model training method in another embodiment;

[0070] Fig.10 is a structural block diagram of a model training device in one embodiment;

[0071] Fig.11 is a structural block diagram of a model training device in another embodiment;

[0072] Fig.12 is a schematic diagram of the internal structure of a communication device in one embodiment;

[0073] Fig.13 FIG. 4 is a schematic diagram of the structure of a chip in an embodiment. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0075] Figure 1 A schematic diagram of an implementation environment of a model training method provided in an embodiment of the present application. Figure 1 As shown, the implementation environment includes a server 102 and multiple data holders ( Figure 1 Only two data holders are shown for example: data holder 104 and data holder 106). A data holder, or a data holding node, may be a server that holds sample data required for model training.

[0076] The data is transmitted between the server 102 and the data holder via the network; the server 102 may be a single server or a server cluster consisting of multiple servers; the data holder may be a single server or a server cluster consisting of multiple servers.

[0077] Among them, server 102 can be an edge server, and of course it can also be any other server in the model training process, and no specific limitation is made here.

[0078] Take Federated Learning (FL) as an example. Federated learning is a type of machine learning that allows data holders to combine their data to establish a shared machine learning model, namely a federated model, without sharing private data.

[0079] At present, during the training process of the federated model, the central server sends the initialized federated model to each data holder, and each data holder uses the federated model as a local model and trains the local model locally according to the same and fixed number of tasks until the model converges. However, since different data holders participating in the local model training often have different training conditions, such as different amounts of computing resources, sample data, network conditions, etc., there are large differences in the number of tasks that different data holders can complete within a given time. Traditional technology assigns the same and fixed number of tasks to different data holders. Then, data holders with poor training conditions may not be able to complete the training tasks in time, and data holders with better training conditions may have the problem of not fully utilizing system resources, which leads to low model training efficiency and low model accuracy.

[0080] In view of this, an embodiment of the present application provides a model training method that can improve model training efficiency and improve model accuracy.

[0081] It should be noted that the beneficial effects or technical problems solved by the embodiments of the present application are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0082] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0083] In one embodiment, Figure 2 As shown, a model training method is provided, which is applied to Figure 1 Taking the server 102 in the example as an example, the following steps are included:

[0084] Step 201: The server receives training report data sent by the data holder.

[0085] In the embodiment of the present application, the server acts as the "central server" in the federated model training process, and is responsible for issuing training tasks, aggregating and updating federated model parameters, etc. Among them, edge computing (EC) refers to a computing method that deploys workloads at the edge, which aims to migrate data processing to servers closer to users, effectively alleviating network congestion.

[0086] As an implementation method, the embodiment of the present application can train the federated model in communication rounds, that is, the server can set up communication rounds, one communication round corresponds to a time period, and allocate a target number of training times corresponding to the communication round to each data holder, wherein the target number of training times refers to the number of training tasks corresponding to the communication round, that is, the task amount of the training times of the communication round, and the target number of training times represents the target number of training times for the data holder to iteratively train the local model within the communication round.

[0087] For the first training process, the server sends the parameter-initialized federated model to each data holder. The data holder uses the parameter-initialized federated model as the local model and iterates the local model according to the communication rounds and number of tasks set by the server until the communication round ends.

[0088] It is understandable that the data holder may be able to complete the task within the communication round, that is, the number of training times actually completed by the data holder within the communication round can reach or exceed the set target number of training times, but of course, the data holder may not be able to complete the task in time.

[0089] After the communication round is over, the data holder sends training report data to the server, and the training report data includes at least the actual number of training times completed by the data holder in iteratively training the local model in the current communication round. After the communication round is over, if the data holder completes the task set by the server, the actual number of training times is the target number of training times set by the server for the data holder in the current communication round; if the data holder fails to complete the task set by the server, the actual number of training times is the number of iterative training times actually completed by the data holder in the current communication round.

[0090] In an embodiment of the present application, the training reporting data also includes the model parameters obtained by the data holder after iteratively training the local model in the current communication round. The server updates the parameters of the global model according to the model parameters reported by each data holder after the end of the current communication round to obtain updated model parameters.

[0091] Then, in the non-first training process, the server reassigns the target number of training times corresponding to the next communication round to each data holder, and sends the updated model parameters to each data holder. The data holder continues to iteratively train the local model locally based on the target number of training times assigned by the server and the updated model parameters in the next time period.

[0092] After the next communication round, the data holder continues to send training report data to the server, and so on, until the local model converges.

[0093] It should be noted that in step 201, the server receives the training report data sent by the data holder, and the training report data may be the training report data sent by the data holder to the server after any communication round is completed during the local model training process.

[0094] Step 202: The server determines the target number of training times for the data holder in the next communication round according to the training reporting data.

[0095] The actual number of training times completed by the data holder in iteratively training the local model in the current communication round can reflect the training conditions of the data holder, such as the storage space, computing power, network conditions, etc. of the data holder. It can be understood that the greater the actual number of training times, the better the training conditions of the data holder, and the smaller the actual number of training times, the worse the training conditions of the data holder.

[0096] In an embodiment of the present application, the server can set the target number of training times for the data holder in the next communication round based on the actual number of training times completed by the data holder in the current communication round, that is, the server sets the target number of training times for the next communication round based on the training conditions of the data holder reflected in the current communication round.

[0097] As an implementation method, if the data holder completes fewer rounds of training in the current communication round, it means that its training conditions are poor, and its target training times are reduced accordingly in the next communication round to ensure that it successfully completes the training task. If the data holder completes more rounds of training in the current communication round, it means that its training conditions are good, and its target training times are increased accordingly in the next communication round to make full use of the data holder's system resources.

[0098] In a possible implementation of step 202, the server may compare the actual number of training times with the target number of training times set for the current communication round. If the actual number of training times is greater than or equal to the target number of training times for the current communication round, the server increases the actual number of training times to obtain the target number of training times for the next communication round. If the actual number of training times is less than the target number of training times for the current communication round, the server decreases the actual number of training times to obtain the target number of training times for the next communication round, or the server may directly use the actual number of training times as the target number of training times for the next communication round, and so on.

[0099] Other possible implementations of step 202 will be described in the following examples.

[0100] Step 203: The server sends the target number of training times in the next communication round to the data holder.

[0101] The target number of training times in the next communication round is the target number of training times that the data holder iteratively trains the local model in the next communication round.

[0102] In this way, after the data holder receives the target number of training times of the data holder in the next communication round sent by the server, the data holder iteratively trains the local model according to the target number of training times in the next communication round.

[0103] After the next communication round ends, the data holder continues to send training report data to the server, and so on, until the local model converges.

[0104] In order to more clearly understand the data interaction procedures between the server and the data holder in the embodiment of the present application. Figure 3 , Figure 3 The figure is a schematic diagram of data interaction during an exemplary model training process.

[0105] After the current communication round of training is completed, the data holder uploads training report data to the server. The training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round. The server receives the training report data, and then adjusts the target number of training times of the data holder in the next communication round according to the training report data. The server sends the target number of training times in the next communication round to the data holder.

[0106] The above embodiment receives training report data sent by the data holder, and the training report data at least includes the actual number of training times completed by the data holder in the iterative training of the local model in the current communication round, and then determines the target number of training times of the data holder in the next communication round according to the training report data, and then sends the target number of training times in the next communication round to the data holder. In this way, since different data holders participating in the local model training often have different training conditions, such as different amounts of computing resources, sample data amounts, network conditions, etc., there are large differences in the target number of training times that different data holders can complete within a given time. The related technology assigns the same and fixed number of tasks to different data holders. Then, the data holders with poor training conditions may not be able to complete the training tasks in time, and the data holders with better training conditions may not be able to complete the training tasks in time. Some may have the problem of not fully utilizing system resources, which leads to low model training efficiency and low model accuracy. In the embodiment of the present application, the target number of training times of the data holder in the next communication round is dynamically determined based on the training reporting data actually completed by the data holder in the previous training time period (i.e., the current communication round). For example, if the data holder completes fewer rounds of training in the current communication round, indicating that its training conditions are poor, then its target number of training times in the next communication round is also reduced accordingly to ensure that it successfully completes the training task. If the data holder completes more rounds of training in the current communication round, indicating that its training conditions are better, then its target number of training times in the next communication round is increased accordingly to make full use of the data holder's system resources, thereby improving the model training efficiency and model accuracy.

[0107] In one embodiment, based on Figure 2 The embodiment shown, see Figure 4 In this embodiment, the training report data also includes the loss value of the local model of the data holder in the current communication round. This embodiment involves how the server determines the target number of training times of the data holder in the next communication round based on the training report data. Figure 4 As shown, step 202 includes Figure 4 Step 301, step 302 and step 303 shown:

[0108] Step 301: The server compares the actual number of training times with the target number of training times in the current communication round to obtain a comparison result.

[0109] Step 302: The server determines the model loss change value obtained by the data holder after the current communication round ends.

[0110] As described above, the target number of training times in the current communication round is the target number of training times for the data holder to iteratively train the local model in the current communication round. When the server determines the target number of training times for the data holder in the next communication round, it first compares the actual number of training times with the target number of training times in the current communication round to obtain a comparison result. The comparison result may include: the actual number of training times is less than the target number of training times in the current communication round, or the actual number of training times is greater than or equal to the target number of training times in the current communication round.

[0111] It can be understood that if the actual number of training times is less than the target number of training times in the current communication round, it means that the data holder has not completed the training task assigned by the server in the current time period and has fallen behind. Therefore, the server will reduce the target number of training times for the data holder in the next communication round, that is, it will reduce the training task of the data holder in the next communication round; and if the actual number of training times is greater than or equal to the target number of training times in the current communication round, it means that the data holder has completed the training task assigned by the server in the current time period. Therefore, the server will increase the target number of training times for the data holder in the next communication round.

[0112] Next, under each comparison result (the actual number of training sessions is less than the target number of training sessions in the current communication round, or the actual number of training sessions is greater than or equal to the target number of training sessions in the current communication round), the server needs to determine the corresponding number of training sessions that needs to be reduced or increased.

[0113] In the embodiment of the present application, the number of training times that needs to be reduced and the number of training times that needs to be increased are determined based on the change in the loss value of the data holder after the current communication round. Therefore, the server first determines the model loss change value obtained by the data holder after the current communication round, and the model loss change value is the difference between the loss value of the local model of the data holder in the current communication round and the loss value of the local model of the data holder in the previous communication round.

[0114] As an implementation method, the loss function of the local model may be a cross-entropy loss function, and the data holder may obtain a cross-entropy loss value each time the local model is trained. In the embodiment of the present application, the loss value obtained by the data holder in iteratively training the local model in the current communication round may refer to the cross-entropy loss value obtained by the data holder in the last training of the local model in the current communication round; the loss value obtained by the data holder in iteratively training the local model in the previous communication round may refer to the cross-entropy loss value obtained by the data holder in the last training of the local model in the previous communication round.

[0115] The server uses the cross entropy loss value obtained in the current communication round to subtract the cross entropy loss value obtained in the previous communication round to obtain the model loss change value.

[0116] Step 303: The server obtains the target number of training times in the next communication round according to the actual number of training times, the comparison result, and the model loss change value.

[0117] After the server obtains the comparison results between the actual number of trainings, the target number of trainings in the current communication round corresponding to the current communication round, and the model loss change value, it can determine the corresponding number of trainings that need to be reduced or the number of trainings that need to be increased under each comparison result. Then, the server subtracts the number of trainings that need to be reduced from the actual number of trainings or adds the number of trainings that need to be increased to the actual number of trainings to obtain the target number of trainings in the next communication round.

[0118] In a possible implementation of step 303, the server may perform the following steps A1 and A2 to implement the process of step 303:

[0119] Step A1, if the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, the server reduces the actual number of training times to obtain the target number of training times in the next communication round, and the reduction in the number of training times is positively correlated with the change in model loss.

[0120] The above-mentioned reduction in the number of training times refers to the number of training times reduced by the server from the actual number of training times.

[0121] Step A2: If the comparison result is that the actual number of training times is greater than or equal to the number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in model loss.

[0122] The above-mentioned increase in the number of training times refers to the number of training times increased by the server to the actual number of training times.

[0123] See the following formula for calculating the target number of training sessions:

[0124]

[0125] in, is the target number of training times for data holder k in the next communication round (the next communication round corresponds to t+1 round of training), is the actual number of training times of data holder k in the current communication round (the current communication round corresponds to t rounds of training), is the target number of training times corresponding to the data holder k in the current communication round, is the loss value obtained by iteratively training the local model in the current communication round, It is the loss value obtained by iteratively training the local model in the previous communication round (the previous communication round corresponds to t-1 round of training).

[0126] In this way, when the actual number of training times is less than the target number of training times in the current communication round, the server uses the first formula above to substitute the model loss change value and the actual number of training times into the first formula to obtain the target number of training times in the next communication round. It can be understood that in the first formula, the reduction in the number of training times is:

[0127]

[0128] It can be seen from the first formula that when the actual number of training times is less than the target number of training times in the current communication round:

[0129] 1) If Data holder k did not complete the target number of training times in the current communication round, indicating that compared with the previous communication round, the loss value obtained by data holder k in training the local model in the current communication round has increased, which means that the training effect of the local model has deteriorated. The larger the value of , that is, the greater the change in model loss, the worse the training effect of the local model, and the greater the number of training times that need to be reduced (that is, the reduction in the number of training times); The smaller the value of , that is, the smaller the change in model loss, the smaller the number of training times that need to be reduced (that is, the reduction in the number of training times) is.

[0130] 2) If Data holder k has not completed the target number of training times in the current communication round, indicating that compared with the previous communication round, the loss value obtained by data holder k in training the local model in the current communication round has become smaller, which means that although data holder k has not completed the target number of training times in the current communication round, the training effect of the local model has improved. The smaller the value of is, the smaller the change in model loss is, the better the training effect of the local model is, and the smaller the number of training times that need to be reduced (i.e., the reduction in the number of training times) is. On the contrary, The larger the value of , that is, the greater the change in model loss, the more the number of training times needs to be reduced.

[0131] To sum up, when the actual number of training times is less than the target number of training times in the current communication round, the greater the change in model loss, the greater the number of training times that need to be reduced, and the smaller the change in model loss, the smaller the number of training times that need to be reduced, that is, the reduction in the number of training times is positively correlated with the change in model loss.

[0132] When the actual number of training times is greater than or equal to the target number of training times in the current communication round, the server uses the second formula above to substitute the model loss change value and the actual number of training times into the second formula to obtain the target number of training times in the next communication round. It can be understood that in the second formula, the increase in the number of training times is:

[0133]

[0134] It can be seen from the second formula that when the actual number of training times is greater than or equal to the target number of training times in the current communication round:

[0135] 1) If Although data holder k has completed the target number of training times in the current communication round, the loss value obtained by data holder k in training the local model in the current communication round is larger than that in the previous communication round, which means that the training effect of the local model is worse. The larger the value (i.e. The smaller the value of , that is, the greater the change in model loss, the worse the training effect of the local model, and the smaller the number of training times that need to be increased (that is, the increase in the number of training times); The smaller the value (i.e. The larger the value of ), that is, the smaller the change in model loss, the larger the number of training times that need to be increased (that is, the increase in the number of training times) is.

[0136] 2) If Data holder k has completed the target number of training times in the current communication round, and compared with the previous communication round, the loss value obtained by data holder k in training the local model in the current communication round has become smaller, which means that the training effect of the local model has improved. The smaller the value (i.e. The larger the value of , that is, the smaller the change in model loss, the better the training effect of the local model, and the greater the number of training times that need to be increased (that is, the increase in the number of training times). Conversely, The larger the value (i.e. The smaller the value of ), that is, the greater the change in model loss, the smaller the number of training times required.

[0137] To sum up, when the actual number of training times is greater than or equal to the target number of training times in the current communication round, the greater the change in model loss, the smaller the number of training times that need to be increased, and the smaller the change in model loss, the greater the number of training times that need to be reduced, that is, the increase in the number of training times is negatively correlated with the change in model loss.

[0138] In this way, when the server determines the target number of training times for the data holder in the next communication round, in addition to the actual number of training times completed by the data holder in the current time period, it also needs to consider the change in the loss value of the data holder's training of the local model: if the data holder does not complete the target number of training times in the current time period, and the model loss change value becomes larger, then the number of training times reduced will be larger; if the data holder does not complete the target number of training times in the current time period, but the model loss change value becomes smaller, indicating that the training effect is still optimistic, then the number of training times reduced will be smaller; if the data holder can complete the target number of training times in the current time period, but the model loss change value becomes larger, then the number of training times increased will be smaller; if the data holder can complete the target number of training times in the current time period, and the model loss change value becomes smaller, indicating that the training effect is very good, then the number of training times increased will be larger, thereby improving the flexibility and rationality of setting the target number of training times, which is conducive to making full use of the training capabilities of each data holder and improving the efficiency of model training.

[0139] In one embodiment, based on Figure 2 The embodiment shown, see Figure 5 The model training method of this embodiment may further include step 401:

[0140] Step 401: The server sends the time length corresponding to the next communication round to the data holder.

[0141] In an embodiment of the present application, the server may also set different time lengths for each communication round, wherein the setting of the time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0142] The amount of computing resources represents the computing power of the data holder. The larger the amount of computing resources, the shorter the communication round will be. The amount of sample data corresponding to the local model refers to the amount of samples owned by the data holder for participating in federated training. The larger the amount of sample data, the longer the communication round will be. The amount of model parameters of the local model may refer to the number of model parameters of the local model. The larger the amount of model parameters, the more complex the local model is, and the longer the communication round will be. The better the communication quality, the shorter the communication round will be.

[0143] The server combines the computing resources of each data holder, the amount of sample data corresponding to the local model, the model parameters of the local model, and the communication quality, and sets the time length corresponding to the communication round of each data holder in a targeted manner, and then allocates the target number of training times based on the set time length, further improving the resource utilization of each data holder, which is conducive to improving the training efficiency of the federated model.

[0144] The server sends the time length corresponding to the next communication round to the data holder. After the data holder receives the time length corresponding to the next communication round sent by the server, it iteratively trains the local model in the next communication round according to the target number of training times in the next communication round.

[0145] Of course, in other possible embodiments, the server may also set a fixed time length for each data holder during the first training process, and directly reuse the time length in the subsequent model training process, without making specific restrictions here.

[0146] In one embodiment, based on Figure 2 The embodiment shown, see Figure 6 The training report data of this embodiment also includes the model parameters obtained by the data holder after iteratively training the local model in the current communication round. The model training method of this embodiment also includes steps 501 and 502:

[0147] Step 501: The server updates the parameters of the global model according to the model parameters to obtain updated model parameters.

[0148] After the data holder iteratively trains the local model in the current communication round, the data holder obtains the model parameters, and sends the model parameters and the actual number of training times completed by iteratively training the local model in the current communication round as training report data to the server.

[0149] After receiving the training report data, in addition to determining the target number of training times for the data holder in the next communication round based on the actual number of training times, the server also needs to update the parameters of the global model based on the model parameters to obtain the updated model parameters. That is, the server aggregates the local global model based on the received model parameters to complete the update of the global model.

[0150] As an implementation method, there are multiple data holders, that is, there are multiple data holders in the current communication round who train local models locally respectively. In this way, after the current communication round ends, the server can receive the training report data sent by each data holder. In this way, the server can perform statistical processing on the model parameters sent by each data holder to obtain updated model parameters, thereby realizing the process of step 501.

[0151] Exemplarily, the server may average the model parameters sent by each data holder to obtain updated model parameters.

[0152] Step 502: The server sends the updated model parameters to the data holder.

[0153] The updated model parameters are used to instruct the data holder to train the local model in the next communication round according to the updated model parameters and the target training number in the next communication round.

[0154] The server will also send the target training number of the data holder in the next communication round determined based on the actual training number to the data holder. After the data holder receives the updated model parameters and the target training number in the next communication round sent by the server, the data holder will train the local model according to the updated model parameters and the target training number in the next communication round.

[0155] After the next communication round, the data holder sends the trained model parameters and the actual number of training times completed by the data holder in iteratively training the local model in the next communication round to the server, and so on, until the model converges, thus achieving flexible allocation of training tasks.

[0156] In one embodiment, based on Figure 2 The embodiment shown, see Figure 7 , the model training method of this embodiment further includes step 601 before step 201:

[0157] Step 601: The server sends the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder.

[0158] So that the data holder uses the initialized model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0159] As described above, the initialization model is a federated model with parameter initialization; the time length corresponding to the initial communication round is the first training time set by the server for the data holder, and the setting of the time length is related to at least one of the computing resources of the data holder, the sample data amount corresponding to the local model, the model parameter amount of the local model, and the communication quality corresponding to the data holder.

[0160] The initial training times are the target training times set by the server for the data holder during the first training process, i.e., the first communication round. During non-first training, the target training times corresponding to the communication round are determined based on the actual training times of the previous round. The difference is that the initial training times can be determined based on factors such as the sample data volume and computing resources of each data holder to ensure that all data holders have the opportunity to effectively train local models.

[0161] After the data holder receives the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder, the initialization model is iteratively trained according to the initial training times within the initial communication round.

[0162] After the initial communication round, the data holder sends the trained model parameters and the actual number of training times completed by the data holder in iteratively training the local model in the initial communication round to the server, and so on, until the model converges, thus achieving flexible allocation of training tasks.

[0163] In one embodiment, see Figure 8 , providing a model training method for a data holder, the method comprising:

[0164] Step 701: The data holder sends training report data to the server.

[0165] The training reporting data includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round.

[0166] Step 702: The data holder receives the target number of training times of the data holder in the next communication round sent by the server.

[0167] The target number of training times in the next communication round is determined by the server based on the training report data.

[0168] Step 703: The data holder iteratively trains the local model in the next communication round according to the target number of training times in the next communication round.

[0169] In one embodiment, before step 703, the model training method of this embodiment further includes:

[0170] Step B1, the data holder receives the time length corresponding to the next communication round sent by the server;

[0171] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0172] In one embodiment, the training report data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round. The model training method of this embodiment also includes:

[0173] Step B2: The data holder receives the updated model parameters sent by the server. The updated model parameters are obtained by the server updating the parameters of the local model according to the model parameters.

[0174] Further, the data holder may train the local model according to the updated model parameters and the target number of training times in the next communication round to implement the process of step 703 .

[0175] In one embodiment, before sending the training report data to the server, the model training method of this embodiment further includes:

[0176] Step B3, the data holder receives the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder sent by the server;

[0177] Step B4: The data holder iteratively trains the initialization model according to the initial training times in the initial communication round.

[0178] The specific implementation methods and beneficial effects of the model training method applied to the data holder are similar to the specific implementation methods and beneficial effects of the model training method of the server applied above. Please refer to the relevant records of the model training method of the server applied above, and no further details will be given here.

[0179] In one embodiment, Fig. 9 As shown, a model training method is provided, including:

[0180] Step 801: The data holder sends training report data to the server.

[0181] Among them, the training reporting data includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round, the loss value of the local model of the data holder in the current communication round, and the model parameters obtained after the data holder iteratively trains the local model in the current communication round.

[0182] Before step 801, the server may also send the initialization model, the time length corresponding to the initial communication round, and the initial training number corresponding to the data holder to the data holder; the data holder receives the initialization model, the time length corresponding to the initial communication round, and the initial training number corresponding to the data holder sent by the server; the data holder uses the initialization model as the local model in the initial communication round, and iteratively trains the initialization model according to the initial training number.

[0183] Step 802: The server receives training report data sent by the data holder.

[0184] Step 803: The server compares the actual number of training times with the target number of training times in the current communication round, obtains a comparison result, and determines a model loss change value of the data holder.

[0185] Among them, the target number of training times in the current communication round is the target number of training times for the data holder to iteratively train the local model in the current communication round, and the model loss change value is the difference between the loss value of the local model of the data holder in the current communication round and the loss value of the local model of the data holder in the previous communication round.

[0186] Step 804: If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, the actual number of training times is reduced to obtain the target number of training times in the next communication round, and the reduction in the number of training times is positively correlated with the change in the model loss.

[0187] Step 805, if the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, then the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss.

[0188] The target number of training times in the next communication round is the target number of training times for the data holder to iteratively train the local model in the next communication round.

[0189] Step 806: The server performs statistical processing on the model parameters sent by each data holder to obtain updated model parameters.

[0190] Step 807: The server sends the target number of training times in the next communication round and the updated model parameters to the data holder.

[0191] Among them, the server can also send the time length corresponding to the next communication round to the data holder; the time length is related to at least one of the computing resources of the data holder, the sample data amount corresponding to the local model, the model parameter amount of the local model and the communication quality corresponding to the data holder.

[0192] Step 808: The data holder receives the target number of training times and updated model parameters of the data holder in the next communication round sent by the server.

[0193] Step 809 : The data holder trains the local model according to the updated model parameters and the target number of training times in the next communication round.

[0194] In this way, since different data holders participating in the local model training often have different training conditions, such as different amounts of computing resources, sample data volumes, network conditions, etc., there are large differences in the number of target training times that different data holders can complete within a given time. The related technology assigns the same and fixed number of tasks to different data holders. Then, data holders with poor training conditions may not be able to complete the training tasks in time, and data holders with good training conditions may have the problem of not fully utilizing system resources, which leads to low model training efficiency. In the embodiment of the present application, the data holder is in the next communication The target number of training rounds is dynamically determined based on the training reporting data actually completed by the data holder in the previous training period (i.e., the current communication round). For example, if the data holder completes fewer rounds of training in the current communication round, it means that its training conditions are poor. In this case, the target number of training rounds will be reduced accordingly in the next communication round to ensure that it successfully completes the training task. If the data holder completes more rounds of training in the current communication round, it means that its training conditions are good. In this case, the target number of training rounds will be increased accordingly in the next communication round to make full use of the data holder's system resources, thereby improving the model training efficiency.

[0195] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0196] In one embodiment, Fig.10 As shown, a model training device is provided, which is arranged on a server, and the device comprises:

[0197] The receiving module 901 is used to receive the training report data sent by the data holder, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0198] A determination module 902, configured to determine a target number of training times of the data holder in the next communication round according to the training reporting data;

[0199] The sending module 903 is used to send the target number of training times in the next communication round to the data holder.

[0200] In one embodiment, the training report data also includes the loss value of the local model of the data holder in the current communication round; the determination module 902 includes:

[0201] A comparing unit, used for comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0202] a determining unit, configured to determine a model loss change value of the data holder, wherein the model loss change value is a difference between a loss value of a local model of the data holder in the current communication round and a loss value of a local model of the data holder in a previous communication round;

[0203] An acquisition unit is used to acquire the target number of training times in the next communication round according to the actual number of training times, the comparison result and the model loss change value.

[0204] In one of the embodiments, the acquisition unit is specifically used to, if the comparison result is that the actual training number is less than the target training number in the current communication round, reduce the actual training number to obtain the target training number in the next communication round, and the reduction in the training number is positively correlated with the change value of the model loss; if the comparison result is that the actual training number is greater than or equal to the target training number in the current communication round, increase the actual training number to obtain the target training number in the next communication round, and the increase in the training number is negatively correlated with the change value of the model loss.

[0205] In one of the embodiments, the sending module 903 is further used to send the time length corresponding to the next communication round to the data holder;

[0206] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0207] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the device also includes:

[0208] A parameter updating module, used to update the parameters of the local model according to the model parameters to obtain updated model parameters;

[0209] The sending module 903 is also used to send the updated model parameters to the data holder, and the updated model parameters are used to instruct the data holder to train the local model in the next communication round according to the updated model parameters and the target training number in the next communication round.

[0210] In one of the embodiments, there are multiple data holders, and the parameter updating module is specifically used to perform statistical processing on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0211] In one of the embodiments, the sending module 903 is also used to send the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder, so that the data holder iteratively trains the initialization model according to the initial training times within the initial communication round.

[0212] For the specific definition of the model training device provided on the server, please refer to the definition of the model training method for the server in the above text, which will not be repeated here. Each module in the above-mentioned model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the server in the form of hardware, or can be stored in the memory in the server in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0213] In one embodiment, Fig.11 As shown, a model training device is provided, which is arranged in a data holder, and the device comprises:

[0214] A sending module 1001 is used to send training report data to a server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0215] A receiving module 1002 is used to receive a target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0216] The training module 1003 is used to iteratively train the local model in the next communication round according to the target training times in the next communication round.

[0217] In one of the embodiments, before iteratively training the local model according to the target number of training times in the next communication round, the receiving module 1002 is further used to receive a time length corresponding to the next communication round sent by the server;

[0218] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0219] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the receiving module 1002 is further used to receive updated model parameters sent by the server, and the updated model parameters are obtained by the server updating the parameters of the local model according to the model parameters;

[0220] The training module 1003 is specifically used to train the local model according to the updated model parameters and the target number of training times in the next communication round.

[0221] In one embodiment, before sending the training report data to the server, the receiving module 1002 is further used to receive the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder;

[0222] The training module 1003 is further configured to use the initialization model as a local model in the initial communication round, and iteratively train the local model according to the initial training times.

[0223] For the specific limitations on the model training device provided in the data holder, please refer to the limitations on the model training method for the data holder mentioned above, which will not be repeated here. Each module in the above-mentioned model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the data holder in the form of hardware, or can be stored in the memory in the data holder in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0224] Fig.12 It is a structural diagram of the communication device provided in an embodiment of the present application. Fig.12 The communication device 1100 shown includes: at least one processor 1101, a memory 1102, and at least one network interface 1104. The various components in the communication device 1100 are coupled together via a bus system 1105. It is understood that the bus system 1105 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1105 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 1105 is not described in detail. Fig.12Various buses are labeled as bus system 1105. In addition, in the embodiment of the present application, a transceiver 1106 is also included. The transceiver 1106 can be multiple components, that is, including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium.

[0225] It can be understood that the memory 1102 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 1102 of the system and method described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory.

[0226] In some implementations, the memory 1102 stores the following elements, executable modules or data structures, or their subsets, or their extensions: operating system 1103. The operating system 1103 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks.

[0227] In an embodiment of the present application, by calling the program or instruction stored in the memory 1102, the receiver is used to receive the training report data sent by the data holder, and the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; the processor is used to determine the target number of training times of the data holder in the next communication round according to the training report data; the transmitter is used to send the target number of training times in the next communication round to the data holder.

[0228] Part or all of the methods disclosed in the above embodiments of the present application may also be applied to the processor 1101, or implemented by the processor 1101, or implemented by the processor 1101 in cooperation with other components (such as a transceiver). The processor 1101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware or software instructions in the processor 1101. The above processor 1101 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1102, and the processor 1101 reads the information in the memory 1102 and completes the steps of the above method in combination with its hardware.

[0229] It is understood that the embodiments described in the embodiments of the present application can be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0230] For software implementation, the technology described in the embodiments of the present application can be implemented by a module (such as a process, function, etc.) that performs the functions described in the embodiments of the present application. The software code can be stored in a memory and executed by the processor 1101. The memory can be implemented in the processor 1101 or outside the processor 1101.

[0231] In one embodiment, the training reporting data also includes the loss value of the local model of the data holder in the current communication round; the processor is specifically used to compare the actual number of training times with the target number of training times in the current communication round, obtain a comparison result, and determine the model loss change value of the data holder, the model loss change value being the difference between the loss value of the local model of the data holder in the current communication round and the loss value of the local model of the data holder in the previous communication round; according to the actual number of training times, the comparison result and the model loss change value, the target number of training times in the next communication round is obtained.

[0232] In one embodiment, the processor is specifically configured to reduce the actual number of training times to obtain the target number of training times in the next communication round if the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, and the reduction amount of the number of training times is positively correlated with the change value of the model loss;

[0233] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0234] In one implementation, the transmitter is further configured to send the time length corresponding to the next communication round to the data holder;

[0235] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0236] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the processor is further used to update the parameters of the local model according to the model parameters to obtain updated model parameters;

[0237] The transmitter is also used to send the updated model parameters to the data holder, and the updated model parameters are used to instruct the data holder to train the local model in the next communication round according to the updated model parameters and the target training number in the next communication round.

[0238] In one embodiment, there are multiple data holders, and the processor is specifically configured to perform statistical processing on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0239] In one embodiment, the transmitter is also used to send the initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0240] Alternatively, in an embodiment of the present application, by calling the program or instruction stored in the memory 1102, the transmitter is used to send training reporting data to the server, and the training reporting data includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; the receiver is used to receive the target training times of the data holder in the next communication round sent by the server, and the target training times in the next communication round is determined by the server based on the training reporting data; the processor is used to iteratively train the local model in the next communication round according to the target training times in the next communication round.

[0241] In one embodiment, the receiver is further configured to receive a time length corresponding to the next communication round sent by the server;

[0242] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0243] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the receiver is further used to receive updated model parameters sent by the server, where the updated model parameters are obtained by the server updating the parameters of the local model according to the model parameters;

[0244] The processor is specifically configured to train the local model according to the updated model parameters and the target number of training times in the next communication round.

[0245] In one embodiment, the receiver is further used to receive the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder;

[0246] The processor is further configured to use the initialization model as a local model in the initial communication round, and iteratively train the local model according to the initial training times.

[0247] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0248] Receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0249] Determining a target number of training times for the data holder in the next communication round according to the training reporting data;

[0250] The target number of training times in the next communication round is sent to the data holder.

[0251] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0252] Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0253] Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in a previous communication round;

[0254] The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result and the model loss change value.

[0255] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0256] If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change value of the model loss;

[0257] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0258] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0259] Sending the time length corresponding to the next communication round to the data holder;

[0260] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0261] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the computer program, when executed by the processor, further implements the following steps:

[0262] Update the parameters of the local model according to the model parameters to obtain updated model parameters;

[0263] The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and a target number of training times in the next communication round.

[0264] In one embodiment, the number of the data holders is multiple, and when the computer program is executed by the processor, the following steps are also implemented:

[0265] Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0266] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0267] The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0268] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0269] Sending training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0270] receiving a target training number of the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0271] In the next communication round, the local model is iteratively trained according to the target training number in the next communication round.

[0272] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0273] Receive the time length corresponding to the next communication round sent by the server;

[0274] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0275] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the computer program, when executed by the processor, further implements the following steps:

[0276] Receiving updated model parameters sent by the server, where the updated model parameters are obtained by the server updating parameters of the local model according to the model parameters;

[0277] The iterative training of the local model according to the target training times in the next communication round includes:

[0278] The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

[0279] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0280] Receiving the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder;

[0281] In the initial communication round, the initialization model is used as the local model, and the local model is iteratively trained according to the initial training number.

[0282] Fig.13 It is a schematic structural diagram of the chip of an embodiment of the present application. Fig.13 The chip 1200 shown includes a processor 1210, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0283] Alternatively, if Fig.13 As shown, the chip 1200 may further include a memory 1220. The processor 1210 may call and run a computer program from the memory 1220 to implement the method in the embodiment of the present application.

[0284] The memory 1220 may be a separate device independent of the processor 1210 , or may be integrated into the processor 1210 .

[0285] Optionally, the chip 1200 may further include an input interface 1230. The processor 1210 may control the input interface 1230 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0286] Optionally, the chip 1200 may further include an output interface 1240. The processor 1210 may control the output interface 1240 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0287] Optionally, the chip 1200 can be applied to the network element device in the embodiment of the present application, and the chip 1200 can implement the corresponding processes implemented in the various methods in the embodiment of the present application, which will not be described here for the sake of brevity.

[0288] It should be understood that the chip 1200 mentioned in the embodiment of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0289] The present application also provides a computer program product including instructions, including a computer program, which implements the following steps when executed by a processor:

[0290] Receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0291] Determining a target number of training times for the data holder in the next communication round according to the training reporting data;

[0292] The target number of training times in the next communication round is sent to the data holder.

[0293] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0294] Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result;

[0295] Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in a previous communication round;

[0296] The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result and the model loss change value.

[0297] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0298] If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change value of the model loss;

[0299] If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

[0300] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0301] Sending the time length corresponding to the next communication round to the data holder;

[0302] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0303] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the computer program, when executed by the processor, further implements the following steps:

[0304] Update the parameters of the local model according to the model parameters to obtain updated model parameters;

[0305] The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and a target number of training times in the next communication round.

[0306] In one embodiment, the number of the data holders is multiple, and when the computer program is executed by the processor, the following steps are also implemented:

[0307] Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

[0308] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0309] The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

[0310] The present application also provides a computer program product including instructions, including a computer program, which implements the following steps when executed by a processor:

[0311] Sending training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round;

[0312] receiving a target training number of the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data;

[0313] In the next communication round, the local model is iteratively trained according to the target training number in the next communication round.

[0314] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0315] Receive the time length corresponding to the next communication round sent by the server;

[0316] The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

[0317] In one embodiment, the training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the computer program, when executed by the processor, further implements the following steps:

[0318] Receiving updated model parameters sent by the server, where the updated model parameters are obtained by the server updating parameters of the local model according to the model parameters;

[0319] The iterative training of the local model according to the target training times in the next communication round includes:

[0320] The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

[0321] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0322] Receiving the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder;

[0323] In the initial communication round, the initialization model is used as the local model, and the local model is iteratively trained according to the initial training number.

[0324] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0325] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0326] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A model training method, characterized in that: Applied to a server, the method comprises: Receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; Determining a target number of training times for the data holder in the next communication round according to the training reporting data; The target number of training times in the next communication round is sent to the data holder.

2. The method according to claim 1, characterized in that The training report data also includes a loss value of the local model of the data holder in the current communication round; and determining a target number of training times of the data holder in the next communication round according to the training report data includes: Comparing the actual number of training times with the target number of training times in the current communication round to obtain a comparison result; Determine a model loss change value of the data holder, where the model loss change value is a difference between a loss value of the local model of the data holder in the current communication round and a loss value of the local model of the data holder in a previous communication round; The target number of training times in the next communication round is obtained according to the actual number of training times, the comparison result and the model loss change value.

3. The method according to claim 2, characterized in that The step of obtaining the target number of training times in the next communication round according to the actual number of training times, the comparison result, and the model loss change value includes: If the comparison result is that the actual number of training times is less than the target number of training times in the current communication round, then reducing the actual number of training times to obtain the target number of training times in the next communication round, and the amount of reduction in the number of training times is positively correlated with the change value of the model loss; If the comparison result is that the actual number of training times is greater than or equal to the target number of training times in the current communication round, the actual number of training times is increased to obtain the target number of training times in the next communication round, and the increase in the number of training times is negatively correlated with the change in the model loss value.

4. The method according to claim 1, characterized in that: The method further comprises: Sending the time length corresponding to the next communication round to the data holder; The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

5. The method according to claim 1, characterized in that The training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method also includes: Update the parameters of the global model according to the model parameters to obtain updated model parameters; The updated model parameters are sent to the data holder, where the updated model parameters are used to instruct the data holder to train a local model in the next communication round according to the updated model parameters and a target number of training times in the next communication round.

6. The method according to claim 5, characterized in that The number of the data holders is multiple, and the updating of the global model parameters according to the model parameters to obtain the updated model parameters includes: Statistical processing is performed on the model parameters sent by each of the data holders to obtain the updated model parameters.

7. The method according to claim 1, characterized in that Before receiving the training report data sent by the data holder, the method further includes: The initialization model, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder are sent to the data holder, so that the data holder uses the initialization model as the local model in the initial communication round and iteratively trains the local model according to the initial training times.

8. A model training method, characterized in that: Applied to a data holder, the method comprises: Sending training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; receiving a target training number of the data holder in a next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data; In the next communication round, the local model is iteratively trained according to the target training number in the next communication round.

9. The method according to claim 8, characterized in that Before iteratively training the local model according to the target number of training times in the next communication round, the method further includes: Receive the time length corresponding to the next communication round sent by the server; The time length is related to at least one of the amount of computing resources of the data holder, the amount of sample data corresponding to the local model, the amount of model parameters of the local model, and the communication quality corresponding to the data holder.

10. The method according to claim 8, characterized in that The training reporting data also includes model parameters obtained by the data holder after iteratively training the local model in the current communication round, and the method also includes: Receiving updated model parameters sent by the server, wherein the updated model parameters are obtained by the server updating parameters of the global model according to the model parameters; The iterative training of the local model according to the target training times in the next communication round includes: The local model is trained according to the updated model parameters and the target number of training times in the next communication round.

11. The method according to claim 8, characterized in that Before sending the training report data to the server, the method further includes: Receiving the initialization model sent by the server, the time length corresponding to the initial communication round, and the initial training times corresponding to the data holder; In the initial communication round, the initialization model is used as the local model, and the local model is iteratively trained according to the initial training number.

12. A model training device, characterized in that: Set on the server, the device includes: A receiving module, configured to receive training report data sent by a data holder, wherein the training report data includes at least an actual number of training times completed by the data holder in iteratively training the local model in the current communication round; A determination module, used to determine a target number of training times of the data holder in the next communication round according to the training reporting data; A sending module is used to send the target number of training times in the next communication round to the data holder.

13. A model training device, characterized in that: Set at the data holder, the device includes: A sending module, used to send training report data to a server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; A receiving module, used to receive a target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data; The training module is used to iteratively train the local model in the next communication round according to the target number of training times in the next communication round.

14. A communication device, characterized in that: include: transmitter, processor, and receiver; The receiver is used to receive training report data sent by the data holder, where the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; The processor is used to determine a target number of training times of the data holder in a next communication round according to the training reporting data; The transmitter is used to send the target training times in the next communication round to the data holder.

15. A communication device, characterized in that: include: transmitter, processor, and receiver; The transmitter is used to send training report data to the server, wherein the training report data at least includes the actual number of training times completed by the data holder in iteratively training the local model in the current communication round; The receiver is used to receive the target training number of the data holder in the next communication round sent by the server, where the target training number in the next communication round is determined by the server according to the training reporting data; The processor is used to iteratively train the local model in the next communication round according to the target training number in the next communication round.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images