Model training method, device, system and storage medium

Abstract

The application discloses a model training method, device, system and storage medium. The method comprises the following steps: in the process of federated training of a first local model based on local business sample data and in combination with other second participants, the first local model is subjected to reinforcement feedback learning training based on the relevant evaluation conditions of a corresponding global model and / or the first local model evaluated by an evaluation model issued by a coordinator, so as to obtain a target model; the reinforcement feedback learning training makes the preset evaluation indicators of the first local model after training and the second local model after training of the second participant have a difference not greater than a preset difference; wherein, the global model is associated with the first local model and the second local model; and the target business prediction model is used for performing prediction processing on the to-be-processed business data to obtain a business prediction result. The application aims to improve the joint training effect of different participants in a federated scenario.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of finance, and particularly relates to a model training method, device and system and a storage medium. BACKGROUND

[0002] With the development of financial technology, various large models are emerging. However, due to the different basic models, training data and model adjustment methods used by different institutions, enterprises or individuals, the performances of models of different institutions, or models of different fields or different versions of the same institution are inconsistent (such as inconsistent understanding ability, tone style, etc.), which leads to poor joint training effect of different participants in a federal scenario. SUMMARY

[0003] Therefore, the embodiments of the present application provide a model training method, device, system and storage medium, aiming at solving the problem of poor joint training effect of different participants in a federal scenario in the related art.

[0004] The embodiments of the present application provide a model training method applied to a first participant, and the method comprises:

[0005] In the process of federated training of a first local model based on local business sample data and in combination with other second participants, a corresponding global model and / or the first local model is subjected to relevant evaluation based on an evaluation model issued by a coordinator, and the first local model is subjected to reinforcement feedback learning training to obtain a target model; wherein the reinforcement feedback learning training makes the first local model after training and a second local model after training of the second participant have a difference in a preset evaluation index not greater than a preset difference.

[0006] The global model is associated with the first local model and the second local model; and the target business prediction model is used to perform prediction processing on the to-be-processed business data to obtain a business prediction result when receiving the to-be-processed business data.

[0007] In a possible implementation of the present application, the evaluation model is obtained by training a preset initial model based on an evaluation training data set, the evaluation training data set is generated based on a prediction situation when a plurality of third-party evaluation corresponding models are used to predict a corresponding preset test data, and corresponding second evaluation results tend to be consistent, the evaluation model uses the same model part of each participant as the preset initial model.

[0008] The application also provides a model training method. In the process of federated training of a first local model based on local business sample data and in combination with other second participants, the first local model is subjected to reinforcement feedback learning training based on an evaluation model issued by a coordinator, the evaluation model being used to evaluate the relevant evaluation situation of a corresponding global model and / or the first local model, so as to obtain a target model, comprising the following steps:

[0009] Training the first local model based on local business sample data to obtain a trained first local model;

[0010] Sending the first model parameter of the trained first local model to the coordinator, so that the coordinator aggregates the first model parameter and a second model parameter corresponding to the second participant to obtain a global model, wherein the second model parameter is a model parameter of a trained second local model;

[0011] Receiving the global model and the evaluation model sent by the coordinator, so as to determine a target score based on the evaluation model, a global prediction result and an intermediate prediction result, wherein the global model predicts a result generated by the business sample data as the global prediction result, and the trained first intermediate model predicts a result generated by the business sample data as the intermediate prediction result;

[0012] Based on the target score, iteratively training the first local model to obtain the target model.

[0013] In a possible implementation of the application, the step of iteratively training the first local model based on the target score to obtain the target model comprises the following steps:

[0014] Based on the target score, determining a loss value of a loss function corresponding to the reinforcement feedback learning training;

[0015] Based on the loss value of the loss function corresponding to the reinforcement feedback learning training, returning to the step of federated training of the trained first local model based on the local business sample data and in combination with other second participants to obtain the target model.

[0016] In a possible implementation of the application, the evaluation model comprises a reward model, wherein the reward model is obtained by iteratively training a corresponding initial model based on an evaluation training data set having a preset score label, and the preset score label is used to represent the prediction effect of a corresponding local model on corresponding test data in the evaluation training data set.

[0017] In a possible implementation of the present application, if the evaluation model is a reward model, the step of training the first local model based on the evaluation of the corresponding global model and / or the first local model by the evaluation model issued by the coordinator to obtain a target model includes:

[0018] determining the score of the first local model based on the reward model issued by the coordinator;

[0019] adjusting the first related loss function value corresponding to the reinforcement feedback learning training based on the first reward data determined based on the score, to train the first local model based on the adjusted first related loss function value to obtain the target model.

[0020] In a possible implementation of the present application, the evaluation model includes a classification model, wherein the classification model is obtained by iteratively training a corresponding initial model based on an evaluation training data set with preset classification labels, and the preset classification labels are used to represent the prediction ranking of different models in the evaluation training data set on corresponding test data.

[0021] In a possible implementation of the present application, if the evaluation model is a classification model, the step of training the first local model based on the evaluation of the corresponding global model and / or the first local model by the evaluation model issued by the coordinator to obtain a target model includes:

[0022] determining the comparison between the corresponding global model and the first local model based on the classification model issued by the coordinator;

[0023] adjusting the second related loss function value corresponding to the reinforcement feedback learning training based on the second reward data determined based on the comparison, to train the first local model based on the adjusted second related loss function value to obtain the target model.

[0024] In a possible implementation of the present application, the first related loss function value or the second related loss function value is adjusted by a preset learning intensity.

[0025] In a possible implementation of the present application, after the step of training the first local model based on the evaluation of the corresponding global model and / or the first local model by the evaluation model issued by the coordinator to obtain a target model, the method further includes:

[0026] when receiving to-be-processed business data, performing prediction processing on the to-be-processed business data based on the trained target model to obtain a business prediction result.

[0027] The application also provides a model training method, applied to a coordination party, and the method comprises:

[0028] determining preset test data;

[0029] After a plurality of third parties evaluate the prediction of the corresponding model on the preset test data, if it is determined that the corresponding evaluation results tend to be consistent, generating an evaluation training data set based on the prediction corresponding to the evaluation results that tend to be consistent;

[0030] training an evaluation model based on the evaluation training data set;

[0031] issuing the evaluation model to each participant, so that each participant performs training of reinforcement feedback learning on the corresponding local model based on the relevant evaluation of the corresponding global model and / or the corresponding local model by the evaluation model;

[0032] The global model is associated with the local model of each participant.

[0033] In a possible implementation of the application, the step of training an evaluation model based on the evaluation training data set comprises:

[0034] iteratively training a preset initial model based on the evaluation training data set to obtain an evaluation model;

[0035] The same model part of each participant is used as the preset initial model.

[0036] The application also provides a model training device, applied to a first participant, and the device comprises:

[0037] A first training module is configured to perform training of reinforcement feedback learning on the first local model based on the relevant evaluation of the corresponding global model and / or the first local model by an evaluation model issued by a coordination party during the process of federated training of the first local model by the first participant based on local business sample data in combination with other second participants, to obtain a target model; wherein the training of reinforcement feedback learning makes the preset evaluation indicators of the first local model after training and the second local model after training of the second participant have a difference not greater than a preset difference.

[0038] The global model is associated with the first local model and the second local model; and the target business prediction model is configured to perform prediction processing on the to-be-processed business data to obtain a business prediction result when receiving the to-be-processed business data.

[0039] The application also provides a model training device, applied to a coordination party, and the device comprises:

[0040] The first determining module is configured to determine preset test data.

[0041] The second determining module is configured to, after the multiple third-party evaluation models evaluate the prediction of the preset test data, if it is determined that the corresponding evaluation results tend to be consistent, generate an evaluation training data set based on the prediction corresponding to the evaluation results that tend to be consistent.

[0042] The second training module is configured to train an evaluation model based on the evaluation training data set.

[0043] The issuing module is configured to issue the evaluation model to each participant, so that each participant performs reinforcement feedback learning training on the corresponding local model based on the evaluation model and the relevant evaluation of the corresponding global model and / or the corresponding local model.

[0044] The global model is associated with the local model of each participant.

[0045] The present application also provides a model training device, which is an entity node device. The model training device comprises a memory, a processor, and a program of the model training method stored in the memory and executable on the processor. When the program of the model training method is executed by the processor, the steps of the model training method described above can be implemented.

[0046] To achieve the above object, a storage medium is also provided, which stores a model training program. When the model training program is executed by a processor, the steps of any of the above model training methods are implemented.

[0047] The application provides a model training method, device, system and storage medium. Compared with the poor joint training effect of different participants in the prior art under a federal scenario, in the application, during the process of federal training, the first local model is trained through reinforcement feedback learning based on the relevant evaluation of the evaluation model of the corresponding global model (the global model is associated with the first local model and the second local model) and / or the first local model issued by the coordinator, and the relevant evaluation. The difference between the preset evaluation indicators of the first local model trained based on the relevant evaluation and the second local model trained by the second participant is not greater than a preset difference. It can be understood that the evaluation model evaluates the model prediction of the corresponding global model and / or the first local model, and after evaluation, the first participant trains the first local model through reinforcement feedback learning based on the relevant evaluation (it is obvious that the global model is associated with the first local model and the second local model, that is, the global model integrates the conditions of other models. When the data of different parties is greatly different or the models are heterogeneous, the global model integrates the conditions of other models. Therefore, even if the data of some models is greatly different or the models are heterogeneous, the influence degree or adjustment pace of the current model will not be too large. In addition, in the embodiment, the global model is not directly used to replace the local model for subsequent training, but the relevant evaluation is obtained based on the evaluation model. Therefore, the first local model adjusts its local model based on better evaluation results), further, in the embodiment, the reinforcement feedback learning training makes the difference between the preset evaluation indicators of different parties not greater than the preset difference, that is, the reinforcement feedback learning training adjusts some evaluation indicators of different parties through the relevant evaluation of the evaluation model, which needs to meet the requirement that the difference is not greater than the preset difference, that is, in the embodiment, a common training target (reinforcement feedback learning training) is added on the basis of the local training target of each party through the global model and / or the evaluation model. The common training target needs to meet the requirement that the difference is not greater than the preset difference, that is, in the embodiment, a new constraint (not only the constraint of the local training task, but also the new constraint) is added. The constraint essentially controls the adjustment pace of different models. Even if the data of a participant is particularly large, without this constraint, the first participant needs to make a large adjustment, but due to this constraint, it is not allowed to make a large adjustment (equivalent to limiting the adjustment range), and thus the corresponding model will not be excessively deviated from abnormal data, thereby improving the federal modeling effect. In the embodiment, the problem of poor federal modeling effect caused by the large difference between the behaviors of different models due to the large difference in the data of the models or the model heterogeneity is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 Flowchart of a first embodiment of the model training method of the present application;

[0049] Figure 2 Flowchart of a second embodiment of the model training method of the present application;

[0050] Figure 3 Flowchart of a third embodiment of the model training method of the present application;

[0051] Figure 4 Device structure diagram of the hardware running environment involved in the embodiment of the present application;

[0052] Figure 5 Flowchart of a fourth embodiment of the model training method of the present application;

[0053] Figure 6 System architecture diagram of the horizontal federated learning system involved in the present application;

[0054] Figure 7 Schematic diagram of the participants A and B implementing encrypted sample alignment involved in the embodiment of the present application;

[0055] Figure 8 System architecture diagram of the vertical federated learning system involved in the present application;

[0056] Figure 9 Schematic diagram of the overall scheme framework involved in the embodiment of the present application;

[0057] Figure 10 Structure block diagram of an embodiment of the model training device involved in the present application. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein merely serve to explain the present application and are not intended to limit the present application.

[0059] Reference Figure 1 , Figure 1 Flowchart of a first embodiment of the model training method of the present application.

[0060] In the first embodiment, the model training method comprises the following steps:

[0061] Step S10, in the process of federated training of the first local model based on the local business sample data and in combination with other second participants, the first local model is subjected to reinforcement feedback learning training based on the evaluation model issued by the coordinator to evaluate the relevant evaluation of the corresponding global model and / or the first local model, to obtain a target model; wherein the reinforcement feedback learning training makes the pre-set evaluation index of the first local model after training and the second local model after training of the second participant not more than a pre-set gap.

[0062] The global model is associated with the first local model and the second local model; and the target business prediction model is used to perform prediction processing on the to-be-processed business data to obtain a business prediction result.

[0063] It should be noted that the application scenario in this embodiment can be that the models of several institutions have large differences in data or different model structures, resulting in poor federated modeling effect.

[0064] For example, in the era of large models, due to different model bases, training data and model adjustment methods used by institutions, enterprises and individuals, the performances (understanding ability, tone style) of models of different institutions and models of different fields or different versions of the same institution are inconsistent.

[0065] For example, there are now 8 institutions (participants), institution 1 is the first participant, and the other 7 institutions are the second participants. Due to large data differences between institution 1 and institution 3, different model structures between institution 2 and institution 4, and large data differences between institution 4 and institution 5, etc., when the 8 institutions perform federated modeling, there is a problem of poor effect.

[0066] It should be noted that the execution subject of the method of this embodiment is any first participant, which can be an institution, an enterprise or a company, and the present application does not make specific limitations.

[0067] It should be understood that the second participant can also be an institution, an enterprise or a company.

[0068] It should be understood that there are 2 or more than 2 second participants.

[0069] It can be understood that different participants can perform horizontal federated learning or vertical federated learning.

[0070] In this embodiment, the concept of federated learning (including vertical federated learning and horizontal federated learning) is first described.

[0071] (1) Federated learning refers to a method of machine learning that involves different participants (or parties, also known as data owners, clients, or model parties) working together. In federated learning, participants (model parties) do not need to expose their data to other participants and the coordinator (also known as parameter servers, aggregation servers, or coordinators), thus federated learning can effectively protect user privacy and ensure data security.

[0072] Horizontal federated learning involves extracting the data from participants with largely overlapping features but minimal user overlap, and then performing joint machine learning on the portion of data where participants share similar features but users are not entirely identical. For example, consider two banks in different regions, each with its own customer base and minimal overlap. However, their businesses are very similar, and their recorded user data features are largely the same. Horizontal federated learning can be used to help these two banks build a joint model to predict their customer behavior.

[0073] like Figure 6 In the illustrated horizontal federated learning system architecture, step ① is as follows: After participant A completes the model parameter update locally, participant A can send the updated model parameters to the coordinator. Participant A can send the model parameter update to the coordinator in an encrypted manner, for example, using homomorphic encryption. The model parameters can be parameters of the federated learning model, such as the weight parameters of the connections between nodes in a neural network; or the joint model parameters can be gradient information of the federated learning model, such as the gradient information in the gradient descent algorithm of a neural network. Step ② is as follows: The coordinator merges the received model parameter updates from different participants, for example, by calculating a weighted average. Step ③ is as follows: The coordinator redistributes the merged model parameter updates (also called global model parameters) to each participant. The coordinator can also transmit the global model parameters in an encrypted manner. Step ④ is as follows: Participants can use the received global model parameters as the starting point for local model training or as the latest model parameters for their local model, in order to start training or continue training the model locally.

[0074] Participants and coordinators iterate through the above steps until the loss function converges, the model parameters converge, the maximum number of iterations is reached, or the maximum training time is reached, thus completing the entire model training process.

[0075] Longitudinal federated learning is a kind of federated learning in which the data characteristics of participants overlap less, and the users overlap more. For example, there are two participants A and B belonging to the same region, where participant A is a bank and participant B is an e-commerce platform. Participants A and B have more same users in the same region, but A and B have different businesses and record different user data characteristics. In particular, the user data characteristics recorded by A and B can be complementary. In such a scenario, longitudinal federated learning can be used to help A and B build a joint machine learning prediction model to help A and B provide better services to customers.

[0076] As shown in the longitudinal federated learning system architecture, Figure 8 To help A and B jointly model, a coordinator C is needed. The first part: participants A and B implement encrypted sample alignment, as shown in Figure 7 Since the user groups of the two enterprises A and B do not completely coincide, the system uses an encrypted user sample alignment technology to confirm the common users of A and B without disclosing their respective data, and does not expose the users that do not overlap with each other, so as to jointly model the features of these users.

[0077] The second part: encrypted model training. After determining the common user group, a machine learning model can be trained using these data. To ensure the confidentiality of the data during training, the coordinator C is needed to perform encrypted training. Taking a linear regression model as an example, the training process can be divided into the following four steps. Step ①: The coordinator C distributes public keys to A and B to encrypt the data that needs to be exchanged during training. Step ②: Participants A and B exchange intermediate results for calculating gradients in an encrypted form. Step ③: Participants A and B calculate based on the encrypted gradient values, while participant B calculates the loss function based on its label data and aggregates the results to the coordinator C. The coordinator C calculates the total gradient value by aggregating the results and decrypts it. Step ④: The coordinator C returns the decrypted gradient to participants A and B, and participants A and B update the parameters of their respective models based on the gradient.

[0078] Participants and coordinators iterate the above steps until the loss function converges or the model parameters converge or the maximum number of iterations is reached or the maximum training time is reached, thus completing the entire model training process.

[0079] In this embodiment, it should be noted that the different participants can have a relationship in which the data characteristics overlap less and the users overlap more, or a relationship in which the data characteristics overlap more and the users overlap less.

[0080] It should be noted that the first participant jointly trains the first local model with other second participants based on local business sample data (where the trained model is a large model).

[0081] The first participant jointly trains the first local model with other second participants based on local business sample data through the coordinator.

[0082] It should be understood that the coordinator is a coordinator, and the relationship between the coordinator, the first participant and the second participant is as shown in Figure 9 .

[0083] In this embodiment, the business sample data can be credit data, and the first participant jointly trains the first local model with other second participants based on local credit data to obtain a target model for predicting whether a corresponding user can be given credit when receiving other credit data to be processed.

[0084] Or in this embodiment, the business sample data can be car loan data, and the first participant jointly trains the first local model with other second participants based on local car loan data to obtain a target model for predicting whether a corresponding user can be given a car loan when receiving car loan data to be processed.

[0085] As shown in Figure 9 , the framework relationship between the global model, the first local model and the evaluation model is shown, and based on the global model, the first local model and the evaluation model, a target model can be obtained.

[0086] The coordinator determines the preset test data; then after a plurality of third parties evaluate the prediction of the corresponding model on the preset test data, if it is determined that the corresponding evaluation results tend to be consistent, based on the prediction corresponding to the evaluation results that tend to be consistent, an evaluation training data set is generated (for example, model 1 and model 2 answer A question, corresponding to answer 1 and answer 2, 5 third parties think that the answer 2 corresponding to model 2 is better, then the prediction is that answer 2 is better, that is, a piece of data in the evaluation training data set can be generated); after obtaining the evaluation training data set, the evaluation model is trained based on the evaluation training data set; the evaluation model is distributed to each participant, so that each participant can perform relevant evaluation of the corresponding global model and / or corresponding local model based on the evaluation model, and perform reinforcement feedback learning training on the corresponding local model; wherein the global model is associated with the local model corresponding to each participant.

[0087] Specifically, as shown in Figure 9As shown, the answers of the target data set are evaluated by the multi-model under the federal scenario, and the evaluation results of the multiple models are screened to obtain evaluation results with high consistency / significance, which are used as the evaluation training data set. Based on the evaluation training data set, the evaluation model for a specific scenario is trained by the coordinator using a parameter-efficient training method. After obtaining the evaluation model, the evaluation model is distributed to each participant; each participant locally performs reinforcement feedback training based on the evaluation model (the first local model of the first participant is trained by full fine-tuning or efficient parameter fine-tuning; the corresponding coordinator aggregation model; in this process, the evaluation training data set has been screened, and the evaluation model is trained; the coordinator distributes the evaluation model and the global model to each participant; each participant performs reinforcement feedback learning based on the evaluation model and the global model).

[0088] Specifically, in this embodiment, it should be understood that the evaluation model is used to evaluate which model's answer is better. For example, after the first participant answers question A, answer 1 is obtained, and after the second participant answers question A, answer 2 is obtained. The evaluation model can evaluate which of answer 1 and answer 2 is a better answer and output an evaluation result, for example, the evaluation model outputs [question, answer 1, answer 2, evaluation result (1 is better)], or the evaluation model outputs [question, answer 1, answer 2, evaluation result (2 is better)], or the evaluation model outputs [question, answer 1, answer 2, evaluation result (tie)].

[0089] Alternatively, it can be understood that the evaluation model is used to evaluate whether the answer of the corresponding model is good. For example, after the first participant answers question A, answer 1 is obtained. The evaluation model can evaluate whether answer 1 is good and output an evaluation result, for example, the evaluation model outputs [question, answer 1, evaluation result (good)], or the evaluation model outputs [question, answer 1, evaluation result (bad)].

[0090] As shown in Figure 7 The evaluation model is obtained by training a preset initial model based on the evaluation training data set. The evaluation model uses the same model part of each participant (which is small and improves efficiency) as the preset initial model, for example, the preset initial model is a small model (low model complexity and small calculation amount), which can improve transmission efficiency.

[0091] It can be understood that the evaluation training data set is generated based on the prediction of the corresponding model for the corresponding preset test data using multiple third-party evaluations when the corresponding second evaluation results tend to be consistent.

[0092] For example, there are 2 large models, which answer the target question to obtain answer 1 and answer 2, and other 6 large models evaluate the answers of the above 2 large models, and the corresponding second evaluation results tend to be consistent, and the corresponding second evaluation results tend to be consistent all think that answer 1 is better, or at least three quarters think that answer 1 is better, so the corresponding second evaluation results tend to be consistent (with high significance and consistency), and the evaluation training data set is generated based on the prediction of the consistent situation, and one piece of data in the evaluation training data set can be: [question, answer 1, answer 2, evaluation result (1 better)].

[0093] Or for example, for example, there is 1 large model, which answers the target question to obtain answer 1, and other 6 large models evaluate the answer of the above one large model, and the corresponding second evaluation results tend to be consistent, and the corresponding second evaluation results tend to be consistent all think that answer 1 is good, or at least three quarters think that answer 1 is good, so the corresponding second evaluation results tend to be consistent (with high significance and consistency), and the evaluation training data set is generated based on the prediction of the consistent situation, and one piece of data in the evaluation training data set can be: [question, answer 1, answer 2, evaluation result (1 good)].

[0094] Based on this, the evaluation model can determine the relevant evaluation situation of the corresponding global model and / or the first local model (such as who is better, or whether a certain model is good).

[0095] It can be understood that the global model parameters corresponding to the global model are jointly determined by the first model parameters corresponding to the first local model and the second model parameters corresponding to the second local model.

[0096] It can be understood that the global model can also be a pre-trained base model.

[0097] It can be understood that in this embodiment, based on the relevant evaluation situation determined by the evaluation model, the first local model is trained for reinforcement feedback learning to obtain a target model.

[0098] Or it can be understood that in this embodiment, through the global model and / or the evaluation model, a common training target is added on the basis of the local training target of each party.

[0099] Overall, in this embodiment, the global model affects the corresponding first local model based on the relevant evaluation situation, and the global model also affects the corresponding second local model of other participants, and then the global model can make the pre-set evaluation index of the first local model after training and the pre-set evaluation index of the second local model after training of the second participant not greater than a pre-set gap.

[0100] Or it can be understood that the training of different participant models can be adjusted through a global model, so that the difference between the preset evaluation indicators (parameter distribution of each model) of the trained different corresponding models is not greater than the preset difference.

[0101] In this embodiment, the relevant evaluation conditions determined by the evaluation model can be used to adjust the loss value corresponding to the loss function of the reinforcement feedback learning training, so as to affect the corresponding first local model.

[0102] It can be understood that the preset evaluation indicator can be a model parameter indicator or an indicator for model interpretation, and the specific implementation is not limited.

[0103] It can be understood that the preset evaluation indicator can be the parameter distribution of each model.

[0104] The global model parameters corresponding to the global model are determined by aggregating the first model parameters corresponding to the first local model and the second model parameters corresponding to the second local model.

[0105] In the process of federated training of the first local model based on local business sample data and other second participant, the relevant evaluation conditions of the corresponding global model and / or the first local model based on the evaluation model issued by the coordinator are used to perform reinforcement feedback learning training on the first local model to obtain the target model. The step comprises:

[0106] Training the first local model based on local business sample data to obtain a trained first local model;

[0107] The first model parameters of the trained first local model are sent to the coordinator for the coordinator to aggregate the first model parameters and the second model parameters corresponding to other second participants, to obtain a global model, wherein the second model parameters are model parameters of the trained second local model;

[0108] The global model and the evaluation model sent by the coordinator are received to determine a target score based on the evaluation model, the global prediction result, and the intermediate prediction result, wherein the global model predicts the result generated by the business sample data as the global prediction result, and the trained first intermediate model predicts the result generated by the business sample data as the intermediate prediction result.

[0109] Based on the target score, the first local model is iteratively trained to obtain the target model.

[0110] In this embodiment, it is emphasized that the global model is not a substitute for any local model of the participants, which is a distinguishing point of the present application from other federated learning (the difference between the preset evaluation indicators of different models can be made not greater than a preset difference).

[0111] In this embodiment, after obtaining the global model, the global model will make a prediction based on the business sample data to obtain a global prediction result.

[0112] Similarly, the trained local model will also make a prediction based on the business sample data to obtain an intermediate prediction result.

[0113] For example, the global model will answer the target question to obtain answer 3, and the trained first local model will also answer the target question to obtain answer 4.

[0114] Then, the evaluation model can determine a target score based on the global prediction result and the intermediate prediction result, determine a loss value of a corresponding loss function according to the target score, and use the loss value to adjust the corresponding trained local model to continue iterative training of the first local model to obtain the target model.

[0115] For example, the first participant determines a target score based on answer 3 and answer 4, and continues iterative training of the first local model to obtain the target model;

[0116] The step of continuing iterative training of the first local model based on the target score to obtain the target model includes:

[0117] Determining a loss value of a reinforcement feedback learning training corresponding loss function based on the target score;

[0118] Based on the loss value of the reinforcement feedback learning training corresponding loss function, the step of returning the local-based business sample data to jointly train the trained first local model with other second participants to obtain the target model.

[0119] It can be understood that if the target score can update the loss value (the trained first local model adjusts the corresponding vector expression or matrix expression based on the loss value), and correspondingly adjusts the model parameters of the trained first local model.

[0120] In this embodiment, the evaluation model is also used to constrain that the difference between the global model and the corresponding trained first local model cannot be too large (preset difference degree, etc.).

[0121] In this embodiment, since the evaluation model is also used to constrain that the global model cannot be too different from the corresponding trained second local model (constrained by the corresponding loss function of the reinforcement feedback learning), the pace of model adjustment of different participants is not too large, but tends to be consistent, so as to avoid the problem of poor training effect that may be caused during the federated training of the model.

[0122] The application provides a model training method, device, system and storage medium. Compared with the poor joint training effect of different participants in the prior art under a federal scenario, in the application, during the process of federal training, the first local model is trained based on the evaluation model issued by the coordinator and the relevant evaluation of the corresponding global model (the global model is associated with the first local model and the second local model) and / or the first local model. The relevant evaluation is used for training the first local model based on the reinforcement feedback learning. The difference between the preset evaluation indicators of the first local model after the training and the second local model after the corresponding training of the second participant is not greater than a preset difference. It can be understood that the evaluation model evaluates the model prediction of the corresponding global model and / or the first local model, and after the evaluation, the first participant trains the first local model based on the relevant evaluation for reinforcement feedback learning (it is obvious that the global model is associated with the first local model and the second local model, that is, the global model integrates the conditions of other models. When the data of different parties is greatly different or the models are heterogeneous, the global model integrates the conditions of other models. Therefore, even if the data of some models is greatly different or the models are heterogeneous, the influence degree or adjustment pace of the current model will not be too large. In addition, in the embodiment, the global model is not directly used to replace the local model for subsequent training, but the relevant evaluation is obtained based on the evaluation model. Therefore, the first local model adjusts its local model based on better evaluation results). Further, in the embodiment, the reinforcement feedback learning training makes the difference between the preset evaluation indicators of different parties not greater than the preset difference, that is, the reinforcement feedback learning training adjusts the evaluation indicators of different parties through the relevant evaluation of the evaluation model, and needs to meet the requirement that the difference is not greater than the preset difference, that is, in the embodiment, a common training target (reinforcement feedback learning training) is added on the basis of the local training target of each party through the global model and / or the evaluation model. The common training target needs to meet the requirement that the difference is not greater than the preset difference, that is, in the embodiment, a new constraint (not only the constraint of the local training task, but also the new constraint) is added. The constraint essentially controls the adjustment pace of different models. Even if the data of a participant is particularly large, without the constraint, the first participant needs to make a large adjustment. However, due to the constraint, the first participant is not allowed to make a large adjustment (equivalent to limiting the adjustment range), and the corresponding model will not be excessively deviated from the abnormal data, thereby improving the federal modeling effect. In the embodiment, the problem of poor federal modeling effect caused by the large difference between the data of different models or the large difference between the behaviors of different models is avoided.

[0123] As Figure 2As shown, the second embodiment of the model training method is proposed based on the first embodiment. In this embodiment, if the evaluation model is a reward model, step S10 specifically includes:

[0124] Step S11, based on the reward model issued by the coordinator, determining the score of the first local model;

[0125] In this embodiment, it should be noted that the evaluation model can be a reward model.

[0126] The reward model is obtained by iteratively training the corresponding initial model based on the evaluation training data set with the preset score label (better answer, higher score), and the preset score label is used to represent the prediction effect (such as good or bad) of the corresponding local model on the corresponding test data in the evaluation training data set.

[0127] Specifically, one training method of the reward model is based on the InstructGPT method, which allows different models to enter the evaluation model with two answers to the same question, and the evaluation model gives a score. The loss value of a certain loss function is calculated based on the scores of the two answers, and the better answer should have a higher score. In this way, a reward model can be trained, wherein the loss function can be:

[0128] Loss(loss(θ)=-E (x,yj,yk) ~D[log(σ(r θ (x,yj)-r θ (x,yk)))]);

[0129] Where r is the score given by the evaluation model, yj is the better answer, and the higher the score, the better the answer.

[0130] Step S12, based on the first reward data determined based on the score, adjusting the first related loss function value of the reinforcement feedback learning training, to train the first local model based on the adjusted first related loss function value. Reinforcement feedback learning training to obtain a target model.

[0131] It should be noted that in this embodiment, based on the score, the score of the corresponding prediction result of the first local model corresponding to the first participant (the score of the first participant's answer 1) is determined, and based on the score and the first related loss function value corresponding to the preset reinforcement feedback learning training, the first local model is trained. Reinforcement feedback learning training to obtain a target model, wherein the first related loss function value can be specifically:

[0132] Lr1=PPO(r(xt, yp))-βKL(yr, yp), wherein PPO is a PPO algorithm, xt is a question, yp is an answer M(xt) generated by the first local model after training, yr is an output answer of the global model (a pre-trained base model). KL is a KL-divergence, used to constrain that the difference between the model output of the global model and the first local model after training cannot be too large.

[0133] The overall embodiments are described below:

[0134] The first participant has a first local model M and an evaluation model R, and the target task / pre-training task of the first participant is Dt=(Xt, Yt), wherein Yt is a label answering a corresponding question, and Xt is a question;

[0135] wherein the target task loss function is Lt=L(M(xt), yt);

[0136] And the reinforcement feedback task loss function is:

[0137] Lr1=PPO(r(xt, yp))-βKL(yr, yp), wherein PPO is a PPO algorithm, xt is a question, yp is an answer M(xt) generated by the first local model after training, yr is an output answer of the global model (a pre-trained base model). KL is a KL-divergence, used to constrain that the difference between the model output of the global model and the first local model after training cannot be too large.

[0138] In this embodiment, the total loss function is Ltotal=Lt+βLr1, wherein β is a hyperparameter, controlling the strength of reinforcement feedback learning.

[0139] In this embodiment, the first local model is trained by reinforcement feedback learning through the value of the reinforcement feedback task loss function of the reward model, to obtain a target model, avoiding the problem that the federal modeling effect is not good due to large differences in data or large differences in behaviors of different models caused by model heterogeneity.

[0140] As Figure 3 shown, based on the first embodiment, the third embodiment of the model training method of the present application is proposed, and in this embodiment, if the evaluation model is a classification model, the step S10 comprises:

[0141] Step S13, based on the classification model issued by the coordinator, determining the comparison between the corresponding global model and the first local model;

[0142] In this embodiment, it should be noted that the evaluation model can be a classification model.

[0143] The classification model is obtained by iteratively training a corresponding initial model based on an evaluation training data set with preset classification labels, and the preset classification labels are used to represent the prediction ranking of different models on the corresponding test data (for example, which model has a better answer).

[0144] That is, it can be understood that the training method of the classification model can obtain two answers (answers output by different models) at the same time and give the better result (which answer is better) it thinks. Compared with the reward model, the classification model can fully utilize the ability of the pre-trained model and obtain two answers of different models. The evaluation ability can improve the granularity of the model.

[0145] Specifically, if the prompt information is received:

[0146] The two answers are evaluated, and it is determined which answer is better in terms of helpfulness, relevance, accuracy, and detail level. The output of the evaluation model can be '1' or '2'.

[0147] That is, in this application, the output of the evaluation model contains only one value in a line, indicating which answer is better. '1' indicates that answer 1 is better, and '2' indicates that answer 2 is better. If the results are similar or cannot be judged, '0' is output. The purpose of using '0' is to ensure that a single token output can cover the target result and reduce the complexity of the algorithm.

[0148] In this embodiment, when the classification model is used for evaluation, '0', '1', and '2' probabilities are obtained, and after normalization, normalized probabilities p0, p1, and p2 are obtained (the sum of the three is 1).

[0149] In this embodiment, the evaluation result is converted into a classification problem, and a cross-entropy loss function is used.

[0150] In step S14, based on the second reward data determined according to the comparison, the second related loss function value of the reinforcement feedback learning training is adjusted, and the first local model is trained based on the adjusted second related loss function value to obtain a target model.

[0151] It should be noted that in this embodiment, the second reward data is determined based on the comparison (which answer is better), that is, the second reward data r = r(xt, yp, yc) can be calculated based on the comparison of the answers. According to the second reward data, the second related loss function value of the reinforcement feedback learning training is adjusted, and the first local model is trained based on the adjusted second related loss function value to obtain a target model.

[0152] The second correlation loss function value can be specifically:

[0153] Lr2=PPO(r(xt,yp,yc))-βKL(yr,yp),wherein PPO is a PPO algorithm, xt is a problem, yp is an answer M(xt) generated by the first local model after training, yr is an output answer of a global model (a pre-trained base model), yc is a comparison case (e.g., answer 1 is better than answer 2), and KL is a KL-divergence used to constrain that the difference between the model output of the global model and the first local model after training cannot be too large.

[0154] The overall embodiments are described below:

[0155] The first participant has a first local model M and an evaluation model R, and a target task / pre-training task Dt=(Xt,Yt).

[0156] A target task loss function is Ltt=L(M(xt),yt).

[0157] In this embodiment, the global model and the corresponding first local model after training generate a comparison answer yc, and based on the comparison answer, a reward r=r(xt,yp,yc) can be calculated, where yp is an answer M(xt) generated by the first local model after training. Then, based on the reinforcement learning PPO algorithm, model parameter updating can be completed: Lr=PPO(r(xt,yp,yc))-βKL(yr,yp), where yr is an output answer of a global model (a pre-trained base model). KL is a KL-divergence used to constrain that the difference between the model output of the global model and the first local model after training cannot be too large.

[0158] A total loss function is Ltotal=Lt+βLr2, where β is a hyperparameter that controls the strength of reinforcement feedback learning.

[0159] In this embodiment, the first local model is trained through the reinforcement feedback task loss function of the classification model to obtain a target model, thereby avoiding the problem of poor federal modeling effect caused by large differences in different model data or large differences in behaviors of different models due to model heterogeneity.

[0160] As shown in Figure 5 The fourth embodiment of the model training method of the present application is proposed based on the above embodiments. In this embodiment, the method is applied to a coordinator, and the method comprises:

[0161] Step S40, determining preset test data;

[0162] Step S50, after the plurality of third-party evaluation models evaluate the prediction of the preset test data, if it is determined that the corresponding evaluation results tend to be consistent, generate an evaluation training data set based on the prediction corresponding to the evaluation results that tend to be consistent;

[0163] Step S60, based on the evaluation training data set, train an evaluation model;

[0164] Step S70, distribute the evaluation model to each participant, so that each participant performs reinforcement feedback learning training on the corresponding local model based on the evaluation model and / or the relevant evaluation of the corresponding global model and / or the corresponding local model;

[0165] The global model is associated with the local model of each participant.

[0166] Optionally, the step of training an evaluation model based on the evaluation training data set comprises:

[0167] iteratively training a preset initial model based on the evaluation training data set to obtain the evaluation model;

[0168] The same model part of each participant is used as the preset initial model (the same model part model is small, which improves efficiency).

[0169] The specific implementation of the model training apparatus of the present application is basically the same as that of the above-mentioned model training method, and will not be repeated here.

[0170] Referring to Figure 4 , Figure 4 is a device structure diagram of a hardware running environment involved in the embodiment scheme of the present application.

[0171] As Figure 4 shown, the model evaluation device can include a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection communication between the processor 1001 and the memory 1005.

[0172] Optionally, the model evaluation device can also include a user interface, a network interface, a camera, an RF (Radio Frequency, Radio Frequency) circuit, a sensor, a WiFi module, etc. The user interface can include a display screen (Display), an input sub-module such as a keyboard (Keyboard), and the optional user interface can also include a standard wired interface, a wireless interface. The network interface can include a standard wired interface, a wireless interface (such as a WI-FI interface).

[0173] Those skilled in the art can understand that Figure 4The model evaluation device structure shown in the figures is not intended to limit the model evaluation device, which can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0174] As shown in Figure 4 The memory 1005 as a storage medium can include an operating system, a network communication module, and a model training program. The operating system is a program that manages and controls the hardware and software resources of the model evaluation device, supports the running of the model training program and other software and / or programs. The network communication module is used to realize the communication between the cameras inside the memory 1005, and the communication with other hardware and software in the device.

[0175] In Figure 4 In the model evaluation device shown in the figures, the processor 1001 is used to execute the model training program stored in the memory 1005, and implement the steps of the model training method described in any of the above embodiments.

[0176] The model evaluation device specific embodiments of the present application are basically the same as the above-mentioned model training method embodiments, and will not be repeated here.

[0177] An embodiment of the present application provides a model training device, which is applied to a first participant and includes Figure 10

[0178] The first training module 10 is configured to perform reinforcement feedback learning training on the first local model based on the evaluation model issued by the coordinator during the process of federated training of the first local model based on local business sample data and in conjunction with other second participants, to obtain a target model, wherein the reinforcement feedback learning training makes the difference between the first local model after training and the second local model after training of the second participant not greater than a preset difference.

[0179] The global model is associated with the first local model and the second local model; and the target business prediction model is configured to perform prediction processing on the to-be-processed business data to obtain a business prediction result.

[0180] In a possible implementation of the present application, the evaluation model is obtained by training a preset initial model based on an evaluation training data set, the evaluation training data set is generated based on the prediction situation when the prediction situation of the corresponding model of a plurality of third parties is consistent with the corresponding second evaluation result, and the evaluation model uses the same model part of each participant as the preset initial model. ​

[0181] The application also provides a model training method, and the model training device is used for implementing:

[0182] training the first local model based on the local business sample data to obtain a trained first local model;

[0183] sending first model parameters of the trained first local model to a coordinator, so that the coordinator aggregates the first model parameters and second model parameters corresponding to a second participant to obtain a global model, wherein the second model parameters are model parameters of a trained second local model;

[0184] receiving the global model and the evaluation model sent by the coordinator, to determine a target score based on the evaluation model, a global prediction result and an intermediate prediction result, wherein the global model predicts a result generated by the business sample data as the global prediction result, and the trained first intermediate model predicts a result generated by the business sample data as the intermediate prediction result;

[0185] continuing to iteratively train the first local model based on the target score to obtain the target model.

[0186] In a possible implementation of the application, the model training device is used for implementing:

[0187] determining a loss value of a loss function corresponding to the reinforcement feedback learning training based on the target score;

[0188] returning the local business sample data based on the loss value of the loss function corresponding to the reinforcement feedback learning training, and jointly training the trained first local model with other second participants to obtain the target model.

[0189] In a possible implementation of the application, the evaluation model includes a reward model, wherein the reward model is obtained by iteratively training a corresponding initial model based on an evaluation training data set with a preset score label, and the preset score label is used to represent a prediction effect of a corresponding local model on corresponding test data in the evaluation training data set.

[0190] In a possible implementation of the application, the model training device is used for implementing:

[0191] determining a score of the first local model based on the reward model issued by the coordinator;

[0192] Based on the first reward data determined according to the score, a first related loss function value corresponding to the reinforcement feedback learning training is adjusted, so that the first local model is trained based on the adjusted first related loss function value to obtain a target model.

[0193] In a possible implementation of the present application, the evaluation model comprises a classification model, wherein the classification model is obtained by iteratively training a corresponding initial model based on an evaluation training data set with preset classification labels, and the preset classification labels are used to represent the prediction ranking of different models in the evaluation training data set on corresponding test data.

[0194] In a possible implementation of the present application, the model training apparatus is configured to implement:

[0195] Based on the classification model issued by the coordinator, the comparison between the corresponding global model and the first local model is determined;

[0196] Based on the second reward data determined according to the comparison, a second related loss function value corresponding to the reinforcement feedback learning training is adjusted, so that the first local model is trained based on the adjusted second related loss function value to obtain a target model.

[0197] In a possible implementation of the present application, the first related loss function value or the second related loss function value is adjusted by a preset learning intensity.

[0198] In a possible implementation of the present application, the model training apparatus is configured to implement:

[0199] When receiving the to-be-processed service data, the trained target model is used to predict and process the to-be-processed service data to obtain a service prediction result.

[0200] The specific embodiments of the model training apparatus of the present application are basically the same as those of the above-mentioned model training method, and will not be repeated here.

[0201] The present application also provides a model training apparatus applied to a coordinator, comprising:

[0202] A first determination module is configured to determine preset test data.

[0203] A second determination module is configured to, after determining the prediction of the corresponding model of a plurality of third parties on the preset test data, generate an evaluation training data set based on the prediction corresponding to the consistent evaluation results when the corresponding evaluation results tend to be consistent.

[0204] a second training module, configured to train an evaluation model based on the evaluation training data set;

[0205] a delivery module, configured to deliver the evaluation model to each participant, so that each participant performs reinforcement feedback learning training on the corresponding local model based on the evaluation model and the relevant evaluation situation of the corresponding global model and / or the corresponding local model;

[0206] The global model is associated with the local model of each participant.

[0207] The second training module is configured to implement:

[0208] iteratively train a preset initial model based on the evaluation training data set to obtain the evaluation model;

[0209] The model part of each participant is used as the preset initial model.

[0210] The specific implementation of the model training device of the present application is basically the same as that of the above-mentioned model training method, and will not be repeated here.

[0211] The present application also provides a model training system, comprising the first participant as described above and the coordinator as described above.

[0212] The specific implementation of the model training system of the present application is basically the same as that of the above-mentioned model training method, and will not be repeated here.

[0213] The embodiment of the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of the model training method of any one of the above.

[0214] The specific implementation of the storage medium of the present application is basically the same as that of the above-mentioned model training method, and will not be repeated here.

[0215] The present application also provides a computer program product, comprising a computer program, which is executed by a processor to implement the steps of the above-mentioned model training method.

[0216] The specific implementation of the computer program product of the present application is basically the same as that of the above-mentioned model training method, and will not be repeated here.

[0217] It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0218] The above-mentioned sequence numbers of embodiments of the present application are only for description, and do not represent advantages or disadvantages of the embodiments.

[0219] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of a software and hardware platform, or by hardware, but in many cases, the former is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM), and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0220] The above is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings of the present application, is also included in the patent protection scope of the present application.

Claims

1. A model training method, characterized in that, Applied to the first participant, the method includes: In the process of federated training of the first local model based on local business sample data and in conjunction with other second participants, the first local model is also trained with reinforcement feedback learning based on the evaluation model issued by the coordinator and the relevant evaluation of the corresponding global model and / or the first local model to obtain the target model; wherein, the reinforcement feedback learning training makes the difference between the first local model after training and the preset evaluation index of the corresponding second local model of the second participant not greater than the preset difference. The global model is associated with the first local model and the second local model; the target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, so as to obtain the business prediction result. The evaluation model is obtained by training a preset initial model based on the evaluation training dataset. The evaluation training dataset is a training dataset generated based on the prediction results when multiple third-party evaluation models predict the corresponding preset test data and the corresponding second evaluation results tend to be consistent during the process of obtaining the global model. The evaluation model uses the same model part of each participant as the preset initial model. The step of training the first local model in a federated manner based on local business sample data and in conjunction with other second participants, and further training the first local model using reinforcement feedback learning based on the evaluation model issued by the coordinator for the relevant evaluation of the corresponding global model and / or the first local model to obtain the target model, includes: Based on local business sample data, the first local model is trained to obtain the trained first local model. The first model parameters of the trained first local model are sent to the coordinator so that the coordinator can aggregate the first model parameters with the second model parameters corresponding to the other second participants to obtain the global model. The second model parameters are the model parameters of the trained second local model. The system receives the global model and the evaluation model sent by the coordinator, and determines the target score based on the evaluation model, the global prediction result, and the intermediate prediction result. The result generated by the global model predicting the business sample data is the global prediction result, and the result generated by the first intermediate model after training predicting the business sample data is the intermediate prediction result. Based on the target score, the first local model is iteratively trained to obtain the target model.

2. The model training method as described in claim 1, characterized in that, The step of iteratively training the first local model based on the target score to obtain the target model includes: Based on the target score, determine the loss value of the loss function corresponding to the reinforcement feedback learning training; Based on the loss value of the loss function corresponding to the reinforcement feedback learning training, the first local model is federatedly trained in conjunction with other second participants, based on local business sample data, to obtain the target model.

3. The model training method as described in claim 1, characterized in that, The evaluation model includes a reward model, wherein the reward model is obtained by iteratively training the corresponding initial model based on an evaluation training dataset with preset rating labels. The preset rating labels are used to characterize the prediction performance of the corresponding local model in the evaluation training dataset on the corresponding test data.

4. The model training method as described in claim 3, characterized in that, If the evaluation model is a reward model, the step of training the first local model to obtain the target model based on the evaluation results of the corresponding global model and / or the first local model using the evaluation model issued by the coordinator, and performing reinforcement feedback learning, includes: Based on the reward model issued by the coordinator, the score of the first local model is determined; Based on the first reward data determined by the scoring, the first relevant loss function value corresponding to the reinforcement feedback learning training is adjusted, and the first local model is trained for reinforcement feedback learning based on the adjusted first relevant loss function value to obtain the target model.

5. The model training method as described in claim 1, characterized in that, The evaluation model includes a classification model, wherein the classification model is obtained by iteratively training the corresponding initial model based on the evaluation training dataset with preset classification labels. The preset classification labels are used to characterize the prediction ranking of different models in the evaluation training dataset for the corresponding test data.

6. The model training method as described in claim 5, characterized in that, If the evaluation model is a classification model, the step of training the first local model to obtain the target model based on the evaluation results of the corresponding global model and / or the first local model based on the evaluation model issued by the coordinator includes: Based on the classification model issued by the coordinator, determine the comparison between the corresponding global model and the first local model; Based on the second reward data determined by the comparison, the value of the second relevant loss function corresponding to the reinforcement feedback learning training is adjusted. Based on the adjusted second relevant loss function value, the first local model is trained for reinforcement feedback learning to obtain the target model.

7. The model training method as described in any one of claims 4 or 6, characterized in that, The value of the first or second correlation loss function can be adjusted by setting the learning intensity.

8. The model training method as described in claim 1, characterized in that, After the step of training the first local model to obtain the target model based on the evaluation results of the corresponding global model and / or the first local model issued by the coordinator, the following steps are included: Upon receiving business data to be processed, the trained target model is used to perform prediction processing on the business data to obtain business prediction results.

9. A model training method, characterized in that, Applied to the coordinating party, the method includes: Determine the preset test data; After multiple third-party evaluations of the model's prediction performance on the preset test data, if it is determined that the corresponding evaluation results tend to be consistent, an evaluation training dataset is generated based on the prediction performance corresponding to the consistent evaluation results. Based on the aforementioned evaluation training dataset, an evaluation model is trained. The evaluation model is distributed to each participant so that each participant can train the corresponding local model for reinforcement feedback learning based on the evaluation results of the corresponding global model and / or the corresponding local model. The global model is associated with the local model corresponding to each participant; The step of training an evaluation model based on the evaluation training dataset includes: Based on the evaluation training dataset, the preset initial model is iteratively trained to obtain the evaluation model; Specifically, the same model portion is used by all participating parties as the preset initial model.

10. A model training device, characterized in that, Applied to the first participant, the device includes: The first training module is used to train the first local model through reinforcement feedback learning based on the evaluation model issued by the coordinator and the relevant evaluation results of the corresponding global model and / or the first local model during the federated training of the first local model in conjunction with other second participants, based on local business sample data. This training aims to obtain the target model. The reinforcement feedback learning training ensures that the difference between the trained first local model and the corresponding trained second local model of the second participant is not greater than a preset difference. The global model is associated with the first local model and the second local model; the target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, so as to obtain the business prediction result. The evaluation model is obtained by training a preset initial model based on the evaluation training dataset. The evaluation training dataset is a training dataset generated based on the prediction results when multiple third-party evaluation models predict the corresponding preset test data and the corresponding second evaluation results tend to be consistent during the process of obtaining the global model. The evaluation model uses the same model part of each participant as the preset initial model. The model training device is used to achieve: Based on local business sample data, the first local model is trained to obtain the trained first local model. The first model parameters of the trained first local model are sent to the coordinator so that the coordinator can aggregate the first model parameters with the second model parameters corresponding to the other second participants to obtain the global model. The second model parameters are the model parameters of the trained second local model. The system receives the global model and the evaluation model sent by the coordinator, and determines the target score based on the evaluation model, the global prediction result, and the intermediate prediction result. The result generated by the global model predicting the business sample data is the global prediction result, and the result generated by the first intermediate model after training predicting the business sample data is the intermediate prediction result. Based on the target score, the first local model is iteratively trained to obtain the target model.

11. A model training device, characterized in that, Applied to the coordinating party, the device includes: The first determining module is used to determine the preset test data; The second determining module is used to generate an evaluation training dataset based on the prediction results corresponding to the consistent evaluation results when multiple third-party evaluations of the corresponding model's prediction of the preset test data are determined to be consistent. The second training module is used to train an evaluation model based on the evaluation training dataset. The distribution module is used to distribute the evaluation model to each participant, so that each participant can train the corresponding local model for reinforcement feedback learning based on the evaluation of the corresponding global model and / or the corresponding local model based on the evaluation of the evaluation model. The global model is associated with the local model corresponding to each participant; The model training device is used to achieve: Based on the evaluation training dataset, the preset initial model is iteratively trained to obtain the evaluation model; Specifically, the same model portion is used by all participating parties as the preset initial model.

12. A model training system, characterized in that, This includes the first participant as described in claim 10 and the coordinator as described in claim 11.

13. A storage medium, characterized in that, The storage medium stores a model training program, which, when executed by a processor, implements the steps of the model training method as described in any one of claims 1 to 9; or the storage medium stores a model training program, which, when executed by a processor, implements the steps of the model training method as described in claim 11.

Images