Model training method and device, electronic equipment and computer program product

Abstract

The invention provides a model training method and device, electronic equipment and a computer program product, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring an intermediate model parameter combination obtained by training a to-be-trained model by a plurality of participating nodes; based on the intermediate model parameter combination and the multi-objective function, an optimal solution set of the multi-objective function is determined, the multi-objective function is constructed based on a global average loss function, a Bayesian optimization function, a model interpretability function and a robustness function, and the optimal solution set comprises a model parameter combination of the to-be-trained model. And determining a target model parameter combination based on the optimal solution set of the multi-objective function. And sending the target model parameter combination to the plurality of participating nodes, so that the plurality of participating nodes continue to train the to-be-trained model based on the target model parameter combination. The method is suitable for federated learning, and can select model parameters conforming to various targets in each iteration to carry out the next round of model training so as to achieve the data fusion effect of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a model training method, device, electronic device and computer program product. Background Art

[0002] With the development of information technology, data has become an important resource for innovation and development in various industries. However, in practical applications, data is stored in different organizations or institutions, forming "data islands", making it difficult to achieve direct data sharing and integration.

[0003] As a new distributed machine learning framework, the core idea of ​​federated learning is to allow multiple participating nodes to jointly train a global model through encryption and secure computing protocols without leaving the local data. This method can not only effectively utilize data resources scattered across the country, but also ensure the privacy and security of the data.

[0004] However, in the actual application of federated learning, due to its distributed training method and the possible differences in computing device performance and training data among multiple participating nodes, the characteristics of the model parameters trained by each participating node are different, which may affect the training of the model and the final fusion results of federated learning. Summary of the invention

[0005] The present application provides a model training method, device, electronic device and computer program product, which are suitable for federated learning and can select model parameters with better quality that meet multiple objectives in each iteration for the next round of model training, thereby optimizing the data fusion effect of federated learning.

[0006] In a first aspect, the present application provides a model training method, the method comprising: obtaining an intermediate model parameter combination obtained by training a model to be trained by multiple participating nodes. Based on the intermediate model parameter combination and the multi-objective function, an optimal solution set of the multi-objective function is determined, the multi-objective function is constructed based on a global average loss function, a Bayesian optimization function, a model interpretability function, and a robustness function, and the optimal solution set includes a model parameter combination of the model to be trained. Based on the optimal solution set of the multi-objective function, a target model parameter combination is determined. The target model parameter combination is sent to multiple participating nodes so that the multiple participating nodes continue to train the model to be trained based on the target model parameter combination.

[0007] The present application provides a model training method, which can construct a multi-objective function based on a global average loss function, a Bayesian optimization function, a model interpretability function, and a robustness function. Among them, the global average loss function can measure the overall prediction error of the model on the global data, the Bayesian optimization function can quickly find the optimal model parameter combination by searching the model parameter space through the Bayesian method, the model interpretability function can enhance the clarity and understandability of the model output results, make the model decision process easier to understand and accept, and the robustness function can improve the model's resistance to noise, outliers or data changes, and ensure the stability and reliability of the model in different environments. By constructing a multi-objective function through the above specific functions, the model training performance can be considered from different levels, and the characteristics of the node model training of all parties involved can be comprehensively considered. Compared with the prior art central node simply weighted summing the intermediate model parameter combination, the present application obtains the optimal solution set by solving the multi-objective function, and can find the target model parameters that can meet multiple screening angles from the intermediate model parameter combination. The selected target model parameter combination has a higher quality and is used for the next round of model training, which can improve the efficiency of model training and optimize the final fusion effect of federated learning.

[0008] A possible implementation method is that the optimal solution set includes multiple optimal solution sets, and the target model parameter combination is determined based on the optimal solution set of the multi-objective function, specifically including: based on multiple optimal solution sets, determining the confidence interval set and coefficient of variation corresponding to each optimal solution set, the confidence interval set includes the confidence interval corresponding to each model parameter in the corresponding optimal solution set, and the confidence interval is used to represent the value range of the corresponding model parameter under the preset confidence level. The coefficient of variation is used to represent the degree of discreteness between the prediction results of the to-be-trained model for multiple test samples under the model parameter combination corresponding to the optimal solution set. Based on the confidence interval set and coefficient of variation corresponding to each optimal solution set, the target model parameter combination is determined from multiple optimal solution sets.

[0009] Another possible implementation method is to determine the target model parameter combination from multiple optimal solution sets based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set, including: determining the confidence interval score corresponding to each optimal solution set based on the confidence interval set corresponding to each optimal solution set, the confidence interval score is obtained by averaging the reference confidence interval score corresponding to each model parameter in the optimal solution set, and the reference confidence interval score is negatively correlated with the width of the confidence interval of the corresponding model parameter. Based on the coefficient of variation corresponding to each optimal solution set, determine the coefficient of variation score corresponding to each optimal solution set, and the coefficient of variation score is negatively correlated with the coefficient of variation. Perform weighted summation on the coefficient of variation score and the confidence interval score to obtain the comprehensive score corresponding to each optimal solution set. Select the target optimal solution set with the highest corresponding comprehensive score from multiple optimal solution sets to obtain the target model parameter combination.

[0010] Another possible implementation method is to determine the target model parameter combination from multiple optimal solution sets based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set, including: determining the confidence interval width corresponding to each optimal solution set based on the average width of the confidence interval corresponding to each model parameter in the optimal solution set. Select the optimal solution set whose corresponding confidence interval width is less than the width threshold from the multiple optimal solution sets to obtain the first candidate set. Select the optimal solution set whose corresponding coefficient of variation is less than the coefficient of variation threshold from the multiple optimal solution sets to obtain the second candidate set. Determine the target model parameter combination based on the intersection of the first candidate set and the second candidate set.

[0011] Another possible implementation method is to determine the target model parameter combination based on the intersection of the first candidate set and the second candidate set, including: when the intersection of the first candidate set and the second candidate set is not empty, randomly selecting an optimal solution set from the intersection to obtain the target model parameter combination.

[0012] Another possible implementation method is to determine the target model parameter combination based on the intersection of the first candidate set and the second candidate set, including: when the intersection of the first candidate set and the second candidate set is empty, select the optimal solution set with the smallest product of the corresponding confidence interval width and the corresponding coefficient of variation from multiple optimal solution sets to obtain the target model parameter combination.

[0013] Another possible implementation is to send the target model parameter combination to multiple participating nodes, specifically including: verifying the target model parameter combination based on the verification sample set to obtain a performance evaluation index corresponding to the target model parameter combination. When the performance evaluation index is greater than the index threshold, the target model parameter combination is sent to the multiple participating nodes.

[0014] In another possible implementation, the method further includes: obtaining a gradient and a performance evaluation index corresponding to the intermediate model parameter combination. Based on the gradient and the performance evaluation index, adjusting the learning rate and / or batch size of the model to be trained, the batch size is used to indicate the number of training samples used by the participating nodes in one model training iteration.

[0015] Another possible implementation method: Based on the gradient and the performance evaluation index, the learning rate and / or the batch size of the model to be trained are adjusted, including: based on the difference between the norm of the gradient and the norm of the historical gradient being greater than the difference threshold, and the performance evaluation index being less than the historical performance evaluation index, the learning rate of the model to be trained is reduced and / or the batch size of the model to be trained is increased. Based on the gradient being less than the gradient threshold, and the performance evaluation index being less than the historical performance evaluation index, the learning rate of the model to be trained is reduced and / or the batch size of the model to be trained is reduced.

[0016] In a second aspect, the present application provides a data processing device, which includes various functional modules used for the method described in the first aspect above.

[0017] In a third aspect, the present application provides a computer program product, including: computer instructions; when the computer instructions are executed on an electronic device, the electronic device implements the method described in the first aspect above.

[0018] In a fourth aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method described in the first aspect above.

[0019] In a fifth aspect, the present application provides a readable storage medium, which includes: software instructions; when the software instructions are executed in an electronic device, the electronic device implements the method described in the first aspect above.

[0020] The beneficial effects of the second to fifth aspects mentioned above can be referred to the first aspect and will not be elaborated on again. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flow chart of a model training method provided for this application;

[0023] Figure 2 A schematic diagram of a target model parameter combination determination method provided in this application;

[0024] Figure 3 A flow chart of a method for determining target model parameter combination based on confidence interval and coefficient of variation weighted scoring provided in this application;

[0025] Figure 4 A schematic diagram of a method for determining a target model parameter combination based on double threshold screening of confidence interval width and coefficient of variation provided in this application;

[0026] Figure 5 A schematic diagram of a target model parameter combination verification method provided in this application;

[0027] Figure 6 A flowchart of a method for adjusting the training strategy of a model to be trained provided in this application;

[0028] Figure 7 A schematic diagram of a model training device provided in this application;

[0029] Figure 8 A schematic diagram of the composition of an electronic device provided in this application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] It should be noted that, in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0032] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical items or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.

[0033] In the field of federated learning, as application scenarios become increasingly complex, multi-objective optimization has become a research hotspot. Traditional federated learning often focuses on a single goal, such as improving model accuracy. However, in practical applications, in addition to accuracy, many other factors need to be considered. For example, the interpretability of the model, the model's ability to resist risks, the efficiency of model training, and fairness between different clients.

[0034] Currently, many studies are devoted to solving the multi-objective problem in federated learning. Some studies construct multi-objective functions by introducing regularization terms. This method adds regularization terms to the loss function, aiming to consider multiple objectives at the same time, such as controlling the complexity and accuracy of the model. However, the setting of regularization coefficients often relies on experience and lacks scientific basis, which leads to inaccurate trade-offs between multiple objectives and difficulty in achieving an effective balance.

[0035] On the other hand, there are also studies that use model compression technology to construct multi-objective functions, aiming to improve efficiency by reducing the amount of data for model training and transmission, while taking into account other goals. However, in practical applications, model compression not only requires a lot of resources and time investment, but also often focuses too much on compression efficiency, thereby ignoring the performance of the model on other goals.

[0036] In general, the existing methods for constructing multi-objective functions in federated learning have certain deficiencies in terms of scientificity and efficiency. The failure to properly handle the key elements and causal relationships in multi-objective construction leads to a lack of precision in the trade-offs between objectives, which in turn leads to problems such as imbalance and inefficiency. Therefore, how to construct a reasonable multi-objective function to achieve the optimal balance of the model in multiple dimensions has become a major challenge that needs to be solved urgently.

[0037] Based on this, the embodiments of the present application provide a model training method, device, electronic device and computer program product, which can construct a multi-objective function based on multiple screening angles, select high-quality model parameter combinations from the intermediate model parameter combinations obtained from the participating node training, and optimize the data fusion effect of federated learning.

[0038] The model training method provided in the embodiment of the present application can be applied to a central node in a federated learning system, where the central node is communicatively connected to multiple participating nodes.

[0039] The central node can be an electronic device with computing and processing capabilities, such as a computer or a server.

[0040] Among them, the server can be a single server, or it can be a server cluster composed of multiple servers. In some implementations, the server cluster can also be a distributed cluster. Optionally, the server can also be implemented on a cloud platform, for example, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, and a multi-cloud, etc., or any combination thereof. The embodiments of the present application do not limit the specific device form of the central node.

[0041] The specific equipment forms of the participating nodes can refer to the description of the central node and will not be repeated here.

[0042] Multiple participating nodes can perform iterative training on a machine learning model separately, and send the intermediate model parameter combinations obtained from each iterative training to the central node.

[0043] The central node can receive the intermediate model parameters obtained from the training of the participating nodes, and select the target model parameter combination for the next iterative training and send it to multiple participating nodes, thereby completing multiple iterative training. The specific selection process can be referred to in the model training method provided in the following embodiment, and will not be repeated here.

[0044] Figure 1 A flow chart of a model training method provided in this application. Figure 1 As shown, the present application provides a model training method, which can be applied to the above-mentioned central node, and the method specifically includes the following steps:

[0045] S101, obtaining a combination of intermediate model parameters obtained by multiple participating nodes from training a model to be trained.

[0046] It should be understood that the participating nodes use local data and their own computing resources to train the models to be trained issued by the central node to obtain intermediate model parameters, which can improve the utilization value of local data while ensuring data privacy and security. The central node aggregates multiple intermediate model parameters to obtain intermediate parameter combinations, which facilitates the subsequent construction of multi-objective functions.

[0047] It should be noted that in the scenario of federated learning, the participating nodes need to perform data preprocessing operations on the model before model training, which may include: data cleaning, normalization and feature selection to ensure data quality and consistency, and provide a reliable data foundation for subsequent model training. In addition, during the model training process, encryption technology is also needed to encrypt the data to ensure that data privacy is fully protected during the data fusion process. Because data preprocessing technology and data privacy encryption technology can be found in relevant documents, this application will not go into details.

[0048] It should also be noted that the central node can obtain the metadata generated by the participating nodes during the training process while ensuring the privacy and security of the participating nodes' data. The metadata may include the model loss function, the feature importance score of the training data, and the model performance indicators, etc., so that the central node can subsequently construct multi-objective functions and adjust the training strategies of the participating nodes.

[0049] In some embodiments, each participating node may actively send intermediate model parameters to the central node after the current round of model training is completed, or may enter a waiting state after the training is completed and send the intermediate model parameters to the central node after receiving an acquisition instruction from the central node.

[0050] S102: Based on the intermediate model parameter combination and the multi-objective function, determine the optimal solution set of the multi-objective function.

[0051] Specifically, the multi-objective function is constructed based on the global average loss function, the Bayesian optimization function, the model interpretability function and the robustness function. The optimal solution set includes the model parameter combination of the model to be trained.

[0052] It should be understood that after each round of training, the central node can also collect the model loss functions of multiple participating nodes and then calculate the global average loss function. Subsequently, the central node derives the Bayesian optimization function based on the global average loss function and the intermediate model parameters through the Bayesian optimization method, which can explore the model parameter space more quickly and find model parameters that make the model performance better.

[0053] Furthermore, on the basis of the global average loss function and the Bayesian optimization function, a multi-objective function is constructed by incorporating a model interpretability function that can improve the transparency and understandability of model decisions, as well as a robustness function that can enhance the model's ability to resist risks. The multi-objective function can consider the model training performance from different levels, integrate the characteristics of all parties involved in the node model training, and by solving the multi-objective function, a set of optimal solutions can be obtained to achieve a better balance between multiple objectives.

[0054] Exemplarily, the construction of the multi-objective function F(θ) can be expressed as:

[0055] F(θ)=λ1L global (θ)+λ2O Bayes (θ)+λ3I(θ)+λ4R(θ) Formula (1)

[0056] Among them: λ1, λ2, λ3, λ4 are weight coefficients of each target, which are used to balance the importance of different targets. The weight coefficients can be determined according to the actual application scenario. global (θ) represents the global average loss function, O Bayes (θ) represents the Bayesian optimization function, I(θ) represents the model interpretability function, and R(θ) represents the robustness function.

[0057] Exemplarily, the global average loss function can be expressed as:

[0058]

[0059] Where n represents the total number of participating nodes, l(x i ,y i ,θ) represents the model loss function of the i-th participating node, which is used to measure the training model of the participating node based on the intermediate model parameter θ for the training data x i and the true label y i The degree of difference between them.

[0060] Exemplarily, the model loss function of the participating nodes can be calculated using the mean square error method.

[0061] Exemplarily, the derivation process of the Bayesian optimization function can be expressed as:

[0062] θt +1 =AdaptiveBayesOpt({θt,Lglobal(θt)},feedbackt) Formula (3)

[0063] Among them, t represents the current training round, θ t represents the model parameter combination obtained by the central node in round t, L global represents the global average loss function calculated in round t, feedbac t k represents the real-time feedback of each participating node after t rounds of training. The real-time feedback may include the performance of the model, the training speed, the performance of the training data, the accuracy of the validation set, etc.

[0064] Specifically, the algorithm receives model parameters, loss functions, and real-time feedback from each participating node in round t. First, the algorithm uses a proxy model to approximate the behavior of the loss function and updates the proxy model through Bayesian reasoning. Second, after updating the proxy model, the algorithm selects the model parameters that have the greatest probability of reducing the loss function value and improves the real-time feedback. This process is repeated until the loss function value converges or the maximum number of iterations is reached to obtain the model parameters for the next round.

[0065] Exemplarily, the model interpretability function can be based on the parameter importance ranking function, which can be specifically expressed as:

[0066]

[0067] in:

[0068] I(θ,x) represents the model interpretability function, which can deduce the model interpretability based on the intermediate model parameter combination θ and the prediction sample set x. p represents the number of parameters in the model that are considered important after screening or evaluation by the central node, and λ k Represents the weight of the kth important model parameter, which is used to adjust the contribution of different important parameters in the explanatory power and can be determined by the parameter importance evaluation method. k * represents the kth most important model parameter. x represents the prediction sample set, which includes the input data samples and prediction results.

[0069] It should be noted that the model parameters are calculated by weighted summation to calculate the performance score of each important model parameter on the prediction sample set to calculate the degree of interpretability of the intermediate model parameter combination.

[0070] Exemplarily, the importance of model parameters may be evaluated based on a model compression method.

[0071] Exemplarily, the robustness function may be a robustness function based on parameter sensitivity analysis, which may be expressed as:

[0072]

[0073] Among them, R sensitivity Represents the robustness measure of the model. The larger the value, the more sensitive the model is to parameter changes and the worse the robustness. L is the global average loss function, θ i is the i-th parameter in the intermediate model parameter combination. By calculating the sum of the absolute values ​​of the partial derivatives of the loss function with respect to each parameter, the sensitivity of the model to parameter changes can be evaluated. If the absolute value of the partial derivative of a model parameter is large, it means that a small change in the parameter will have a greater impact on the loss function, thereby affecting the performance of the model, that is, the model is sensitive to the parameter and its robustness is relatively weak.

[0074] Exemplarily, the Pareto method can be used to solve the multi-objective function, and a set of Pareto optimal solutions can be obtained.

[0075] S103. Determine a target model parameter combination based on an optimal solution set of the multi-objective function.

[0076] In some embodiments, Figure 2 As shown, the step of determining the target model parameter combination based on the optimal solution set of the multi-objective function in S103 specifically includes:

[0077] S1031. Based on multiple optimal solution sets, determine the confidence interval set and coefficient of variation corresponding to each optimal solution set.

[0078] The confidence interval set includes the confidence interval corresponding to each model parameter in the optimal solution set, and the confidence interval is used to indicate the value range of the corresponding model parameter under the preset confidence level. The coefficient of variation is used to indicate the degree of dispersion between the prediction results of the training model for multiple test samples under the model parameter combination corresponding to the optimal solution set.

[0079] It should be noted that the confidence interval is an estimation interval that can characterize the degree of uncertainty of the optimal solution set. A wider confidence interval indicates that the value range of the optimal solution set is wider, thus reflecting a greater uncertainty. On the contrary, a narrower confidence interval indicates that the value of the optimal solution set is limited to a relatively compact range, reflecting a smaller uncertainty.

[0080] The coefficient of variation is obtained by calculating the ratio of the standard deviation to the mean of the prediction results of the model to be trained for multiple test samples under the application of the optimal solution set. It can characterize the degree of dispersion of each optimal solution set. The optimal solution set with a smaller coefficient of variation indicates that its prediction results are relatively concentrated and the model performance is more stable. On the contrary, the optimal solution set with a larger coefficient of variation indicates that the prediction results are more dispersed and the performance is less stable.

[0081] For example, the central node can use uncertainty quantification to find the confidence interval and coefficient of variation for each optimal solution set, which can be expressed as:

[0082] CI(θ * ),CV(θ * )=UncertaintyQuantification(θ * ) Formula (6)

[0083] Among them, CI(θ * ) represents the confidence interval. For each parameter combination θ, a confidence interval can be calculated separately. The confidence interval gives a range of parameter estimates that contains the true parameter value with a certain confidence level. CV(θ * ) represents the coefficient of variation. The coefficient of variation is the ratio of the standard deviation to the mean. The coefficient of variation provides a dimensionless measure of the variability of the model predictions and helps to compare the uncertainty levels under different parameter combinations.

[0084] S1032. Determine a target model parameter combination from multiple optimal solution sets based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set.

[0085] It should be understood that by solving multi-objective functions, multiple optimal solution sets can be obtained. By calculating the confidence interval and coefficient of variation of the optimal solution set, the model parameter certainty of each optimal solution set and the stability of the model applied can be evaluated, and then the target optimal model that meets the current model performance can be selected from multiple optimal solution sets.

[0086] Specifically, the intelligent selection method is used to determine the target model parameter combination according to the confidence interval and coefficient of variation of the optimal solution set. The process can be expressed as:

[0087] θ opt = SmartSelection({θ*},{CI(θ * ),CV(θ * )}) Formula (7)

[0088] Among them, θ opt represents the target model parameters, {θ *},{CI(θ * ),CV(θ * )} represents the mapping set between each optimal solution set and the confidence interval and coefficient of variation.

[0089] In some embodiments, based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set, a target model parameter combination is determined from multiple optimal solution sets, such as Figure 3 As shown, the intelligent selection method may include:

[0090] S10321a. Based on the confidence interval set corresponding to each optimal solution set, determine the confidence interval score corresponding to each optimal solution set.

[0091] Specifically, the confidence interval score is obtained by averaging the reference confidence interval scores corresponding to each model parameter in the optimal solution set, and the reference confidence interval score is negatively correlated with the width of the confidence interval of the corresponding model parameter.

[0092] It should be noted that the confidence interval can characterize the degree of uncertainty of the optimal solution set. The larger the confidence interval, the larger the range of the optimal solution set and the smaller the certainty of the value, so the confidence interval is negatively correlated with the reference confidence interval score.

[0093] As an example, the central node may specifically use the inverse of the confidence interval corresponding to each model parameter as the reference confidence interval score corresponding to the model parameter.

[0094] S10322a. Based on the coefficient of variation corresponding to each optimal solution set, determine the coefficient of variation score corresponding to each optimal solution set.

[0095] Specifically, the coefficient of variation score was negatively correlated with the coefficient of variation.

[0096] It should be noted that the coefficient of variation can characterize the degree of discreteness of each optimal solution set. The larger the coefficient of variation, the greater the degree of discreteness of the optimal solution set, and the greater the uncertainty in its application to the model to be trained. Therefore, the size of the coefficient of variation is negatively correlated with the coefficient of variation score.

[0097] As an example, the central node may specifically use the inverse of the coefficient of variation corresponding to each optimal solution set as the coefficient of variation score corresponding to the optimal solution set.

[0098] S10323a. Perform weighted summation on the coefficient of variation score and the confidence interval score to obtain a comprehensive score corresponding to each optimal solution set.

[0099] S10324a. Select the target optimal solution set with the highest comprehensive score from multiple optimal solution sets to obtain the target model parameter combination.

[0100] It should be understood that by calculating the reference confidence interval score and the coefficient of variation score of each optimal solution set, the degree of uncertainty and reliability of the optimal solution set can be determined. Then, by weighted summing the coefficient of variation score and the confidence interval score to calculate the comprehensive score, the optimal solution set with the highest comprehensive score can be determined, and the target optimal solution set with higher certainty and higher reliability can be selected.

[0101] In other embodiments, based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set, a target model parameter combination is determined from multiple optimal solution sets, such as Figure 4 As shown, the intelligent selection method may include:

[0102] S10321b. Determine the width of the confidence interval corresponding to each optimal solution set based on the average value of the width of the confidence interval corresponding to each model parameter in the optimal solution set.

[0103] S10322b. Select an optimal solution set whose corresponding confidence interval width is less than a width threshold from multiple optimal solution sets to obtain a first candidate set.

[0104] It should be noted that, from the aforementioned step S10322a, it can be seen that the confidence interval is negatively correlated with the optimal solution set. By screening the optimal solution set that is smaller than the width threshold, the certainty of the optimal solution set can be ensured.

[0105] S10323b. Select an optimal solution set whose corresponding coefficient of variation is less than a threshold value of the coefficient of variation from multiple optimal solution sets to obtain a second candidate set.

[0106] It should be noted that, as can be seen from the aforementioned step S10323a, the coefficient of variation is negatively correlated with the optimal solution set. By screening the optimal solution set that is less than the coefficient of variation threshold, the stability of the optimal solution set applied to the model can be ensured.

[0107] S10324b. Determine a target model parameter combination based on the intersection of the first candidate set and the second candidate set.

[0108] In some embodiments, based on the intersection of the first candidate set and the second candidate set, determining the target model parameter combination includes: when the intersection of the first candidate set and the second candidate set is not empty, randomly selecting an optimal solution set from the intersection to obtain the target model parameter combination.

[0109] In other embodiments, based on the intersection of the first candidate set and the second candidate set, a target model parameter combination is determined, including: when the intersection of the first candidate set and the second candidate set is empty, an optimal solution set with the smallest product of the corresponding confidence interval width and the corresponding coefficient of variation is selected from multiple optimal solution sets to obtain the target model parameter combination.

[0110] It should be noted that, for determining the target model parameters in step S1032, the present application provides two methods for selecting the target optimal model, namely: a method for determining the target model parameter combination based on the confidence interval and the weighted score of the coefficient of variation, such as Figure 3 As shown, the other is a method to determine the target model parameter combination based on the confidence interval width and the coefficient of variation double threshold screening, such as Figure 4 As shown in the figure. The first method requires more computing resources and can more accurately select the optimal target model parameters. The second solution requires less computing resources and can quickly select the optimal model parameters. In view of the characteristics of the above two solutions, in actual applications, we can determine which solution to use based on the scale of federated learning and the complexity of the model.

[0111] S104: Send the target model parameter combination to multiple participating nodes, so that the multiple participating nodes continue to train the to-be-trained model based on the target model parameter combination.

[0112] Specifically, the target model parameter combination obtained in this round is sent to multiple participating nodes, so that the participating nodes can use the target model parameter combination for the next round of training, which can improve the model training efficiency and optimize the model.

[0113] Exemplarily, the process of updating the model according to the target model parameters by the participating nodes can be expressed as:

[0114] θ new =ModelUpdate(θ opt ,new data) formula (8)

[0115] Among them, θ opt Represents the target model parameters, new data represents the training data for the next round. By updating the model with the target model parameters and training, the intermediate model parameters θ generated in the next round can be obtained. new .

[0116] In some embodiments, after determining the target model parameter combination, the central node may send it directly to multiple participating nodes.

[0117] In other embodiments, the target model parameter combination sent by the central node to the participating nodes may also be a model parameter combination verified by the performance evaluation index. Figure 5 As shown, the above S104 may further specifically include S1041 and S1042.

[0118] S1041. Verify the target model parameter combination based on the verification sample set to obtain a performance evaluation index corresponding to the target model parameter combination.

[0119] Exemplarily, the performance evaluation indicators may include any one or more of the following: accuracy, recall rate, F1 value, etc.

[0120] S1042: When the performance evaluation index is greater than the index threshold, send a target model parameter combination to multiple participating nodes.

[0121] It should be understood that by using the verification sample set to verify the determined target model parameter combination, the actual application effect of the target model parameters and the specific performance evaluation index can be obtained. Further, the obtained performance evaluation index is compared with the index threshold. If it is greater than the index threshold, it means that the target model parameter combination can meet multiple target conditions and can improve the efficiency of model training, and it can be sent to multiple participating nodes. If not, S101-S103 can be executed again to recalculate the target model parameter combination.

[0122] In some embodiments, after step S101, Figure 6 As shown, the model training method may also include S105 and S106.

[0123] S105. Obtain the gradient and performance evaluation index corresponding to the intermediate model parameter combination.

[0124] Specifically, the gradient and performance evaluation index corresponding to the intermediate model parameter combination can be obtained from multiple participating nodes. The gradient can be the gradient of the model loss function with respect to the intermediate model parameter combination. The gradient is a vector whose direction points to the direction where the loss function value increases fastest, while the negative gradient points to the direction where the loss function value decreases fastest.

[0125] S106. Based on the gradient and the performance evaluation index, adjust the learning rate and / or batch size of the model to be trained.

[0126] Specifically, the batch size can be used to indicate the number of training samples used by the participating nodes in one model training iteration, and the learning rate is used to indicate the step size of the participating nodes in updating the parameters in another model training iteration.

[0127] It is understandable that by obtaining the gradients and performance evaluation indicators of the participating nodes, the central node can understand the model training process of the participating nodes, and then adjust the next training strategy of the model. It can optimize the differences among the participating nodes to a certain extent, thereby improving the fusion efficiency of federated learning.

[0128] For example, a meta-learner can be deployed on the central node to learn the training process of the participating nodes based on the gradient and performance evaluation indicators, and make corresponding adjustments. The specific adjustment process of the meta-learner can be expressed as:

[0129] Meta-Learner({θ i ,▽L i (θ i ),metrics})→{adjusted strategy} Formula (9)

[0130] Among them, Meta-Learner represents the meta-learner, θ i represents the intermediate model parameters of the ith participating node, L i (θ i ) represents the loss function of the ith participating node, ▽L i (θ i ) represents the loss function L i (θ i ) about the model parameter θ i The gradient is a vector whose direction points to the direction where the loss function value increases the fastest, while the negative gradient direction points to the direction where the loss function value decreases the fastest. Metrics represent the performance indicators of the participating node model training process. These indicators are used to measure the performance of the model on the test set or validation set, such as accuracy, recall, F1 score, etc.

[0131] In some embodiments, based on the gradient and the performance evaluation index, adjusting the learning rate and / or batch size of the model to be trained includes: based on the difference between the norm of the gradient and the norm of the historical gradient being greater than a difference threshold, and the performance evaluation index being less than the historical performance evaluation index, reducing the learning rate of the model to be trained and / or increasing the batch size of the model to be trained. Based on the gradient being less than the gradient threshold, and the performance evaluation index being less than the historical performance evaluation index, reducing the learning rate of the model to be trained and / or reducing the batch size of the model to be trained.

[0132] It should be noted that the norm of the gradient reflects the size of the gradient. When the difference between the norm of the gradient and the historical gradient is greater than the difference threshold, it means that the current gradient has changed significantly. If the gradient is less than the gradient threshold, it means that the current gradient has become very small, which may mean that the model is close to a local optimal solution or saddle point. Near these points, the gradient is almost zero, and the model parameter update becomes very slow or even stagnant. Both of these situations represent the instability of model training. At the same time, if the performance evaluation index is less than the historical performance evaluation index, it can also prove that the performance of the model has declined. When the above situation occurs, reducing the learning rate of the model to be trained / or reducing the batch size of the model to be trained can reduce the complexity of model training and improve the performance of the model.

[0133] Exemplarily, the historical gradient can be obtained by calculating the average value of the gradient of all previous rounds of training of the participating node. Correspondingly, the historical performance evaluation index can also be obtained by calculating the average value of the performance evaluation index of all previous rounds of training of the participating node.

[0134] Specifically, after executing step S106, steps S102 to S104 may be continued to be executed.

[0135] The present application provides a model training method. First, the method can construct a multi-objective function based on multiple screening angles, and select higher-quality model parameters from the intermediate model parameter combination obtained from the participating node training as the target model parameters for the next round, which can optimize the data fusion effect of federated learning.

[0136] In addition, the method can also use the verification sample set to verify the target model parameters, which can test the actual application effect of the target model parameters and effectively reduce the loss of computing resources caused by the poor adaptation of the model parameters to the actual application. Finally, this method also provides a model training strategy adjustment method. Through the loss function and model performance indicators sent by the participating nodes, the central node can learn the training process of the node, and then analyze the problems in the training process of the participating node, and adjust the learning rate and / or batch size of its training process, which can optimize the differences of the participating nodes to a certain extent.

[0137] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed in this article, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0138] In an exemplary embodiment, the present application also provides a model training device, which can be applied to the above-mentioned central node. Figure 7 As shown, the device includes: an acquisition module 701, a processing module 702 and a sending module 703.

[0139] The acquisition module 701 is used to obtain the intermediate model parameter combination obtained by multiple participating nodes from training the to-be-trained model.

[0140] The processing module 702 is used to determine the optimal solution set of the multi-objective function based on the intermediate model parameter combination and the multi-objective function.

[0141] The processing module 702 is further used to determine a target model parameter combination based on an optimal solution set of the multi-objective function.

[0142] The sending module 703 is used to send the target model parameter combination to multiple participating nodes, so that the multiple participating nodes continue to train the to-be-trained model based on the target model parameter combination.

[0143] In a possible implementation, the processing module 702 is specifically used to determine the confidence interval set and the coefficient of variation corresponding to each optimal solution set based on multiple optimal solution sets, the confidence interval set includes the confidence interval corresponding to each model parameter in the corresponding optimal solution set, the confidence interval is used to represent the value range of the corresponding model parameter under the preset confidence level, and the coefficient of variation is used to represent the degree of discreteness between the prediction results of the to-be-trained model for multiple test samples under the model parameter combination corresponding to the optimal solution set. Based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set, the target model parameter combination is determined from the multiple optimal solution sets.

[0144] In another possible implementation, the processing module 702 is specifically used to determine the confidence interval score corresponding to each optimal solution set based on the confidence interval set corresponding to each optimal solution set. The confidence interval score is obtained by averaging the reference confidence interval score corresponding to each model parameter in the optimal solution set, and the reference confidence interval score is negatively correlated with the width of the confidence interval of the corresponding model parameter. Based on the coefficient of variation corresponding to each optimal solution set, the coefficient of variation score corresponding to each optimal solution set is determined, and the coefficient of variation score is negatively correlated with the coefficient of variation. The coefficient of variation score and the confidence interval score are weighted and summed to obtain the comprehensive score corresponding to each optimal solution set. The target optimal solution set with the highest corresponding comprehensive score is selected from multiple optimal solution sets to obtain a target model parameter combination.

[0145] In another possible implementation, the processing module 702 is specifically used to determine the width of the confidence interval corresponding to each optimal solution set based on the average width of the confidence interval corresponding to each model parameter in the optimal solution set. An optimal solution set whose corresponding confidence interval width is less than a width threshold is selected from multiple optimal solution sets to obtain a first candidate set. An optimal solution set whose corresponding coefficient of variation is less than a coefficient of variation threshold is selected from multiple optimal solution sets to obtain a second candidate set. Based on the intersection of the first candidate set and the second candidate set, a target model parameter combination is determined.

[0146] In another possible implementation, the processing module 702 is specifically configured to, when the intersection of the first candidate set and the second candidate set is not empty, randomly select an optimal solution set from the intersection to obtain a target model parameter combination.

[0147] In another possible implementation, the processing module 702 is specifically used to select an optimal solution set with the smallest product of the corresponding confidence interval width and the corresponding coefficient of variation from multiple optimal solution sets when the intersection of the first candidate set and the second candidate set is empty, to obtain a target model parameter combination.

[0148] In another possible implementation, the processing module 702 is further configured to verify the target model parameter combination based on the verification sample set to obtain a performance evaluation index corresponding to the target model parameter combination. When the performance evaluation index is greater than the index threshold, the target model parameter combination is sent to multiple participating nodes.

[0149] In another possible implementation, the processing module 702 is further used to obtain the gradient and performance evaluation index corresponding to the intermediate model parameter combination. Based on the gradient and performance evaluation index, the learning rate and / or batch size of the training model to be trained are adjusted, and the batch size is used to indicate the number of training samples used by the participating nodes in one model training iteration.

[0150] In another possible implementation, the processing module 702 is specifically used to reduce the learning rate of the model to be trained and / or increase the batch size of the model to be trained based on the difference between the norm of the gradient and the norm of the historical gradient being greater than the difference threshold, and the performance evaluation index being less than the historical performance evaluation index. Based on the gradient being less than the gradient threshold, and the performance evaluation index being less than the historical performance evaluation index, reduce the learning rate of the model to be trained and / or reduce the batch size of the model to be trained.

[0151] It should be noted that Figure 7 The division of modules in the example is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or software function modules.

[0152] In an exemplary embodiment, as described above, the central node may be a computer or a server or other electronic device with computing and processing functions. In this case, the present application also provides an electronic device, Figure 8 This is a schematic diagram of an electronic device provided in this application. Figure 8 As shown, the electronic device includes: a processor 10 , a memory 20 , a communication line 30 , a communication interface 40 , and an input / output interface 50 .

[0153] The processor 10 , the memory 20 , the communication interface 40 and the input / output interface 50 may be connected via a communication line 30 .

[0154] The processor 10 is used to execute the instructions stored in the memory 20 to implement the model training method provided in the above embodiments of the present application. The processor 10 can be a CPU, a general-purpose processor network processor (network processor, NP), a digital signal processor (digital signal processing, DSP), a microprocessor, a microcontroller (micro control unit, MCU) / single-chip microcomputer / single-chip microcomputer, a programmable logic device (programmable logic device, PLD) or any combination thereof. The processor 10 can also be any other device with processing functions, such as a circuit, a device or a software module, which is not limited in the embodiments of the present application. In one example, the processor 10 may include one or more CPUs, such as Figure 8 As an optional implementation, the electronic device may include multiple processors, for example, in addition to the processor 10, it may also include a processor 60 ( Figure 8 The dashed line is used as an example.

[0155] The memory 20 is used to store instructions. For example, the instructions may be computer programs. Optionally, the memory 20 may be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage devices, etc., and the embodiments of the present application are not limited to this.

[0156] It should be noted that the memory 20 may exist independently of the processor 10, or may be integrated with the processor 10. The memory 20 may be located inside the electronic device, or may be located outside the electronic device, which is not limited in the embodiment of the present application.

[0157] The communication line 30 is used to transmit information between various components included in the electronic device.

[0158] The communication interface 40 is used to communicate with other devices or other communication networks. The other communication networks may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 40 may be a module, a circuit, a transceiver or any device capable of achieving communication.

[0159] The input / output interface 50 is used to implement human-computer interaction between a user and an electronic device, for example, to implement action interaction or information interaction between a user and an electronic device.

[0160] Exemplarily, the input / output interface 50 may be a mouse, a keyboard, a display screen, or a touch display screen, etc. Action interaction or information interaction between a user and an electronic device may be achieved through a mouse, a keyboard, a display screen, or a touch display screen, etc.

[0161] It should be noted that Figure 8 The structure shown in the figure does not constitute a limitation on the electronic device, except Figure 8 In addition to the components shown, the electronic device may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

[0162] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed in an electronic device, the electronic device implements the method in the aforementioned method embodiment.

[0163] In an exemplary embodiment, the present application also provides a readable storage medium, which includes software instructions. When the software instructions are executed in an electronic device, the electronic device implements the method in the aforementioned method embodiment. The computer-readable storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.

[0164] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When loading and executing computer-executable instructions on a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer-executable instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0165] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0166] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0167] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training method, characterized in that: The method is applied to a central node; The central node is communicatively connected with a plurality of participating nodes; the method comprises: Obtaining a combination of intermediate model parameters obtained by the multiple participating nodes from training the model to be trained; Based on the intermediate model parameter combination and the multi-objective function, determining the optimal solution set of the multi-objective function; the multi-objective function is constructed based on the global average loss function, the Bayesian optimization function, the model interpretability function and the robustness function; the optimal solution set includes the model parameter combination of the model to be trained; Determining a target model parameter combination based on an optimal solution set of the multi-objective function; The target model parameter combination is sent to the multiple participating nodes, so that the multiple participating nodes continue to train the to-be-trained model based on the target model parameter combination.

2. The method according to claim 1, characterized in that The optimal solution set includes multiple ones; the optimal solution set based on the multi-objective function, determining the target model parameter combination, includes: Based on multiple optimal solution sets, determine the confidence interval set and coefficient of variation corresponding to each optimal solution set; the confidence interval set includes the confidence interval corresponding to each model parameter in the corresponding optimal solution set; the confidence interval is used to represent the value range of the corresponding model parameter under a preset confidence level; the coefficient of variation is used to represent the degree of discreteness between the prediction results of the to-be-trained model for multiple test samples under the model parameter combination corresponding to the optimal solution set; The target model parameter combination is determined from the multiple optimal solution sets based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set.

3. The method according to claim 2, characterized in that The step of determining the target model parameter combination from the multiple optimal solution sets based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set comprises: Based on the confidence interval set corresponding to each optimal solution set, a confidence interval score corresponding to each optimal solution set is determined; the confidence interval score is obtained by averaging the reference confidence interval score corresponding to each model parameter in the optimal solution set; the reference confidence interval score is negatively correlated with the width of the confidence interval of the corresponding model parameter; Based on the coefficient of variation corresponding to each optimal solution set, determining the coefficient of variation score corresponding to each optimal solution set; the coefficient of variation score is negatively correlated with the coefficient of variation; The coefficient of variation score and confidence interval score are weighted and summed to obtain the comprehensive score corresponding to each optimal solution set; A target optimal solution set with the highest comprehensive score is selected from the multiple optimal solution sets to obtain the target model parameter combination.

4. The method according to claim 2, characterized in that: The step of determining the target model parameter combination from the multiple optimal solution sets based on the confidence interval set and the coefficient of variation corresponding to each optimal solution set comprises: Based on the average width of the confidence interval corresponding to each model parameter in the optimal solution set, determine the width of the confidence interval corresponding to each optimal solution set; Select an optimal solution set whose corresponding confidence interval width is less than a width threshold from multiple optimal solution sets to obtain a first candidate set; Select an optimal solution set whose corresponding variation coefficient is less than a variation coefficient threshold from multiple optimal solution sets to obtain a second candidate set; The target model parameter combination is determined based on the intersection of the first candidate set and the second candidate set.

5. The method according to claim 4, characterized in that The determining the target model parameter combination based on the intersection of the first candidate set and the second candidate set includes: When the intersection of the first candidate set and the second candidate set is not empty, an optimal solution set is randomly selected from the intersection to obtain the target model parameter combination.

6. The method according to claim 4, characterized in that The determining the target model parameter combination based on the intersection of the first candidate set and the second candidate set includes: When the intersection of the first candidate set and the second candidate set is empty, an optimal solution set with the smallest product of the corresponding confidence interval width and the corresponding coefficient of variation is selected from multiple optimal solution sets to obtain the target model parameter combination.

7. The method according to any one of claims 1 to 6, characterized in that: The sending the target model parameter combination to the multiple participating nodes includes: Verifying the target model parameter combination based on the verification sample set to obtain a performance evaluation index corresponding to the target model parameter combination; When the performance evaluation index is greater than an index threshold, the target model parameter combination is sent to the multiple participating nodes.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtaining the gradient and performance evaluation index corresponding to the intermediate model parameter combination; Based on the gradient and the performance evaluation index, the learning rate and / or batch size of the model to be trained is adjusted; the batch size is used to indicate the number of training samples used by the participating nodes in one model training iteration.

9. The method according to claim 8, characterized in that The adjusting the learning rate and / or batch size of the model to be trained based on the gradient and the performance evaluation index includes: Based on the difference between the norm of the gradient and the norm of the historical gradient being greater than a difference threshold, and the performance evaluation index being less than a historical performance evaluation index, reducing the learning rate of training the model to be trained and / or increasing the batch size of training the model to be trained; Based on the gradient being less than a gradient threshold and the performance evaluation index being less than a historical performance evaluation index, reducing a learning rate for training the model to be trained and / or reducing a batch size for training the model to be trained.

10. A model training device, characterized in that: The device is applied to a central node; The central node is in communication connection with a plurality of participating nodes; the device comprises: an acquisition module, a processing module and a sending module; The acquisition module is used to acquire the intermediate model parameter combination obtained by the plurality of participating nodes from training the model to be trained; The processing module is used to determine the optimal solution set of the multi-objective function based on the intermediate model parameter combination and the multi-objective function; the multi-objective function is constructed based on the global average loss function, the Bayesian optimization function, the model interpretability function and the robustness function; the optimal solution set includes the model parameter combination of the model to be trained; based on the optimal solution set of the multi-objective function, determine the target model parameter combination; The sending module is used to send the target model parameter combination to the multiple participating nodes, so that the multiple participating nodes continue to train the model to be trained based on the target model parameter combination.

11. An electronic device, characterized in that: include: Processor and memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 9.

12. A readable storage medium, characterized in that: include: Software instructions; When the software instructions are executed in an electronic device, the electronic device implements the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that include: Computer instructions; When the computer instructions are executed in an electronic device, the electronic device is enabled to implement the method according to any one of claims 1 to 9.

Images