Business forecast optimization method, system, device and storage medium

Abstract

This application discloses a business prediction optimization method, apparatus, device, and storage medium. The method includes: federated training of a preset gradient boosting tree model to be trained with at least one second terminal device based on the first parameter values ​​of a first hyperparameter set and local business sample data; and evaluating multiple objectives corresponding to the first hyperparameter set; the multiple objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability; optimizing the first hyperparameter set based on the evaluation results of the multiple objectives; and / or optimizing and / or iteratively training the multiple objectives; and determining a target business prediction model based on the target hyperparameter set that meets the preset evaluation criteria; and inputting the business data to be processed into the target business prediction model to perform prediction processing on the business data to be processed, thereby obtaining a business prediction result. This application aims to improve data processing efficiency.

Description

Technical Field

[0001] This application relates to the field of communication and computer technology, and in particular to a service forecasting and optimization method, system, device and storage medium. Background Technology

[0002] Protecting personal data, especially in sensitive industries like finance and healthcare, during model training is crucial, ensuring privacy while providing convenience. Federated learning addresses the shortcomings of traditional centralized machine learning or deep learning in protecting data privacy, while also leveraging distributed data to collaboratively train a global model.

[0003] Currently, when using models to process data in sensitive industries such as finance and healthcare, the poor performance of these models often results in ineffective data processing. Summary of the Invention

[0004] In view of this, this application provides a business forecasting optimization method, system, device and storage medium, which aims to solve the technical problem of poor data processing effect in the prior art.

[0005] This application provides a service forecasting optimization method applied to a first terminal device, the method comprising:

[0006] Based on the first parameter values ​​of the first hyperparameter group and local business sample data, federated training is performed on the preset gradient boosting tree model to be trained with at least one second terminal device, and the multi-objectives corresponding to the first hyperparameter group are evaluated; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability.

[0007] Based on the evaluation results of the multi-objectives, the first hyperparameter set is optimized, and / or the multi-objectives are optimized and / or federated iterative training is performed. Based on the evaluation results, the target hyperparameter set corresponding to the preset evaluation criteria is reached, and a target business prediction model is determined. The target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and to obtain the business prediction result.

[0008] In one possible implementation of this application, the step of optimizing the first hyperparameter set based on the multi-objective evaluation results, performing federated iterative training, and determining the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria based on the evaluation results includes:

[0009] Based on the evaluation values ​​of the multi-objectives, if it is determined that the corresponding preset evaluation criteria have not been met, the first hyperparameter set is optimized to generate a new second hyperparameter set.

[0010] Based on the second hyperparameter set and local sample service data, the process iteratively executes the step of continuing federated training of the preset gradient boosting tree model with other second terminal devices until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard. Based on the evaluation result, the target hyperparameter set corresponding to the preset evaluation standard is reached, and the target service prediction model is determined.

[0011] In one possible implementation of this application, the step of iteratively performing federated training of the preset gradient boosting tree model to be trained together with other second terminal devices based on the second hyperparameter set and local sample service data, until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard, and determining the target service prediction model based on the target hyperparameter set corresponding to the evaluation result reaching the corresponding preset evaluation standard, includes:

[0012] Based on the second hyperparameter set and local sample service data, the preset gradient boosting tree model to be trained is federated and trained together with other second terminal devices, and it is determined whether the node depth of the decision tree constructed during the federated training process reaches the preset maximum depth.

[0013] If not, determine whether the target purity of the node that has not reached the preset maximum depth is greater than the preset purity threshold, wherein the purity is used to characterize the ratio of the number of samples of the preset category in the node to the total number of samples in the node.

[0014] If the purity threshold is greater than or equal to the preset purity threshold, statistical information is calculated based on the first gradient corresponding to the local sample. If the purity threshold is less than the preset purity threshold, statistical information is calculated together with the second gradient corresponding to the samples of other second terminal devices.

[0015] Based on the statistical information, the splitting points for splitting the nodes that have not reached the preset maximum depth are determined, and the step of determining whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth is returned until the node depth of the decision tree has reached the preset maximum depth.

[0016] If the depth of all decision tree nodes reaches the preset maximum depth, the weights of the corresponding leaf nodes are determined. Based on the gradient boosting tree model after determining the weights of the leaf nodes, it is determined whether the evaluation values ​​of the corresponding multi-objectives meet the corresponding preset evaluation criteria. The process is then iteratively executed to continue federated training of the preset gradient boosting tree model with other second terminal devices until the evaluation values ​​of the corresponding multi-objectives meet the corresponding preset evaluation criteria. Based on the evaluation results, the target hyperparameter set corresponding to the preset evaluation criteria is determined to establish the target business prediction model.

[0017] In one possible implementation of this application, the step of performing federated training on a preset gradient boosting tree model based on the second hyperparameter set and local sample service data, together with other second terminal devices, and determining whether the node depth of the decision tree constructed during the federated training process reaches a preset maximum depth includes:

[0018] Based on the second hyperparameter set and local sample business data, the local preset gradient boosting tree model to be trained is trained to reduce the residual between the predicted value corresponding to the local sample and the sample label.

[0019] If the local training meets the preset local training completion conditions, it will work with other second terminal devices to perform federated training on the preset gradient boosting tree model to be trained, and determine whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth.

[0020] In one possible implementation of this application, the step of optimizing the first hyperparameter set and generating a new second hyperparameter set based on the multi-objective evaluation value if it is determined that the corresponding preset evaluation standard has not been met includes:

[0021] Based on the evaluation values ​​of the multiple objectives, the corresponding hypervolume is determined.

[0022] When the super-volume is less than the preset super-volume threshold, it is determined that the corresponding preset evaluation standard has not been met.

[0023] If it is determined that the corresponding preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set.

[0024] In one possible implementation of this application, the multi-objective includes multiple objectives to be optimized, and the method for determining the evaluation values ​​of the multi-objective includes at least one of the following:

[0025] If the target to be optimized is training overhead, determine the time and number of operations occupied by the preset homomorphic encryption operation during model training, and based on the time and number of operations, determine the evaluation value of the training overhead in the process of obtaining the target model.

[0026] If the target to be optimized is model accuracy loss, an evaluation value of the model accuracy loss is determined based on the target model and the corresponding test dataset during the process of obtaining the target model;

[0027] If the target to be optimized is the degree of model privacy leakage, the samples are clustered based on the sample similarity of the target model. Based on the inferred labels of the samples after clustering and the labels of the corresponding samples themselves, the evaluation value of the degree of model privacy leakage in the process of obtaining the target model is determined.

[0028] If the target to be optimized is the interpretability of the model, the evaluation value of the interpretability of the model is determined based on the number of leaf nodes in the decision tree of the target model.

[0029] In one possible implementation of this application, the step of optimizing the first hyperparameter set to generate a new second hyperparameter set includes:

[0030] Determine the dominance relationship of each first hyperparameter group, and perform non-dominated sorting of each first hyperparameter group based on the dominance relationship;

[0031] Determine the distance between each first hyperparameter group and its neighboring first hyperparameter groups;

[0032] Based on the non-dominated sort and the distance, the next generation first hyperparameter set is selected;

[0033] The next-generation first hyperparameter set is subjected to preset crossover and mutation operations to obtain a new second hyperparameter set.

[0034] In one possible implementation of this application, prior to the step of federated training of a preset gradient boosting tree model based on the first parameter values ​​of the first hyperparameter set and local business sample data with at least one second terminal device, the method includes:

[0035] The first hyperparameter set is determined by using the trained Pareto front model and the corresponding user preference features;

[0036] Before the step of determining the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria achieved by the evaluation results, the following steps are included:

[0037] The evaluation values ​​of multiple objectives in the target model process are estimated by the trained surrogate model. Each of the multiple objectives corresponds to a Gaussian process model (GP). The corresponding Gaussian process model (GP) is optimized using a preset Bayesian optimization until a model that accurately predicts the evaluation values ​​of the corresponding multiple objectives is obtained. This model is then set as the surrogate model.

[0038] This application also provides a business forecasting optimization apparatus, applied to a first terminal device, the apparatus comprising:

[0039] The training module is used to perform federated training on a preset gradient boosting tree model to be trained with at least one second terminal device based on the first parameter values ​​of the first hyperparameter group and local business sample data, and to evaluate the multi-objectives corresponding to the first hyperparameter group; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability.

[0040] An optimization module is used to optimize the first hyperparameter set based on the evaluation results of the multi-objectives, and / or optimize the multi-objectives and / or perform federated iterative training, and determine the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria reached by the evaluation results. The target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and obtain the business prediction result.

[0041] This application also provides a business forecasting optimization device, which is a physical node device. The business forecasting optimization device includes: a memory, a processor, and a program of the business forecasting optimization method stored in the memory and executable on the processor. When the program of the business forecasting optimization method is executed by the processor, it can implement the steps of the business forecasting optimization method as described above.

[0042] To achieve the above objectives, a storage medium is also provided, on which a business forecasting optimization program is stored, which, when executed by a processor, implements the steps of any of the business forecasting optimization methods described above.

[0043] This application provides a business prediction optimization method, apparatus, device, and storage medium. Compared with the poor data processing effect of existing technologies, this application, based on the first parameter values ​​of a first hyperparameter set and local business sample data, performs federated training on a preset gradient boosting tree model to be trained with at least one second terminal device, and evaluates multiple objectives corresponding to the first hyperparameter set; wherein, the multiple objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability; based on the evaluation results of the multiple objectives, the first hyperparameter set is optimized, and / or the multiple objectives are optimized and / or federated iterative training is performed, and based on the evaluation results, a target business prediction model is determined by reaching the target hyperparameter set corresponding to the preset evaluation criteria; wherein, the target business prediction model is used to perform prediction processing on the business data to be processed when receiving the business data to be processed, to obtain the business prediction result. It is understood that in this application, the target business prediction model is used to predict and process the business data to be processed. The target business prediction model is obtained by the first terminal device based on the target hyperparameter set. Since the target hyperparameter set is a set of hyperparameters that makes the evaluation value of the corresponding target to be optimized reach the corresponding preset evaluation standard (based on the evaluation results of the multiple targets, the first hyperparameter set is optimized, and / or the multiple targets are optimized and / or federated iterative training is performed, and the target hyperparameter set corresponding to the preset evaluation standard is reached based on the evaluation results, thus determining the target business prediction model), the target business prediction model optimized based on the target hyperparameter set... (In addition to good data processing performance), since the optimization objectives include at least two of the following: training overhead, model accuracy loss, model privacy leakage, and model interpretability, and in the process of obtaining multiple objective evaluation values, the corresponding hyperparameter set is determined through multiple iterations rather than empirical selection or grid search when performing federated training on the preset gradient boosting tree model to be trained. Therefore, it is possible to accurately determine whether the evaluation value of the corresponding objective meets the corresponding preset evaluation standard. Thus, when the target business prediction model after the target hyperparameter set optimization processes data, it can take into account efficiency, privacy, and model performance, thereby improving the data processing effect. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the first embodiment of the business forecasting optimization method of this application;

[0045] Figure 2 This is a detailed flowchart of step S20 in the first embodiment of the business forecasting optimization method of this application;

[0046] Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application;

[0047] Figure 4 This is a schematic diagram of the overall process involved in the business forecasting optimization method of this application;

[0048] Figure 5 This is a flowchart illustrating the federated training process involved in the business forecasting optimization method of this application. Detailed Implementation

[0049] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0050] This application provides a service forecasting optimization method. In one embodiment of this service forecasting optimization method, it is applied to a first terminal device, as shown below. Figure 1 The method includes:

[0051] Step S10: Based on the first parameter values ​​of the first hyperparameter group and local business sample data, federated training is performed on the preset gradient boosting tree model to be trained with at least one second terminal device, and the multi-objectives corresponding to the first hyperparameter group are evaluated; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability.

[0052] Step S20: Based on the evaluation results of the multi-objective, optimize the first hyperparameter set, and / or optimize the multi-objective and / or perform federated iterative training, and determine the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation standard reached by the evaluation results. The target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and obtain the business prediction result.

[0053] In this embodiment, the basic concepts are explained first:

[0054] Gradient boosting tree model: It is an iterative decision tree algorithm (which can fit the true distribution). The algorithm consists of multiple decision trees, and the conclusions of all trees are summed to determine the final answer.

[0055] Leaf node: A branch node in a decision tree (a node with no children);

[0056] Node purity: Characterizes the ratio of the number of samples of the preset category (majority category) in a node to the total number of samples in that node;

[0057] Hyperparameters: Machine learning models generally have two types of parameters: one type needs to be learned and estimated from the data, called model parameters—that is, the parameters of the model itself. For example, the weighting coefficients (slope) and the bias term (intercept) of a linear regression line are model parameters. The other type are tuning parameters in the machine learning algorithm, which need to be set manually, called hyperparameters. For example, the depth of the tree in a decision tree model.

[0058] Hyperparameter set: formed by the combination of multiple hyperparameters;

[0059] Hypervolume: The S-measure or Lebesgue measure is used to evaluate the results of multi-objective problems.

[0060] In this embodiment, the optimal set of hyperparameters is found for the gradient boosting tree model to be trained, so that the corresponding optimization target meets the requirements (not just one optimization target), thereby meeting the application needs of real-world scenarios.

[0061] The background involved in this embodiment is as follows:

[0062] In recent years, both the consumer economy and the digital transformation have experienced rapid development, and the technological advancements behind these new forms are undeniable. Artificial intelligence, with machine learning as its primary technology, has made a significant contribution. Machine learning can automatically unearth the vast wealth of information hidden within data, train this information on large datasets to create mature models, and then gradually apply these models to various scenarios (all aspects of life), such as facial recognition, voice recognition, recommendation algorithms, and clinical diagnostic assistance. However, with such big data-driven models, the protection of privacy data has gradually become a major concern.

[0063] In other words, big data is a fundamental support in the model training process. Machines rely on learning from massive amounts of data, and in this process, personal data, industry data, and even national data are inevitably used. Therefore, protecting personal and industry data—ensuring privacy while allowing people to enjoy convenience—has become particularly important. To address this issue, federated learning, as a new learning framework, has emerged. Federated learning can overcome the shortcomings of traditional centralized machine learning or deep learning in protecting data privacy, while also utilizing distributed data to jointly train a global model (where, the vertical federated gradient boosting tree uses a vertical expansion approach to simultaneously train from multiple data sources, such as banking, insurance, and healthcare, to obtain more feature data from the training sample set, thereby improving model accuracy).

[0064] Currently, when using models to process data in sensitive industries such as finance and healthcare, the poor performance of these models often results in ineffective data processing.

[0065] Gradient boosting tree models are a type of machine learning model that is widely used in fields such as finance and healthcare due to their advantages such as good interpretability and model simplicity. However, existing gradient boosting tree models have the following problems:

[0066] First, during gradient boosting tree training, model performance (accuracy) is usually considered only or used as the only evaluation metric. However, in federated scenarios, efficiency and privacy are equally important. If only model performance (accuracy) is considered, it is difficult to meet the application needs of real-world scenarios.

[0067] Second, the federated gradient boosting tree model uses homomorphic encryption to protect sample gradient information. The model is too complex and will bring huge communication and computational overhead, making it unsuitable for real-world scenarios.

[0068] Third, although the current mainstream algorithms only protect the sample gradient information using homomorphic encryption, the sample distribution during the model training process is not protected;

[0069] This embodiment aims to: use a trained target business prediction model to predict and process business data. Since the target business prediction model is obtained by optimizing a preset training model based on a target hyperparameter set, and since the target hyperparameter set is a set of hyperparameters that makes the evaluation value of the corresponding target to be optimized reach the corresponding preset evaluation standard, the target business prediction model obtained by optimization based on the target hyperparameter set can improve the data processing effect.

[0070] Furthermore, before obtaining the target business prediction model, multiple objectives to be optimized corresponding to the multi-objective optimization task are used as evaluation indicators. This allows for the simultaneous minimization of objectives such as model performance, efficiency, privacy, and interpretability, i.e., finding the Pareto optimal solution set. This enables the target business prediction model to be applied efficiently and accurately in real federated learning scenarios without leaking privacy.

[0071] Furthermore, in this embodiment, by adding a node purity limit (if it is greater than or equal to a preset purity threshold, statistical information is calculated based on the first gradient corresponding to the local sample; if it is less than the preset purity threshold, statistical information is calculated together with the second gradient corresponding to the samples of other second terminal devices), excessive information leakage from the sample distribution is avoided, effectively reducing the risk of privacy leakage.

[0072] Furthermore, in this embodiment, the local training is only federated when the local training meets the preset local training completion conditions (such as the number of decision trees reaching the required level). In other words, this embodiment adds a local training stage, which reduces the residual between the predicted value and the sample label of the local sample. Therefore, the risk of privacy leakage is effectively reduced.

[0073] In this embodiment, it should be emphasized that the focus of this application is not on improving the internal parameters of the federated model, but on how to screen parameters (hyperparameter sets) based on multiple objectives, or to screen the optimal hyperparameter set based on the resource conditions and business needs of multiple terminal devices.

[0074] The specific steps are as follows:

[0075] Step S10: Based on the first parameter values ​​of the first hyperparameter group and local business sample data, federated training is performed on the preset gradient boosting tree model to be trained with at least one second terminal device, and the multi-objectives corresponding to the first hyperparameter group are evaluated; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability.

[0076] As an example, the business sample data can be banking business sample data or advertising business sample data, or other data, without any specific limitations.

[0077] As an example, sample data for advertising operations could be user browsing data on social networks obtained within the previous 6 months (which is timely).

[0078] Specifically, the sample data for banking business can be the loan click data on the app of Bank A obtained within the previous 6 months (which is timely).

[0079] As an example, when the business sample data is banking business sample data, the corresponding target business prediction model is a banking business prediction model. Then, based on this banking business prediction model, it is predicted whether a loan should be given to the corresponding user. (In this case, in the financial industry, if all terminal devices are banks and have the same user set, the multiple objectives involved can be model privacy leakage, model accuracy loss, and model interpretability. Therefore, the resulting set of target hyperparameters is the set of hyperparameters that makes the evaluation values ​​of the corresponding optimization objectives (model privacy leakage, model accuracy loss, and model interpretability) reach the corresponding preset evaluation criteria. Furthermore, since the objectives include...) The study assesses the degree of model privacy leakage, model accuracy loss, and model interpretability. Furthermore, in obtaining the corresponding evaluation values, the federated training of the preset gradient boosting tree model involves multiple iterations rather than empirical selection or grid search to determine the corresponding hyperparameter set. This allows for accurate determination of whether the evaluation value of the target meets the preset evaluation criteria. Consequently, when using the target business prediction model optimized with the corresponding target hyperparameter set to process data, privacy, accuracy, and model interpretability can be balanced, thereby improving the data processing performance in terms of privacy, accuracy, and model interpretability.

[0080] When the business sample data is advertising business sample data, the corresponding target business prediction model is the advertising business prediction model. Then, based on this advertising business prediction model, it is predicted whether the corresponding advertisement should be recommended to the corresponding user. (In the financial industry, if all terminal devices are advertisers and have the same user set, the multiple objectives involved can be training cost, model accuracy loss, and model privacy leakage. Therefore, the obtained target hyperparameter set is the set of hyperparameters that makes the evaluation values ​​of the corresponding optimization objectives (training cost, model accuracy loss, and model privacy leakage) reach the corresponding preset evaluation criteria. Furthermore, since the optimization objectives include model privacy leakage, model...) This approach reduces model accuracy loss and interpretability. Furthermore, in obtaining the corresponding evaluation value, because the federated training of the preset gradient boosting tree model involves multiple iterations rather than empirical selection or grid search, the corresponding hyperparameter set is determined. Therefore, it accurately determines whether the evaluation value of the corresponding optimization target meets the preset evaluation standard. Consequently, when using the advertising business prediction model optimized with the corresponding hyperparameter set of the optimization target to process data, it can balance training overhead, model accuracy loss, and model privacy leakage, thereby improving the data processing effectiveness in terms of training overhead, model accuracy loss, and model privacy leakage.

[0081] In this embodiment, it should be noted that the target business prediction model may be embedded in a server deployed in a distributed manner within the computer system of the first terminal device.

[0082] In this embodiment, it should be noted that the target business prediction model can also be deployed in the GPU hardware parallel computing accelerator inside the computer system of the first terminal device to improve computing efficiency.

[0083] That is, in this embodiment, the optimized target business prediction model is specifically associated with the internal structure of the computer system.

[0084] In this embodiment, a multi-objective optimization task can be determined by a first terminal device, and multiple first hyperparameter sets corresponding to the multi-objective optimization task can be determined. The multi-objective includes at least two objectives to be optimized from training overhead, model accuracy loss, model privacy leakage degree, and model interpretability degree.

[0085] As an example, the business forecasting optimization method is applied to the first terminal device;

[0086] As an example, the service sample data of the first terminal device is tagged;

[0087] As an example, a first terminal device (acting as a server) and other second terminal devices are federated to train a pre-set gradient boosting tree model to be trained.

[0088] As an example, the first terminal device could be a bank, an advertiser, or something similar; there are no specific limitations.

[0089] As an example, the first terminal device determines a multi-objective optimization task based on its own needs.

[0090] As an example, the needs of the first terminal device are different, and the corresponding multi-objective optimization tasks have different objectives.

[0091] As an example, in the financial industry, if all terminal devices are banks and have the same user set, the multi-objectives involved in the multi-objective optimization task initiated by the first terminal device can be the degree of model privacy leakage, model accuracy loss, and model interpretability.

[0092] At this point, the multiple objectives involved in the multi-objective optimization task can be the degree of model privacy leakage, model accuracy loss, and model interpretability. Therefore, the objective hyperparameter is the parameter that makes the degree of model privacy leakage, model accuracy loss, and model interpretability reach the corresponding evaluation criteria. Then, the objective hyperparameter is used as the model parameter to perform federated training on the preset gradient boosting tree model to be trained, thereby obtaining the banking business prediction model. Then, based on the banking business prediction model, the bank data is predicted and processed to determine whether a loan can be issued to the corresponding user (improving the efficiency of determining whether a loan can be issued to the corresponding user).

[0093] As an example, in an advertising recommendation task, the terminal device is a multi-advertising provider with different feature data, one of which has labels. Since the advertising recommendation task focuses more on real-time performance, the multi-objectives involved by the first terminal device (initiating the multi-objective optimization task) can be training overhead, model accuracy loss, and the degree of model privacy leakage.

[0094] At this point, since the multi-objectives involved in the multi-objective optimization task can be training overhead, model accuracy loss, and model privacy leakage, the objective hyperparameter is the parameter that makes the training overhead, model accuracy loss, and model privacy leakage reach the corresponding evaluation criteria. Therefore, the objective hyperparameter is used as the model parameter to perform federated training on the preset gradient boosting tree model to be trained, thereby obtaining the advertising business prediction model. Then, based on the advertising business prediction model, the advertising data or social data is predicted and processed to determine whether the advertisement can be recommended to the corresponding user (improving the efficiency of determining whether the advertisement can be recommended to the corresponding user).

[0095] As an example, the multi-objective includes at least two of the following, rather than one, objectives: training cost, model accuracy loss, model privacy leakage, and model interpretability.

[0096] In other words, if the model needs to satisfy multiple objectives rather than just one objective, the method in this embodiment can be used.

[0097] As an example, training cost ∈ c This could refer to the computational and time overhead during model training.

[0098] As an example, model accuracy loss ∈ u Used to evaluate model performance, it can be used to characterize model accuracy;

[0099] As an example, the degree of model privacy leakage ∈ p Used to assess the extent of label leakage;

[0100] As an example, the interpretability of the model is ∈ i It can be determined by calculating the number of leaf nodes in the decision tree of model M.

[0101] As an example, in this embodiment, multiple first hyperparameter groups corresponding to the multi-objective optimization task are also determined;

[0102] The first hyperparameter group {Xi} can be multiple groups, and each group of hyperparameters contains the number of locally trained decision trees n. l Number of federal training decision trees n f The maximum depth of the tree is d, the sample sampling rate is r, the node purity threshold is p, and the learning rate is η.

[0103] Multiple sets of first hyperparameters can actually be [(n l 1,n_r1,...,d1),(n_l2 n_r2,...,d2),...,(n_l,3n_r3,...,d3)].

[0104] As an example, different terminal devices train federated gradient boosting tree models separately for each set of hyperparameters.

[0105] Prior to the step of federated training of the preset gradient boosting tree model based on the first parameter values ​​of the first hyperparameter group and local business sample data with at least one second terminal device, the following steps are included:

[0106] Step S011: Determine the first hyperparameter set using the trained Pareto front model and the corresponding user's preference features;

[0107] In this embodiment, the Pareto front model is already trained. The specific training method for the Pareto front model can be: based on the optimization objective ∈ c ,∈ u ,∈ p ,∈ i (reflecting user preference features λ), and the hyperparameter set {X} i} t-1 The Pareto front model is obtained by training the previous hyperparameter set (using a multi-layer fully connected network as the model to be trained) using stochastic gradient descent.

[0108] Furthermore, multiple first hyperparameter sets are determined by using the trained Pareto front model and the corresponding user's preference features (such as inputting three optimization objectives into the trained Pareto front model, and outputting a set of hyperparameters).

[0109] Based on the first parameter values ​​of the first hyperparameter group and local business sample data, federated training is performed on the preset gradient boosting tree model to be trained with at least one second terminal device, and the multi-objectives corresponding to the first hyperparameter group are evaluated; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability.

[0110] In this embodiment, the initial parameter value (first parameter value) corresponding to each hyperparameter in the first hyperparameter group is obtained, that is, the parameter value of each hyperparameter in each hyperparameter group is randomly initialized.

[0111] After obtaining the first parameter value corresponding to each hyperparameter in the first hyperparameter group, based on the first parameter value, the preset gradient boosting tree model to be trained is federatedly trained together with other second terminal devices until the training is completed (such as when the number of decision trees reaches the required level), and the multi-objective corresponding to the first hyperparameter group is evaluated.

[0112] Step S20: Based on the evaluation results of the multi-objective, optimize the first hyperparameter set, and / or optimize the multi-objective and / or perform federated iterative training, and determine the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation standard reached by the evaluation results. The target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and obtain the business prediction result.

[0113] After federated training of the preset gradient boosting tree model to be trained with at least one second terminal device based on the first parameter values ​​of the first hyperparameter set and local business sample data, the evaluation values ​​of the corresponding multi-objectives are determined. Based on the evaluation results of the multi-objectives, the first hyperparameter set is optimized, and / or the multi-objectives are optimized and / or federated iterative training is performed. Based on the evaluation results, the target hyperparameter set corresponding to the preset evaluation standard is reached, and the target business prediction model is determined.

[0114] As an example, if the multi-objective optimization task is an advertising recommendation task, the evaluation value of the multi-objective is the evaluation value of training cost, model accuracy loss, and model privacy leakage. If the multi-objective optimization task is a financial task, the evaluation value of the multi-objective is the model privacy leakage, model accuracy loss, and model interpretability, etc.

[0115] After determining the evaluation values ​​of multiple objectives, based on the evaluation values ​​of multiple objectives, it is determined whether their overall performance meets the corresponding preset evaluation criteria. If it is determined that the corresponding preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set.

[0116] In this embodiment, if the multi-objective optimization task is an advertising recommendation task, the evaluation values ​​of training overhead, model accuracy loss, and model privacy leakage are comprehensively determined to meet the corresponding preset evaluation criteria. If the preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set. If the preset evaluation criteria are met, a target hyperparameter set is obtained and used as the model hyperparameters to train the corresponding model to be trained. Finally, an advertising business prediction model that can process advertising data is obtained, and then the advertising data received by the terminal device is processed to improve the advertising data processing effect.

[0117] If the multi-objective optimization task is a financial task, then the evaluation values ​​of the model's privacy leakage level, model accuracy loss, and model interpretability are comprehensively determined to meet the corresponding preset evaluation criteria. If it is determined that the corresponding preset evaluation criteria are not met, then a new second hyperparameter set is generated based on the first hyperparameter set. If the preset evaluation criteria are met, then the target hyperparameter set is obtained and used as the model hyperparameters to train the corresponding model to be trained. Finally, a banking business prediction model that can process bank data or loan data is obtained, and then the loan data or bank data received by the terminal device is processed to improve the bank data processing effect.

[0118] Prior to the step of determining the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria achieved by the evaluation results, the following steps are included:

[0119] The evaluation values ​​of multiple objectives in the target model process are estimated by the trained surrogate model. Each of the multiple objectives corresponds to a Gaussian process model (GP). The corresponding Gaussian process model (GP) is optimized using a preset Bayesian optimization until a model that accurately predicts the evaluation values ​​of the corresponding multiple objectives is obtained. This model is then set as the surrogate model.

[0120] It should be noted that in this application, during the process of selecting the optimal hyperparameter set based on the resource status and operation status of multiple terminal devices, the multi-objective evaluation results are affected by factors such as the computing memory capacity and computing speed of the first terminal device or the second terminal device, or the preset evaluation criteria for multiple objectives are affected by factors such as the computing memory capacity and computing speed of the relevant devices.

[0121] As an example, in this application, when the multi-objective evaluation result still fails to meet the standard (does not meet the preset evaluation standard) after a certain number of training iterations, the Pareto optimal solution set can be obtained by calculating the difference between the multi-objective evaluation result and the preset evaluation standard, as well as the computing resources consumed by obtaining the multi-objective evaluation result and the corresponding computing speed. By applying for GPU hardware acceleration processing locally (increasing the computing resource memory capacity through hardware accelerator to improve computing speed), the Pareto optimal solution set can be obtained.

[0122] As an example, in this application, under existing computing resources, it is not necessary to actually train the model to obtain the corresponding multi-objective evaluation results. Instead, the multi-objective evaluation results can be predicted by a trained surrogate model to save computing resources.

[0123] In this embodiment, it should be noted that training a federated learning model is too costly. By estimating the optimization objective in the federated learning model using Pareto frontier and surrogate models, the number of training iterations can be reduced, thus saving costs.

[0124] The steps of optimizing the first hyperparameter set based on the multi-objective evaluation results, performing federated iterative training, and determining the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria based on the evaluation results include:

[0125] Step S21: Based on the evaluation values ​​of the multi-objectives, if it is determined that the corresponding preset evaluation criteria have not been met, the first hyperparameter group is optimized to generate a new second hyperparameter group.

[0126] Step S22: Based on the second hyperparameter set and local sample service data, iteratively execute the step of continuing federated training of the preset gradient boosting tree model to be trained together with other second terminal devices until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard, and determine the target service prediction model based on the target hyperparameter set corresponding to the evaluation result reaching the corresponding preset evaluation standard.

[0127] In this embodiment, it should be clarified that after obtaining the second hyperparameter set, it is necessary to iteratively execute federated training of the preset gradient boosting tree model to be trained together with other second terminal devices, using the second hyperparameter set and local sample service data, until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard.

[0128] In this embodiment, based on the evaluation results reaching the target hyperparameter set corresponding to the preset evaluation criteria, a target business prediction model is determined. After obtaining the target business prediction model, when receiving business data to be processed, the business data to be processed is subjected to prediction processing to obtain a business prediction result.

[0129] The step of optimizing the first hyperparameter set and generating a new second hyperparameter set based on the multi-objective evaluation value if it is determined that the corresponding preset evaluation standard has not been met includes:

[0130] Step A1: Based on the evaluation values ​​of the multiple targets, determine the common hypervolume;

[0131] Step A2: When the super-volume is less than the preset super-volume threshold, it is determined that the corresponding preset evaluation standard has not been met.

[0132] Specifically, when the hypervolume is less than a preset hypervolume threshold, it is determined that the corresponding preset evaluation standard has not been met, and the corresponding Pareto optimal solution set (corresponding to the hyperparameter set) does not converge. When the hypervolume is greater than or equal to the preset hypervolume threshold, it is determined that the corresponding preset evaluation standard has been met, and the corresponding Pareto optimal solution set (corresponding to the hyperparameter set) converges.

[0133] Step A3: If it is determined that the corresponding preset evaluation criteria have not been met, a new second hyperparameter set is generated based on the first hyperparameter set.

[0134] If the corresponding preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set.

[0135] Among them, a new set of second hyperparameters can be generated using methods such as multi-objective genetic algorithms and multi-objective Bayesian optimization algorithms.

[0136] The step of optimizing the first hyperparameter set to generate a new second hyperparameter set includes:

[0137] Step B1: Determine the dominance relationship of each first hyperparameter group, and perform non-dominated sorting on each first hyperparameter group based on the dominance relationship;

[0138] Step B12: Determine the distance between each first hyperparameter group and its neighboring first hyperparameter groups;

[0139] Step B13: Based on the non-dominated sort and the distance, select the next generation of the first hyperparameter set;

[0140] Step B4: Perform preset crossover and mutation operations on the next-generation first hyperparameter set to obtain a new second hyperparameter set.

[0141] In this embodiment, the generation of a new second hyperparameter set through a multi-objective genetic algorithm is used as an example for specific explanation.

[0142] As an example, based on ∈ c ,∈ u ,∈ p ,∈ i The dominance relationship between them is used to perform a non-dominated sort on the first hyperparameter set, i.e., based on ∈ c ,∈ u ,∈ p ,∈ i The magnitude of the evaluation value divides each first hyperparameter group into multiple levels. The higher the level, the better. First hyperparameter groups dominated by other first hyperparameter groups are ranked lower.

[0143] After determining the non-dominated sorting of each first hyperparameter group, the crowding degree between each first hyperparameter group and its neighboring first hyperparameter groups is determined:

[0144] As an example, crowding is calculated for the first hyperparameter group at each level. Crowding measures the density around the first hyperparameter group, i.e., the distance between the first hyperparameter group and its neighboring first hyperparameter groups. A higher crowding indicates that the area where the individual is located is more crowded. Specifically, based on multi-objective f k For the first hyperparameter group i after sorting, calculate its crowding distance:

[0145] After determining the crowding distance, a selection operation is performed (selecting the first hyperparameter group of the next generation based on the non-dominated ranking and the distance): the next generation individuals are calculated and selected according to the non-dominated ranking and the crowding.

[0146] As an example, the first hyperparameter set with a high non-dominated ranking is chosen because it represents superiority over other first hyperparameter sets across multiple objectives. Then, within the same level, the first hyperparameter set with higher crowding is selected to increase diversity. The above steps are repeated until a preset hyperparameter set size (next-generation first hyperparameter set) is reached.

[0147] Perform crossover and mutation operations: Perform preset crossover and mutation operations on the next-generation first hyperparameter set to obtain a new second hyperparameter set, for discrete variables (n) l n f For the first hyperparameter set (r, p, η), single-point crossover with a crossover probability of 0.9 and gene locus inversion mutation with a flipping probability of 0.1 are used. For continuous variables (r, p, η), simulated binary crossover with a crossover probability of 0.9 and multinomial mutation with a mutation probability of 0.1 are used. The resulting progeny hyperparameter set is merged with the original first hyperparameter set to form a new second hyperparameter set. If the size of the new second hyperparameter set exceeds the predetermined size, a crowding selection operation is performed to ensure the consistency of hyperparameter set size.

[0148] The multi-objective includes multiple objectives to be optimized, and the method for determining the evaluation value of the multi-objective includes at least one of the following:

[0149] Step C1: If the target to be optimized is training overhead, determine the time and number of operations occupied by the preset homomorphic encryption operation during model training, and determine the evaluation value of the training overhead in the process of obtaining the target model based on the time and the number of operations.

[0150] As an example, one way to determine the evaluation value of the training cost is to count the number of homomorphic encryption, decryption, and addition operations, respectively, as c. enc c dec and c add Estimate the time required for homomorphic encryption, decryption, and addition as t, respectively. enc t dec and t add Then the cost evaluation value ∈ c Alternatively, the formula for calculating model training overhead is as follows:

[0151] ∈ c =c enc ×t enc +c dec ×t dec +cadd ×t add

[0152] Step C2: If the target to be optimized is model accuracy loss, determine the evaluation value of model accuracy loss in the process of obtaining the target model based on the target model and the corresponding test dataset;

[0153] In this embodiment, M represents the trained model, D represents the test dataset, and U represents the evaluation method. For binary classification tasks, AUC is used for evaluation, and for multi-class classification tasks, accuracy is used for evaluation. The specific formula for calculating the model accuracy loss is as follows:

[0154] ∈ u =1-U(M,D)

[0155] Step C3: If the target to be optimized is the degree of model privacy leakage, the samples are clustered based on the sample similarity in the target model. Based on the inferred labels of the samples after clustering and the labels of the corresponding samples themselves, the evaluation value of the degree of model privacy leakage in the process of obtaining the target model is determined.

[0156] As an example, the assessment value for the degree of privacy breach in a model can be determined as follows:

[0157] (1) Sample sampling: Uniformly sample from all training samples on different terminal devices to obtain a sample space of I. pl ,

[0158] (2) Calculate sample similarity: Based on the sample distribution I in the decision tree i,j The sample similarity is calculated using the following formulas (1) and (2):

[0159]

[0160]

[0161] The above formula indicates that if samples a and b are both on the same leaf node of a decision tree, the similarity is increased by 1; otherwise, it is 0.

[0162] (3) Sample clustering: Based on the sample similarity matrix sim, the samples are clustered to obtain multiple clusters.

[0163] (4) Sample inference: Assuming that there is a known sample in each cluster, use the label of that sample as the label of all samples in that cluster.

[0164] (5) Calculation of privacy leakage metric: Calculate the average accuracy of sample inference using the following formula:

[0165]

[0166] Where N pl =|I pl |, For the indicator function, y i Indicates the actual label. Labels indicating inferences.

[0167] Step C4: If the target to be optimized is the interpretability of the model, determine the evaluation value of the interpretability of the model in the process of obtaining the target model based on the number of leaf nodes in the decision tree of the target model.

[0168] In this embodiment, the number of leaf nodes in the decision tree of model M is calculated, and the sum is obtained as ∈ i , which indicates the interpretability of the model.

[0169] In this embodiment, based on the second hyperparameter set, the step of continuing federated training of the preset gradient boosting tree model to be trained together with other second terminal devices is returned until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard.

[0170] In this embodiment, the overall process is as follows: Figure 4 As shown, the individuals that have been evaluated are the first hyperparameter group that have been evaluated.

[0171] This application provides a business prediction optimization method, apparatus, device, and storage medium. Compared with the poor data processing effect of existing technologies, this application, based on the first parameter values ​​of a first hyperparameter set and local business sample data, performs federated training on a preset gradient boosting tree model to be trained with at least one second terminal device, and evaluates multiple objectives corresponding to the first hyperparameter set; wherein, the multiple objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability; based on the evaluation results of the multiple objectives, the first hyperparameter set is optimized, and / or the multiple objectives are optimized and / or federated iterative training is performed, and based on the evaluation results, a target business prediction model is determined by reaching the target hyperparameter set corresponding to the preset evaluation criteria; wherein, the target business prediction model is used to perform prediction processing on the business data to be processed when receiving the business data to be processed, to obtain the business prediction result. It is understood that in this application, the target business prediction model is used to predict and process the business data to be processed. The target business prediction model is obtained by the first terminal device based on the target hyperparameter set. Since the target hyperparameter set is a set of hyperparameters that makes the evaluation value of the corresponding target to be optimized reach the corresponding preset evaluation standard (based on the evaluation results of the multiple targets, the first hyperparameter set is optimized, and / or the multiple targets are optimized and / or federated iterative training is performed, and the target hyperparameter set corresponding to the preset evaluation standard is reached based on the evaluation results, thus determining the target business prediction model), the target business prediction model optimized based on the target hyperparameter set... (In addition to good data processing performance), since the optimization objectives include at least two of the following: training overhead, model accuracy loss, model privacy leakage, and model interpretability, and in the process of obtaining multiple objective evaluation values, the corresponding hyperparameter set is determined through multiple iterations rather than empirical selection or grid search when performing federated training on the preset gradient boosting tree model to be trained. Therefore, it is possible to accurately determine whether the evaluation value of the corresponding objective meets the corresponding preset evaluation standard. Thus, when the target business prediction model after the target hyperparameter set optimization processes data, it can take into account efficiency, privacy, and model performance, thereby improving the data processing effect.

[0172] Furthermore, based on the first embodiment of this application, another embodiment of this application is provided.

[0173] The step of iteratively executing the federated training of the preset gradient boosting tree model based on the second hyperparameter set and local sample service data, together with other second terminal devices, until the evaluation values ​​of the corresponding multi-objectives reach the corresponding preset evaluation criteria, and determining the target service prediction model based on the target hyperparameter set corresponding to the evaluation results reaching the corresponding preset evaluation criteria, includes:

[0174] Step D1: Based on the second hyperparameter group and local sample service data, federated training is performed on the preset gradient boosting tree model to be trained together with other second terminal devices, and it is determined whether the node depth of the decision tree constructed during the federated training process reaches the preset maximum depth.

[0175] Specifically, the first terminal device and other second terminal devices jointly perform federated training on a preset gradient boosting tree model to be trained. During this process, it is determined whether there are nodes with a depth less than a preset depth *d* (to determine whether the node depth of the decision tree constructed during federated training reaches a preset maximum depth). Figure 5 As shown.

[0176] It should be noted that the cutoff condition for training a gradient boosting tree is that the number of decision trees meets the requirement, and the condition for constructing a decision tree is that the node depth reaches the maximum depth. A gradient boosting tree is trained one decision tree at a time, sequentially.

[0177] Step D2: If not, determine whether the target purity of the node that has not reached the preset maximum depth is greater than the preset purity threshold, wherein the purity is used to characterize the ratio of the number of samples of the preset category in the node to the total number of samples in the node.

[0178] If not, determine whether the target purity of the node that has not reached the preset maximum depth is greater than a preset purity threshold, wherein the purity is used to characterize the ratio of the number of samples of a preset category in the node to the total number of samples in the node, specifically, wherein the node purity p j The calculation method can be:

[0179] (1) The label y′ that appears most frequently in the statistical node.

[0180] (2) Count the number of samples n' with label y', and the total number of samples in each node is n. Calculate the sample purity as follows:

[0181] Step D3: If the purity threshold is greater than or equal to the preset purity threshold, then the statistical information is calculated based on the first gradient corresponding to the local sample. If the purity threshold is less than the preset purity threshold, the statistical information is calculated together with the second gradient corresponding to the samples of other second terminal devices.

[0182] If it is determined that the target purity of the node that has not reached the preset maximum depth is greater than or equal to the preset purity threshold, then statistical information is calculated based on the first gradient corresponding to the local sample. If it is less than the preset purity threshold, statistical information is calculated together with the second gradient corresponding to the sample of the other second terminal device to avoid leakage of sample information.

[0183] The formula for calculating statistical information is (statistical information) <gkv> , <hkv>The calculation method is to accumulate the sample gradient based on the sample feature data:

[0184]

[0185]

[0186] Step D4: Based on the statistical information, determine the splitting points for the nodes that have not reached the preset maximum depth, and return to the step of determining whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth, until the node depth of the decision tree has reached the preset maximum depth.

[0187] The first terminal device calculates the information gain based on statistical information, using the following formula:

[0188] Information gain

[0189] Where I_l and I_r are the sample sets of the left and right child nodes, G and H represent the gradients of the aggregated samples, and g and h represent the gradients of each terminal device sample. The split point corresponding to the maximum information gain is selected to divide the nodes, and the steps to determine whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth are returned until the node depth of the decision tree has reached the preset maximum depth.

[0190] Step D5: If the depth of all decision tree nodes reaches the preset maximum depth, determine the weight of the corresponding leaf node, and based on the gradient boosting tree model after determining the weight of the leaf node, determine whether the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard, and iteratively execute the step of continuing federated training of the preset gradient boosting tree model to be trained together with other second terminal devices until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard, and determine the target business prediction model based on the target hyperparameter set corresponding to the evaluation result reaching the corresponding preset evaluation standard.

[0191] In this embodiment, if the depth of all decision tree nodes reaches a preset maximum depth, the weights of the corresponding leaf nodes are determined, and the trained target business prediction model is obtained.

[0192] The active side calculates the weights of the leaf nodes using the following formula:

[0193]

[0194] The leaf node weights are stored in the first terminal device. Once the leaf node weights are determined, the training is considered complete, resulting in a complete target business prediction model.

[0195] In this application, by adding a node purity limit (if it is greater than or equal to a preset purity threshold, statistical information is calculated based on the first gradient corresponding to the local sample; if it is less than the preset purity threshold, statistical information is calculated together with the second gradient corresponding to the sample of the other second terminal device), excessive information leakage from the sample distribution is avoided, and the risk of privacy leakage is effectively reduced.

[0196] Furthermore, based on the above embodiments of this application, another embodiment of this application is provided. In this embodiment, the step of performing federated training on a preset gradient boosting tree model to be trained together with other second terminal devices based on initialization parameter values, and determining whether the node depth of the decision tree constructed during the federated training process reaches a preset maximum depth, includes:

[0197] Step E1: Based on the second hyperparameter set and the local sample business data, train the local preset gradient boosting tree model to be trained in order to reduce the residual between the predicted value and the sample label of the local sample.

[0198] In this embodiment, the difference from the prior art is that training the local preset gradient boosting tree model is to reduce the residual between the predicted value and the sample label of the local sample (the difference between the true label and the predicted label).

[0199] Step E2: If the local training meets the preset local training completion conditions, perform federated training on the preset gradient boosting tree model to be trained together with other second terminal devices, and determine whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth.

[0200] If the local training meets the preset local training completion conditions (the number of decision trees meets the requirements), it will work with other second terminal devices to perform federated training on the preset gradient boosting tree model to be trained, and determine whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth.

[0201] In this embodiment, if the local training meets the preset local training completion conditions, the preset gradient boosting tree model to be trained is federated and trained together with other second terminal devices. That is, a local training stage is added, and this stage reduces the residual between the predicted value corresponding to the local sample and the sample label. Therefore, the risk of privacy leakage is effectively reduced.

[0202] Reference Figure 3 , Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0203] like Figure 3 As shown, the business prediction optimization device may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005.

[0204] Optionally, the service prediction optimization device may also include a user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, a WiFi module, etc. The user interface may include a display screen and an input submodule such as a keyboard; optional user interfaces may also include standard wired or wireless interfaces. The network interface may include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0205] Those skilled in the art will understand that Figure 3 The business forecasting optimization device structure shown does not constitute a limitation on the business forecasting optimization device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0206] like Figure 3 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and a business forecasting optimization program. The operating system is a program that manages and controls the hardware and software resources of the business forecasting optimization device, supporting the operation of the business forecasting optimization program and other software and / or programs. The network communication module is used to enable communication between the cameras within the memory 1005, as well as communication with other hardware and software within the device.

[0207] exist Figure 3 In the business forecasting optimization device shown, the processor 1001 is used to execute the business forecasting optimization program stored in the memory 1005 to implement the steps of the business forecasting optimization method described above.

[0208] The specific implementation of the business forecasting optimization device in this application is basically the same as the embodiments of the business forecasting optimization method described above, and will not be repeated here.

[0209] This application also provides a business forecasting optimization apparatus, applied to a first terminal device, the apparatus comprising:

[0210] The training module is used to perform federated training on a preset gradient boosting tree model to be trained with at least one second terminal device based on the first parameter values ​​of the first hyperparameter group and local business sample data, and to evaluate the multi-objectives corresponding to the first hyperparameter group; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability.

[0211] An optimization module is used to optimize the first hyperparameter set based on the evaluation results of the multi-objectives, and / or optimize the multi-objectives and / or perform federated iterative training, and determine the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria reached by the evaluation results. The target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and obtain the business prediction result.

[0212] In one possible implementation of this application, the business forecasting optimization device is used to achieve:

[0213] Based on the evaluation values ​​of the multi-objectives, if it is determined that the corresponding preset evaluation criteria have not been met, the first hyperparameter set is optimized to generate a new second hyperparameter set.

[0214] Based on the second hyperparameter set and local sample service data, the process iteratively executes the step of continuing federated training of the preset gradient boosting tree model with other second terminal devices until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard. Based on the evaluation result, the target hyperparameter set corresponding to the preset evaluation standard is reached, and the target service prediction model is determined.

[0215] In one possible implementation of this application, the business forecasting optimization device is used to achieve:

[0216] Based on the second hyperparameter set and local sample service data, the preset gradient boosting tree model to be trained is federated and trained together with other second terminal devices, and it is determined whether the node depth of the decision tree constructed during the federated training process reaches the preset maximum depth.

[0217] If not, determine whether the target purity of the node that has not reached the preset maximum depth is greater than the preset purity threshold, wherein the purity is used to characterize the ratio of the number of samples of the preset category in the node to the total number of samples in the node.

[0218] If the purity threshold is greater than or equal to the preset purity threshold, statistical information is calculated based on the first gradient corresponding to the local sample. If the purity threshold is less than the preset purity threshold, statistical information is calculated together with the second gradient corresponding to the samples of other second terminal devices.

[0219] Based on the statistical information, the splitting points for splitting the nodes that have not reached the preset maximum depth are determined, and the step of determining whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth is returned until the node depth of the decision tree has reached the preset maximum depth.

[0220] If the depth of all decision tree nodes reaches the preset maximum depth, the weights of the corresponding leaf nodes are determined. Based on the gradient boosting tree model after determining the weights of the leaf nodes, it is determined whether the evaluation values ​​of the corresponding multi-objectives meet the corresponding preset evaluation criteria. The process is then iteratively executed to continue federated training of the preset gradient boosting tree model with other second terminal devices until the evaluation values ​​of the corresponding multi-objectives meet the corresponding preset evaluation criteria. Based on the evaluation results, the target hyperparameter set corresponding to the preset evaluation criteria is determined to establish the target business prediction model.

[0221] In one possible implementation of this application, the business forecasting optimization device is used to achieve:

[0222] Based on the second hyperparameter set and local sample business data, the local preset gradient boosting tree model to be trained is trained to reduce the residual between the predicted value corresponding to the local sample and the sample label.

[0223] If the local training meets the preset local training completion conditions, it will work with other second terminal devices to perform federated training on the preset gradient boosting tree model to be trained, and determine whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth.

[0224] In one possible implementation of this application, the business forecasting optimization device is used to achieve:

[0225] Based on the evaluation values ​​of the multiple objectives, the corresponding hypervolume is determined.

[0226] When the super-volume is less than the preset super-volume threshold, it is determined that the corresponding preset evaluation standard has not been met.

[0227] If it is determined that the corresponding preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set.

[0228] In one possible implementation of this application, the multi-objective includes multiple objectives to be optimized, and the business forecasting optimization device is used to achieve at least one of the following:

[0229] If the target to be optimized is training overhead, determine the time and number of operations occupied by the preset homomorphic encryption operation during model training, and based on the time and number of operations, determine the evaluation value of the training overhead in the process of obtaining the target model.

[0230] If the target to be optimized is model accuracy loss, an evaluation value of the model accuracy loss is determined based on the target model and the corresponding test dataset during the process of obtaining the target model;

[0231] If the target to be optimized is the degree of model privacy leakage, the samples are clustered based on the sample similarity of the target model. Based on the inferred labels of the samples after clustering and the labels of the corresponding samples themselves, the evaluation value of the degree of model privacy leakage in the process of obtaining the target model is determined.

[0232] If the target to be optimized is the interpretability of the model, the evaluation value of the interpretability of the model is determined based on the number of leaf nodes in the decision tree of the target model.

[0233] In one possible implementation of this application, the business forecasting optimization device is used to achieve:

[0234] Determine the dominance relationship of each first hyperparameter group, and perform non-dominated sorting of each first hyperparameter group based on the dominance relationship;

[0235] Determine the distance between each first hyperparameter group and its neighboring first hyperparameter groups;

[0236] Based on the non-dominated sort and the distance, the next generation first hyperparameter set is selected;

[0237] The next-generation first hyperparameter set is subjected to preset crossover and mutation operations to obtain a new second hyperparameter set.

[0238] In one possible implementation of this application, the business forecasting optimization device is used to achieve:

[0239] The first hyperparameter set is determined by using the trained Pareto front model and the corresponding user preference features;

[0240] The business forecasting optimization device is used to achieve:

[0241] The evaluation values ​​of multiple objectives in the target model process are estimated by the trained surrogate model. Each of the multiple objectives corresponds to a Gaussian process model (GP). The corresponding Gaussian process model (GP) is optimized using a preset Bayesian optimization until a model that accurately predicts the evaluation values ​​of the corresponding multiple objectives is obtained. This model is then set as the surrogate model.

[0242] The specific implementation of the business forecasting optimization device in this application is basically the same as the embodiments of the business forecasting optimization method described above, and will not be repeated here.

[0243] This application provides a storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the business prediction optimization method described above.

[0244] The specific implementation of the storage medium in this application is basically the same as the embodiments of the above-described business forecasting optimization method, and will not be repeated here.

[0245] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described business forecasting optimization method.

[0246] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-mentioned business forecasting optimization method, and will not be repeated here.

[0247] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0248] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of a software plus hardware platform, or by hardware, but in many cases the former is a better implementation. Based on this understanding, the technical solution of the present invention, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0250] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.< / hkv> < / gkv>

Claims

1. A business forecasting optimization method, characterized in that, Applied to a first terminal device, the method includes: Based on the first parameter values ​​of the first hyperparameter group and local business sample data, federated training is performed on the preset gradient boosting tree model to be trained with at least one second terminal device, and the multi-objectives corresponding to the first hyperparameter group are evaluated; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability. Based on the evaluation results of the multi-objectives, the first hyperparameter set is optimized, and / or the multi-objectives are optimized and / or federated iterative training is performed, and the target hyperparameter set corresponding to the preset evaluation criteria is reached based on the evaluation results, and the target business prediction model is determined, wherein the target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and to obtain the business prediction result. Prior to the step of federated training of the preset gradient boosting tree model based on the first parameter values ​​of the first hyperparameter set and local business sample data with at least one second terminal device, the following steps are included: Before the step of determining the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria achieved by the evaluation results, the following steps are included: The evaluation values ​​of multiple objectives in the target model process are estimated by using a trained surrogate model. Each of the multiple objectives corresponds to a Gaussian process model (GP). The corresponding Gaussian process model (GP) is optimized using a preset Bayesian optimization until a model that accurately predicts the evaluation values ​​of the corresponding multiple objectives is obtained. This model is then set as the surrogate model. The first hyperparameter set is determined by using the trained Pareto front model and the corresponding user preference features; The step of optimizing the first hyperparameter set and generating a new second hyperparameter set if the evaluation value based on the multi-objective criteria is determined not to meet the corresponding preset evaluation standard includes: Based on the evaluation values ​​of the multiple objectives, the corresponding hypervolume is determined. When the hypervolume is less than the preset hypervolume threshold, it is determined that the corresponding preset evaluation standard has not been met, and the Pareto optimal solution set corresponding to the hyperparameter group does not converge. If it is determined that the corresponding preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set.

2. The business forecasting optimization method as described in claim 1, characterized in that, The steps of optimizing the first hyperparameter set based on the multi-objective evaluation results, performing federated iterative training, and determining the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria based on the evaluation results include: Based on the evaluation values ​​of the multi-objectives, if it is determined that the corresponding preset evaluation criteria have not been met, the first hyperparameter set is optimized to generate a new second hyperparameter set. Based on the second hyperparameter set and local sample service data, the process iteratively executes the step of continuing federated training of the preset gradient boosting tree model with other second terminal devices until the evaluation value of the corresponding multi-objective reaches the corresponding preset evaluation standard. Based on the evaluation result, the target hyperparameter set corresponding to the preset evaluation standard is reached, and the target service prediction model is determined.

3. The business forecasting optimization method as described in claim 2, characterized in that, The step of iteratively executing the federated training of the preset gradient boosting tree model based on the second hyperparameter set and local sample service data, together with other second terminal devices, until the evaluation values ​​of the corresponding multi-objectives reach the corresponding preset evaluation criteria, and determining the target service prediction model based on the target hyperparameter set corresponding to the evaluation results reaching the corresponding preset evaluation criteria, includes: Based on the second hyperparameter set and local sample service data, the preset gradient boosting tree model to be trained is federated and trained together with other second terminal devices, and it is determined whether the node depth of the decision tree constructed during the federated training process reaches the preset maximum depth. If not, determine whether the target purity of the node that has not reached the preset maximum depth is greater than the preset purity threshold, wherein the purity is used to characterize the ratio of the number of samples of the preset category in the node to the total number of samples in the node. If the purity threshold is greater than or equal to the preset purity threshold, statistical information is calculated based on the first gradient corresponding to the local sample. If the purity threshold is less than the preset purity threshold, statistical information is calculated together with the second gradient corresponding to the samples of other second terminal devices. Based on the statistical information, the splitting points for splitting the nodes that have not reached the preset maximum depth are determined, and the step of determining whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth is returned until the node depth of the decision tree has reached the preset maximum depth. If the depth of all decision tree nodes reaches the preset maximum depth, the weights of the corresponding leaf nodes are determined. Based on the gradient boosting tree model after determining the weights of the leaf nodes, it is determined whether the evaluation values ​​of the corresponding multi-objectives meet the corresponding preset evaluation criteria. The process is then iteratively executed to continue federated training of the preset gradient boosting tree model with other second terminal devices until the evaluation values ​​of the corresponding multi-objectives meet the corresponding preset evaluation criteria. Based on the evaluation results, the target hyperparameter set corresponding to the preset evaluation criteria is determined to establish the target business prediction model.

4. The business forecasting optimization method as described in claim 3, characterized in that, The step of performing federated training on a preset gradient boosting tree model based on the second hyperparameter set and local sample service data, together with other second terminal devices, and determining whether the node depth of the decision tree constructed during the federated training process reaches a preset maximum depth includes: Based on the second hyperparameter set and local sample business data, the local preset gradient boosting tree model to be trained is trained to reduce the residual between the predicted value and the sample label of the local sample. If the local training meets the preset local training completion conditions, it will work with other second terminal devices to perform federated training on the preset gradient boosting tree model to be trained, and determine whether the node depth of the decision tree constructed during the federated training process has reached the preset maximum depth.

5. The business forecasting optimization method as described in claim 1, characterized in that, The multi-objective includes multiple objectives to be optimized, and the method for determining the evaluation value of the multi-objective includes at least one of the following: If the target to be optimized is training overhead, determine the time and number of operations occupied by the preset homomorphic encryption operation during model training, and based on the time and number of operations, determine the evaluation value of the training overhead in the process of obtaining the target model. If the target to be optimized is model accuracy loss, an evaluation value of the model accuracy loss is determined based on the target model and the corresponding test dataset during the process of obtaining the target model; If the target to be optimized is the degree of model privacy leakage, the samples are clustered based on the sample similarity of the target model. Based on the inferred labels of the samples after clustering and the labels of the corresponding samples themselves, the evaluation value of the degree of model privacy leakage in the process of obtaining the target model is determined. If the target to be optimized is the interpretability of the model, the evaluation value of the interpretability of the model is determined based on the number of leaf nodes in the decision tree of the target model.

6. The business forecasting optimization method as described in claim 2, characterized in that, The step of optimizing the first hyperparameter set to generate a new second hyperparameter set includes: Determine the dominance relationship of each first hyperparameter group, and perform non-dominated sorting of each first hyperparameter group based on the dominance relationship; Determine the distance between each first hyperparameter group and its neighboring first hyperparameter groups; Based on the non-dominated sort and the distance, the next generation first hyperparameter set is selected; The next-generation first hyperparameter set is subjected to preset crossover and mutation operations to obtain a new second hyperparameter set.

7. A business forecasting optimization device, characterized in that, Applied to a first terminal device, the device includes: The training module is used to perform federated training on a preset gradient boosting tree model to be trained with at least one second terminal device based on the first parameter values ​​of the first hyperparameter group and local business sample data, and to evaluate the multi-objectives corresponding to the first hyperparameter group; wherein, the multi-objectives include at least two objectives: system training overhead, model accuracy loss, model privacy leakage degree, and model interpretability. An optimization module is used to optimize the first hyperparameter set based on the evaluation results of the multi-objectives, and / or optimize the multi-objectives and / or perform federated iterative training, and determine the target business prediction model based on the target hyperparameter set corresponding to the preset evaluation criteria reached by the evaluation results. The target business prediction model is used to perform prediction processing on the business data to be processed when the business data to be processed is received, and obtain the business prediction result. The business forecasting optimization device is used to achieve: The first hyperparameter set is determined by using the trained Pareto front model and the corresponding user preference features; The evaluation values ​​of multiple objectives in the target model process are estimated by using a trained surrogate model. Each of the multiple objectives corresponds to a Gaussian process model (GP). The corresponding Gaussian process model (GP) is optimized using a preset Bayesian optimization until a model that accurately predicts the evaluation values ​​of the corresponding multiple objectives is obtained. This model is then set as the surrogate model. Based on the evaluation values ​​of the multiple objectives, the corresponding hypervolume is determined. When the hypervolume is less than the preset hypervolume threshold, it is determined that the corresponding preset evaluation standard has not been met, and the Pareto optimal solution set corresponding to the hyperparameter group does not converge. If it is determined that the corresponding preset evaluation criteria are not met, a new second hyperparameter set is generated based on the first hyperparameter set.

8. A business forecasting optimization device, characterized in that, The method includes a memory, a processor, and a business forecasting optimization program stored in the memory and executable on the processor, wherein the processor, when executing the business forecasting optimization program, implements the steps of the business forecasting optimization method according to any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a business forecasting optimization program, which, when executed by a processor, implements the steps of the business forecasting optimization method as described in any one of claims 1 to 6.

Images