Model optimization method and device, equipment, storage medium and product

Abstract

The invention discloses a model optimization method and device, equipment, a storage medium and a product, and relates to the technical field of artificial intelligence, and the method comprises the steps: training a decision tree model according to a main training feature set and an auxiliary training feature set; acquiring parameter updating information and feature importance information; performing aggregation updating of the global model according to the parameter updating information and the feature importance information, and optimizing the decision tree model according to a current aggregation updating result; through the above mode, the decision tree model is trained by using the training feature set subjected to scientific feature selection and sampling, all features can be effectively and fully utilized, and a target algorithm combining federal learning feature influence aggregation and model cooperative training is introduced to perform aggregation updating of a global model. And the decision tree model is optimized according to the current aggregation updating result to ensure that the decision tree model can effectively perform cooperative work with the global model, so that the accuracy of model optimization can be effectively improved, and the federal learning performance is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to model optimization methods, devices, equipment, storage media and products. Background Art

[0002] Federated Learning is a distributed machine learning method that allows multiple participants to train models while maintaining data privacy. At present, the common way to use federated learning for model training is that each participant first identifies the features that have a greater impact on the model's predictive performance through feature selection, and then the participant uses a subset of the selected features to train the model and sends the model update to the central server. The central server then merges the model updates of all participants into a global model. However, it is unable to fully utilize the information of all available features, lacks awareness of the importance of different features, and does not dynamically adjust the sampling strategy, so it is impossible to maximize the model performance. Based on this, the accuracy of the above method in optimizing the model is low.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a model optimization method, device, equipment, storage medium and product, aiming to solve the technical problem of low accuracy of optimization models in the prior art.

[0005] To achieve the above objectives, the present application proposes a model optimization method, which comprises:

[0006] Train a decision tree model based on the main training feature set and the auxiliary training feature set;

[0007] Obtaining parameter update information and feature importance information of the decision tree model;

[0008] Performing aggregation update of the global model according to the parameter update information and the feature importance information to obtain a current aggregation update result;

[0009] The decision tree model is optimized according to the current aggregate update result.

[0010] In one embodiment, before the step of training the decision tree model according to the main training feature set and the auxiliary training feature set, the method further includes:

[0011] Obtaining the current training sample set of each participant, and determining the proportion of target class samples in the current training sample set and discrete attributes;

[0012] Calculate the information entropy of the current training sample set according to the proportion of the target class samples;

[0013] Dividing the current training sample set according to the discrete attributes to obtain a plurality of branch nodes;

[0014] Count the number of subset samples and the total number of samples respectively;

[0015] Calculate the conditional information entropy of the current training sample set under the discrete attribute according to the number of subset samples, the total number of samples and the multiple branch nodes;

[0016] A main training feature set and an auxiliary training feature set are determined according to the information entropy and the conditional information entropy.

[0017] In one embodiment, the step of determining the main training feature set and the auxiliary training feature set according to the information entropy and the conditional information entropy includes:

[0018] Calculate the information gain of the discrete attribute for the current training sample set according to the information entropy and the conditional information entropy;

[0019] Determine the importance value of each training feature in the current training sample set according to the information gain;

[0020] Sorting the training features according to the importance values ​​according to a preset sorting relationship;

[0021] Selecting a main training feature set from the current training sample set according to the training feature sorting result;

[0022] The auxiliary training feature set is determined based on the remaining training features in the current training sample set through a random sampling strategy.

[0023] In one embodiment, the step of performing aggregate update of the global model according to the parameter update information and the feature importance information to obtain a current aggregate update result includes:

[0024] Obtain the weight of each participant and the local importance value reported by each participant;

[0025] The central server calculates the global importance value of each feature according to the feature importance information and the weight of each participant;

[0026] Performing feature importance aggregation of the global model according to the global importance values ​​and feature importance information of each feature through a target algorithm;

[0027] Get the gradient of the loss function with respect to the model parameters;

[0028] The parameters of the global model are updated according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information by the target algorithm;

[0029] Get the current aggregation update result based on the aggregation result and update result.

[0030] In one embodiment, after the step of updating the parameters of the global model according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information by the target algorithm, the step further includes:

[0031] After detecting that each participant receives the global model, the newly added feature importance value is obtained;

[0032] The target evaluation algorithm is used to evaluate the newly added feature importance value and the local importance value to obtain a new local importance value;

[0033] Calculating new global importance values ​​of each feature based on the central server according to the new local importance values ​​and the weights of each participant;

[0034] Performing feature importance aggregation of the global model according to the new global importance values ​​of each feature and the new local importance values ​​through a target algorithm;

[0035] The current aggregation update result is updated according to the new aggregation result.

[0036] In one embodiment, the step of optimizing the decision tree model according to the current aggregate update result includes:

[0037] Obtaining global model parameter information and feature information gain according to the current aggregation update result;

[0038] According to the global model parameter information, the feature usage information and the feature weight of the decision tree model are adjusted respectively;

[0039] Updating the parameters of the decision tree model according to the feature information gain;

[0040] The training strategy of the tree-planning model is adjusted according to the performance evaluation result of the global model.

[0041] In addition, to achieve the above purpose, the present application also proposes a model optimization device, which includes:

[0042] A training module, used for training a decision tree model according to a main training feature set and an auxiliary training feature set;

[0043] An acquisition module, used to obtain parameter update information and feature importance information of the decision tree model;

[0044] An aggregation update module, used to perform aggregation update of the global model according to the parameter update information and the feature importance information to obtain a current aggregation update result;

[0045] An optimization module is used to optimize the decision tree model according to the current aggregation update result.

[0046] In addition, to achieve the above-mentioned purpose, the present application also proposes a model optimization device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the model optimization method described above.

[0047] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model optimization method described above are implemented.

[0048] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the model optimization method described above are implemented.

[0049] One or more technical solutions proposed in the present application have at least the following technical effects: training a decision tree model based on a main training feature set and an auxiliary training feature set; obtaining parameter update information and feature importance information of the decision tree model; performing aggregation update of the global model based on the parameter update information and the feature importance information to obtain a current aggregation update result; optimizing the decision tree model based on the current aggregation update result; in the above manner, training a decision tree model using a target training feature set that has been scientifically feature selected and sampled can more effectively and fully utilize all features, and introduce a target algorithm that combines federated learning feature influence aggregation and model collaborative training to perform aggregation update of the global model, and then optimize the decision tree model using the current aggregation update result to ensure that the local decision tree model can effectively perform collaborative work tasks with the global model, thereby effectively improving the accuracy of the optimized model and improving the performance of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the model optimization method of the present application;

[0053] Figure 2 A schematic diagram of a flow chart provided for the second embodiment of the model optimization method of the present application;

[0054] Figure 3 A schematic diagram of decision-making of each participant in the model optimization method provided in Example 2 of the present application;

[0055] Figure 4 A swimming lane diagram of a model optimization method provided in one embodiment of the present application;

[0056] Figure 5 This is a schematic diagram of the module structure of the model optimization device of the embodiment of the present application;

[0057] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the model optimization method in the embodiment of the present application.

[0058] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0059] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a model optimization device, etc. The following takes the model optimization device as an example to illustrate this embodiment and the following embodiments.

[0060] Based on this, the present application embodiment provides a model optimization method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the model optimization method of the present application.

[0061] In this embodiment, the model optimization method includes steps S10 to S40:

[0062] Step S10, training a decision tree model according to the main training feature set and the auxiliary training feature set.

[0063] It should be noted that the main training feature set refers to the training feature set determined from the current training sample sets of each participant through scientific feature selection. The importance value of each training feature in the main training feature set is higher than the respective initial reference threshold. Each participant sets a different threshold based on its own data situation, and then comprehensively considers the results of all parties to negotiate and scientifically select and determine the main training feature set. This can adapt to the characteristics of the data of all parties. In order to increase the diversity and generalization ability of the model, it is also necessary to determine the auxiliary training feature set. The auxiliary training feature set refers to the feature set of the auxiliary training decision tree model.

[0064] It should be understood that after obtaining the main training feature set and the auxiliary training feature set, each participant uses the main training feature set and the auxiliary training feature set in the local system to train the decision tree model to achieve the purpose of establishing an accurate prediction model. For example, participant 1 uses features a, b, c...x, y, z, etc. to train the decision tree model, and participant 2 uses features c, a, e...u, v, x, etc. to train the decision tree model. In horizontal federated learning, although each participant has a feature space with the same structure, they can use different feature subsets to train their own tree models.

[0065] Step S20, obtaining parameter update information and feature importance information of the decision tree model.

[0066] It is understandable that for the participants, after the decision tree model is trained, the parameter update information of the decision tree model and the feature importance information of each feature will be exported in the local system. At the same time, the learning success of the local decision tree model and the usage of the training features will be encrypted and sent to the global model for secure aggregation.

[0067] Step S30, performing aggregation update of the global model according to the parameter update information and the feature importance information to obtain a current aggregation update result.

[0068] It should be understood that for the central server, after receiving the parameter update information and feature importance information sent by all participants, the target algorithm that combines federated learning feature impact aggregation and model collaborative training is used to aggregate and update the global model. The target algorithm can be the EFedImpact algorithm, which aims to improve the performance and stability of the federated learning model by optimizing the aggregation of feature importance and the update of model parameters. This algorithm not only takes into account the feature importance information of different participants, but also introduces a new model parameter update mechanism, which can better address the challenge of model heterogeneity.

[0069] It should be noted that the target algorithm can further optimize model performance and take into account model heterogeneity, because different participants may use different features and sampling methods, which is mainly reflected in the following aspects: (1) Differences in feature selection: Although the participants have the same feature set, they select different feature subsets to train their respective decision tree models. This difference in selectivity directly leads to the heterogeneity of the input information of each model. (2) Heterogeneity of feature importance: Even for commonly used features, their importance in each model may be different due to the differences in the training process and data of different models. This difference reflects the heterogeneity of the model in understanding and applying these features. (3) The impact of unselected features: After receiving the importance values ​​of unselected features returned by the central server, participant 1 needs to integrate this information into its own model. This process may introduce new variability because the distribution of local data may be different from the data distribution of participant 2. This heterogeneity may affect the aggregation of the model and the final prediction results, so it is necessary to adopt the method of this embodiment to handle it to ensure the validity and reliability of the research results.

[0070] Furthermore, in order to effectively improve the accuracy and comprehensiveness of the current aggregated update result, step S30 includes: obtaining the weight of each participant and the local importance value reported by each participant; calculating the global importance value of each feature based on the feature importance information and the weight of each participant based on the central server; performing feature importance aggregation of the global model according to the global importance value and feature importance information of each feature through the target algorithm; obtaining the gradient of the loss function with respect to the model parameters; updating the parameters of the global model according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information through the target algorithm; and obtaining the current aggregated update result according to the aggregation result and the update result.

[0071] It is understandable that for each feature, the global importance value of the feature can be calculated by weighted average, which is explained by taking participant i and feature f as an example:

[0072]

[0073] Among them, I f represents the global importance value of feature f, w i represents the weight of participant i, I f，i Represents the local importance value reported by participant i.

[0074] It should be noted that the central server can calculate the weighted average information gain of each feature based on the sample size and reduce the weights of features that differ greatly between different participants. The calculation method of the local importance values ​​reported by the participants usually depends on the machine learning algorithm used. For example, when using an integrated algorithm based on a decision tree, the local importance values ​​reported by the participants can be determined by the average decrease in impurity caused by the feature when dividing the data. The integrated algorithm can be a random forest, LightGBM algorithm, etc., and the impurity can be the Gini coefficient. After calculating the global importance values ​​of each feature, the target algorithm is used to aggregate the feature importance of the global model.

[0075] It should be understood that when updating the parameters of the global model, the central server introduces a proximal term penalty coefficient to reduce the deviation between the local decision tree model and the global model. After obtaining the gradient of the loss function with respect to the model parameters, the target algorithm is used to update the parameters of the global model. Each iterative update can be expressed as:

[0076]

[0077] Among them, θ new represents the iterative update parameters, represents the learning rate, ablaL(θ) represents the gradient of the loss function with respect to , θ represents the parameters of the global model, λ represents the proximal term penalty coefficient, θ old Represents iterative model parameters, for example, the model parameters of the previous iteration.

[0078] Furthermore, in order to effectively improve the accuracy of parameter updating of the global model, after the step of updating the parameters of the global model according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information through the target algorithm, it also includes: after detecting that each participant has received the global model, obtaining a newly added feature importance value; evaluating the newly added feature importance value and the local importance value through the target evaluation algorithm to obtain a new local importance value; calculating new global importance values ​​of each feature based on the central server according to the new local importance value and the weights of each participant; performing feature importance aggregation of the global model according to the new global importance values ​​and the new local importance values ​​of each feature through the target algorithm; and updating the current aggregation update result according to the new aggregation result.

[0079] It should be understood that the newly added feature importance value refers to the importance value of the feature introduced by the global model in the current iteration but not yet considered by the local decision tree model. The newly added feature importance value is calculated by the global model in the aggregation stage and fed back to each participant along with the global model. For each participant, after detecting and receiving the global model fed back by the central server, it will be evaluated through the target evaluation algorithm, specifically:

[0080]

[0081] Among them, I' f,i Represents the new local importance value, Eval() represents the evaluation function, I f,i Represents the local importance value reported by participant i.

[0082] It should be noted that in the local evaluation phase, the participants will use the above-mentioned target evaluation algorithm to evaluate based on the newly added feature importance values ​​and local importance values. Specifically, the participants can use the newly added feature importance values ​​to adjust the parameters or structure of the local decision tree model to better fit the needs of the global model. Then, the participants send the new local importance values ​​to the central server, which will perform the next round of aggregation.

[0083] In addition, compared with the original FedImpact algorithm, the target algorithm is enhanced by integrating asynchronous communication, device sampling and fault-tolerant mechanisms, which can more comprehensively solve the heterogeneity problem of the model. Asynchronous communication: The original FedImpact algorithm needs to synchronize the information of all participants during global updates, which may cause the system to be less efficient when waiting for slow devices. By introducing the asynchronous communication mechanism, the central server can process updates from any participant immediately after receiving them, without waiting for all devices to complete the calculation. This method can reduce the idle time of the system and improve the overall training speed. Device sampling: In response to the heterogeneity problem between devices, the EFedImpact algorithm adopts a device sampling strategy, that is, only a part of the devices are selected for training and updating in each iteration. This not only reduces the communication cost in each iteration, but also dynamically adjusts the set of devices participating in the training according to the computing power and current status of the devices, thereby improving the flexibility and scalability of the system. Fault-tolerant mechanism: In order to deal with possible device failures or data loss problems, EFedImpact has added a fault-tolerant mechanism. When a device fails to successfully upload a model update, the system can use data from other devices or previous model parameters to compensate and ensure the stability and accuracy of the global model. This mechanism enhances the robustness of the algorithm in the face of uncertain environments.

[0084] Step S40: Optimize the decision tree model according to the current aggregate update result.

[0085] It is understandable that, for each participant, after obtaining the current aggregate update result, they will optimize the decision tree model based on their own parameters to improve the performance of the decision tree model and make the final model more in line with actual application needs.

[0086] Furthermore, in order to effectively improve the accuracy of the optimization model, step S40 includes: obtaining global model parameter information and feature information gain according to the current aggregation update result; adjusting the feature usage information and feature weight of the decision tree model according to the global model parameter information; updating the parameters of the decision tree model according to the feature information gain; and adjusting the training strategy of the decision tree model according to the performance evaluation result of the global model.

[0087] It should be understood that the current aggregate update results include but are not limited to the model parameter information, feature information gain, etc. of the global model, and the dimensions to be optimized include but are not limited to feature usage information, feature weights, feature integration, and training strategies, which may be learning rate, tree depth, etc. In addition, if the global model indicates that certain features that are not used in the local model are very important, you can consider integrating these features into the local model, use the performance evaluation results of the global model as feedback, and adjust the training strategy of the tree-determining model.

[0088] It should be noted that the above optimization will continue to iterate until the function converges or reaches the upper limit of the allowed number of iterations or the allowed training time. At the same time, the central server will evaluate the performance of the current global model. If the expected effect is not achieved, the next round of local training and model aggregation will be continued. In multiple rounds of iterations, the estimation of the global feature importance value is gradually optimized through iterative refinement. Each round of iteration can adjust the feature usage information and feature weights according to the global model performance and feature importance value of the previous round.

[0089] This embodiment trains a decision tree model according to a main training feature set and an auxiliary training feature set; obtains parameter update information and feature importance information of the decision tree model; performs aggregation update of a global model according to the parameter update information and the feature importance information to obtain a current aggregation update result; optimizes the decision tree model according to the current aggregation update result; in the above manner, a decision tree model is trained using a target training feature set that has been scientifically feature selected and sampled, which can more effectively and fully utilize all features, and introduces a target algorithm that combines federated learning feature influence aggregation and model collaborative training to perform aggregation update of the global model, and then optimizes the decision tree model using the current aggregation update result to ensure that the local decision tree model can effectively perform collaborative work tasks with the global model, thereby effectively improving the accuracy of the optimization model and improving the performance of federated learning.

[0090] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , before step S10, it also includes steps S001 to S106:

[0091] Step S001, obtaining the current training sample set of each participant, and determining the proportion of target class samples in the current training sample set and discrete attributes.

[0092] It should be understood that in order to ensure the authenticity and integrity of the data of each participant, the current training sample set is obtained through a series of processing such as cleaning, screening, missing values, etc. After obtaining the current training sample set of each participant, the proportion of the target class samples and the discrete attributes are further determined.

[0093] Step S002, calculating the information entropy of the current training sample set according to the proportion of the target class samples.

[0094] It can be understood that after obtaining the proportion of target class samples, the information entropy of the current training sample set is calculated using a decision tree learning algorithm. The decision tree learning algorithm may be an Iterative Dichotomiser 3 algorithm, and the specific calculation formula may be:

[0095]

[0096] Where Ent(D) represents the information entropy of the current training sample set D, k represents the target class sample, k = 1, 2, .., |Y|, P k Indicates the proportion of target class samples.

[0097] Step S003: divide the current training sample set according to the discrete attributes to obtain multiple branch nodes.

[0098] It should be understood that if the discrete attribute in the current training sample set has multiple values, multiple branch nodes will be generated when the current training sample set is divided according to the discrete attribute. For example, if the discrete attribute has v values, v branch nodes will be generated.

[0099] Step S004, counting the number of subset samples and the total number of samples respectively.

[0100] Step S005, calculating the conditional information entropy of the current training sample set under the discrete attribute according to the number of subset samples, the total number of samples and the multiple branch nodes.

[0101] It can be understood that if the vth branch node contains all the nodes in the current training sample set whose value is a on the discrete attribute a v The number of samples in the subset is expressed as |Dv|, and then the conditional information entropy of the current training sample set under discrete attributes is calculated in combination with the total number of samples, specifically:

[0102]

[0103] Among them, Ent(D|a) represents the conditional information entropy of the current training sample set D under discrete attribute a, |Dv| represents the number of subset samples, |D| represents the total number of samples, and v represents the sequence number of the branch node.

[0104] Step S006: determining a main training feature set and an auxiliary training feature set according to the information entropy and the conditional information entropy.

[0105] It should be understood that the main training feature set refers to the training feature set determined by scientific feature selection from the current training sample sets of each participant, and the auxiliary training feature set refers to the feature set of the auxiliary training decision tree model. After obtaining the conditional information entropy, the main training feature set and the auxiliary training feature set are comprehensively determined in combination with the conditional information entropy.

[0106] Furthermore, in order to effectively improve the accuracy of determining the main training feature set and the auxiliary training feature set, step S006 includes: calculating the information gain of the discrete attribute for the current training sample set based on the information entropy and the conditional information entropy; determining the importance value of each training feature in the current training sample set based on the information gain; sorting the training features according to the importance value according to a preset sorting relationship; selecting the main training feature set from the current training sample set according to the training feature sorting result; and determining the auxiliary training feature set based on the remaining training features in the current training sample set through a random sampling strategy.

[0107] It can be understood that information gain represents the reduction in uncertainty of a random variable after a certain condition is known. In this embodiment, it describes the degree of reduction in uncertainty of data set D after attribute a is known. After obtaining information entropy and conditional information entropy, the information gain of discrete attributes for the current training sample set is calculated, specifically:

[0108] Gain(D,a)=Ent(D)-Ent(D|a).

[0109] Wherein, Gain(D,a) represents the information gain of the discrete attribute for the current training sample set, Ent(D) represents the information entropy of the current training sample set D, and Ent(D|a) represents the conditional information entropy of the current training sample set D under the discrete attribute a.

[0110] It should be understood that after calculating the information gain of the discrete attribute for the current training sample set, the importance value of each training feature in the current training sample set is further determined, and then the feature importance ranking of all parties and the needs of global collaboration are comprehensively considered. According to the training feature ranking results, the highest-ranked training feature is selected from the current training sample set as the main training feature set. Figure 3 , Figure 3 Schematic diagram of the decision-making of each participant, specifically: Taking participant 1 and participant 2 as examples, for participant 1, the calculated information gain is: Gain feature a>Gain feature b>Gain feature c, at this time feature a is the most important, followed by feature b, and finally feature c. For participant 2, the calculated information gain is: Gain feature c>Gain feature a>Gain feature e, at this time feature c is the most important, followed by feature a, and finally feature e.

[0111] This embodiment obtains the current training sample set of each participant and determines the proportion of target class samples in the current training sample set and discrete attributes; calculates the information entropy of the current training sample set according to the proportion of the target class samples; divides the current training sample set according to the discrete attributes to obtain multiple branch nodes; counts the number of subset samples and the total number of samples respectively; calculates the conditional information entropy of the current training sample set under the discrete attributes according to the number of subset samples, the total number of samples and the multiple branch nodes; determines the main training feature set and the auxiliary training feature set according to the information entropy and the conditional information entropy; in the above manner, the decision tree learning algorithm is used to effectively process the current training sample set with classification labels, and then after calculating the information entropy and the conditional information entropy respectively, the main training feature set and the randomly sampled auxiliary training feature set are further selected, so as to effectively improve the accuracy of determining the main training feature set and the auxiliary training feature set, thereby increasing the diversity and generalization ability of the decision tree model.

[0112] For example, to help understand the implementation process of the model optimization method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A swimlane schematic diagram of a model optimization method is provided, specifically:

[0113] Taking Participant 1 and Participant 2 as examples, specifically: Participant 1 and Participant 2 respectively calculate the conditional information entropy of the current training sample set under discrete attributes according to the number of subset samples, the total number of samples and multiple branch nodes, and then calculate the information gain of the discrete attribute for the current training sample set in combination with the information entropy, and then sort each training feature according to the preset sorting relationship, and select the main training feature set from the current training sample set according to the training feature sorting result, and sample the auxiliary training feature set according to the remaining training features in the current training sample set through a random sampling strategy, and then train the decision tree model according to the main training feature set and the auxiliary training feature set respectively. For Participant 1, train decision tree model 1, and for Participant 2, train decision tree model 2, and then Participant 1 and Participant 2 respectively send the parameter update information and feature importance information of their respective decision tree models to the central server, and perform aggregation update of the global model through the central server. At this time, Participant 1 and Participant 2 perform model optimization according to the current aggregation update result, thereby effectively improving the accuracy of the optimization model and improving the performance of federated learning.

[0114] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the model optimization method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0115] This application also provides a model optimization device, please refer to Figure 5 , the model optimization device comprises:

[0116] The training module 10 is used to train the decision tree model according to the main training feature set and the auxiliary training feature set.

[0117] The acquisition module 20 is used to obtain parameter update information and feature importance information of the decision tree model.

[0118] The aggregation update module 30 is used to perform aggregation update of the global model according to the parameter update information and the feature importance information to obtain a current aggregation update result.

[0119] The optimization module 40 is used to optimize the decision tree model according to the current aggregation update result.

[0120] This embodiment trains a decision tree model according to a main training feature set and an auxiliary training feature set; obtains parameter update information and feature importance information of the decision tree model; performs aggregation update of a global model according to the parameter update information and the feature importance information to obtain a current aggregation update result; optimizes the decision tree model according to the current aggregation update result; in the above manner, a decision tree model is trained using a target training feature set that has been scientifically feature selected and sampled, which can more effectively and fully utilize all features, and introduces a target algorithm that combines federated learning feature influence aggregation and model collaborative training to perform aggregation update of the global model, and then optimizes the decision tree model using the current aggregation update result to ensure that the local decision tree model can effectively perform collaborative work tasks with the global model, thereby effectively improving the accuracy of the optimization model and improving the performance of federated learning.

[0121] The model optimization device provided by the present application adopts the model optimization method in the above embodiment, which can solve the technical problem of low accuracy of the optimization model in the prior art. Compared with the prior art, the beneficial effects of the model optimization device provided by the present application are the same as the beneficial effects of the model optimization method provided by the above embodiment, and the other technical features in the model optimization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0122] In one embodiment, the training module 10 is also used to obtain the current training sample set of each participant, and determine the proportion of target class samples in the current training sample set and the discrete attributes; calculate the information entropy of the current training sample set according to the proportion of the target class samples; divide the current training sample set according to the discrete attributes to obtain multiple branch nodes; count the number of subset samples and the total number of samples respectively; calculate the conditional information entropy of the current training sample set under the discrete attributes according to the number of subset samples, the total number of samples and the multiple branch nodes; determine the main training feature set and the auxiliary training feature set according to the information entropy and the conditional information entropy.

[0123] In one embodiment, the training module 10 is also used to calculate the information gain of the discrete attribute for the current training sample set based on the information entropy and the conditional information entropy; determine the importance value of each training feature in the current training sample set based on the information gain; sort the training features according to the importance value according to a preset sorting relationship; select a main training feature set from the current training sample set based on the training feature sorting result; and determine an auxiliary training feature set based on the remaining training features in the current training sample set through a random sampling strategy.

[0124] In one embodiment, the aggregation update module 30 is also used to obtain the weight of each participant and the local importance value reported by each participant; calculate the global importance value of each feature based on the feature importance information and the weight of each participant based on the central server; perform feature importance aggregation of the global model according to the global importance value and feature importance information of each feature through the target algorithm; obtain the gradient of the loss function with respect to the model parameters; update the parameters of the global model according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information through the target algorithm; and obtain the current aggregation update result according to the aggregation result and the update result.

[0125] In one embodiment, the aggregation update module 30 is also used to obtain a newly added feature importance value after detecting that each participant has received the global model; evaluate the newly added feature importance value and the local importance value through a target evaluation algorithm to obtain a new local importance value; calculate new global importance values ​​of each feature based on the central server according to the new local importance value and the weights of each participant; aggregate the feature importance of the global model according to the new global importance values ​​and the new local importance values ​​of each feature through a target algorithm; and update the current aggregation update result according to the new aggregation result.

[0126] In one embodiment, the optimization module 40 is also used to obtain global model parameter information and feature information gain based on the current aggregation update result; adjust the feature usage information and feature weight of the decision tree model according to the global model parameter information; update the parameters of the decision tree model according to the feature information gain; and adjust the training strategy of the decision tree model according to the performance evaluation result of the global model.

[0127] The present application provides a model optimization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model optimization method in the above-mentioned embodiment one.

[0128] Reference below Figure 6 , which shows a schematic diagram of the structure of a model optimization device suitable for implementing an embodiment of the present application. The model optimization device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The model optimization device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0129] like Figure 6As shown, the model optimization device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the model optimization device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the model optimization device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a model optimization device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0130] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0131] The model optimization device provided by the present application adopts the model optimization method in the above embodiment, which can solve the technical problem of low accuracy of the optimization model in the prior art. Compared with the prior art, the beneficial effects of the model optimization device provided by the present application are the same as the beneficial effects of the model optimization method provided by the above embodiment, and the other technical features in the model optimization device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0132] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0133] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0134] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the model optimization method in the above-mentioned embodiment.

[0135] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0136] The computer-readable storage medium may be included in the model optimization device; or may exist independently without being assembled into the model optimization device.

[0137] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0140] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned model optimization method, and can solve the technical problem of low accuracy of the optimization model in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the model optimization method provided in the above-mentioned embodiment, and will not be repeated here.

[0141] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned model optimization method when executed by a processor.

[0142] The computer program product provided by this application can solve the technical problem of low accuracy of the optimization model in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the model optimization method provided by the above embodiment, which will not be repeated here.

[0143] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A model optimization method, characterized in that: The method comprises: Train a decision tree model based on the main training feature set and the auxiliary training feature set; Obtaining parameter update information and feature importance information of the decision tree model; Performing aggregation update of the global model according to the parameter update information and the feature importance information to obtain a current aggregation update result; The decision tree model is optimized according to the current aggregate update result.

2. The method according to claim 1, characterized in that Before the step of training the decision tree model according to the main training feature set and the auxiliary training feature set, the method further includes: Obtaining the current training sample set of each participant, and determining the proportion of target class samples in the current training sample set and discrete attributes; Calculate the information entropy of the current training sample set according to the proportion of the target class samples; Dividing the current training sample set according to the discrete attributes to obtain a plurality of branch nodes; Count the number of subset samples and the total number of samples respectively; Calculate the conditional information entropy of the current training sample set under the discrete attribute according to the number of subset samples, the total number of samples and the multiple branch nodes; A main training feature set and an auxiliary training feature set are determined according to the information entropy and the conditional information entropy.

3. The method according to claim 2, characterized in that The step of determining the main training feature set and the auxiliary training feature set according to the information entropy and the conditional information entropy comprises: Calculate the information gain of the discrete attribute for the current training sample set according to the information entropy and the conditional information entropy; Determine the importance value of each training feature in the current training sample set according to the information gain; Sorting the training features according to the importance values ​​according to a preset sorting relationship; Selecting a main training feature set from the current training sample set according to the training feature sorting result; The auxiliary training feature set is determined based on the remaining training features in the current training sample set through a random sampling strategy.

4. The method according to claim 1, characterized in that The step of performing aggregate update of the global model according to the parameter update information and the feature importance information to obtain a current aggregate update result includes: Obtain the weight of each participant and the local importance value reported by each participant; The central server calculates the global importance value of each feature according to the feature importance information and the weight of each participant; Performing feature importance aggregation of the global model according to the global importance values ​​and feature importance information of each feature through a target algorithm; Get the gradient of the loss function with respect to the model parameters; The parameters of the global model are updated according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information by the target algorithm; Get the current aggregation update result based on the aggregation result and update result.

5. The method according to claim 4, characterized in that After the step of updating the parameters of the global model according to the gradient of the loss function with respect to the model parameters, the learning rate, the proximal term penalty coefficient, the iterative model parameters and the parameter update information by the target algorithm, the step further includes: After detecting that each participant receives the global model, the newly added feature importance values ​​are obtained; The target evaluation algorithm is used to evaluate the newly added feature importance value and the local importance value to obtain a new local importance value; Calculating new global importance values ​​of each feature based on the central server according to the new local importance values ​​and the weights of each participant; Performing feature importance aggregation of the global model according to the new global importance values ​​of each feature and the new local importance values ​​through a target algorithm; The current aggregation update result is updated according to the new aggregation result.

6. The method according to any one of claims 1 to 5, characterized in that The step of optimizing the decision tree model according to the current aggregate update result includes: Obtaining global model parameter information and feature information gain according to the current aggregation update result; According to the global model parameter information, the feature usage information and the feature weight of the decision tree model are adjusted respectively; Updating the parameters of the decision tree model according to the feature information gain; The training strategy of the tree-planning model is adjusted according to the performance evaluation result of the global model.

7. A model optimization device, characterized in that: The device comprises: A training module, used for training a decision tree model according to a main training feature set and an auxiliary training feature set; An acquisition module, used to obtain parameter update information and feature importance information of the decision tree model; An aggregation update module, used to perform aggregation update of the global model according to the parameter update information and the feature importance information to obtain a current aggregation update result; An optimization module is used to optimize the decision tree model according to the current aggregation update result.

8. A model optimization device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the model optimization method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model optimization method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the model optimization method according to any one of claims 1 to 6 are implemented.

Images