Federated Modeling Method, Device, Medium and Program Product of Random Forest Model

Abstract

The present application discloses a federated modeling method, device, medium, and program product for a random forest model, relating to the field of artificial intelligence. The method includes: receiving different model information sent by different federated participants after respectively establishing an interactive random forest model; merging the different model information to obtain an aggregated random forest model, and distributing the aggregated random forest model to different federated participants for the corresponding different federated participants to receive the aggregated random forest model and feedback whether a preset federated training end condition is reached based on the aggregated random forest model; if feedback information indicating that the preset federated training end condition is not reached is received, returning to the step of receiving different model information sent by different federated participants after respectively establishing an interactive random forest model until feedback information indicating that the preset federated training end condition is reached is received from the corresponding federated participant to obtain a target federated random forest model. The present invention can improve the modeling efficiency, and ensure the model performance and data privacy and security.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular, to a federated modeling method, device, computer-readable storage medium, and computer program product for a random forest model. Background Art

[0002] With the continuous development of fintech, especially Internet technology finance, more and more technologies (such as distributed, blockchain, artificial intelligence, etc.) are applied in the financial field. However, the financial industry also poses higher requirements for technologies. For example, the financial industry also has higher requirements for modeling.

[0003] Due to the good interpretability of the random forest model, it is widely used in fields such as finance and healthcare. That is, financial institutions will use the random forest model for financial risk control modeling, and medical institutions will use the interactive random forest model for medical data analysis modeling, etc. However, with the increasing emphasis on data privacy in recent years and the release of corresponding policies and regulations, the traditional approach of aggregating data from various participating parties and performing machine learning modeling on the data is no longer feasible.

[0004] In summary, how to further ensure the privacy and security of modeling data while improving the modeling efficiency of building a random forest model and ensuring the accuracy and robustness of the model has become an urgent technical problem in the industry. Summary of the Invention

[0005] The main purpose of this application is to provide a federated modeling method, device, storage medium, and program product for a random forest model, aiming to further ensure the privacy and security of modeling data based on multi-person collaborative modeling while improving the modeling efficiency of building a random forest model and ensuring the accuracy and robustness of the model.

[0006] To achieve the above object, this application provides a federated modeling method for a random forest model, which is applied to a server. The federated interactive modeling method for the random forest model includes:

[0007] Receiving different model information sent by different federated participants after respectively establishing an interactive random forest model;

[0008] Merging the different model information to obtain an aggregated random forest model, and distributing the aggregated random forest model to the different federated participants for the corresponding different federated participants to receive the aggregated random forest model and feedback whether a preset federated training end condition is reached based on the aggregated random forest model;

[0009] If feedback information indicating that the preset end condition for federated training is not met is received, return to the step of receiving different model information sent after different federated participants respectively establish interactive random forest models until feedback information indicating that the preset end condition for federated training is met is received from the corresponding federated participant, so as to obtain a target federated random forest model.

[0010] Optionally, the step of combining the different model information to obtain an aggregated random forest model includes:

[0011] Determine the same first decision trees in the interactive random forest models of different federated participants based on the different model information;

[0012] Determine the different second decision trees in the interactive random forest models based on the different model information;

[0013] Extract any target decision tree from each of the first decision trees, and combine the target decision tree and each of the second decision trees to obtain an aggregated random forest model.

[0014] Optionally, the step of receiving different model information sent after different federated participants respectively establish interactive random forest models includes:

[0015] Receive different model information sent after different federated participants respectively establish interactive random forest models, where the different federated participants obtain a random forest model to be operated and an operation task for operating the random forest model to be operated, and operate the random forest model to be operated according to the operation task to obtain an interactive random forest model, and determine model information according to the interactive random forest model.

[0016] Optionally, the step of distributing the aggregated random forest model to the different federated participants for the corresponding different federated participants to receive the aggregated random forest model and feedback whether the preset end condition for federated training is met based on the aggregated random forest model includes:

[0017] Distribute the aggregated random forest model to the different federated participants for the corresponding different federated participants to continue local iterative training and federated training on the aggregated random forest model after receiving the aggregated random forest model, and feedback whether the preset end condition for federated training is met, where the preset end condition for federated training includes reaching a preset model metric threshold or reaching a preset maximum number of federated training iteration rounds.

[0018] Optionally, after the step of returning to receive different model information sent after different federal participants establish interactive random forest models respectively until the feedback information indicating that the preset federal training end condition is reached by the corresponding federal participant to obtain the target federal random forest model when the feedback information indicating that the preset federal training end condition is not reached is received, the method includes:

[0019] Obtain the data to be processed, and input the data to be processed into the target federal random forest model;

[0020] Perform prediction processing on the data to be processed based on the target federal random forest model to obtain a prediction result.

[0021] Optionally, the data to be processed includes data to be processed for loans or data to be processed for medical treatment;

[0022] The step of performing prediction processing on the data to be processed based on the target federal random forest model to obtain a prediction result includes:

[0023] When the data to be processed is data to be processed for loans, perform prediction processing on the data to be processed for loans based on the target federal random forest model to obtain a prediction result on whether a loan can be obtained; or,

[0024] When the data to be processed is data to be processed for medical treatment, perform prediction processing on the data to be processed for medical treatment based on the target federal random forest model to obtain a probability value that the data to be processed for medical treatment is a preset result.

[0025] In addition, to achieve the above object, the present application further provides a method for federated modeling of a random forest model, which is applied to different federal participants in the method for federated modeling of a random forest model as described above. The method for federated modeling of a random forest model includes:

[0026] After establishing an interactive random forest model, upload the model information of the interactive random forest model to the server;

[0027] Receive the aggregated random forest model obtained by the server by merging the model information, and based on the aggregated random forest model, feedback whether the preset federal training end condition is reached;

[0028] If the feedback information indicating that the preset federal training end condition is not reached is sent to the server, return to the step of uploading the model information of the interactive random forest model to the server after establishing the interactive random forest model until the feedback information indicating that the preset federal training end condition is reached is sent to the server for the server to obtain the target federal random forest model.

[0029] In addition, to achieve the above object, the present application further provides a federated modeling device for a random forest model, which is applied to a server. The federated interactive modeling device for the random forest model includes:

[0030] A first receiving module, configured to receive different model information sent by different federated participants after respectively establishing an interactive random forest model;

[0031] A merging and distributing module, configured to merge the different model information to obtain an aggregated random forest model, and distribute the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model and feedback whether a preset federated training end condition is reached based on the aggregated random forest model;

[0032] A first judging module, configured to, if feedback information indicating that the preset federated training end condition is not reached is received, return to the step of receiving different model information sent by different federated participants after respectively establishing an interactive random forest model, until feedback information indicating that the preset federated training end condition is reached is received from the corresponding federated participant, so as to obtain a target federated random forest model;

[0033] Among them, the federated interactive modeling device for the random forest model is further applied to different federated participants connected to the server. The federated interactive modeling device for the random forest model further includes:

[0034] A model building module, configured to upload the model information of the interactive random forest model to the server after establishing the interactive random forest model;

[0035] A second receiving module, configured to receive the aggregated random forest model obtained by the server by merging the model information, and feedback whether a preset federated training end condition is reached based on the aggregated random forest model;

[0036] A second judging module, configured to, if feedback information indicating that the preset federated training end condition is not reached is sent to the server, return to the step of uploading the model information of the interactive random forest model to the server after establishing the interactive random forest model, until feedback information indicating that the preset federated training end condition is reached is sent to the server, so that the server can obtain a target federated random forest model.

[0037] The present application further provides a federated modeling device for a random forest model. The federated modeling device for the random forest model is an entity device. The federated modeling device for the random forest model includes: a memory, a processor, and a program of the federated modeling method for the random forest model stored on the memory and executable on the processor. When the program of the federated modeling method for the random forest model is executed by the processor, the steps of the federated modeling method for the random forest model as described above can be implemented.

[0038] The present application also provides a computer-readable storage medium, on which a program for implementing the federated modeling method of the random forest model is stored. When the program of the federated modeling method of the random forest model is executed by a processor, the steps of the federated modeling method of the random forest model as described above are implemented.

[0039] The present application also provides a computer program product, including a computer program, which when executed by a processor implements the steps of the method for federated modeling of the random forest model described in any one of the above.

[0040] A method, device, equipment, computer-readable storage medium and computer program product for federated modeling of a random forest model provided by the present application, wherein the server receives different model information sent by different federated participants after respectively establishing an interactive random forest model; combines the different model information to obtain an aggregated random forest model, and distributes the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model and feedback whether a preset federated training end condition is reached based on the aggregated random forest model; if feedback information indicating that the preset federated training end condition is not reached is received, return to the step of receiving different model information sent by different federated participants after respectively establishing an interactive random forest model, until feedback information indicating that the preset federated training end condition is reached is fed back by the corresponding federated participant, so as to obtain a target federated random forest model.

[0041] The present application realizes the joint modeling of an interactive random forest model by multiple federated participants, so that the modeling process of the interactive random forest model can use more training data on the basis of ensuring data privacy and security, thereby improving the construction efficiency of the random forest model, ensuring the accuracy and robustness of the model, and more importantly, ensuring the privacy and security of the modeling data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 It is a schematic flowchart of the first embodiment of the method for federated modeling of the random forest model of the present application;

[0045] Figure 2 It is a schematic diagram of an interactive operation interface in the application scenario involved in the federated modeling method of the random forest model of this application;

[0046] Figure 3 It is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of this application.

[0047] The realization of the purpose, functional characteristics and advantages of this application will be further described in combination with the embodiments with reference to the accompanying drawings. Specific Embodiments

[0048] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0049] The embodiment of this application provides a federated modeling method for a random forest model. In the first embodiment of the federated modeling method of the random forest model of this application, refer to Figure 1 , the federated modeling method of the random forest model includes:

[0050] Step S10, receiving different model information sent by different federated participants after respectively establishing an interactive random forest model;

[0051] Step S20, merging the different model information to obtain an aggregated random forest model, and distributing the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model, and based on the aggregated random forest model, feedback whether the preset federated training end condition is reached;

[0052] Step S30, if feedback information that does not reach the preset federated training end condition is received, return to the step of receiving different model information sent by different federated participants after respectively establishing an interactive random forest model, until the feedback information that reaches the preset federated training end condition is fed back by the corresponding federated participant, so as to obtain a target federated random forest model.

[0053] The specific steps are as follows:

[0054] Step S10, receiving different model information sent by different federated participants after respectively establishing an interactive random forest model;

[0055] In this embodiment, it should be noted that the federated modeling method of the random forest model is applied to the federated modeling device of the random forest model. The federated modeling device of the random forest model belongs to the federated modeling device of the random forest model. At present, with the increasing attention to data privacy and the release of corresponding policies and regulations, the traditional method of aggregating the data of each participating party to one place and performing machine learning modeling on the data is no longer feasible. Therefore, how to further ensure the privacy and security of the modeling data while improving the modeling efficiency of the random forest model and ensuring the accuracy and robustness of the model has become an urgent technical problem in the industry.

[0056] In this embodiment, it should be noted that the interactive random forest model can include three layers of interaction methods or three layers of interaction meanings:

[0057] Interaction method 1: It means that the user can operate on the random forest model displayed in the visual interaction interface to trigger an interaction operation instruction. Then, based on the interaction operation instruction, an interactive random forest model is obtained. Among them, the random forest model displayed in the visual interaction interface can be one or more;

[0058] Interaction method 2: Through manual intervention or human interaction, conventional interaction operations such as retaining, deleting, and screening decision trees can be performed on the displayed random forest model;

[0059] Interaction method 3: Through manual intervention or human interaction, operations such as adjusting the composition of decision trees and adjusting the decision tree merging composition rules can be performed on the displayed random forest model.

[0060] It should be noted that in this embodiment, the application scenario of the federated modeling method of the random forest model can be: a random forest model that needs to be effectively predicted and is subject to manual intervention. In order to improve the generalization ability of the model, during the modeling process, the personnel participating in the modeling are divided into multiple groups. That is, through multiple groups of manual intervention, each group of participating parties obtains an interactive random forest model corresponding to the group based on the intervention method of the group and the data within the group. Then, the random forest models obtained by federating or combining multiple groups are used to obtain the target federated random forest model. It should be emphasized that during the local iterative training process of the participating parties, manual intervention is performed, such as the selection of manual intervention reference factors or the selection of the reference factor range.

[0061] It should be noted that in this embodiment, the application scenario of the federated modeling method of the random forest model can also be: a classification random forest model that needs to be effectively predicted and intervened by humans. In order to improve the generalization ability of the model, during the modeling process, the people participating in the modeling are divided into multiple groups, that is, through multiple groups of human interventions. Each participating party obtains the classification random forest model corresponding to its own group based on the intervention method of its own group and the classification data within its own group. Then, the classification random forest models obtained from multiple groups are federated or combined to obtain the target classification federated random forest model.

[0062] Specifically, each participating party has a classification model dataset: X, and X contains n data items x1, x2, x3,..., xi,..., xn, and each data item xi contains k numerical values (xi1, xi2, xi3, xi4,..., xik). The main participating party's corresponding model dataset has a classification label Y: containing m data items y1, y2, y3, y4,..., yi, etc. For the classification model (binary classification or multi-classification), yi is discrete.

[0063] Or the application scenario of the federated modeling method of the random forest model can also be: a regression random forest model that needs to be effectively predicted and intervened by humans. In order to improve the generalization ability of the model, during the modeling process, the people participating in the modeling are divided into multiple groups, that is, through multiple groups of human interventions. Each participating party obtains the regression random forest model corresponding to its own group based on the intervention method of its own group and the regression data within its own group. Then, the regression random forest models obtained from multiple groups are federated or combined to obtain the target federated regression random forest model.

[0064] Specifically, each participating party has a model dataset: X, and X contains n data items x1, x2, x3,..., xi,..., xn, and each data item xi contains k numerical values (xi1, xi2, xi3, xi4,..., xik). In this embodiment, the model dataset has a label Y: containing m data items y1, y2, y3, y4,..., yi, etc. For the regression model (binary classification or multi-classification), yi is continuous.

[0065] Receive different model information sent by different federal participants after they respectively establish interactive random forest models. That is, different federal participants respectively establish interactive random forest models through interaction and then respectively send the corresponding local model information to the server. Among them, the model information can be information such as model parameters. Specifically, the model parameters can be max_features (the maximum number of features that a single decision tree in the random forest is allowed to use), Auto / None (simply select all features, and each tree in the random forest can utilize these selected features), sqrt (this option means that each subtree can utilize the square root of the total number of features), 0.2 (allow each subtree of the random forest to utilize 20% of the number of variables (features)), and n_estimators (the number of subtrees that the user wants to build before predicting using the maximum vote or average), and so on. Specifically, after Party A establishes Interactive Random Forest Model 1, it sends the model information 1 corresponding to Interactive Random Forest Model 1 to the server. After Party B establishes Interactive Random Forest Model 2, it also sends the model information 2 corresponding to Interactive Random Forest Model 2 to the server.

[0066] Further, in a feasible embodiment, the above step S10, receiving different model information sent by different federal participants after they respectively establish interactive random forest models, may include:

[0067] Step S11, receiving different model information sent by different federal participants after they respectively establish interactive random forest models;

[0068] It should be noted that, in this embodiment, the different federal participants obtain the random forest model to be operated and the operation task for operating the random forest model to be operated, and operate the random forest model to be operated according to the operation task to obtain an interactive random forest model, and determine the model information according to the interactive random forest model.

[0069] In addition, in this embodiment, the manner in which each participant establishes an interactive random forest model is described. Specifically, the training data for training the interactive random forest model is respectively stored in multiple participants. That is, each participant locally stores some training data. The server side can be pre-connected to multiple participants, or the server side can establish a connection with the participant when the participant requests a connection. It should be noted that this embodiment does not limit the organization method of each participant.

[0070] During the interactive modeling process of each participant, the user of the corresponding participant can, for example, attach Figure 2An interactive operation interface for the visualized output, where operations are performed on the random forest model displayed in this interactive interface to trigger interactive operation instructions. The participating parties can determine the random forest model to be operated on and the operation tasks for operating on the random forest model based on these interactive operation instructions. Among them, depending on the different interactive operation instructions triggered by the user, the operation tasks are different. For example, if the user wants to delete a certain tree of the random forest model displayed on the current interface and triggers an interactive operation instruction through the interactive interface, then the random forest model to be operated on obtained by the participating parties is the random forest model displayed on the current interface, and the operation task is to delete a certain decision tree in this random forest model. In order to establish a unified federated model, each participating party user needs to determine the corresponding and consistent random forest model to be operated on and the operation tasks.

[0071] Certainly, during the interactive modeling process of each participating party, to ensure the unity of federated modeling, the corresponding user on the server can perform operations on the random forest model displayed in the interactive interface of the visualized output to trigger interactive operation instructions, and then determine the random forest model to be operated on and the operation tasks for operating on the random forest model based on the interactive operation instructions. Furthermore, the server distributes the obtained random forest model and the corresponding operation tasks to each participating party. Among them, depending on the different interactive operation instructions triggered by the user, the operation tasks are different.

[0072] Specifically, in this embodiment, the operation tasks can include conventional interactive operations such as visualizing the output of the random forest model, deleting one or more decision trees in the random forest model, retaining one or more decision trees in the random forest model, and screening one or more decision trees in the random forest model, etc., and can also include interactive operations such as classification prediction and regression prediction. According to different operation tasks, the participating parties perform different operations on the random forest model. For example, when the operation task is classification prediction, the participating party needs to determine the classification prediction rule and then make a prediction based on this classification prediction rule, or when the operation task is regression prediction, the participating party needs to determine the regression prediction rule and then make a prediction based on this regression prediction rule.

[0073] After each participating party performs local operations, i.e., local training, on the random forest model, the participating party determines the model information of the federation based on the random forest model after the operation. Specifically, each participating party continuously performs local iterative training on the random forest model after the operation with local training data, and during the iterative training process, obtains the model information after manual intervention. For example, the model information after manual intervention includes the number of decision trees in the random forest and max_features of each decision tree, etc. When the local training is completed, the model information of the trained interactive random forest model is obtained.

[0074] Specifically, for example, after the A participant finishes training, the model information of the interactive random forest model 1 includes: the model contains 10 decision trees, and each decision tree includes reference factors or node information such as age, income, and education level. The branching conditions of the reference factors or node information are respectively that the age is below or above 35 years old, the income is below or above 200,000 yuan per year, and the education level is below or above undergraduate degree. Or, for example, after the B participant finishes training, the model information of the interactive random forest model 2 includes: the model contains 8 decision trees, and each decision tree includes reference factors or node information such as age, household register, income, position, and credit. The branching conditions of the reference factors or node information are respectively that the age is below or above 40 years old, the income is below or above 150,000 yuan per year, the position is below or above supervisor, the household register is urban household register or non-urban household register, and the credit is greater than or less than a preset score, etc.

[0075] Step S20: Merge the different model information to obtain an aggregated random forest model, and distribute the aggregated random forest model to the different federal participants, so that the corresponding different federal participants can receive the aggregated random forest model and feedback whether the preset federal training end condition is met based on the aggregated random forest model;

[0076] In this embodiment, after the server obtains the different model information of different federal participants, it merges the different model information to obtain an aggregated random forest model. Among them, the merging method includes: taking the union. After the server obtains the aggregated random forest model, it distributes the aggregated random forest model to different federal participants, so that the corresponding different federal participants can receive the aggregated random forest model and feedback whether the preset federal training end condition is met based on the aggregated random forest model. Among them, the preset federal training end condition can be that the training reaches a preset number of times, or the accuracy rate reaches a preset value, and meets preset indicators such as the KS value (Kolmogorov-Smirnov) and AUC (Area Under the Curve) value.

[0077] It should be noted that in this embodiment, the feedback on whether the preset federal training end condition is met can be provided by the main participant, that is, it is provided by the participant with a preset labeled dataset.

[0078] In a feasible embodiment, in the above step S20, the step of "merging the different model information to obtain an aggregated random forest model" may include:

[0079] Step S21: Determine the same first decision trees in the interactive random forest models of different federal participants based on the different model information;

[0080] Step S22: Determine each different second decision tree in the interactive random forest model based on the different model information.

[0081] It should be noted that in this embodiment, each decision tree included in the random forest model has various multi-level branches, branches at each level, etc. Thus, after the server obtains the different model information, it can determine the first decision trees with the same same-level node information and the second decision trees with different same-level node information among the decision trees included in the interactive random forest models established locally by different federated participants.

[0082] Specifically, the server can determine whether the reference factors of the same-level nodes of the decision trees included in the interactive random forest models established locally by different federated participants are the same, so as to determine the first decision trees with the same same-level node information and the second decision trees with different same-level node information among the decision trees included in the interactive random forest models established locally by different federated participants.

[0083] Step S23: Extract any one target decision tree from each of the first decision trees, and merge the target decision tree and each of the second decision trees to obtain an aggregated random forest model.

[0084] It should be noted that in this embodiment, when the server determines that there are two or more identical first decision trees among the decision trees included in the interactive random forest models established locally by different federal participants, the server immediately uses any one of the two or more identical first decision trees as the target decision tree, and combines the target decision tree with one or more other second decision trees that are different among the decision trees included in the interactive random forest models respectively to form an aggregated random forest model. Specifically, for example, the interactive random forest model 1 established locally by Party A includes 10 decision trees: Decision Tree 1, Decision Tree 2... Decision Tree 10, and the interactive random forest model 2 established locally by Party B includes 8 decision trees: Decision Tree 11, Decision Tree 12... Decision Tree 18. When the server determines the reference factors of the same-level nodes of Decision Tree 1 to Decision Tree 18 respectively, it is found that the reference factors of Decision Tree 3 and Decision Tree 17 are both age, income, education level, position, and credit. Therefore, the server determines that among Decision Tree 1 to Decision Tree 18, Decision Tree 3 and Decision Tree 17 are the first decision trees with the same same-level node information, and each of the decision trees other than Decision Tree 3 and Decision Tree 17 among Decision Tree 1 to Decision Tree 18 is a second decision tree with different same-level node information. Furthermore, the server extracts only one first decision tree from Decision Tree 3 and Decision Tree 17: Decision Tree 3 or Decision Tree 17, and then combines Decision Tree 3 or Decision Tree 17 with each of the second decision trees other than Decision Tree 3 and Decision Tree 17 among Decision Tree 1 to Decision Tree 18 to construct an aggregated random forest model.

[0085] Step S30, if feedback information indicating that the preset federal training end condition is not reached is received, return to the step of receiving different model information sent after different federal participants respectively establish interactive random forest models until feedback information indicating that the preset federal training end condition is reached is received from the corresponding federal participant, so as to obtain the target federal random forest model.

[0086] When each participant feedbacks that the preset federal training end condition is not reached, or when the main participant (which can be preset) among each participant feedbacks that the preset federal training end condition is not reached, that is, when the server receives feedback information indicating that the preset federal training end condition is not reached, return to the step of receiving different model information sent after different federal participants respectively establish interactive random forest models. That is, each participant retrains the aggregated random forest model distributed by the server based on the local training data until the corresponding federal participant feedbacks that the preset federal training end condition is reached, so as to obtain the target federal random forest model.

[0087] In this embodiment, it should be noted that the preset end condition of the federated training can be that the training reaches a preset number of times or reaches the AUC index, etc.

[0088] A federated modeling method for a random forest model provided by this application. The server receives different model information sent by different federated participants after they respectively establish an interactive random forest model; combines the different model information to obtain an aggregated random forest model, and distributes the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model and feedback whether the preset federated training end condition is reached based on the aggregated random forest model; if feedback information indicating that the preset federated training end condition is not reached is received, return to the step of receiving different model information sent by different federated participants after they respectively establish an interactive random forest model, until feedback information indicating that the preset federated training end condition is reached is received from the corresponding federated participant, so as to obtain a target federated random forest model. This application realizes the joint modeling of an interactive random forest model by multiple federated participants, enabling the modeling process of the interactive random forest model to use more training data while ensuring data privacy and security, thereby improving the construction efficiency of the random forest model, ensuring the accuracy and robustness of the model, and more importantly, ensuring the privacy and security of the modeling data.

[0089] Further, based on the first embodiment in this application, a second embodiment of the federated modeling method for a random forest model provided by this application is proposed. In the second embodiment of the federated modeling method for a random forest model provided by this application, in the above step S20, the step of "distributing the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model and feedback whether the preset federated training end condition is reached based on the aggregated random forest model" may include:

[0090] Step A1: Distribute the aggregated random forest model to the different federated participants, so that after the corresponding different federated participants receive the aggregated random forest model, continue to perform local iterative training and federated training on the aggregated random forest model, and feedback whether the preset federated training end condition is reached, where the preset federated training end condition includes reaching a preset model metric threshold or reaching a preset maximum number of federated training iteration rounds.

[0091] It should be noted that, in this embodiment, the aggregated random forest model is distributed to the different federal participants, so that after receiving the random forest model, the corresponding different federal participants continue to perform local iterative training on the aggregated random forest model. After the number of local iterative training reaches the preset maximum number of federal training iterations, they perform federal training with the server again to feedback whether the preset federal training end condition is reached. That is, in this embodiment, after the server aggregates the aggregated random forest model each time, the participants continue to perform local training based on the aggregated random forest model and the local training data until the local iterative training reaches the preset number of local training times, and then perform aggregation of model parameters, and continuously repeat the process of local training and model parameter aggregation until the preset federal training end condition is reached.

[0092] In addition, in another feasible embodiment, after receiving the random forest model, the corresponding different federal participants can also continue to perform local iterative training on the aggregated random forest model, and after the model metrics of the aggregated random forest model after local iterative training reach the preset model metric threshold, they perform federal training with the server again to feedback whether the preset federal training end condition is reached.

[0093] In this embodiment, by receiving different model information sent after different federal participants respectively establish interactive random forest models; wherein, the different federal participants obtain the random forest model to be operated and the operation tasks for operating the random forest model to be operated, and operate the random forest model to be operated according to the operation tasks to obtain an interactive random forest model, and determine model information according to the interactive random forest model. This lays a foundation for accurately obtaining the target federal random forest model.

[0094] Further, based on the first and second embodiments of a federal modeling method for a random forest model provided in the present application, a third embodiment of a federal modeling method for a random forest model provided in the present application is proposed. In the third embodiment of a federal modeling method for a random forest model provided in the present application, in step S30 above, if feedback information that does not reach the preset federal training end condition is received, return to the step of receiving different model information sent after different federal participants respectively establish interactive random forest models until feedback information indicating that the preset federal training end condition is reached is fed back by the corresponding federal participant to obtain the target federal random forest model. After that, a federal modeling method for a random forest model provided in the present application may further include:

[0095] Step S40, obtain the data to be processed, and input the data to be processed into the target federal random forest model;

[0096] Step S50: Perform prediction processing on the data to be processed based on the target federated random forest model to obtain a prediction result.

[0097] In this embodiment, after the server obtains the target federated random forest model, when it detects data to be processed, it acquires the data to be processed and inputs the data to be processed into the target federated random forest model. Since the target federated random forest model is a trained model, it can perform prediction processing on the data to be processed based on the target federated random forest model to obtain a corresponding prediction result.

[0098] It should be noted that in this embodiment, the prediction result can be a classification result or a predicted probability value.

[0099] In this embodiment, by acquiring the data to be processed, the data to be processed is input into the target federated random forest model; prediction processing is performed on the data to be processed based on the target federated random forest model to obtain a prediction result. In this embodiment, the target federated random forest model is applied to accurately perform prediction processing on the data to be processed.

[0100] Further, in a feasible embodiment, the data to be processed includes data to be processed for loans or data to be processed for medical treatment;

[0101] The above step S50: Perform prediction processing on the data to be processed based on the target federated random forest model to obtain a prediction result, may include:

[0102] Step S51: When the data to be processed is data to be processed for loans, perform prediction processing on the data to be processed for loans based on the target federated random forest model to obtain a prediction result on whether a loan can be obtained;

[0103] Step S52: Or when the data to be processed is data to be processed for medical treatment, perform prediction processing on the data to be processed for medical treatment based on the target federated random forest model to obtain the probability value that the data to be processed for medical treatment is a preset result.

[0104] In this embodiment, after the server models the target federated random forest model based on the above-mentioned federated modeling method of the random forest model, it can be applied to the type prediction of whether to grant a loan or can be applied to the prediction of nodule probability in the medical field. Specifically, when the data to be processed is loan data to be processed, the target federated random forest model is used to perform prediction processing on the loan data to be processed, and a prediction result of whether a loan can be granted is obtained. This prediction result can be that the user can be granted a loan or the user cannot be granted a loan. When the data to be processed is medical data to be processed, the target federated random forest model is used to perform prediction processing on the medical data to be processed, and a probability value that the medical data to be processed is a preset result is obtained. Among them, the preset result can be a lung nodule, and the probability value can be 0.9 or 0.8, etc.

[0105] In this embodiment, when the data to be processed is loan data to be processed, the target federated random forest model is used to perform prediction processing on the loan data to be processed, and a prediction result of whether a loan can be granted is obtained; or when the data to be processed is medical data to be processed, the target federated random forest model is used to perform prediction processing on the medical data to be processed, and a probability value that the medical data to be processed is a preset result is obtained. In this embodiment, the target federated random forest model accurately processes the medical data to be processed and the loan data to be processed.

[0106] Furthermore, based on the first embodiment, the second embodiment, and the third embodiment of the federated modeling method of a random forest model provided in the present application above, an embodiment of the federated modeling method of a random forest model provided in the present application applied to different federated participants connected to the server is proposed. In the embodiment of the present application, the federated modeling method of a random forest model provided in the present application may further include:

[0107] Step A, after establishing an interactive random forest model, upload the model information of the interactive random forest model to the server;

[0108] It should be noted that in this embodiment, the application scenario of the federated modeling method of the random forest model can be: a random forest model that needs to be effectively predicted and intervened by humans. In order to improve the generalization ability of the model, during the modeling process, the personnel participating in the modeling are divided into multiple groups, that is, it is intervened by multiple groups of humans. Each group of participants obtains an interactive random forest model corresponding to the group based on the intervention method of the group and the data within the group. Furthermore, the random forest models obtained by federating or combining multiple groups are used to obtain the target federated random forest model. It should be emphasized that during the local iterative training process of the participants, human intervention is performed, such as the selection of artificial intervention reference factors or the selection of the reference factor range.

[0109] It should be noted that in this embodiment, the application scenario of the federated modeling method of the random forest model can also be: it is necessary to establish an effective prediction and a classification random forest model with manual intervention. In order to improve the generalization ability of the model, therefore, during the modeling process, the population participating in the modeling is divided into multiple groups, that is, through multiple groups of manual intervention. Each group of participants obtains the classification random forest model corresponding to the group based on the intervention method of the group and the classification data within the group. Furthermore, the classification random forest models obtained by federating or combining multiple groups are used to obtain the target classification federated random forest model.

[0110] Specifically, each participant has a classification model dataset: X, and X contains n data items x1, x2, x3,..., xi,..., xn. Each data item xi contains k numerical values (xi1, xi2, xi3, xi4,..., xik). The main participant's corresponding model dataset has a classification label Y: containing m data items y1, y2, y3, y4,..., yi, etc. For a classification model (binary classification or multi-classification), yi is discrete.

[0111] Or the application scenario of the federated modeling method of the random forest model can also be: it is necessary to establish an effective prediction and a regression random forest model with manual intervention. In order to improve the generalization ability of the model, therefore, during the modeling process, the population participating in the modeling is divided into multiple groups, that is, through multiple groups of manual intervention. Each group of participants obtains the regression random forest model corresponding to the group based on the intervention method of the group and the regression data within the group. Furthermore, the regression random forest models obtained by federating or combining multiple groups are used to obtain the target federated regression random forest model.

[0112] Specifically, each participant has a model dataset: X, and X contains n data items x1, x2, x3,..., xi,..., xn. Each data item xi contains k numerical values (xi1, xi2, xi3, xi4,..., xik). In this embodiment, the model dataset has a label Y: containing m data items y1, y2, y3, y4,..., yi, etc. For a regression model (binary classification or multi-classification), yi is continuous.

[0113] After different federal participants respectively establish interactive random forest models locally and send different model information to the server, that is, after different federal participants respectively establish interactive random forest models through interaction and then respectively send the corresponding local model information to the server. Among them, the model information can be information such as model parameters. Specifically, the model parameters can be max_features (the maximum number of features allowed for a single decision tree in a random forest), Auto / None (simply select all features, and each tree in the random forest can utilize these selected features), sqrt (this option is that each subtree can utilize the square root of the total number of features), 0.2 (allow each subtree of a random forest to utilize 20% of the number of variables (features)), and n_estimators (the number of subtrees the user wants to establish before predicting using the majority vote or average), and so on. Specifically, after Party A establishes the interactive random forest model 1, it sends the model information 1 corresponding to the interactive random forest model 1 to the server. After Party B establishes the interactive random forest model 2, it also sends the model information 2 corresponding to the interactive random forest model 2 to the server.

[0114] Step B: Receive the aggregated random forest model obtained by the server by merging the model information, and based on the aggregated random forest model, feedback whether the preset federal training end condition is reached;

[0115] In this embodiment, after the server obtains the different model information of different federal participants, it merges the different model information to obtain an aggregated random forest model. Among them, the merging method includes: taking the union. After the server obtains the aggregated random forest model, it distributes the aggregated random forest model to different federal participants for the corresponding different federal participants to receive the aggregated random forest model and feedback whether the preset federal training end condition is reached based on the aggregated random forest model. Among them, the preset federal training end condition can be that the training reaches a preset number of times, or the accuracy reaches a preset value, and meets preset indicators such as the KS value (Kolmogorov-Smirnov) and AUC (Area Under the Curve) value.

[0116] It should be noted that in this embodiment, feedback on whether the preset federal training end condition is reached can be provided by the main participant, that is, provided by the participant with a preset labeled dataset.

[0117] It should be noted that in this embodiment, the server distributes the aggregated random forest model to the different federal participants, so that after the corresponding different federal participants receive the random forest model, they continue to perform local iterative training on the aggregated random forest model. After the number of local iterative training times reaches the preset local training times, they then perform federal training with the server to feedback again whether the preset federal training end condition is reached. That is, in this embodiment, after the server aggregates the aggregated random forest model each time, the participants continue to perform local training based on the aggregated random forest model and the local training data until the local iterative training reaches the preset local training times, and then perform aggregation of model parameters, and continuously repeat the process of local training and model parameter aggregation until the preset federal training end condition is reached.

[0118] Step C, if the feedback information indicating that the preset federal training end condition is not reached is sent to the server, return to the step of uploading the model information of the interactive random forest model to the server after establishing the interactive random forest model until the feedback information indicating that the preset federal training end condition is reached is sent to the server, so that the server can obtain the target federal random forest model.

[0119] When each participant feedbacks that the preset federal training end condition is not reached, or when the main participant (which can be preset) among each participant feedbacks that the preset federal training end condition is not reached, that is, when the server receives the feedback information indicating that the preset federal training end condition is not reached, return to the step of receiving the different model information sent by different federal participants after they respectively establish the interactive random forest model. That is, each participant re-trains the aggregated random forest model distributed by the server based on the local training data until the corresponding federal participant feedbacks that the preset federal training end condition is reached to obtain the target federal random forest model.

[0120] In this embodiment, it should be noted that the preset federal training end condition can be that the training reaches the preset number of times or reaches the AUC index, etc.

[0121] A federated modeling method for a random forest model provided by this application. After different federated participants locally establish an interactive random forest model, they upload the model information of the interactive random forest model to the server. Receive the aggregated random forest model obtained by the server merging the model information, and based on the aggregated random forest model, feedback whether the preset federated training end condition is reached. If the feedback information indicating that the preset federated training end condition is not reached is sent to the server, return to the step of establishing the interactive random forest model and uploading the model information of the interactive random forest model to the server until the feedback information indicating that the preset federated training end condition is reached is sent to the server, so that the server can obtain the target federated random forest model. This application realizes the joint modeling of an interactive random forest model by multiple federated participants, enabling the modeling process of the interactive random forest model to use more training data while ensuring data privacy and security. Thus, it not only improves the construction efficiency of the random forest model, ensures the accuracy and robustness of the model, but also guarantees the privacy and security of the modeling data.

[0122] Refer to Figure 3 , Figure 3 It is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of this application.

[0123] As Figure 3 shown, the federated modeling device for the random forest model may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0124] Optionally, the federated modeling device for the random forest model may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, sensors, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen and an input sub-module such as a keyboard. Optionally, the rectangular user interface may further include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0125] Those skilled in the art can understand, Figure 3The structure of the federated modeling device of the random forest model shown does not limit the federated modeling device of the random forest model, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0126] As Figure 3 shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, and a program of the federated modeling method of the random forest model. The operating system is a program that manages and controls the hardware and software resources of the federated modeling device of the random forest model, and supports the operation of the program of the federated modeling method of the random forest model and other software and / or programs. The network communication module is used to implement communication between components inside the memory 1005, and communication with other hardware and software in the system of the federated modeling method of the random forest model.

[0127] In Figure 3 the federated modeling device of the random forest model shown, the processor 1001 is used to execute the program of the federated modeling method of the random forest model stored in the memory 1005, and implement the steps of the federated modeling method of the random forest model described in any one of the above.

[0128] The specific implementation manner of the federated modeling device of the random forest model in this application is basically the same as each embodiment of the above federated modeling method of the random forest model, and will not be repeated here.

[0129] This application embodiment also provides a federated modeling device of a random forest model, which is applied to a server. The federated interactive modeling device of the random forest model includes:

[0130] A first receiving module, configured to receive different model information sent by different federated participants after respectively establishing an interactive random forest model;

[0131] A merging and distributing module, configured to merge the different model information to obtain an aggregated random forest model, and distribute the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model, and based on the aggregated random forest model, feedback whether the preset federated training end condition is reached;

[0132] A first judging module, configured to, if feedback information that does not reach the preset federated training end condition is received, return to the step of receiving different model information sent by different federated participants after respectively establishing an interactive random forest model, until the feedback information indicating that the preset federated training end condition is reached is fed back by the corresponding federated participant, so as to obtain a target federated random forest model;

[0133] Among them, the federated interactive modeling device of the random forest model is also applied to different federated participants connected to the server, and the federated interactive modeling device of the random forest model further includes:

[0134] A model construction module, configured to establish an interactive random forest model and upload the model information of the interactive random forest model to the server;

[0135] A second receiving module, configured to receive the aggregated random forest model obtained by the server by merging the model information, and feedback whether a preset federated training end condition is reached based on the aggregated random forest model;

[0136] A second judgment module, configured to, if feedback information indicating that the preset federated training end condition is not reached is sent to the server, return to the step of establishing an interactive random forest model and uploading the model information of the interactive random forest model to the server until feedback information indicating that the preset federated training end condition is reached is sent to the server, so that the server can obtain a target federated random forest model.

[0137] Optionally, the merging and distributing module includes:

[0138] A first determining unit, configured to determine the same first decision trees in the interactive random forest models of different federated participants based on the different model information;

[0139] A second determining unit, configured to determine the different second decision trees in the interactive random forest model based on the different model information;

[0140] A merging unit, configured to extract any one target decision tree from each of the first decision trees, and merge the target decision tree and each of the second decision trees to obtain an aggregated random forest model.

[0141] Optionally, the first receiving module is further configured to receive different model information sent by different federated participants after respectively establishing interactive random forest models, where the different federated participants obtain a random forest model to be operated and an operation task for operating the random forest model to be operated, and operate the random forest model to be operated according to the operation task to obtain an interactive random forest model, and determine model information according to the interactive random forest model.

[0142] Optionally, the merging and distributing module further includes:

[0143] A distribution unit for distributing the aggregated random forest model to the different federal participants, so that after the corresponding different federal participants receive the aggregated random forest model, they can continue to perform local iterative training and federal training on the aggregated random forest model and feedback whether a preset federal training end condition is reached, where the preset federal training end condition includes reaching a preset model metric threshold or reaching a preset maximum number of federal training iteration rounds.

[0144] Optionally, the federal interactive modeling device of the random forest model further includes:

[0145] An acquisition module for acquiring data to be processed and inputting the data to be processed into the target federal random forest model;

[0146] A prediction module for performing prediction processing on the data to be processed based on the target federal random forest model to obtain a prediction result.

[0147] Optionally, the data to be processed includes data to be processed for loans or data to be processed for medical treatment, and the prediction module includes:

[0148] A first prediction unit for, when the data to be processed is data to be processed for loans, performing prediction processing on the data to be processed for loans based on the target federal random forest model to obtain a prediction result on whether a loan can be obtained; or,

[0149] A second prediction unit for, when the data to be processed is data to be processed for medical treatment, performing prediction processing on the data to be processed for medical treatment based on the target federal random forest model to obtain a probability value that the data to be processed for medical treatment is a preset result.

[0150] The specific implementation manners of the federal modeling device of the random forest model in this application are basically the same as those of the various embodiments of the above-mentioned federal modeling method of the random forest model, and will not be elaborated here.

[0151] This application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method for federal modeling of the random forest model described in any one of the above.

[0152] The specific implementation manners of the computer program product of this application are basically the same as those of the various embodiments of the above-mentioned federal modeling method of the random forest model, and will not be elaborated here.

[0153] The above are only the preferred embodiments of this application, and do not limit the patent scope of this application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, is similarly included in the patent scope of this application.

Claims

1. A federated modeling method for a random forest model, characterized in that, Applied to a server, the federated modeling method of the random forest model includes: Receiving different model information sent by different federated participants after they respectively establish an interactive random forest model; Merging the different model information to obtain an aggregated random forest model, and distributing the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model and feedback whether a preset federated training end condition is reached based on the aggregated random forest model, wherein the merging method includes taking the union; If feedback information indicating that the preset federated training end condition is not reached is received, return to the step of receiving different model information sent by different federated participants after they respectively establish an interactive random forest model, until feedback information indicating that the preset federated training end condition is reached is received from the corresponding federated participant, so as to obtain a target federated random forest model; Obtaining data to be processed, and inputting the data to be processed into the target federated random forest model, wherein the data to be processed includes data to be processed for loans or data to be processed for medical treatment; Performing prediction processing on the data to be processed based on the target federated random forest model to obtain a prediction result; The step of merging the different model information to obtain an aggregated random forest model includes: Based on the different model information, determining each first decision tree in the interactive random forest models of different federated participants where the information of nodes in the same layer is the same; Based on the different model information, determining each second decision tree in the interactive random forest model where the information of nodes in the same layer is different; Extracting any one target decision tree from each of the first decision trees, and merging the target decision tree and each of the second decision trees to obtain an aggregated random forest model.

2. The federated modeling method of the random forest model according to claim 1, characterized in that, The step of receiving different model information sent by different federated participants after they respectively establish an interactive random forest model includes: Receiving different model information sent by different federated participants after they respectively establish an interactive random forest model, wherein the different federated participants obtain a random forest model to be operated and an operation task for operating the random forest model to be operated, and operate the random forest model to be operated according to the operation task to obtain an interactive random forest model, and determine model information according to the interactive random forest model.

3. The federated modeling method of the random forest model according to claim 1, wherein, The step of distributing the aggregated random forest model to the different federated participants, so that the corresponding different federated participants receive the aggregated random forest model and feedback whether a preset federated training end condition is reached based on the aggregated random forest model includes: Distributing the aggregated random forest model to the different federated participants, so that after the corresponding different federated participants receive the aggregated random forest model, they continue to perform local iterative training and federated training on the aggregated random forest model and feedback whether a preset federated training end condition is reached, wherein the preset federated training end condition includes reaching a preset model metric threshold or reaching a preset maximum number of federated training iteration rounds.

4. The federated modeling method of the random forest model according to claim 1, wherein, The step of performing prediction processing on the data to be processed based on the target federated random forest model to obtain a prediction result includes: When the data to be processed is loan data to be processed, performing prediction processing on the loan data to be processed based on the target federated random forest model to obtain a prediction result on whether a loan can be obtained; or, When the data to be processed is medical data to be processed, performing prediction processing on the medical data to be processed based on the target federated random forest model to obtain a probability value that the medical data to be processed is a preset result.

5. A federated modeling method for a random forest model, characterized in that, Applied to different federated participants in the federated modeling method of the random forest model according to any one of claims 1 to 4, the federated modeling method of the random forest model includes: After establishing an interactive random forest model, uploading model information of the interactive random forest model to the server; Receiving the aggregated random forest model obtained by the server by merging the model information, and based on the aggregated random forest model, feeding back whether a preset federated training end condition is reached; If feedback information indicating that the preset federated training end condition is not reached is fed back to the server, return to the step of establishing an interactive random forest model and uploading model information of the interactive random forest model to the server until feedback information indicating that the preset federated training end condition is reached is fed back to the server for the server to obtain a target federated random forest model.

6. A federated modeling device for a random forest model, characterized in that, The federated modeling device of the random forest model includes: a memory, a processor, and a program stored on the memory for implementing the federated modeling method of the random forest model, The memory is used for storing a program for implementing the federated modeling method of the random forest model; The processor is used for executing the program for implementing the federated modeling method of the random forest model to implement the steps of the federated modeling method of the random forest model according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, A program for implementing the federated modeling method of the random forest model is stored on the computer-readable storage medium, and the program for implementing the federated modeling method of the random forest model is executed by the processor to implement the steps of the federated modeling method of the random forest model according to any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method for federated modeling of the random forest model according to any one of claims 1 to 5 are implemented.

Images