Vertical federated learning modeling optimization method, equipment, medium and program product

Abstract

The present application discloses a vertical federated learning modeling optimization method, device, medium and program product, which are applied to the first federated participant, and the vertical federated learning modeling optimization method includes: when receiving the training intermediate parameters sent by each second federated participant, using each training intermediate parameter that arrives on time as the current round of federated learning intermediate parameters, and selecting each backup machine from the second federated participant corresponding to each training intermediate parameter that fails to arrive on time; extracting the training intermediate parameters previously sent by each backup machine from the preset backup pool as the compensation federated learning intermediate parameters; and performing vertical federated learning modeling based on each current round of federated learning intermediate parameters and each compensation federated learning intermediate parameters. The present application solves the technical problem of low efficiency of vertical federated learning modeling in the prior art.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology in financial technology (Fintech), and in particular to a vertical federated learning modeling optimization method, device, medium and program product based on feature engineering. Background Art

[0002] With the continuous development of financial technology, especially Internet technology finance, more and more technologies (such as distributed, artificial intelligence, etc.) are applied in the financial field, but the financial industry also puts forward higher requirements on technology, such as higher requirements on the distribution of corresponding to-do items in the financial industry.

[0003] With the continuous development of computer software, artificial intelligence, and big data cloud service applications, in order to solve the "data island" problem, technicians have proposed the concept of federated learning. In the vertical federated learning scenario, there is usually a label provider who actively initiates federated learning and at least one feature provider in the vertical federated learning. The label provider usually waits until the data of all feature providers arrives before starting the next step of operation to achieve vertical federated learning modeling. However, if the local network of each feature provider is a heterogeneous network, the data operation time required for different heterogeneous networks is usually different, so the label provider will always need to wait for a long time for the data of the last feature provider to arrive before the next step of operation can be performed, which seriously affects the efficiency of vertical federated learning modeling. Summary of the invention

[0004] The main purpose of this application is to provide a vertical federated learning modeling optimization method, device, medium and program product, aiming to solve the technical problem of low efficiency of vertical federated learning modeling in the prior art.

[0005] To achieve the above objectives, the present application provides a vertical federated learning modeling optimization method, which is applied to a first federated participant. The vertical federated learning modeling optimization method includes:

[0006] When receiving the training intermediate parameters sent by each second federation participant, each training intermediate parameter that arrives on time is used as the intermediate parameter of this round of federation learning, and each backup machine is selected from the second federation participants corresponding to each training intermediate parameter that fails to arrive on time;

[0007] Extracting the training intermediate parameters previously sent by each backup machine from the preset backup pool as the compensation federated learning intermediate parameters;

[0008] Longitudinal federated learning modeling is performed based on the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensation federated learning.

[0009] The present application also provides a vertical federated learning modeling optimization device, which is applied to a first federated participant, and the vertical federated learning modeling optimization device includes:

[0010] The backup module is used to, when receiving the intermediate training parameters sent by each second federation participant, use the intermediate training parameters that arrive on time as the intermediate parameters of this round of federation learning, and select each backup machine from the second federation participants corresponding to the intermediate training parameters that fail to arrive on time;

[0011] A compensation module, used to extract the training intermediate parameters previously sent by each backup machine from a preset backup pool as compensation federated learning intermediate parameters;

[0012] A modeling module is used to perform longitudinal federated learning modeling based on the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensation federated learning.

[0013] The present application also provides an electronic device, comprising: a memory, a processor, and a program of the longitudinal federated learning modeling optimization method stored in the memory and executable on the processor. When the program of the longitudinal federated learning modeling optimization method is executed by the processor, the steps of the longitudinal federated learning modeling optimization method as described above can be implemented.

[0014] The present application also provides a computer-readable storage medium, on which is stored a program for implementing the longitudinal federated learning modeling optimization method. When the program of the longitudinal federated learning modeling optimization method is executed by a processor, the steps of the longitudinal federated learning modeling optimization method as described above are implemented.

[0015] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the vertical federated learning modeling optimization method as described above.

[0016] The present application provides a method, device, medium and program product for optimizing vertical federated learning modeling. Compared with the technical means adopted in the prior art that the label provider waits until the data of all feature providers arrives before starting the next step of operation to realize vertical federated learning modeling, the present application firstly receives the training intermediate parameters sent by each second federated participant, and uses the training intermediate parameters that arrive on time as the intermediate parameters of this round of federated learning, and selects each backup machine from the second federated participants corresponding to the training intermediate parameters that fail to arrive on time, that is, in each round of iteration of vertical federated learning, first receives a part of the connected intermediate parameters that arrive faster as the intermediate parameters of this round of federated learning, and uses the training intermediate parameters that arrive slower as the intermediate parameters of this round of federated learning. The corresponding second federated participant is set as a backup machine; the training intermediate parameters previously sent by each backup machine are extracted from the preset backup pool as compensation federated learning intermediate parameters, that is, for the federated learning participants whose training intermediate parameters arrive slowly, the federated learning intermediate parameters sent in the previous iteration are selected from the preset backup pool as compensation, and then the next step of operation is directly performed according to the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensation federated learning. Since the parameters of adjacent iterations of the model usually do not change significantly during the iteration process, the vertical federated learning modeling between the first federated participant and each of the second federated participants can also be realized, and the accuracy of the vertical federated learning modeling will not be affected. Therefore, the present application saves the time for the first federated participant to wait for the arrival of the training intermediate parameters without affecting the accuracy of the vertical federated learning modeling, and achieves the purpose of accelerating the federated learning modeling, overcoming the technical defect that if the local network of each feature provider is a heterogeneous network, the data calculation time required for different heterogeneous networks is usually different, so the label provider always needs to wait for a long time for the data of the last feature provider to arrive before the next step of operation can be performed, which seriously affects the efficiency of the vertical federated learning, and improves the efficiency of the vertical federated learning modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 This is a flowchart of the first embodiment of the vertical federated learning modeling optimization method of the present application;

[0020] Figure 2A schematic diagram of the process of constructing a Poisson regression model based on longitudinal federated learning in the longitudinal federated learning modeling optimization method of this application;

[0021] Figure 3 This is a flow chart of the second embodiment of the vertical federated learning modeling optimization method of the present application;

[0022] Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the vertical federated learning modeling optimization method in the embodiment of the present application.

[0023] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0025] Embodiment 1

[0026] At present, in order to solve the problem of "data islands", technicians have proposed the concept of federated learning. Federated learning can be divided into two methods: horizontal federated learning and vertical federated learning. In the vertical federated learning modeling scenario, there is usually a label provider who actively initiates federated learning and at least one feature provider in vertical federated learning, and the label provider needs to receive the training intermediate parameters sent by each feature provider, and then the label provider calculates the feedback intermediate parameters that need to be fed back to each feature provider based on the local data and the training intermediate parameters sent by all feature providers, that is, the label provider and the feature provider need to exchange intermediate parameters, so as to realize vertical federated learning modeling. However, the feature providers are usually heterogeneous networks, and the computing time required for heterogeneous networks to process local sample data to obtain training intermediate parameters is usually different. In addition, since the training intermediate parameters are usually homomorphically encrypted parameters and the data volume is high, the computing time gap between different heterogeneous networks will be further widened. The label provider must collect the training intermediate parameters of all feature providers before it can calculate the feedback intermediate parameters to complete the process of exchanging intermediate parameters between the label provider and the feature provider. Therefore, the label provider always needs to wait for the arrival of the training intermediate parameters sent by the last feature provider before it can complete the calculation of the feedback intermediate parameters. The waiting time greatly affects the efficiency of vertical federated learning modeling.

[0027] The present application embodiment provides a vertical federated learning modeling optimization method, which is applied to the first federated participant. In the first embodiment of the vertical federated learning modeling optimization method of the present application, refer to Figure 1 , the longitudinal federated learning modeling optimization method includes:

[0028] Step S10, when receiving the training intermediate parameters sent by each second federation participant, taking each training intermediate parameter that arrives on time as the intermediate parameter of this round of federation learning, and selecting each backup machine from the second federation participants corresponding to each training intermediate parameter that fails to arrive on time;

[0029] Step S20, extracting the training intermediate parameters previously sent by each backup machine from the preset backup pool as compensation federated learning intermediate parameters;

[0030] Step S30, performing longitudinal federated learning modeling based on each of the intermediate parameters of the current round of federated learning and each of the intermediate parameters of the compensation federated learning.

[0031] The embodiment of the present application provides a longitudinal federated learning modeling method. Compared with the existing longitudinal federated learning modeling method, the embodiment of the present application firstly uses the training intermediate parameters that arrive on time as the intermediate parameters of this round of federated learning when receiving the training intermediate parameters sent by each second federated participant, and selects each backup machine from the second federated participants corresponding to the training intermediate parameters that fail to arrive on time, that is, in each round of iteration of the longitudinal federated learning, firstly receives a part of the connected intermediate parameters that arrive faster as the intermediate parameters of this round of federated learning, and sets the second federated participants corresponding to the training intermediate parameters that arrive slower as the backup machine; and then extracts the latest training intermediate parameters previously sent by each of the backup machines from the preset backup pool as the compensatory federated learning intermediate parameters, that is, for the federated learning participants with slower training intermediate parameters, In the embodiment of the present invention, the latest federated learning intermediate parameters sent in the previous iteration are selected from the preset backup pool as compensation, wherein the step size of iterative update is usually set small during model training, so the parameters of adjacent iteration rounds of the model usually do not change significantly during the iteration process, and then the next step of operation is directly performed according to the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensation federated learning. The feedback intermediate parameters can also be calculated to realize the vertical federated learning modeling between the first federated participant and each of the second federated participants without affecting the accuracy of the vertical federated learning modeling, thereby saving the time for the first federated participant to wait for the arrival of the training intermediate parameters without affecting the accuracy of the vertical federated learning modeling, accelerating the vertical federated learning modeling, and thus improving the efficiency of the vertical federated learning modeling.

[0032] In this embodiment, it should be noted that the first federated participant is a federated participant that actively initiates the vertical federated learning modeling task, and may be a label provider with sample labels. The second federated participant may be a feature provider without sample labels. The training intermediate parameters are parameters sent by the feature provider for calculating the feedback intermediate parameters. The feedback intermediate parameters are parameters used to calculate the federated model gradient. As an example, Figure 2 The figure shows a flow chart of constructing a Poisson regression model based on longitudinal federated learning, wherein the label provider B is the first federated participant, the feature provider A is the second federated participant, the number of second federated participants is at least 1, Weizhong C is a federated server, and wx A and exp(wx A ) are the intermediate parameters of the training, wx B and exp(wx B ) are all intermediate parameters calculated by the label provider, w is the model parameter, x is the sample feature, y is the sample label, d is the feedback intermediate parameter, g B The federated model gradient calculated for the label provider, g A The federated model gradient calculated for the feature provider. [] is a homomorphic encryption symbol, indicating that the data within the symbol is encrypted.

[0033] As an example, steps S10 to S30 include:

[0034] The intermediate training parameters sent by each second federated participant are received within a preset first specified time, and each intermediate training parameter that arrives on time is used as the intermediate training parameter of this round of federated learning, wherein the intermediate training parameter of this round of federated learning is the intermediate training parameter in the calculation of this round of federated learning, and each intermediate training parameter that does not arrive on time is determined, and the second federated participant corresponding to each intermediate training parameter that does not arrive on time is used as a backup machine; the latest intermediate training parameters sent by each backup machine in the previous iteration round are respectively extracted from the preset backup pool as the compensation intermediate training parameters, wherein the compensation intermediate training parameters will replace the corresponding intermediate training parameters that do not arrive on time as the intermediate parameters of this round of federated learning to participate in the next calculation; the preset vertical federated process is continued to be executed by performing the next preset vertical federated learning calculation step according to each intermediate training parameter of this round of federated learning and each compensation intermediate training parameter, wherein the specific content of the preset vertical federated process is the prior art and will not be repeated here.

[0035] As an example, the latest training intermediate parameter is the latest training intermediate parameter in the preset backup pool. If the training intermediate parameter that did not arrive on time in the previous round arrives at the first federated participant before the end of the previous iteration round, the latest training intermediate parameter is the training intermediate parameter sent by the backup machine in the previous iteration round.

[0036] The step of performing vertical federated learning modeling based on each of the intermediate parameters of the current round of federated learning and each of the intermediate parameters of the compensation federated learning includes:

[0037] The feedback intermediate parameters are calculated based on the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensated federated learning, and the first federated model gradient is calculated based on the feedback intermediate parameters and local sample features, and the first local model is updated based on the first federated model gradient; the feedback intermediate parameters are sent to each second federated participant respectively, so that the second federated participant can calculate the second federated gradient based on the feedback intermediate parameters and its own local sample features, and update the second local model based on the second federated gradient; then the first federal participant performs the next round of federal iteration based on the updated first local model and the updated second local models of each second federal participant, until the model converges.

[0038] Among them, in step S10, the steps of using each training intermediate parameter that arrives on time as the intermediate parameter of this round of federated learning, and selecting each backup machine from the second federated participants corresponding to each training intermediate parameter that fails to arrive on time include:

[0039] Step S11, waiting to receive the training intermediate parameters sent by each of the second federated participants within a preset first specified time, and after the preset first specified time is over, using the received training intermediate parameters as the intermediate parameters of the current round of federated learning;

[0040] Step S12, selecting each of the backup machines from the second federation participants corresponding to each of the training intermediate parameters that have not been received.

[0041] Exemplarily, within a preset first specified time, wait to receive the training intermediate parameters sent by each second federal participant. After the preset first specified time ends, use the training intermediate parameters that have been received within the preset first specified time as the intermediate parameters of this round of federal learning; and determine the training intermediate parameters that have not been received within the preset first specified time, and use the second federal learning participants corresponding to the training intermediate parameters that have not been received as backup machines. For example, assuming that there are 5 second federal participants A, B, C, D and E, A needs to send the training intermediate parameter a to the first federal learning participant, B needs to send the training intermediate parameter b to the first federal learning participant, C needs to send the training intermediate parameter c to the first federal learning participant, D needs to send the training intermediate parameter d to the first federal learning participant, and E needs to send the training intermediate parameter e to the first federal learning participant. If only a, b and c reach the first federal participant within the preset first specified time, the training intermediate parameters a, b and c are the intermediate parameters of this round of federal learning, and the second federal learning participants C and D will serve as backup machines.

[0042] Wherein, before the step of waiting to receive the training intermediate parameters sent by each of the second federated participants within the preset first specified time, the vertical federated learning modeling optimization method further includes:

[0043] Step A10, obtaining the arrival time of the training intermediate parameters sent by each of the second federation participants in the previous iteration;

[0044] Step A20, determining the preset first prescribed time according to each arrival time.

[0045] In this embodiment, it should be noted that the arrival time is the time taken from the start time of the iteration round to the time taken for the training intermediate parameters sent by the second federal participant to reach the first federal participant.

[0046] Exemplarily, the arrival time of the training intermediate parameters sent by each of the second federal participants in the previous iteration is obtained; the arrival time is averaged to obtain the preset first specified time.

[0047] As an example, step A20 further includes:

[0048] Obtain the preset number of parameters arriving on time, and set the preset first specified time based on the preset number of parameters arriving on time and the time taken for each arrival, wherein in the previous round of iterations, the number of training intermediate parameters that arrive at the first federal participant within the preset first specified time is the preset number of parameters arriving on time.

[0049] Wherein, before the step of respectively extracting the training intermediate parameters previously sent by each backup machine from the preset backup pool as the compensation federated learning intermediate parameters, the longitudinal federated learning modeling optimization method further includes:

[0050] Step B10, after receiving the training intermediate parameter sent by the second federated participant, obtaining an iteration round label corresponding to the training intermediate parameter;

[0051] Step B20, backing up the iteration round label and the intermediate training parameters to the preset backup pool.

[0052] In this embodiment, it should be noted that the first federal participant locally maintains a preset backup pool. After each intermediate training parameter sent by the second federal participant reaches the first federal participant, the first federal participant will back up the arrived intermediate training parameters to the preset backup pool.

[0053] Exemplarily, after receiving the training intermediate parameters sent by the second federal participant, an iteration round label corresponding to the training intermediate parameters is obtained, wherein the iteration round label is an identifier of the federated learning iteration round corresponding to the training intermediate parameters; the iteration round label and the training intermediate parameters are associated and backed up to a preset backup pool.

[0054] The step of associating and backing up the iteration round label and the training intermediate parameter to a preset backup pool includes:

[0055] Obtain the device serial number of the second federation participant that sends the training intermediate parameters, combine the device serial number and the iteration round label into an index, and back up the index and the training intermediate parameters in the form of a key-value pair to a preset backup pool.

[0056] The vertical federated learning modeling optimization method further includes:

[0057] Step C10, determining the current iteration round, and obtaining the latest training intermediate parameters corresponding to each of the second federation participants in the preset backup pool;

[0058] Step C20, determining the backup iteration rounds corresponding to each of the latest training intermediate parameters;

[0059] Step C30: judging whether each of the second federation participants is in an on-hook state according to each of the backup iteration rounds and the current iteration round.

[0060] Exemplarily, determine the current iteration round, obtain the latest training intermediate parameters corresponding to each of the second federated participants in the preset backup pool, wherein the latest training intermediate parameters are the training intermediate parameters sent by the second federated participants the latest; obtain the backup iteration rounds corresponding to each of the latest training intermediate parameters; respectively calculate the round gaps between each of the backup iteration rounds and the current iteration round, and determine whether each of the round gaps is less than the preset round gap threshold; if the round gaps are all less than the preset round gap threshold, then determine that each of the second federated participants is not in an on-hook state; if the round gaps are not all less than the preset round gap threshold, take the round gap that exceeds the preset round gap threshold as the target round gap, and determine that the second federated participant corresponding to the target round gap is in an on-hook state. The embodiment of the present application can determine whether the second federated participant is in an on-hook state or the network's computing speed is slow by monitoring the gap between the federated learning round of the training intermediate parameters sent by each second federated participant the latest and the current iteration round in real time, and can prevent the on-hook second federated participant from being misjudged as a participant with a slow computing speed, thereby affecting the accuracy of vertical federated learning modeling.

[0061] In this embodiment, it should be noted that in the process of federated learning modeling, such as in the horizontal federated learning modeling scenario, the backup mechanism can select the participant that becomes slow due to the network heterogeneity in each round of training as the backup machine, and ignore the gradient update of the backup machine, thereby improving communication efficiency. However, in the vertical federated learning scenario, it is not the gradient information but the intermediate training parameters that are ignored. Since the feature dimensions of each participant in the vertical federated learning modeling are different, the lack of a certain party's training intermediate parameters will result in the inability to calculate the complete loss function, and the federated learning training will be blocked. Forcibly continuing the training will introduce a large training deviation, which greatly affects the accuracy of the vertical federated modeling. In the embodiment of the present application, a preset backup pool is maintained locally at the first federated learning participant. While setting up the backup machine, the compensation corresponding to the backup machine is selected from the backup pool. The missing training intermediate parameters of the backup machine can be compensated, so the purpose of improving communication efficiency based on the backup mechanism can be achieved without affecting the efficiency of vertical federated learning modeling.

[0062] The embodiment of the present application provides a method for optimizing vertical federated learning modeling. Compared with the technical means adopted in the prior art that the label provider waits until the data of all feature providers arrive before starting the next operation to realize vertical federated learning modeling, the embodiment of the present application firstly receives the training intermediate parameters sent by each second federated participant, and uses the training intermediate parameters that arrive on time as the intermediate parameters of this round of federated learning, and selects each backup machine from the second federated participants corresponding to the training intermediate parameters that fail to arrive on time, that is, in each round of iteration of vertical federated learning, first receives a part of the connected intermediate parameters that arrive faster as the intermediate parameters of this round of federated learning, and uses the corresponding training intermediate parameters that arrive slower The second federated participant is set as a backup machine; the training intermediate parameters previously sent by each backup machine are extracted from the preset backup pool as compensation federated learning intermediate parameters, that is, for the federated learning participants whose training intermediate parameters arrive slowly, the federated learning intermediate parameters sent in the previous iteration are selected from the preset backup pool as compensation, and then the next step of operation is directly performed according to the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensation federated learning. Since the parameters of adjacent iteration rounds of the model usually do not change significantly during the iteration process, the vertical federated learning modeling between the first federated participant and each of the second federated participants can also be realized, and the accuracy of the vertical federated learning modeling will not be affected. Therefore, the present application saves the time for the first federated participant to wait for the arrival of the training intermediate parameters without affecting the accuracy of the vertical federated learning modeling, and achieves the purpose of accelerating the federated learning modeling, overcoming the technical defect that if the local network of each feature provider is a heterogeneous network, the data calculation time required for different heterogeneous networks is usually different, so the label provider always needs to wait for a long time for the data of the last feature provider to arrive before the next step of operation can be performed, which seriously affects the efficiency of the vertical federated learning, and improves the efficiency of the vertical federated learning modeling.

[0063] Embodiment 2

[0064] Further, refer to Figure 3 Based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above description and will not be described in detail later. On this basis, the steps of using each training intermediate parameter that arrives on time as the intermediate parameter of this round of federated learning, and selecting each backup machine from the second federated participants corresponding to each training intermediate parameter that fails to arrive on time include:

[0065] Step D10, obtaining the number of parameters of the received training intermediate parameters;

[0066] Step D20, if the number of parameters is greater than the preset parameter number threshold, then start waiting to receive the training intermediate parameters sent by each of the second federated participants within the preset second specified time, and after the preset second specified time is over, use the received training intermediate parameters as the intermediate parameters of the current round of federated learning;

[0067] Step D30, selecting each backup machine from the second federation participants corresponding to each training intermediate parameter that has not been received;

[0068] Step D40: If the number of parameters is not greater than the preset parameter number threshold, return to the execution step: obtain the number of parameters of the received training intermediate parameters.

[0069] In this embodiment, it should be noted that if in a round of iteration of vertical federated learning modeling, the proportion of the intermediate parameters of the compensating federated learning involved in the calculation is high, then in this round of iteration, it is equivalent to calculating on the intermediate parameters of the training in the previous rounds, which will result in this round of iteration contributing little to the update of the model, and the calculation of this round of iteration is basically invalid calculation. Therefore, in the process of vertical federated modeling in the embodiment of the present application, it is necessary to ensure that in each round of iteration, the proportion of the intermediate parameters of each round of federated learning involved in the calculation.

[0070] Exemplarily, the process continuously waits for receiving the intermediate training parameters sent by each second federal participant, and monitors in real time the number of parameters of the received intermediate training parameters; if the number of parameters is greater than a preset parameter number threshold, the timing is started, and within a preset second specified time, the process waits for receiving the intermediate training parameters sent by each second federal participant, and after the preset second specified time is over, all the received intermediate training parameters are used as the intermediate parameters of this round of federal learning; the intermediate training parameters that have not been received are determined, and the second federal learning participants corresponding to the intermediate training parameters that have not been received are used as backup machines; if the number of parameters is not greater than the preset parameter number threshold, the process returns to the execution step of obtaining the number of parameters of the received intermediate training parameters.

[0071] The step of selecting each backup machine from the second federation participants corresponding to each unreceived intermediate training parameter includes:

[0072] Step E10, taking the second federated participants corresponding to the unreceived intermediate training parameters as the participants to be evaluated, and obtaining the federated learning contribution corresponding to each of the participants to be evaluated;

[0073] Step E20, if the federated learning contribution is greater than the preset federated learning contribution threshold, then start waiting to receive the training intermediate parameters sent by each of the participants to be evaluated within the preset third specified time, and after the preset third specified time is over, use the received training intermediate parameters as the intermediate parameters of the current round of federated learning;

[0074] Step E30, using the second federation participant corresponding to each training intermediate parameter that has not been received as the backup machine;

[0075] Step E40: If the federated learning contribution is not greater than a preset federated learning contribution threshold, the second federated participant corresponding to each unreceived intermediate training parameter is directly used as the backup machine.

[0076] Exemplarily, it should be noted that the contribution of the federated learning is determined by the data quality of the sample data of each of the second federated participants. The data quality of the sample data can be evaluated by statistical means, such as the chi-square test, etc., or by a specific model building method, such as a model building method based on the stepwise regression method, a model building method based on the forward method, or a model building method based on the backward method, etc. The data quality evaluation methods of the sample data listed here are all existing technologies and will not be repeated here.

[0077] In the embodiment of the present application, a preset third prescribed time is set on the basis of the preset first prescribed time or the preset second prescribed time, so as to ensure that the second federal participant with a larger contribution to federal learning can always participate in the current round of iteration technology in time, and the second federal participant with a larger contribution to federal learning will not be used as a backup machine. Since the second federal participant with a larger contribution to federal learning has a larger impact on the overall federated learning model, although the step size of the model update in each round of iteration will not be large, if the federated learning contribution of the second federal learning participant serving as a backup machine is very large, it will still have a slight impact on the training accuracy of the vertical federated learning model. By setting the preset third prescribed time, the embodiment of the present application can prevent the second federal participant with a larger contribution to federal learning from being used as a backup machine, which affects the accuracy of vertical federated learning modeling. While improving the communication efficiency of vertical federated learning modeling based on the backup compensation mechanism, the accuracy of vertical federated learning modeling is further improved.

[0078] An embodiment of the present application provides a method for setting up a backup machine, that is, obtaining the number of parameters of the received training intermediate parameters; if the number of parameters is greater than the preset parameter number threshold, starting to wait for receiving the training intermediate parameters sent by each second federal participant within a preset second specified time, after the preset second specified time ends, using the received training intermediate parameters as the intermediate parameters of this round of federated learning; selecting each backup machine from the second federal participants corresponding to the training intermediate parameters that have not been received; if the number of parameters is not greater than the preset parameter number threshold, returning to the execution step: obtaining the number of parameters of the received training intermediate parameters. Among them, on the basis of the backup compensation mechanism, the embodiment of the present application sets a preset second specified time that starts timing according to the number of parameters arrived, which can ensure that the number of parameters arrived in this round of iteration will not be too small, that is, it can ensure that the ratio of each intermediate parameter participating in the calculation of this round of federated learning will not be too small, and can prevent the situation where most of the intermediate parameters participating in the calculation of this round are compensation federated learning intermediate parameters, thereby affecting the model update effect of this round of iteration, so that the federated learning model in this round of iteration can be updated in the direction of improving the model accuracy, and the model is invalidly updated in this round of iteration compared with the previous round of iteration, thereby further improving the accuracy of vertical federated learning modeling.

[0079] Embodiment 3

[0080] The embodiment of the present application further provides a vertical federated learning modeling optimization device, which is applied to a first federated participant, and the vertical federated learning modeling optimization device includes:

[0081] The backup module is used to, when receiving the intermediate training parameters sent by each second federation participant, use the intermediate training parameters that arrive on time as the intermediate parameters of this round of federation learning, and select each backup machine from the second federation participants corresponding to the intermediate training parameters that fail to arrive on time;

[0082] A compensation module, used to extract the training intermediate parameters previously sent by each backup machine from a preset backup pool as compensation federated learning intermediate parameters;

[0083] A modeling module is used to perform longitudinal federated learning modeling based on the intermediate parameters of the current round of federated learning and the intermediate parameters of the compensation federated learning.

[0084] Optionally, the backup module is further used for:

[0085] Waiting to receive the training intermediate parameters sent by each of the second federated participants within a preset first specified time, and after the preset first specified time is over, using the received training intermediate parameters as the intermediate parameters of the current round of federated learning;

[0086] The backup machines are selected from the second federation participants corresponding to the unreceived intermediate training parameters.

[0087] Optionally, the backup module is further used for:

[0088] Obtaining the arrival time of the training intermediate parameters sent by each of the second federation participants in the previous iteration;

[0089] The preset first specified time is determined according to each of the arrival time.

[0090] Optionally, the backup module is further used for:

[0091] Get the number of parameters of the received training intermediate parameters;

[0092] If the number of parameters is greater than the preset parameter number threshold, then start waiting to receive the training intermediate parameters sent by each of the second federated participants within the preset second specified time, and after the preset second specified time is over, use the received training intermediate parameters as the intermediate parameters of the current round of federated learning;

[0093] Selecting each of the backup machines from the second federation participants corresponding to each of the training intermediate parameters that have not been received;

[0094] If the number of parameters is not greater than the preset parameter number threshold, return to the execution step: obtain the number of parameters of the received training intermediate parameters.

[0095] Optionally, the backup module is further used for:

[0096] The second federated participants corresponding to the unreceived intermediate training parameters are used as the participants to be evaluated, and the federated learning contribution corresponding to each of the participants to be evaluated is obtained;

[0097] If the federated learning contribution is greater than the preset federated learning contribution threshold, then start waiting to receive the training intermediate parameters sent by each participant to be evaluated within the preset third specified time, and after the preset third specified time is over, use the received training intermediate parameters as the intermediate parameters of the current round of federated learning;

[0098] The second federation participant corresponding to each training intermediate parameter that has not been received is used as the backup machine;

[0099] If the federated learning contribution is not greater than a preset federated learning contribution threshold, the second federated participant corresponding to each unreceived intermediate training parameter is directly used as the backup machine.

[0100] Optionally, the vertical federated learning modeling optimization device is also used for:

[0101] After receiving the training intermediate parameter sent by the second federated participant, obtaining an iteration round label corresponding to the training intermediate parameter;

[0102] The iteration round label and the training intermediate parameters are backed up together to the preset backup pool.

[0103] Optionally, the vertical federated learning modeling optimization device is also used for:

[0104] Determine the current iteration round, and obtain the latest training intermediate parameters corresponding to each of the second federation participants in the preset backup pool;

[0105] Determine the backup iteration rounds corresponding to each of the latest training intermediate parameters;

[0106] According to each of the backup iteration rounds and the current iteration round, it is determined whether each of the second federation participants is in an on-hook state.

[0107] The vertical federated learning modeling optimization device provided by the present invention adopts the vertical federated learning modeling optimization method in the above embodiment to solve the technical problem of low efficiency of vertical federated learning modeling. Compared with the prior art, the beneficial effects of the vertical federated learning modeling optimization device provided by the embodiment of the present invention are the same as the beneficial effects of the vertical federated learning modeling optimization method provided by the above embodiment, and other technical features in the vertical federated learning modeling optimization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0108] Embodiment 4

[0109] An embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the vertical federated learning modeling optimization method in the above-mentioned embodiment one.

[0110] Reference below Figure 4 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0111] like Figure 4As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0112] Typically, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0113] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0114] The electronic device provided by the present invention adopts the longitudinal federated learning modeling optimization method in the above embodiment to solve the technical problem of low efficiency of longitudinal federated learning modeling. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present invention are the same as the beneficial effects of the longitudinal federated learning modeling optimization method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0115] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0116] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0117] Embodiment 5

[0118] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the method for vertical federated learning modeling optimization in the above-mentioned embodiment one.

[0119] The computer-readable storage medium provided in the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0120] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0121] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: when receiving the training intermediate parameters sent by each second federal participant, uses each training intermediate parameter that arrives on time as the intermediate parameter of this round of federated learning, and selects each backup machine from the second federal participants corresponding to each training intermediate parameter that fails to arrive on time; extracts the training intermediate parameters previously sent by each backup machine from the preset backup pool as the compensatory federated learning intermediate parameters; and performs longitudinal federated learning modeling based on each of the current round of federated learning intermediate parameters and each of the compensatory federated learning intermediate parameters.

[0122] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0123] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0125] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned vertical federated learning modeling optimization method, which solves the technical problem of low efficiency of vertical federated learning modeling. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present invention are the same as the beneficial effects of the vertical federated learning modeling optimization method provided by the above-mentioned embodiment, and will not be repeated here.

[0126] Embodiment 6

[0127] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the vertical federated learning modeling optimization method as described above.

[0128] The computer program product provided by this application solves the technical problem of low efficiency of vertical federated learning modeling. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the vertical federated learning modeling optimization method provided by the above embodiment, which will not be repeated here.

[0129] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A vertical federated learning modeling optimization method, characterized in that: Applied to the first federated participant, the vertical federated learning modeling optimization method includes: When receiving the training intermediate parameters sent by each second federation participant, each training intermediate parameter that arrives on time is used as the intermediate parameter of this round of federation learning, and each backup machine is selected from the second federation participants corresponding to each training intermediate parameter that fails to arrive on time; Extracting the training intermediate parameters previously sent by each backup machine from the preset backup pool as the compensation federated learning intermediate parameters; Performing longitudinal federated learning modeling according to each of the intermediate parameters of the current round of federated learning and each of the intermediate parameters of the compensation federated learning; The steps of using each training intermediate parameter that arrives on time as the intermediate parameter of this round of federated learning, and selecting each backup machine from the second federated participants corresponding to each training intermediate parameter that fails to arrive on time include: Get the number of parameters of the received training intermediate parameters; If the number of parameters is greater than the preset parameter number threshold, then start waiting to receive the training intermediate parameters sent by each of the second federated participants within the preset second specified time, and after the preset second specified time is over, use the received training intermediate parameters as the intermediate parameters of the current round of federated learning; Selecting each of the backup machines from the second federation participants corresponding to each of the training intermediate parameters that have not been received; If the number of parameters is not greater than the preset parameter number threshold, returning to the execution step: obtaining the number of parameters of the received training intermediate parameters; Or the step of using each training intermediate parameter that arrives on time as the intermediate parameter of this round of federated learning, and selecting each backup machine from the second federated participants corresponding to each training intermediate parameter that fails to arrive on time includes: Waiting to receive the training intermediate parameters sent by each of the second federated participants within a preset first specified time, and after the preset first specified time is over, using the received training intermediate parameters as the intermediate parameters of the current round of federated learning; Selecting each of the backup machines from the second federation participants corresponding to each of the training intermediate parameters that have not been received; The step of selecting each backup machine from the second federation participants corresponding to each unreceived training intermediate parameter comprises: The second federated participants corresponding to the unreceived intermediate training parameters are used as the participants to be evaluated, and the federated learning contribution corresponding to each of the participants to be evaluated is obtained; If the federated learning contribution is greater than the preset federated learning contribution threshold, then start waiting to receive the training intermediate parameters sent by each participant to be evaluated within the preset third specified time, and after the preset third specified time is over, use the received training intermediate parameters as the intermediate parameters of the current round of federated learning; The second federation participant corresponding to each training intermediate parameter that has not been received is used as the backup machine; If the federated learning contribution is not greater than a preset federated learning contribution threshold, the second federated participant corresponding to each unreceived intermediate training parameter is directly used as the backup machine.

2. The vertical federated learning modeling optimization method according to claim 1, characterized in that: Before the step of waiting to receive the training intermediate parameters sent by each of the second federated participants within the preset first specified time, the vertical federated learning modeling optimization method further includes: Obtaining the arrival time of the training intermediate parameters sent by each of the second federation participants in the previous iteration; The preset first specified time is determined according to each of the arrival time.

3. The vertical federated learning modeling optimization method according to claim 1, characterized in that: Before the step of extracting the training intermediate parameters previously sent by each backup machine from the preset backup pool as the compensation federated learning intermediate parameters, the vertical federated learning modeling optimization method further includes: After receiving the training intermediate parameter sent by the second federated participant, obtaining an iteration round label corresponding to the training intermediate parameter; The iteration round label and the training intermediate parameters are backed up together to the preset backup pool.

4. The vertical federated learning modeling optimization method according to claim 1, characterized in that: The vertical federated learning modeling optimization method also includes: Determine the current iteration round, and obtain the latest training intermediate parameters corresponding to each of the second federation participants in the preset backup pool; Determine the backup iteration rounds corresponding to each of the latest training intermediate parameters; According to each of the backup iteration rounds and the current iteration round, it is determined whether each of the second federation participants is in an on-hook state.

5. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the longitudinal federated learning modeling optimization method described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for implementing the longitudinal federated learning modeling optimization method, and the program for implementing the longitudinal federated learning modeling optimization method is executed by a processor to implement the steps of the longitudinal federated learning modeling optimization method as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the longitudinal federated learning modeling optimization method as described in any one of claims 1 to 4 are implemented.

Images