An optimization method and tool for a vertical federated learning model training process

Abstract

The application discloses an optimization method for a longitudinal federated learning model training process, and relates to the technical field of model training. After data input, the optimization method performs the following operations: determining a party with the minimum upper limit and a party with the maximum lower limit in the number range of each party participating in federated learning; calculating the new number range of each party and formulating the intersection range based on the minimum upper limit and the maximum lower limit; dividing the intersection range into multiple sub-intervals using the dichotomy method; performing privacy intersection and longitudinal federated learning for each sub-interval respectively, integrating the longitudinal federated learning results of all sub-intervals, and performing model verification. The application also discloses an optimization tool for the longitudinal federated learning model training process. In combination with the foregoing method, the optimization tool can accelerate the model training process of longitudinal federated learning and improve the training efficiency of the longitudinal federated learning model while ensuring that information is not leaked.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model training, in particular to a method and tool for optimizing the training process of a vertical federated learning model. BACKGROUND

[0002] Federated learning is a distributed machine learning technique, and its core idea is to train a model in a distributed manner among multiple data sources with local data, without exchanging local individual or sample data, but only exchanging model parameters or intermediate results, to build a global model based on virtual fused data, so as to achieve a balance between data privacy protection and data sharing calculation, that is, a new application paradigm of "data available but invisible" and "data not moving model moving".

[0003] The general process of vertical federated learning can be divided into four steps: data input, privacy intersection, vertical federation, and verification. The two most time-consuming parts, privacy intersection and vertical federation, use different resources. Privacy intersection mainly relies on the network resources of the server, and vertical federation mainly relies on the computing resources of the server. Therefore, during the vertical federated training, the network resource usage of the computer is high and the computing resource usage is low during the privacy intersection part, while the computing resource usage of the computer is high and the network resource usage is low during the vertical federation part.

[0004] Based on this, the traditional vertical federated learning model training process is optimized, a vertical federated task is divided into multiple small tasks, and parallel federated learning is used to shorten the training time of vertical federated learning. SUMMARY

[0005] The present application provides a method and tool for optimizing the training process of a vertical federated learning model to meet the needs and deficiencies of current technology development.

[0006] Firstly, the technical solution adopted by the method for optimizing the training process of a vertical federated learning model to solve the above technical problems is as follows:

[0007] A method for optimizing the training process of a vertical federated learning model, after executing data input, performs the following operations:

[0008] Determine the party with the smallest upper limit and the party with the largest lower limit in the number range of each party participating in federated learning;

[0009] Based on the smallest upper limit and the largest lower limit, calculate the new number range of each party and determine the intersection range;

[0010] Divide the intersection range into multiple sub-intervals using the bisection method;

[0011] The privacy intersection and longitudinal federated learning are respectively performed for each sub-interval, the longitudinal federated learning results of all sub-intervals are integrated, and model verification is performed.

[0012] Optionally, the party with the minimum upper limit and the party with the maximum upper limit in the number range of each party participating in federated learning are fitted using oblivious transfer.

[0013] Optionally, based on the minimum upper limit and the maximum upper limit, the new number range of each party is calculated using binary search and oblivious transfer, and the intersection range is formulated.

[0014] Further optionally, the specific operation of formulating the intersection range is as follows:

[0015] (i) If the party with the minimum upper limit and the party with the maximum upper limit are the same party participating in federated learning, the party formulates an intersection range that is not greater than its own range and ends;

[0016] (ii) If the party with the minimum upper limit and the party with the maximum upper limit are different parties participating in federated learning,

[0017] (a) Based on the party with the maximum upper limit, an intermediate value is taken according to the number range of the party, the intermediate value is compared with the upper limits of the remaining parties, if the intermediate value is smaller / larger than the upper limits of the remaining parties, an intermediate value of the intermediate value is taken for comparison, until the taken intermediate value is between the upper limits of the remaining parties, the intermediate value is used as a new upper limit, and the upper limit of the remaining party that is greater than the new upper limit is replaced,

[0018] (b) Based on the party with the minimum upper limit, an intermediate value is taken according to the number range of the party, the intermediate value is compared with the lower limits of the remaining parties, if the intermediate value is smaller / larger than the lower limits of the remaining parties, an intermediate value of the intermediate value is taken for comparison, until the taken intermediate value is between the lower limits of the remaining parties, the intermediate value is used as a new lower limit, and the lower limit of the remaining party that is greater than the new lower limit is replaced.

[0019] Optionally, the privacy intersection and longitudinal federated learning are performed on each sub-interval in a parallel manner.

[0020] Secondly, the technical solution adopted by the optimization tool for the longitudinal federated learning model training process to solve the above technical problems is as follows:

[0021] An optimization tool for the longitudinal federated learning model training process, the tool is deployed in the longitudinal federated learning model training process, and the structure comprises:

[0022] A determination module is configured to determine a party with a minimum upper limit and a party with a maximum upper limit in a number range of each party according to input data of each party.

[0023] a calculating module configured to calculate new number ranges of the parties according to the minimum upper limit and the maximum lower limit, and to formulate an intersection range;

[0024] a range dividing module configured to divide the formulated intersection range into a plurality of sub-intervals using dichotomy;

[0025] a parallel module configured to perform privacy intersection and longitudinal federated learning on the plurality of sub-intervals in parallel;

[0026] a consolidating module configured to consolidate longitudinal federated learning results of all the sub-intervals, and to perform model verification.

[0027] Optionally, the involved determining module uses oblivious transfer to fit a party having the minimum upper limit and a party having the maximum lower limit in the number range of the party participating in federated learning.

[0028] Optionally, according to the minimum upper limit and the maximum lower limit, the involved calculating module uses dichotomy and oblivious transfer to calculate new number ranges of the parties, and to formulate an intersection range.

[0029] Preferably, the specific process of the involved calculating module for formulating an intersection range is as follows:

[0030] determining whether the party having the minimum upper limit and the party having the maximum lower limit are the same party;

[0031] if the party having the minimum upper limit and the party having the maximum lower limit are the same party participating in federated learning, then the party formulates a range not greater than its own range as the intersection range and ends;

[0032] if the party having the minimum upper limit and the party having the maximum lower limit are different parties participating in federated learning,

[0033] (a) based on the party having the maximum lower limit, taking a middle value according to the number range of the party, comparing the taken middle value with the upper limits of the remaining parties, if the taken middle value is smaller / larger than the upper limits of the remaining parties, then taking a middle value of the taken middle value and comparing, until the taken middle value is between the upper limits of the remaining parties, using the middle value as a new upper limit to replace the upper limit of the remaining party which is greater than the new upper limit,

[0034] (b) based on the party having the minimum upper limit, taking a middle value according to the number range of the party, comparing the taken middle value with the lower limits of the remaining parties, if the taken middle value is smaller / larger than the lower limits of the remaining parties, then taking a middle value of the taken middle value and comparing, until the taken middle value is between the lower limits of the remaining parties, using the middle value as a new lower limit to replace the lower limit of the remaining party which is greater than the new lower limit.

[0035] The optimization method and tool for the longitudinal federated learning model training process of the present application have the beneficial effects compared with the prior art:

[0036] The present application can accelerate the model training process of longitudinal federated learning while ensuring that information is not leaked, and improve the training efficiency of the longitudinal federated learning model. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flowchart of the method of the present application; Figure 1

[0038] Figure 2 is a connection block diagram of the present application; Figure 2

[0039] Figure 3 is a traditional longitudinal federated training process. Figure 3 DETAILED DESCRIPTION

[0040] In order to make the technical solutions, technical problems solved and technical effects of the present application clearer, the technical solutions of the present application are described in detail below in combination with specific embodiments.

[0041] Embodiment 1:

[0042] In combination with the accompanying drawings, Figure 1 , 3 The present embodiment proposes an optimization method for the longitudinal federated learning model training process. After executing data input, the following operations are performed:

[0043] The party with the smallest upper limit and the party with the largest upper limit in the number range of each party participating in federated learning are fitted using oblivious transfer;

[0044] Based on the smallest upper limit and the largest upper limit, the new number range of each party is calculated using binary search and oblivious transfer, and the intersection range is proposed;

[0045] The intersection range is divided into multiple subintervals using binary search;

[0046] The privacy intersection and longitudinal federated learning of each subinterval are performed in parallel, the longitudinal federated learning results of all subintervals are integrated, and model verification is performed.

[0047] In the present embodiment, the specific operation of proposing the intersection range is as follows:

[0048] (i) If the party with the smallest upper limit and the party with the largest upper limit are the same party participating in federated learning, the party proposes an intersection range that is not greater than its own range and ends;

[0049] (ii) If the party with the smallest upper limit and the party with the largest upper limit are different parties participating in federated learning, ​​​

[0050] (a) based on the party with the largest lower limit, take the middle value according to the numbering range of the party, compare the middle value with the upper limits of the remaining parties, if the middle value is smaller / larger than the upper limits of the remaining parties, take the middle value of the middle value again for comparison, until the middle value is between the upper limits of the remaining parties, use the middle value as a new upper limit to replace the upper limit of the remaining parties that is larger than the new upper limit,

[0051] (b) based on the party with the smallest upper limit, take the middle value according to the numbering range of the party, compare the middle value with the lower limits of the remaining parties, if the middle value is smaller / larger than the lower limits of the remaining parties, take the middle value of the middle value again for comparison, until the middle value is between the lower limits of the remaining parties, use the middle value as a new lower limit to replace the lower limit of the remaining parties that is larger than the new lower limit.

[0052] In this embodiment, it is assumed that there are three parties, A, B and C, participating in federated learning, and their numbering ranges are [10, 87], [5, 92] and [15, 105] respectively, wherein the party with the smallest upper limit is party A and the party with the largest lower limit is party C.

[0053] Based on the numbering range [15, 105] of party C, the middle value 60 (smaller than the upper limits of parties A and B) is taken, the middle value 82.5 (smaller than the upper limits of parties A and B) of the sub-interval [60, 105] is taken by party C, the middle value 93.75 (larger than the upper limits of parties A and B) of the sub-interval [82.5, 105] is taken by party C, the middle value 88.125 (between the upper limits of parties A and B) of the sub-interval [82.5, 93.75] is taken by party C, and this value is used as the new upper limit of parties B and C; at this time, the new ranges of parties A, B and C are [10, 87], [5, 88.125] and [15, 88.125] respectively, and after rounding down, the new ranges of parties A, B and C are recorded as [10, 87], [5, 88] and [15, 88].

[0054] Based on the numbering range [10, 87] of party A, the middle value 48.5 is taken, (larger than the lower limits of parties B and C), the middle value 29.25 of the sub-interval [10, 48.5] is taken by party A, (larger than the lower limits of parties B and C), the middle value 19.625 of the sub-interval [10, 29.25] is taken by party A, (larger than the lower limits of parties B and C), the middle value 14.8125 (between the lower limits of parties B and C) of the sub-interval [10, 19.625] is taken by party A, and this value is used as the new lower limit of party C; at this time, the new ranges of parties A, B and C are [10, 87], [5, 88] and [14.8125, 88] respectively, and after rounding down, the new ranges of parties A, B and C are recorded as [10, 87], [5, 88] and [14, 88].

[0055] Finally, based on the new ranges of Party A, Party B and Party C [10, 87], [5, 88], [14, 88], it is determined that the actual intersection range of Party A, Party B and Party C is [14, 88], and the intersection range is [15, 87] according to the actual range.

[0056] Embodiment Two:

[0057] In combination with the accompanying Figure 2 , the embodiment proposes an optimization tool for a longitudinal federated learning model training process, which is deployed in the longitudinal federated learning model training process, and the structure thereof comprises:

[0058] A determination module is configured to use the oblivious transfer to fit a party with the smallest upper limit and a party with the largest upper limit in the number range of each party participating in federated learning according to the input data of each party.

[0059] A calculation and preparation module is configured to use the bisection method and the oblivious transfer to calculate the new number range of each party and prepare the intersection range according to the smallest upper limit and the largest upper limit.

[0060] A range division module is configured to divide the prepared intersection range into multiple sub-intervals using the bisection method.

[0061] A parallel module is configured to perform the private intersection and the longitudinal federated learning on the multiple sub-intervals in parallel.

[0062] An integration module is configured to integrate the longitudinal federated learning results of all the sub-intervals and perform model verification.

[0063] In the embodiment, the specific process of the calculation and preparation module to prepare the intersection range is as follows:

[0064] It is determined whether the party with the smallest upper limit and the party with the largest upper limit are the same party.

[0065] If the party with the smallest upper limit and the party with the largest upper limit are the same party participating in federated learning, the party prepares an intersection range that is not greater than its own range and ends.

[0066] If the party with the smallest upper limit and the party with the largest upper limit are different parties participating in federated learning,

[0067] (a) based on the party with the largest upper limit, an intermediate value is taken from the number range of the party, the intermediate value is compared with the upper limits of the remaining parties, if the intermediate value is smaller / larger than the upper limits of the remaining parties, an intermediate value of the intermediate value is taken for comparison, until the taken intermediate value is between the upper limits of the remaining parties, the intermediate value is used as a new upper limit to replace the upper limit of the remaining parties that is greater than the new upper limit,

[0068] (b) based on the minimum upper limit, one party takes the intermediate value according to the number range of the party, compares the intermediate value with the lower limit of the remaining parties, if the intermediate value is smaller / larger than the lower limit of the remaining parties, then take the intermediate value of the intermediate value again, until the intermediate value is between the lower limit of the remaining parties, use the intermediate value as a new lower limit, replace the lower limit of the remaining parties which is larger than the new lower limit.

[0069] In conclusion, the optimization method and tool for the longitudinal federal learning model training process can accelerate the model training process of the longitudinal federal learning while ensuring that information is not leaked.

[0070] The above application specific examples have been described in detail for the principles and implementation modes of the present application, and these examples are only used to help understand the core technical content of the present application. Based on the above specific embodiments of the present application, any improvement and modification of the present application made by the person skilled in the art without departing from the principles of the present application shall fall within the patent protection scope of the present application.

Claims

1. An optimization method for a longitudinal federated learning model training process, the method comprising: After the data input is executed, the following operations are performed: ​ Determine the party with the minimum upper limit and the party with the maximum upper limit in the number range of each party participating in federated learning; Based on the minimum upper limit and the maximum upper limit, use binary search and oblivious transfer to calculate the new number range of each party and determine the intersection range, specifically as follows: (i) If the party with the minimum upper limit and the party with the maximum upper limit are the same party participating in federated learning, the party determines an intersection range that is not greater than its own range and ends; (ii) If the party with the minimum upper limit and the party with the maximum upper limit are different parties participating in federated learning, (a) Based on the party with the maximum upper limit, take the middle value according to the number range of the party, compare the taken middle value with the upper limits of the remaining parties, if the taken middle value is smaller / larger than the upper limits of the remaining parties, take the middle value of the taken middle value again for comparison, until the taken middle value is between the upper limits of the remaining parties, use the middle value as the new upper limit, replace the upper limit of the remaining parties that is greater than the new upper limit, (b) Based on the party with the minimum upper limit, take the middle value according to the number range of the party, compare the taken middle value with the lower limits of the remaining parties, if the taken middle value is smaller / larger than the lower limits of the remaining parties, take the middle value of the taken middle value again for comparison, until the taken middle value is between the lower limits of the remaining parties, use the middle value as the new lower limit, replace the lower limit of the remaining parties that is greater than the new lower limit; Use binary search to divide the intersection range into multiple sub-intervals; Use a parallel method to perform privacy intersection and vertical federated learning on each sub-interval, integrate the vertical federated learning results of all sub-intervals, and perform model verification.

2. The optimization method for a federated learning model training process in the vertical direction according to claim 1, wherein, Use oblivious transfer to fit the party with the minimum upper limit and the party with the maximum upper limit in the number range of each party participating in federated learning.

3. An optimization tool for a longitudinal federated learning model training process, the optimization tool comprising: The tool is deployed in the vertical federated learning model training process, and the structure includes: A determination module for determining the party with the minimum upper limit and the party with the maximum upper limit in the number range of each party according to the input data of each party; A calculation and determination module for calculating the new number range of each party and determining the intersection range using binary search and oblivious transfer according to the minimum upper limit and the maximum upper limit; A range division module for dividing the determined intersection range into multiple sub-intervals using binary search; A parallel module for performing privacy intersection and vertical federated learning on each sub-interval in parallel; An integration module for integrating the vertical federated learning results of all sub-intervals and performing model verification; The specific process of the calculation and determination module for determining the intersection range is as follows: Determine whether the party with the minimum upper limit and the party with the maximum upper limit are the same party; If the party with the minimum upper limit and the party with the maximum upper limit are the same party participating in federated learning, the party determines an intersection range that is not greater than its own range and ends; If the party with the minimum upper limit and the party with the maximum upper limit are different parties participating in federated learning, (a) Based on the party with the maximum upper limit, take the middle value according to the number range of the party, compare the taken middle value with the upper limits of the remaining parties, if the taken middle value is smaller / larger than the upper limits of the remaining parties, take the middle value of the taken middle value again for comparison, until the taken middle value is between the upper limits of the remaining parties, use the middle value as the new upper limit, replace the upper limit of the remaining parties that is greater than the new upper limit, (b) Based on the party with the minimum upper limit, take the middle value according to the number range of the party, compare the taken middle value with the lower limits of the remaining parties, if the taken middle value is smaller / larger than the lower limits of the remaining parties, take the middle value of the taken middle value again for comparison, until the taken middle value is between the lower limits of the remaining parties, use the middle value as the new lower limit, replace the lower limit of the remaining parties that is greater than the new lower limit; (a) based on the party with the largest lower bound, taking the middle value according to the numbering range of the party, comparing the taken middle value with the upper bounds of the remaining parties, if the middle value is smaller / larger than the upper bounds of the remaining parties, taking the middle value of the middle value again for comparison, until the taken middle value is between the upper bounds of the remaining parties, using the middle value as a new upper bound to replace the upper bound of the remaining party which is larger than the new upper bound, (b) based on the party with the smallest upper bound, taking the middle value according to the numbering range of the party, comparing the taken middle value with the lower bounds of the remaining parties, if the middle value is smaller / larger than the lower bounds of the remaining parties, taking the middle value of the middle value again for comparison, until the taken middle value is between the lower bounds of the remaining parties, using the middle value as a new lower bound to replace the lower bound of the remaining party which is larger than the new lower bound.

4. The optimization tool for a federated learning model training process in the vertical direction according to claim 3, wherein, The determining module uses the party with the smallest upper bound and the party with the largest lower bound in the numbering range of the parties participating in federated learning according to the oblivious transmission fitting.

Images