Method and apparatus for jointly training logistic regression models

Abstract

The embodiment of the specification provides a method and device for jointly training a logistic regression model by multiple parties. In the process of training the logistic regression model by the multiple parties using their respective private data, a logistic function is approximated by using a multinomial function or a piecewise function, so as to reduce the complexity of the logistic function calculation. Specifically, different accuracy requirements for the approximation of the logistic function are determined according to the characteristics of the training data and the actual situation of the private data held by each data holder, so that an approximation scheme for the logistic function in the logistic regression model is selected. This method can balance the accuracy and efficiency requirements of the logistic regression model in the multi-party secure calculation, and improve the effectiveness of the joint training of the logistic regression model in the multi-party secure calculation scenario.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the field of computer technology, and in particular, to a method and device for jointly training a logistic regression model by multiple parties. BACKGROUND

[0002] With the development of computer technology, machine learning is increasingly widely used in various business scenarios. Federated learning is a method of joint modeling under the protection of private data. For example, enterprises need to cooperate to build a secure model, and can perform joint machine learning to train the data processing model in collaboration using the data of each party while fully protecting the privacy of enterprise data, so as to more accurately and effectively process business data. Joint learning can effectively break down data silos and achieve joint modeling by multiple parties on the basis of privacy protection.

[0003] However, due to industry competition, data security, user privacy, and other issues, data integration faces great resistance. How to integrate data scattered on various platforms together while ensuring that the data is not leaked is a challenge. In the joint learning process based on multi-party secure computation (MPC), complex functions often use other methods for approximation due to unpredictable errors in the secure computation process. Approximation schemes often need to balance accuracy and efficiency, and the accuracy and efficiency requirements at different stages of modeling and joint learning are different. A single approximation scheme cannot meet the needs of different stages of joint learning. SUMMARY

[0004] One or more embodiments of the present specification describe a method and device for jointly training a logistic regression model by multiple parties, to solve one or more problems mentioned in the background.

[0005] According to a first aspect, a method for jointly training a logistic regression model is provided, wherein the logistic regression model comprises a logistic function, business data used for training the logistic regression model comprises private data distributed among a plurality of data parties, and the method is performed by a first data party of the plurality of data parties and comprises: based on a plurality of attribute features respectively extracted from local business data for each business object, and based on secure processing of the attribute features with other data parties, training the logistic regression model using a first approximation scheme of a plurality of candidate approximation schemes instead of the logistic function; based on a training result, securely training the logistic regression model with other data parties to obtain a first shard of a prediction value of the logistic regression model; based on the first shard of the prediction value of the logistic regression model, securely determining a model loss with other data parties to detect convergence of the logistic regression model based on the model loss; based on a detection result of the convergence of the logistic regression model, determining whether to select a second approximation scheme of the plurality of candidate approximation schemes instead of the logistic function for secure calculation to continue secure joint training of the logistic regression model; and in a case where the logistic regression model converges, obtaining a first parameter shard of the logistic function.

[0006] In one embodiment, the model convergence of the logistic regression model is detected by: jointly normalizing or standardizing the attribute features with other data parties before multi-party secure computation to obtain a first shard of a standardized feature matrix; based on the first shard of the standardized feature matrix, jointly and securely detecting a parameter change rate of the logistic regression model with other data parties to detect the model convergence based on the parameter change rate of the logistic regression model.

[0007] In one embodiment, the parameter change rate of the logistic regression model comprises at least one of: a change rate of a model loss, a change rate of a gradient determined based on the model loss, and a change rate of a pending parameter adjusted based on the gradient.

[0008] In one embodiment, the first approximation scheme is a candidate approximation scheme implemented via a first-order polynomial, and the first-order polynomial is one of: a Taylor first-order expansion polynomial, and a Minimax first-order polynomial.

[0009] In one embodiment, in a case where the detection result of the convergence of the logistic regression model is that the logistic regression model does not have convergence, a candidate approximation scheme of at least one of a higher-order polynomial higher than the first-order polynomial and a piecewise function is selected as the second approximation scheme from the plurality of candidate approximation schemes, and the secure joint training of the logistic regression model is continued.

[0010] In one embodiment, in the case where the logistic regression model converges, obtaining the first parameter segment of the logistic function comprises: in the case where the logistic regression model converges, jointly detecting the prediction accuracy of the trained logistic regression model with other data parties; in the case where the prediction accuracy does not meet a predetermined condition, selecting a third approximation scheme of a high-order polynomial higher than a predetermined order combined with a segmented function to replace the logistic function for security calculation, so as to jointly train the logistic regression model again with other data parties according to the third approximation scheme; until the prediction accuracy meets the predetermined condition, the first parameter segment of the logistic function is obtained.

[0011] In one embodiment, the method further comprises: based on the first parameter segment of the logistic function, performing a coefficient significance test on the logistic regression model in terms of classification effect with other data parties in the case of replacing the logistic function with a fourth approximation scheme.

[0012] In one embodiment, the significance test takes p-value as a test index.

[0013] In one embodiment, the fourth approximation scheme is a high-order polynomial of a least square 7th order polynomial expansion of an input variable of the logistic function.

[0014] According to a second aspect, an apparatus for jointly training a logistic regression model is provided, wherein the logistic regression model comprises a logistic function, and business data used for training the logistic regression model comprises private data distributed among a plurality of data parties, and the apparatus is arranged at a first party among the plurality of data parties, and comprises:

[0015] a training unit configured to train the logistic regression model with other data parties based on a plurality of attribute features respectively extracted from local business data for each business object, in the case of replacing the logistic function with a first approximation scheme among a plurality of candidate approximation schemes;

[0016] a prediction value determination unit configured to determine a prediction value of the logistic regression model with other data parties based on the training result, to obtain a first segment of the prediction value of the logistic regression model;

[0017] a convergence detection unit configured to determine a model loss with other data parties based on the first segment of the prediction value of the logistic regression model, to detect the convergence of the logistic regression model based on the model loss;

[0018] The parameter determination unit is configured to determine whether to select a second approximation scheme instead of the logistic function for secure calculation to continue secure joint training of the logistic regression model in the plurality of candidate approximation schemes based on a detection result of convergence of the logistic regression model, and obtain a first parameter segment of the logistic function in a case where the logistic regression model converges.

[0019] According to a third aspect, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of the first aspect.

[0020] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, characterized in that the memory has stored thereon executable code which, when executed by the processor, implements the method of the first aspect.

[0021] The method and device provided by the embodiments of the present specification can reduce the complexity of the logistic function calculation by using the polynomial, segmented function and the like to approximate the logistic function in the process of training the logistic regression model by multiple parties. Specifically, different accuracy requirements for the approximation calculation of the logistic function are determined according to the characteristics of the training data and the actual situation of the private data held by each data holder, so that a better approximation scheme is selected from the plurality of candidate approximation schemes for the logistic function in the logistic regression model. This way can take into account the accuracy and efficiency requirements of the logistic regression model in the multi-party secure calculation, and improve the effectiveness of the joint training of the logistic regression model in the multi-party secure calculation scenario. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 A specific implementation architecture diagram under the technical concept of the present specification is shown;

[0024] Figure 2a A specific example diagram showing horizontal partitioning of data of each data party in federated learning is shown;

[0025] Figure 2b A specific example diagram showing vertical partitioning of data of each data party in federated learning is shown;

[0026] Figure 3A diagram showing the number of interactions in a multi-party secure computing process for various candidate computing schemes for a logistic regression model;

[0027] Figure 4 A diagram showing a process performed by a single data party in training a logistic regression model according to an embodiment of the present specification;

[0028] Figure 5 A diagram showing an apparatus for training a logistic regression model in a single data party according to an embodiment of the present specification. DETAILED DESCRIPTION

[0029] The schemes provided by the present specification will be described below with reference to the accompanying drawings.

[0030] Figure 1 A diagram showing an implementation scenario of an embodiment disclosed in the present specification. As shown in the diagram, Figure 1 In a joint learning (e.g., federated learning) scenario, a data set is held by multiple holding parties 1, 2, 3, etc. Each holding party can hold a part of the data in the data set, which can also be referred to as a data party, or a member party, a training member, a member, etc. Figure 1 The number of data holding parties shown in the diagram is 3, but in practice, the number of data holding parties can be n, where n is a positive integer greater than 1. The data set here can be a training data set for training a logistic regression model, or a test data set for testing other neural network models, or a data set to be predicted. The data set can include attribute feature data of business objects, which are private data of the data holding parties. The business objects can be various business objects to be analyzed, such as users, merchants, goods, events, etc., e.g., bank users, hospital patients, etc.

[0031] The data distribution in joint learning can be divided into two types. One is horizontal distribution or horizontal distribution, in which each data party holds data of the same attribute items of different business objects. For example, there are 100 samples, and each sample contains 20 attribute items in the private data, which are distributed among 4 holding parties, and each data party holds all 20 attribute data of 25 samples. The samples (corresponding to various business objects) owned by each data party can be the same or different. This data distribution is shown in Figure 2aThe data horizontal distribution diagram is shown. As a specific example, assume that the business object is a user, and a social platform is one of the multiple data parties, which can have q user basic attribute features and label information of p1 users, such as user id, region, number of friends, registration duration, and whether the user is real. Another social platform is one of the multiple data parties, which can have q user basic attribute features and label information of p2 users, such as user id, region, number of friends, registration duration, and whether the user is real. In this way, different samples are distributed horizontally in different data parties.

[0032] The other is longitudinal distribution or vertical distribution, in which each holding party has different attribute data of all business objects (samples). For example, there are 100 samples, and each sample contains 100 attributes in the privacy data. These data are distributed in 4 data parties, and a single holding party can hold 20 attribute data (which can also be 30, 41, or any number of items not exceeding 100) of all 100 samples, and each data party has different attributes. This data distribution mode is shown in Figure 2b The data longitudinal distribution diagram is shown. For example, as a specific example, assume that the business object is a merchant. A certain e-commerce platform is one of the multiple data parties, which can have part of the attribute items of N merchants, such as merchant id and operation duration. Another e-commerce platform is one of the multiple data parties, which can have another part of the attribute items of the N merchants and label information, such as merchant category, cumulative sales, and whether the merchant is honest.

[0033] In some embodiments, the distribution of data in each data party can also be various complex cases, such as a combination of horizontal and vertical (which can be divided into horizontal and vertical), and the like, which will not be described here. Regardless of the data distribution mode, the attribute data of the business object belongs to the privacy data. In order to protect the privacy data, each data party needs to keep its privacy data locally, does not output the plaintext data, and does not perform plaintext aggregation.

[0034] Continuing to refer to Figure 1 As shown, in the joint learning scenario, each data party can communicate and perform various calculations through secure calculation methods such as secret sharing, homomorphic encryption, and garbled circuits, so as to jointly complete model training using the privacy data of each party. Taking secret sharing as an example, each data party processes the local privacy data by random splitting, adding a disturbance item, and the like, and then shares a part of the data as secret sharing to other data parties, so as to complete relevant calculations such as addition, numerical multiplication, and matrix multiplication.

[0035] In practice, there are usually some complex operations in the business model trained jointly, such as operations of activation functions, exponential normalization functions softmax, logistic functions sigmoid, etc. These complex operations may include power operations, exponential operations, and division operations in a shared form, etc. These complex operations have high complexity in the amount of calculation and communication in the process of multi-party secure calculation, and may be prone to errors or difficult to perform. In some optional implementations, these operations can be performed with the help of a trusted third party. In other implementations, without a trusted third party, each data party can approximate and replace these functions by converting them into other operations that can be completed by combination of addition, multiplication, etc. In the specific implementation process, a complex function can have multiple approximation schemes, and these schemes differ in calculation accuracy (proximity to the original function) and efficiency, etc.

[0036] Taking the logistic function sigmoid in the logistic regression model as an example, it is a commonly used classification function and is often used in business prediction of business objects. When the logistic regression model is used to regress the sample features, the calculation formula of the predicted value by sigmoid includes, for example:

[0037]

[0038]

[0039] where x is the input variable of sigmoid, π(x) is the predicted value function of the sample, β is the parameter of sigmoid, and e is the natural constant. As can be seen, the calculation process of the logistic regression model includes exponential operations, division operations, etc.

[0040] In the training of the logistic regression model, n samples and corresponding labels can be used to train the logistic regression model to obtain the predicted value corresponding to each sample. According to the difference between the predicted value and the label, the loss value is determined, and the model parameters are adjusted in the direction of reducing the loss value until the logistic regression model reaches the convergence condition. The predicted value, also known as the estimated value, represents the probability of belonging to a certain category. The label can represent two categories using 0 or 1, and the corresponding predicted value can be a value between 0 and 1. When the sample features include D attribute items, the logistic regression model can correspond to D model parameters. When adjusting the model parameters, the D model parameters can be adjusted. When the training of the logistic regression model is completed, the predicted value of each sample (a total of N predicted values) and D model parameters can be obtained. The n predicted values correspond to each sample, and the D model parameters correspond to each attribute item.

[0041] In the multi-party secure computation process, the combination of operations related to the sigmoid function can introduce unacceptable errors due to the data storage methods such as floating-point numbers, fixed-point numbers, or the sharing form of data sums. Therefore, various approximation schemes can be used to calculate them.

[0042] In this specification, the sigmoid function can be calculated by a polynomial of the input variable of the logistic function or a piecewise function, such as a Taylor series polynomial, a least squares polynomial, a Minimax polynomial, a piecewise function, etc. In the case of high accuracy requirements, the approximation method can also be a combination of piecewise functions and polynomials. Among them:

[0043] The expansion of the Taylor series is, for example:

[0044]

[0045] Since the Taylor series is an infinite polynomial, and the absolute value of each term is getting smaller and smaller, in practical application, its approximate value is usually taken. For example, the first-order approximation n = 1, the second-order approximation n = 2, etc.

[0046] The least squares approximation is based on the minimax principle to minimize the mean square error MSE1 / |I|∫ I (g(x)-f(x)) 2 dx, where |I| is the length of the integral interval I, g(x) and f(x) are the least squares polynomial and the logistic regression function, respectively;

[0047] The piecewise function can be a piecewise linear function with a value in the interval 0-1, for example:

[0048]

[0049] In the combination of piecewise functions and polynomials, Taylor series polynomials and Minimax polynomials can be used to approximate at least one segment of the piecewise function.

[0050] In other embodiments, other ways can be used to approximate the sigmoid function, which will not be described here. It can be understood that the calculation of polynomials, piecewise functions, etc. approximates the sigmoid to an integer power of the input variable, and in the sharing form, it is easier to calculate by secure addition and secure multiplication.

[0051] It is worth noting that the logistic function as a classification function can be used as a separate model or embedded in the processing process of other business models, for example, sigmoid(wx) is a business model containing input layer x, hidden layer (sigmoid layer), and output layer, where the mapping of the input layer to the hidden layer is achieved by w. Here, various models using the logistic function for classification can be referred to as logistic regression models. This specification aims to discuss the approximation of the logistic function in the case of multi-party secure computation.

[0052] In a business process, a single logistic function approximation scheme is usually used, and different business processes or multiple logistic function calculations included in the same business process may have different requirements for the accuracy, efficiency, etc. of the logistic function. Therefore, for high-precision situations, approximation errors can cause large result errors, and for low-precision situations, high-order calculations can increase computational complexity and affect computational efficiency.

[0053] To this end, as shown in Figure 1 This specification provides an approximation scheme for the logistic function, which can provide multiple candidate approximation schemes (such as Figure 1 algorithm 1, algorithm 2, etc. in ) to approximate the logistic function in the logistic regression model. In the multi-party joint learning process, each data party can select an appropriate approximation scheme from the candidate calculation schemes according to the current data characteristics and the needs of the calculation target to perform approximation operations on the sigmoid function calculation process in the current logistic regression model.

[0054] It can be understood that the approximation of the logistic function sigmoid is usually a numerical operation, so the number of communication interactions between data parties can be an important indicator of efficiency. According to the principle of numerical operation under multi-party secure computation, usually, 1 secret sharing of numerical multiplication requires 1 online interaction, and 1 segmented function comparison operation requires 9 interactions (including online 2log2(k) and the generation of a general MUX(1) random number, k is the bit number of a single data, such as 64). Referring to Figure 3 shown is a comparison of the number of communication interactions for some approximation methods of the logistic function (corresponding to the approximation function f(x) column). From Figure 3As can be seen, the higher the polynomial order, the more complex the approximate calculation scheme, the more interactions, the smaller the sample parallelism, and the more parallel interactions. In fact, according to some experiments, under the condition that the sample size is 100,000, the single batch sample size is 64, the cycle number is 50, and the calculation delay is 50 milliseconds, if the interaction number in the sigmoid approximation calculation increases by 10, the training time of the logistic regression model increases by about 15 to 20 minutes. That is, the higher the polynomial order, the higher the accuracy of the approximate calculation scheme, but the more interactions, the lower the training efficiency of the logistic regression model.

[0055] In order to balance between accuracy and efficiency, the present specification proposes an approximate scheme for the logistic function sigmoid during the training process of the logistic regression model by multiple parties. The approximate scheme can be selected according to the characteristics of the current data and different requirements of accuracy and efficiency, so as to balance the accuracy and efficiency. Please refer to Figure 4 As shown in the figure, a flowchart of the joint training of the logistic regression model by multiple parties is given.

[0056] Figure 4 The flowchart shown is a flowchart executed by a single data party in the joint training of multiple data parties. The single data party (single training member) here can be any device, platform or device cluster with computing and processing capabilities. It can be understood that in order to enable multiple data parties to select corresponding approximate schemes, a third party platform can provide multiple candidate approximate schemes of the logistic function sigmoid in advance. The third party platform, for example, is a service party, which can provide candidate approximate schemes for one or more complex calculations. The third party platform can also be an application platform that can be installed on each data party. In short, the third party can provide candidate approximate schemes of the sigmoid function to the data party as a black box, but can not disclose the specific calculation process to the data party. The data party can select the approximate scheme through different interfaces. These candidate calculation schemes can include polynomials of different orders, piecewise functions, etc., in which the coefficients are determined by the third party in advance through simulation. In the simulation process, the logistic function can be approximated with as high a precision as possible under the corresponding order of the corresponding method. The order here can correspond to the corresponding power of the input variable (such as x) of the sigmoid function. For example, the first order indicates that the highest power in the polynomial is one, the second order indicates that the highest power in the polynomial is two, and so on.

[0057] Among them, the secure calculation between multiple data parties can be based on secret sharing, homomorphic encryption and other conventional multi-party secure calculation methods. The present specification improves the approximate scheme of the logistic function sigmoid in multi-party secure calculation, but does not limit the way of secure calculation, so it is not described here.

[0058] For the convenience of description, Figure 4Taking any one of the plurality of data parties (referred to as a first party) as an example, the training process of the logistic regression model is described in combination with one parameter update period. As shown in Figure 4 In the process of jointly training the logistic regression model by the plurality of parties, the steps performed by the first party include: step 401, based on a plurality of attribute features respectively extracted from local business data for each business object, and based on secure processing of the attribute features by other data parties, training the logistic regression model using a first approximation scheme of a plurality of candidate approximation schemes instead of the logistic function; step 402, according to the training result, securely determining the predicted value of the logistic regression model with other data parties to obtain a first slice of the predicted value of the logistic regression model; step 403, according to the first slice of the predicted value of the logistic regression model, securely determining the model loss with other data parties to detect the convergence of the logistic regression model based on the model loss; step 404, based on the detection result of the convergence of the logistic regression model, determining whether to select a second approximation scheme of the plurality of candidate approximation schemes to replace the logistic function for secure calculation to continue secure joint training of the logistic regression model; and in the case of convergence of the logistic regression model, obtaining a first parameter slice of the logistic function.

[0059] First, in step 401, based on a plurality of attribute features respectively extracted from local business data for each business object, and based on secure processing of the attribute features by other data parties, the logistic regression model is trained using a first approximation scheme of a plurality of candidate approximation schemes instead of the logistic function.

[0060] It can be understood that a single data party can extract part or all of the attribute features of the training sample from the local private data. The attribute features here can be numerical values of attribute information used to describe business objects, for example, if a user browses a webpage for 30 seconds, the attribute feature corresponding to the attribute item "browsing time" for the webpage can be the numerical value, i.e. 30 seconds.

[0061] In the case of horizontal distribution of data at each data party, a single data party can extract complete sample information corresponding to a plurality of business objects. A sample information describes a training sample, which can include attribute features and sample labels. For example, a data party holding data of a shopping platform can extract attribute features related to search, browsing, etc. from search information, browsing information, etc. of a user (business object) before a purchase behavior, and obtain the information of the goods purchased in the current purchase behavior as the label.

[0062] In the case that the data of each data party is longitudinally distributed, a single data party can extract part of the attribute features of each business object. These attribute features and the attribute features of other data parties together constitute the training samples. The sample labels can be extracted by a single or multiple data parties according to the local data. Details are not described herein.

[0063] In the federated learning process based on multi-party secure computation, the first party can jointly perform forward secure computation on the attribute features of each business object with other parties. It can be understood that in the logistic regression model, the sigmoid function is used as the classification function, and before that, various attribute features can be processed in various reasonable ways, for example, processed by using a fully connected neural network, processed by using a convolutional network, and the like. Taking the fully connected neural network as an example, assuming that the feature matrix composed of multiple training samples is denoted as X, and the parameter matrix of the fully connected neural network is W, the attribute feature fusion result of WX can be obtained.

[0064] These fusion processing results can be used as the input variables of the logistic function, such as x in the formula (1). It can be understood that in the process of multi-party secure computation, in order to protect data privacy, each data party holds one and a shared slice of various intermediate results and model parameters. For example, in the process of WX computation, the first party can hold one and a shared slice W1 of W, and obtain a slice (WX)1 of the fusion result WX. In the case of sigmoid(WX) for the logistic regression model, it can be understood that the first party obtains a slice of the input variable WX of the logistic function, which is called the first slice. In practice, the logistic regression model can also include other forms of sigmoid, such as sigmoid(WX+b) and the like. In order to protect data privacy, in various forms, the first party can obtain the first slice of the input variable of the logistic function.

[0065] Then, the logistic regression model can be trained by using a first approximation scheme in the multiple candidate approximation schemes to replace the logistic function.

[0066] Under the technical concept of the present specification, multiple candidate approximation schemes can be set in advance. A single candidate approximation scheme can be a polynomial with respect to the input variable of the logistic function, or a piecewise function with respect to the input variable of the logistic function, and on a single segment of the piecewise function, it can be a polynomial function of the input variable (denoted as x) of the logistic function. As a specific example, a candidate approximation scheme of a simple three-segment piecewise function is, for example:

[0067]

[0068] A candidate approximation scheme of a five-segment piecewise function combined with a polynomial is, for example:

[0069]

[0070] wherein x in the above polynomials can represent the input variable of the logistic function. The polynomials in the above piecewise function examples are all of 1st order, in practice, polynomials of more orders can be used according to needs, such as 3rd order, 5th order, 7th order, etc. In addition, the above piecewise functions are examples of polynomials combined with piecewise functions, using lower orders can improve efficiency, in practice, only high-order polynomials can also be used, or the form of piecewise functions combined with high-order polynomials.

[0071] The high-order polynomials approximating the logistic function sigmoid can determine the corresponding expansion form based on the relevant approximation theory, and determine the parameters therein through reasoning or experimental simulation. For example, the polynomial form of Taylor series expansion is:

[0072]

[0073] By replacing f(x) with sigmoid(x) and determining the expansion coefficients through simulation, the following 1st order, 3rd order, and 5th order candidate approximation schemes for sigmoid(x) of Taylor series can be obtained:

[0074] Taylor_1 = 0.5 + 0.25x

[0075] Taylor-3 = 0.5 + 0.25x - 1 / 48x 3

[0076] Taylor-5 = 0.5 + 0.25x - 1 / 48x 3 + 1 / 480x 5

[0077] For another example, in the Minimax method, define ||f|| = max |f(x)|, where x is a value on the interval [a, b], under the constraint f(x) = C[a, b], p ∈ P_d is the d-th Minimax estimate of f ∈ C[a, b] if ||f-p|| = inf ||f-q||, q ∈ P_d. Thus, the first-order and third-order polynomials for approximating sigmoid(x) in the Minimax method can be determined, such as:

[0078] minimax_1 = 0.5 + 0.125x

[0079] minimax_3 = 0.5 + 0.197x - 0.004x 3

[0080] The least squares polynomial is based on minimizing 1 / |I|∫ I(g(x) - f(x)) 2 The variance of dx is the target, and polynomials such as 3rd order, 5th order, 7th order, etc. are determined to approximate sigmoid(x). Where I represents the integral interval, |I| can represent the length of the integral interval, f(x) and g(x) represent the least square polynomial and the logistic function respectively.

[0081] In more polynomial expressions, there can be more polynomial expressions, which will not be described one by one here. Among them, the polynomial expression can be combined with the piecewise function, for example, a first-order or multi-order (such as third-order, fifth-order, etc.) polynomial is used to approximate sigmoid(x) in a certain independent variable interval of the piecewise function. These optional approximation schemes can be used as candidate approximation schemes for each data party to choose.

[0082] The first approximation scheme can be a current approximation scheme selected from the candidate approximation schemes for replacing the sigmoid function calculation in the current parameter update period. Considering the joint learning efficiency in the multi-party secure computing scenario, in the parameter update period, a low-order approximation scheme can be selected to replace the calculation of the sigmoid function. The low-order polynomial can be, for example, a polynomial whose order is not higher than a predetermined order (such as 3rd order). And the polynomial with the lowest order is usually a first-order polynomial, such as Minimax first-order polynomial, Taylor series first-order polynomial, etc. Taking the Minimax first-order polynomial as an example, for example, Minimax-1 = 0.5 + 0.125z, where z is the logistic regression independent variable, such as the logistic regression model in the previous text can be allowed βx = z. Usually, 0.5 is used as the reference quantity of the logistic regression, so the constant term in the polynomial expansion is 0.5.

[0083] The secure calculation of the first-order polynomial can be directly obtained by each data party using the local input variable slice to get the approximate result of the corresponding slice, so it does not require each data party to communicate with each other, and the calculation efficiency is higher. With the progress of the training process, the approximation calculation scheme of the sigmoid function can be replaced according to the characteristics of the business data of each data party. At this time, the first approximation scheme can also be a high-order polynomial, a piecewise function or a combination thereof.

[0084] On the basis of the calculation of the sigmoid function in the forward calculation process by the first approximation scheme, each data party can jointly train the logistic regression model to obtain each parameter of the logistic function. During model training, N samples and corresponding labels can be used to train the logistic regression model to obtain the predicted value corresponding to each sample. According to the difference between the predicted value and the label, the loss value is determined, and the model parameters are adjusted in the direction of reducing the loss value. When the sample features contain D attribute items, the logistic regression model can correspond to D model parameters. When adjusting the model parameters, the D model parameters can be adjusted. After training the logistic regression model, the predicted value of each sample (a total of N predicted values) and D model parameters can be obtained. The N predicted values correspond to each sample, and the D model parameters correspond to each attribute item. Among them, each party can obtain a shared slice (share) of each predicted value and each model parameter.

[0085] Therefore, in step 402, according to the training result, the first party securely determines the predicted value of the logistic regression model with other data parties to obtain the first slice of the predicted value of the logistic regression model.

[0086] Although the training of the logistic regression model pays more attention to efficiency, the basic requirement of model training requires that the training process can make the model have convergence. Therefore, further, in step 403, according to the first slice of the predicted value of the logistic regression model, the first party securely determines the model loss with other data parties to detect the convergence of the logistic regression model based on the model loss.

[0087] It can be understood that the feasibility of model training usually depends on whether the model parameters (to-be-determined parameters) can converge. Specifically, as the training period increases, the model loss tends to 0, or converges near 0. In the case where the model loss tends to 0, the gradient of the model loss with respect to the model parameter tends to 0, the adjustment amplitude of the model parameter tends to 0, and the model parameter tends to a stable value. Therefore, whether the model parameter can be measured by loss convergence, gradient convergence, model parameter value convergence, etc. Here, for the convenience of description, whether the model parameter can tend to converge can be referred to as model convergence.

[0088] The first party can jointly detect the parameter change rate of the logistic regression model with other data parties to detect the model convergence based on the parameter change rate of the logistic regression model. Here, the parameter change rate of the logistic regression model includes at least one of the following: the change rate of the model loss, the change rate of the gradient determined based on the model loss, and the change rate of the pending parameter adjusted based on the gradient. In the case where the change rate converges around 0, it can be determined that the logistic regression model has convergence when the sigmoid function is replaced by the first approximation scheme for calculation, otherwise, the logistic regression model does not have convergence when the sigmoid function is replaced by the first approximation scheme for calculation.

[0089] The parameter change rate of the logistic regression model is usually determined on the basis of multiple parameter update cycle iterations. In an optional embodiment, a reference cycle number, such as 20, can be given, and it is detected whether the parameter change rate of the logistic regression model tends to converge after the reference cycle number.

[0090] It can be understood that the detection result of the model convergence does not leak the local privacy data of each data party, and the detection result can be disclosed to each data party. That is, the first party can obtain the detection result in plaintext.

[0091] Further, in step 404, based on the detection result of the convergence of the logistic regression model, it is determined whether to select a second approximation scheme from the plurality of candidate approximation schemes to replace the logistic function for secure calculation to continue secure joint training of the logistic regression model; and in the case where the logistic regression model converges, the first parameter segment of the logistic function is obtained.

[0092] Based on the detection result of whether the model has convergence, each data party can select to continue to use the first approximation scheme or use other candidate approximation schemes to replace the calculation process of the logistic regression function from the plurality of candidate approximation schemes. Specifically, considering the efficiency requirement of the joint training model, in the case where the model has convergence, it is indicated that the result of approximating the logistic function sigmoid using the first approximation scheme can meet the requirement of the model training (convergent). In the case where the model does not have convergence, it is indicated that the result of approximating the logistic function sigmoid using the first approximation scheme cannot meet the requirement of the model training. Therefore, a candidate approximation scheme with higher accuracy, such as a higher-order polynomial or a segmented function, or a higher-order polynomial embedded in a segmented function, can be selected as a second approximation scheme to replace the calculation of the sigmoid function, and the logistic regression model is further trained.

[0093] In an optional implementation, to ensure efficiency, the complexity of the selected candidate approximation scheme (such as the order or the number of segments) can be increased step by step. For example, the process of selecting the target approximation scheme of the sigmoid function based on model convergence can be an iterative process. In the case where a low-order polynomial cannot make the model have convergence, a way of including a low-order polynomial (such as the 3-segment piecewise function in the foregoing example) in the piecewise function can be selected, the joint processing of the training samples is repeated, and it is detected whether the model has convergence. Further, in the case where the model has convergence, the logistic regression function is approximated using the currently determined low-order polynomial included in the piecewise function. Otherwise, a higher-order polynomial is selected to approximate the logistic regression function. For example, in the case where the use of a first-order candidate approximation scheme cannot make the logistic regression model have model convergence, a piecewise function including a first-order polynomial is selected, and after a number of parameter update periods, it is detected whether the model has convergence. If the model still cannot converge, a 3rd-order, 5th-order, or other multi-order polynomial can be used, or a piecewise function with a larger number of segments can be used, or the piecewise function includes a higher-order polynomial, etc., to replace the calculation of the sigmoid function, until it is detected that the model has convergence.

[0094] According to an embodiment, to facilitate the convergence of the model, according to an embodiment, each data party can further include normalization processing, such as min-max normalization, in the data preprocessing performed for the logistic regression model, such as: In this way, the processing result of the logistic regression model can be more conducive to convergence. Alternatively, whether the product of the current model parameters and the feature matrix (such as wx) converges to a predetermined interval can be used as a detection result of whether the model has convergence.

[0095] According to another optional embodiment, each data party does not pre-process the training samples such as normalization, but directly selects a candidate approximation scheme corresponding to a higher-order (such as 5th-order) polynomial or a candidate approximation scheme in which a piecewise function is combined with a high-order polynomial (such as a piecewise function including a 5th-order least square polynomial) to replace the operation of the sigmoid function. In this way, the candidate approximation scheme that makes the model have convergence can be determined more quickly.

[0096] In the case where the logistic regression model has convergence, the training process of the logistic regression model can be ended. At this time, each data party can jointly obtain each parameter of the logistic function. In the multi-party secure computing scenario, the first party can obtain a segment of each parameter of the logistic function, for example, referred to as the first parameter segment of the logistic function.

[0097] Those skilled in the art can understand that making the model have convergence is a basic condition for the model to be trained. On the basis of ensuring that the model can be trained, the influence of the target approximation scheme for making the model converge instead of the sigmoid function calculation on the precision of the logistic regression model can also be considered. Therefore, in the optional implementation, in order to meet the logistic function calculation precision of the business data for training the logistic regression model, it is also necessary to determine whether to further adjust the target approximation scheme (such as the first approximation scheme or the second approximation scheme, etc.) selected according to the convergence according to the actual situation.

[0098] Specifically, after jointly training the logistic regression model using the target approximation scheme determined according to the model convergence, the first party can also jointly detect the prediction accuracy of the trained logistic regression model with other data parties. In the case where the prediction accuracy does not meet the predetermined condition, the third approximation scheme of combining a high-order polynomial higher than the predetermined order of the input variable of sigmoid with a segmented function is selected instead of the logistic function for secure calculation, so that the logistic regression model is re-securely jointly trained with other data parties according to the third approximation scheme. For example, assuming that the candidate approximation scheme used in the training process is a 5th order polynomial, the third approximation scheme can be a 7th order polynomial, or a segmented function containing a 5th order polynomial, or a segmented function containing a 7th order polynomial, etc. It can be understood that this process of adjusting the approximation scheme based on the prediction accuracy can also be iterated until the prediction accuracy meets the predetermined condition, and the final determined candidate approximation scheme, such as the third approximation scheme, is used to replace the calculation of the sigmoid function to train the logistic regression model with other data parties. And the first parameter piece of the logistic function is obtained when the training is completed.

[0099] It can be understood that the feature data of the machine learning model can come from multiple data parties, for example, for an e-commerce related machine learning model, the multiple data parties can be e-commerce companies, express companies, banks, etc. The data of each data party constitutes a huge data system, and a large number of features can be extracted. Among these features, some are important for business results, and some are not important enough. For example, for the business of pushing goods that may be of interest to consumers, features such as the goods corresponding to the page browsed and searched, the browsing time, the browsing frequency, the search frequency, etc. may be more important, and the time point corresponding to the browsing and searching behavior, such as noon or night, may not be very important.

[0100] At this time, in the case where the prediction accuracy of the trained logistic regression model does not meet the predetermined condition, the third approximation scheme of combining a high-order polynomial higher than the predetermined order of the input variable of sigmoid with a segmented function is selected instead of the logistic function for secure calculation, so that the logistic regression model is re-securely jointly trained with other data parties according to the third approximation scheme. Figure 4On the basis of the shown flow for training the logistic regression model, a significance test of the feature coefficients of the logistic regression model can also be performed. Then, after step 404, a step (not shown) of significance test can also be included: based on the first parameter slice of the logistic function, the coefficient significance test on the logistic regression model in the classification effect is safely performed with other data parties in the case of replacing the logistic function with the fourth approximation scheme.

[0101] A significance test is generally to make a hypothesis on the parameter or the form of the population distribution of the population (random variable) in advance, and then use sample information to judge whether the hypothesis (alternative hypothesis) is reasonable, that is, to judge whether the true situation of the population is significantly different from the original hypothesis. Or, the significance test wants to judge whether the difference between the sample and the hypothesis made on the population is purely a chance variation, or is caused by the inconsistency between the hypothesis made and the true situation of the population. The conventional significance test methods are, for example, t-test, U-test, variance analysis, Wald test, etc.

[0102] In the multi-party secure computing scenario, it is hoped that more useful features are jointly modeled by less computing amount. In this way, each data party can perform a significance test on the features of the model, so as to select the features that are more important to the business target, and filter out the less important features. The significance test of the present specification can adopt the Wald test.

[0103] The principle of the Wald test for the significance test of the logistic regression model is introduced below.

[0104] For the logistic regression model given in the foregoing, the null hypothesis and the alternative hypothesis of the Wald test are respectively:

[0105] H0:ω j = 0 (j = 1, 2, …, k) indicates that the independent variable has no effect on the probability of event occurrence;

[0106] H1:ω j ≠ 0

[0107] If the null hypothesis is rejected, it means that the change of the dependent variable depends on the independent variable j

[0108] The test statistic of the Wald test is:

[0109]

[0110] It conforms to the chi-square distribution with 1 degree of freedom. Where SE(β k ) is the standard error of β k , which is also equal to the square root of the diagonal element of the variance-covariance matrix, that is:

[0111]

[0112] The variance and covariance matrix H of the model parameters -1 , the value of the negative Hessian matrix of the log-likelihood function at

[0113]

[0114] where:

[0115]

[0116] where the indices k and r are natural numbers less than m, x ik and x iT are elements in the joint feature matrix X, x i is the feature vector of the i-th sample in the joint feature matrix X. The joint feature matrix X is a matrix composed of feature vectors of each sample. As denoted as:

[0117]

[0118] From the derivation of the above formula, it can be seen that the H matrix can be expressed as H = X T AX, where:

[0119]

[0120] where m is the total number of samples, n is the feature dimension of the sample, π(x m ) is the predicted value of the logistic regression model for the sample x m , and A is a diagonal matrix obtained based on the predicted value matrix.

[0121] From the above formula , it can be seen that for the k-th model parameter, when the standard error of the model parameter is larger, that is, the value of the k-th row and the k-th column in the variance covariance matrix is larger, it means that the model parameter will make the volatility of the logistic regression model larger, and the Wald test value corresponding to the model parameter is smaller.

[0122] After determining the significance test value Wald k of the k-th model parameter, the corresponding p-value (p-value) can also be determined according to the Wald k value, that is, the Wald k ​The p-value is unified to the same level. When the p-value is less than a given significance level threshold a, the null hypothesis is rejected, and the model parameters can be retained for modeling. The significance level threshold a can be, for example, 0.05 or 0.01, etc. The Wald test can test whether the parameters related to a set of explanatory variables (i.e., attributes) are zero. If the Wald test shows that they are not zero for a particular explanatory variable or set of explanatory variables, it can be concluded that these explanatory variables are important and should be included in the model. If the Wald test shows that these explanatory variables are not significant, they can be omitted from the model. The determination of the p-value can be determined by looking up a conversion table or by other methods. As in an optional embodiment, the p-value can be negatively correlated with the statistical quantity z as follows: p = 2(1 - norm.cdf(|z|)), where norm.cdf(|z|) represents the integral of z in a predetermined interval.

[0123] According to the principle of the Wald significance test described above, the significance test requires model training to adjust the model parameters, and the significance test is performed when the model parameters tend to converge. In the significance test process, the predicted value of the sample is needed, which is determined based on the logistic regression model. The demand for the logistic regression model in the training stage needs to consider both efficiency and accuracy, and the accuracy requirement of the sigmoid function in the logistic regression model is higher in the significance detection stage.

[0124] In the significance test process, for example, the p-value is used for significance test. The p-value can be obtained from the predicted value of the training sample set determined by the logistic regression model trained in step 404. For example, according to an embodiment, each data party can obtain the estimated value of the model parameter corresponding to the feature owned by each data party based on secure computation and the sharing form of the sum of the predicted values, for example, when the number of data parties is 2, the predicted value wherein are the shards of the sum and the sharing form of the predicted value, respectively, and are stored in two data parties, respectively. Each data party can jointly determine the Hessian matrix based on the predicted value, thereby obtaining the statistical quantity z with a normal distribution According to the foregoing, the square of the z value is the Wald statistic with a chi-square distribution. The p-value can be determined by integrating the z value in a predetermined interval, such as p = 2(1 - norm.cdf(|z|)). In order to determine the integral interval corresponding to the p-value, the fitting curve of various intervals can be simulated and compared with the logistic regression curve. Since the value of the logistic function is very small when it exceeds the interval [-6, 6], the least squares polynomial can be used to fit the logistic function in the interval [-6, 6].

[0125] Since the Hessian matrix is sensitive to accuracy, the approximation scheme of the logistic function requires higher accuracy. According to the candidate approximation scheme provided in the specification, the 7th order polynomial of least squares, such as a specific 7th order polynomial of least squares 0.5+0.2168x-0.00819x 3 +0.00016583x 5 -1.1×10 -06 x 7 , the accuracy loss (error) of the simulation of the logistic regression model on multiple public data sets can be kept within 0.001, which can meet the calculation requirements of the Hessian accuracy. The accuracy loss can be determined by the square difference between the polynomial approximation value and the logistic function calculation value on the corresponding data set. In an optional embodiment, the 7th order least squares polynomial or the segmented function containing the 7th order least squares polynomial can be selected as the fourth approximation scheme to approximate the logistic function in the logistic regression model.

[0126] Based on this significance test, the parameters that can be retained in the logistic regression model can be determined to provide a reference for multiple data parties to delete insignificant parameters according to the significance test results to construct more effective business models. Thus, the rationality of the logistic regression model jointly trained by multiple data parties is improved. In addition, in the case of continuing to train the logistic regression model according to the significance test results, a reasonable approximation scheme can be selected from the candidate calculation scheme to approximate the logistic function, thereby improving the effectiveness of the model in processing specific business.

[0127] In summary of the above process, during the joint training of the logistic regression model by multiple data parties, a candidate approximation scheme is provided under various accuracy and efficiency balances. Through the candidate approximation scheme of the logistic function at various accuracies simulated in advance according to various polynomial expansion theories, multiple data parties can select a more efficient approximation scheme according to the accuracy and efficiency requirements at the current stage while meeting the accuracy requirements during the secure training of the logistic regression model, without the need to simulate the approximation scheme themselves. Thus, the problem of balancing accuracy and efficiency in the process of using a single scheme to approximate the logistic function in secure multi-party computation is solved.

[0128] According to another aspect of the embodiment, the specification also provides a device for jointly training a logistic regression model by multiple parties. Here, the private data is business data used by multiple data parties to classify business objects based on the logistic regression model using multi-party secure computation. The device can be provided in any of the multiple data parties. Assuming that the first party is any one of the multiple data parties, the device for jointly training the logistic regression model by multiple parties can be provided in the first party to cooperate with other data parties to complete the significance test of the logistic regression model.

[0129] Figure 5 An apparatus 500 for training a logistic regression model by a plurality of parties in a service side is shown. As shown, the apparatus 500 comprises: Figure 5

[0130] a training unit 501 configured to train the logistic regression model based on a plurality of attribute features respectively extracted from local business data for each business object and based on a secure processing of the attribute features by other data parties, in a case that a first approximation scheme of a plurality of candidate approximation schemes is used to replace a logistic function;

[0131] a prediction value determination unit 502 configured to securely determine a prediction value of the logistic regression model according to the training result and the other data parties, to obtain a first shard of the prediction value of the logistic regression model;

[0132] a convergence detection unit 503 configured to securely determine a model loss according to the first shard of the prediction value of the logistic regression model and the other data parties, to detect a convergence of the logistic regression model based on the model loss;

[0133] a parameter determination unit 504 configured to determine whether to select a second approximation scheme of the plurality of candidate approximation schemes to replace the logistic function for secure calculation based on a detection result of the convergence of the logistic regression model, to continue the secure joint training of the logistic regression model, and to obtain a first parameter shard of the logistic function in a case that the logistic regression model converges.

[0134] It is worth noting that, Figure 5 the apparatus 500 shown respectively corresponds to Figure 4 the method embodiments shown, and thus, Figure 4 the related descriptions of the method for training the logistic regression model in the foregoing embodiments can also be applicable to Figure 5 the apparatus 500 shown, and will not be repeated here.

[0135] According to embodiments of another aspect, the present specification can also provide an apparatus for providing a plurality of candidate approximation schemes of a logistic regression model for data parties. The apparatus can be located outside each data party, and can also be located in each data party. The apparatus can be implemented by hardware or software, and the present specification does not limit the apparatus. Specifically, the apparatus can contain a plurality of candidate approximation schemes of the logistic regression model. Each candidate approximation scheme can be pre-packaged and retain a corresponding calling interface. Each data party can determine a corresponding requirement based on secure multi-party computation, so as to call the related interface to use the approximation scheme contained therein.

[0136] ​According to another aspect, an embodiment also provides a computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in conjunction with Figure 4

[0137] According to another aspect, an embodiment also provides a computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in conjunction with Figure 4

[0138] Those skilled in the art should be aware that, in one or more examples described above, the functions described in the embodiments of the present specification can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium.

[0139] The above detailed description describes the technical concept, technical solutions and beneficial effects of the present specification. It should be understood that the above description is only a specific implementation of the technical concept of the present specification, and is not intended to limit the protection scope of the technical concept of the present specification. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present specification shall be included in the protection scope of the technical concept of the present specification.​​

Claims

1. A method of jointly training for a logistic regression model, wherein, The logistic regression model comprises a logistic function, business data used for training the logistic regression model comprises privacy data distributed in a plurality of data parties, and the method is performed by a first party in the plurality of data parties, comprising: based on a plurality of attribute features respectively extracted from local business data for each business object, and based on secure processing of the attribute features by other data parties, training the logistic regression model using a first approximation scheme in a plurality of candidate approximation schemes instead of the logistic function, and a single candidate approximation scheme is a polynomial with respect to an input variable of the logistic function, or a segmented function with respect to the input variable of the logistic function; based on the training result, securely determining the predicted value of the logistic regression model with other data parties to obtain a first slice of the predicted value of the logistic regression model; based on the first slice of the predicted value of the logistic regression model, securely determining the model loss with other data parties to detect the convergence of the logistic regression model based on the model loss; based on the detection result of the convergence of the logistic regression model, determining whether to select a second approximation scheme in the plurality of candidate approximation schemes to replace the logistic function for secure calculation to continue secure joint training of the logistic regression model; and in the case where the logistic regression model converges, obtaining a first parameter slice of the logistic function.

2. The method of claim 1, wherein, The model convergence of the logistic regression model is detected by: jointly and securely detecting the parameter change rate of the logistic regression model with other data parties to detect the model convergence based on the parameter change rate of the logistic regression model; wherein the parameter change rate of the logistic regression model comprises at least one of the following: a change rate of the model loss, a change rate of the gradient determined based on the model loss, and a change rate of the pending parameter adjusted based on the gradient.

3. The method of claim 1, wherein, The training of the logistic regression model based on a plurality of attribute features respectively extracted from local business data for each business object, and based on secure processing of the attribute features by other data parties, using a first approximation scheme in a plurality of candidate approximation schemes instead of the logistic function, comprises: jointly normalizing or standardizing the attribute features under the premise of multi-party secure calculation with other data parties to obtain a first slice of the standardized feature matrix; based on the first slice of the standardized feature matrix, securely processing the attribute features with other data parties to obtain a first slice of the input variable of the logistic function for secure calculation of the first approximation scheme instead of the logistic function.

4. The method of claim 1, wherein, The first approximation scheme is a candidate approximation scheme implemented via a first-order polynomial with respect to the input variable, and the first-order polynomial is one of the following: a Taylor first-order expansion polynomial, a Minimax first-order polynomial.

5. The method of claim 1, wherein: In a case where the detection result of the convergence of the logistic regression model is no convergence, a candidate approximation scheme of at least one of a high-order polynomial higher than the first order and a segmented function is selected from the plurality of candidate approximation schemes as the second approximation scheme, and the secure joint training of the logistic regression model is continued.

6. The method of claim 1, wherein, The first parameter piece of the logistic function is obtained in a case where the logistic regression model converges. In a case where the logistic regression model converges, the prediction accuracy of the trained logistic regression model is detected jointly with other data parties. In a case where the prediction accuracy does not satisfy a predetermined condition, a third approximation scheme of a high-order polynomial higher than a predetermined order combined with a segmented function is selected to replace the logistic function for secure calculation, so that the logistic regression model is retrained securely with other data parties according to the third approximation scheme. The first parameter piece of the logistic function is obtained until the prediction accuracy satisfies a predetermined condition.

7. The method of claim 1, wherein, The method further comprises: Based on the first parameter piece of the logistic function, a coefficient significance test on the classification effect of the logistic regression model is performed securely with other data parties in a case where a fourth approximation scheme is used to replace the logistic function.

8. The method of claim 7, wherein, The significance test uses a p-value as a test index, and the p-value is a probability of a result more extreme than a sample observation result if a null hypothesis is correct.

9. The method of claim 7, wherein, The fourth approximation scheme is a high-order polynomial of a least square 7-order polynomial expansion of an input variable of the logistic function.

10. An apparatus for jointly training a logistic regression model, wherein, The logistic regression model includes a logistic function, and business data used for training the logistic regression model includes private data distributed in a plurality of data parties, and the device is arranged in a first party of the plurality of data parties, and comprises: A training unit is configured to train the logistic regression model based on a plurality of attribute features respectively extracted from local business data for each business object, and securely process the attribute features with other data parties, and use a first approximation scheme of a plurality of candidate approximation schemes to replace the logistic function, and each candidate approximation scheme is a polynomial of an input variable of the logistic function or a segmented function of the input variable of the logistic function. A prediction value determination unit is configured to securely determine a prediction value of the logistic regression model with other data parties according to a training result, so as to obtain a first piece of the prediction value of the logistic regression model. A convergence detection unit is configured to securely determine a model loss with other data parties according to the first piece of the prediction value of the logistic regression model, so as to detect the convergence of the logistic regression model based on the model loss. A parameter determination unit is configured to determine whether to select a second approximation scheme from the plurality of candidate approximation schemes to replace the logistic function for secure calculation, so as to continue the secure joint training of the logistic regression model based on a detection result of the convergence of the logistic regression model, and obtain a first parameter piece of the logistic function in a case where the logistic regression model converges. 11.A computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of claims 1-9.

12. A computing device comprising a memory and a processor, wherein: The memory stores executable code, and the processor executes the executable code to implement the method of claims 1-9.

Images