A method and system for training a longitudinal logistic regression model for privacy protection

Abstract

The application provides a longitudinal logistic regression model training method and system for privacy protection. A model initiator outputs a first local linear function output result based on local training sample data. A joint training request is sent to a feature provider, and a second local linear function output result is received. The two local linear function output results are aggregated and input into an activation function layer to obtain a prediction value. Residuals are calculated with labels held by the model initiator, and the initiator target model is updated in parameters. In each round of parameter iteration update process, the encrypted residuals are sent to the feature provider to calculate the gradient, and the first mask gradient is obtained after encryption. The first mask gradient is received, the second mask gradient is obtained after removing the second multiplication mask, and the second mask gradient is sent to the feature provider to calculate the model intermediate parameter containing the mask. The model intermediate parameter is received, the fourth addition mask is removed, the first multiplication mask of the current iteration round is added, and the model intermediate parameter is sent to the feature provider to update the provider target model parameter with the mask.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model training, in particular to a longitudinal logistic regression model training method and system for privacy protection. BACKGROUND

[0002] At present, data as a key driving force has important value, and the performance of machine learning model is highly dependent on a large amount of data training. The more the data is, the more the dimension is, and the stronger the accuracy and generalization ability of the model is. However, with the increase of data value, the problem of privacy protection is increasingly prominent. The federated learning emerges as the times require, and multiple participants can use data and machine learning modeling under the premise of meeting user privacy protection, data security and regulatory requirements.

[0003] In the prior art, logistic regression is a machine learning algorithm commonly used in data analysis and data mining, which is used to solve classification problems. Different institutions such as e-commerce, communication and bank can jointly train models using different dimension features of the same user. Further, stochastic gradient descent (SGD) is widely used for iterative training of logistic regression model. When training a high-precision logistic regression model, it is essential to have sufficient high-quality data. Data is often distributed in the form of longitudinal division among multiple data holders, that is, the data set is consistent in the sample space, but the features contained are different. However, due to the existence of individual privacy information in the data, the data holder is unwilling to share the data, which is not conducive to building a high-precision logistic regression model. The existing method based on secure multi-party computation (MPC) or secret sharing technology to ensure privacy protection of multi-party machine learning has the disadvantages of low calculation efficiency and large communication overhead. Therefore, how to efficiently and safely realize the longitudinal logistic regression training with privacy protection is a problem to be solved. SUMMARY

[0004] In view of this, the embodiments of the present application provide a longitudinal logistic regression model training method and system for privacy protection to eliminate or improve one or more defects in the prior art, and solve the problem that the longitudinal logistic regression training cannot realize privacy protection and is low in efficiency in the prior art.

[0005] One aspect of the present application provides a longitudinal logistic regression model training method for privacy protection, the method comprising the following steps:

[0006] The model initiator outputs a first local linear function output result based on the local training sample data using the initiator target model output;

[0007] sending a joint training request to at least one feature provider and receiving a second local linear function output result sent by the at least one feature provider, the model initiator being connected with the at least one feature provider through a preset communication protocol; the second local linear function output result being obtained by the feature provider based on local training sample data by using a provider target model, and a provider target model parameter of the provider target model being encrypted by a first multiplication mask of a previous iteration round since a second iteration round;

[0008] removing the first multiplication mask in the second local linear function output result and aggregating the first local linear function output result and the second local linear function output result, inputting an activation function layer to obtain a prediction value for a target task;

[0009] computing a residual based on the prediction value and a label held by the model initiator for the target task and constructing a loss function, and performing parameter iterative updating on the initiator target model to obtain the target model with a target of minimizing the loss function;

[0010] wherein, in each parameter iterative updating process, the method further synchronously performs the following steps:

[0011] encrypting the residual by a second multiplication mask and sending it to the feature provider, so that the feature provider encrypts a first mask gradient by a third multiplication mask after calculating a gradient of the provider target model;

[0012] receiving the first mask gradient, adding a fourth addition mask after removing the second multiplication mask, and adding a first multiplication mask of a previous iteration round to obtain a second mask gradient since a second iteration round; sending the second mask gradient to the feature provider so that the provider target model performs back propagation calculation on a model intermediate parameter containing a mask;

[0013] receiving the model intermediate parameter, adding a first multiplication mask of a current iteration round after removing the fourth addition mask, and sending it to the feature provider, so that the feature provider updates the provider target model parameter encrypted by the first multiplication mask after removing the third multiplication mask.

[0014] In some embodiments, in the first iteration round, the second mask gradient is sent to the feature provider so that the provider target model performs back propagation calculation on a model intermediate parameter containing a mask, and the calculation satisfies the following expression:

[0015] ;

[0016] wherein, denotes a feature provider, denotes an iteration round, denotes a third random matrix mask, denotes updated provider target model parameters, denotes the fourth additive mask, denotes a gradient, denotes the provider target model parameters before updating.

[0017] In some embodiments, the local training sample data of the model initiator and the local training sample data of the feature provider include financial transaction data or medical data; the financial transaction data includes transaction records, credit rating information and card holding status of a user; the medical data includes medical records, genetic history and allergy history of a user.

[0018] In some embodiments, the method further comprises parameter iteration updating until a preset iteration termination condition is met, and the preset iteration termination condition satisfies the following expression:

[0019] ;

[0020] wherein, denotes the number of iteration updates, denotes the set of feature providers, and i denotes a participating feature provider, denotes the number of continuous features held by the ith participating feature provider.

[0021] In some embodiments, the method further comprises preprocessing the local training sample data of the model initiator and the local training sample data of the feature provider, wherein the local training sample data of the model initiator contains discrete feature data and continuous feature data, the local training sample data of the feature provider contains discrete feature data and continuous feature data, the discrete feature data is one-hot encoded, and the continuous feature data is max-min normalized.

[0022] In some embodiments, the method further comprises storing the parameters of the initiator target model, the residual, the loss function, the gradient and the parameters of the provider target model as a training log and saving them in a cloud storage system, and adding a target task identifier and a timestamp identifier to the training log.

[0023] In some embodiments, the model initiator and at least one feature provider are connected through a preset communication protocol, and the preset communication protocol includes a TCP / IP protocol, a UDP / IP protocol, an HTTP protocol, an HTTPS protocol, a remote procedure call protocol or a representational state transfer protocol.

[0024] In another aspect, the present application also provides a longitudinal logistic regression model training system for privacy protection, comprising a processor, a memory and computer programs / instructions stored on the memory, the processor being configured to execute the computer programs / instructions, and the system implementing the steps of the method according to any one of the above aspects when the computer programs / instructions are executed.

[0025] In another aspect, the present application also provides a computer readable storage medium having computer programs / instructions stored thereon, the programs / instructions being executed by a processor to implement the steps of the method according to any one of the above aspects.

[0026] In another aspect, the present application also provides a computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement the steps of the method according to any one of the above aspects.

[0027] In the longitudinal logistic regression model training method for privacy protection, the training efficiency of the logistic regression is effectively improved based on infinite solutions, a plurality of masks are added to the intermediate results in the training process to meet the infinite solution security, the secure transmission of the intermediate results is realized, the huge overhead required by the public key cryptography technology is avoided, and the homomorphic property of the random mask realizes the lossless transmission of the intermediate results in the training process, so that the target model is accurately obtained through the longitudinal logistic regression model training method.

[0028] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following detailed description and drawings in which

[0029] Those skilled in the art will appreciate that the objects and advantages of the application can not be limited to the specifically described above, and the above and other objects of the application can be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0030] The drawings described herein are used to provide further understanding of the application, form part of the present application, and do not constitute a limitation of the application. In the drawings:

[0031] Figure 1 The structure diagram of the longitudinal logistic regression model training method for privacy protection according to an embodiment of the present application.

[0032] Figure 2 The effect comparison diagram of the longitudinal logistic regression model training method for privacy protection according to an embodiment of the present application and other training methods. DETAILED DESCRIPTION

[0033] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail below with reference to the embodiments and drawings. Here, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but are not intended to limit the present application.

[0034] Here, it should also be noted that, in order to avoid the present application being obscured by unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0035] It should be emphasized that the term "comprises / comprising" when used in this text indicates the presence of the stated features, elements, steps or components, but does not exclude one or more other features, elements, steps or components being present or added.

[0036] Here, it should also be noted that, if not specifically stated, the term "connected" in this text can not only mean direct connection, but also indirect connection with an intermediate.

[0037] In the following, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0038] In the prior art, data holders are unwilling to share data due to the existence of individual privacy information in the data, which is not conducive to the construction of a high-precision logistic regression model; the existing methods for ensuring privacy protection of multi-party machine learning based on secure multi-party computation or secret sharing technology have the disadvantages of low computational efficiency and large communication overhead; the present application provides a longitudinal logistic regression model training method and system for privacy protection, the method comprising: outputting a first local linear function output result by a model initiator based on local training sample data using an initiator target model; sending a joint training request to at least one feature provider and receiving a second local linear function output result sent by at least one feature provider, the model initiator and at least one feature provider being connected through a preset communication protocol; the second local linear function output result is obtained by processing the local training sample data using a provider target model by the feature provider, and the provider target model parameter of the provider target model from the second iteration round is encrypted by the first multiplication mask of the previous iteration round; remove the first multiplication mask of the previous iteration round and aggregate the first local result and the second local result, input the activation function layer to obtain the prediction value for the target task; calculate the residual according to the prediction value and the label and construct the loss function, and update the parameters of the initiator target model to obtain the target model by minimizing the loss function; in each parameter iteration update process, the residual is encrypted by the second multiplication mask and sent to the feature provider to calculate the gradient of the provider target model, and the first mask gradient is obtained by encrypting the third multiplication mask; receive the first mask gradient and add the fourth addition mask after removing the second multiplication mask, and add the second mask gradient obtained by the first multiplication mask of the previous iteration round from the second iteration round; send the second mask gradient to the feature provider to execute the back propagation calculation of the model intermediate parameter containing the mask by the provider target model; receive the model intermediate parameter and add the first multiplication mask of the current iteration round after removing the fourth addition mask, and send it to the feature provider to update the provider target model parameter with the mask after removing the third multiplication mask.

[0039] Figure 1 The structure diagram of the longitudinal logistic regression model training method for privacy protection according to an embodiment of the present application is shown. Specifically, one aspect of the present application provides a longitudinal logistic regression model training method for privacy protection, which comprises the following steps S101-S104:

[0040] Step S101: outputting a first local linear function output result by a model initiator based on local training sample data using an initiator target model.

[0041] Step S102: sending a joint training request to at least one feature provider and receiving second local linear function output results sent by the at least one feature provider, the model initiator being connected with the at least one feature provider through a preset communication protocol; the second local linear function output results being obtained by the feature provider based on local training sample data by using a provider target model, and the provider target model parameters of the provider target model being encrypted by the first multiplication mask of the previous iteration round since the second iteration round.

[0042] Step S103: removing the first multiplication mask in the second local linear function output results and aggregating the first local linear function output results and the second local linear function output results, and inputting an activation function layer to obtain a prediction value for the target task.

[0043] Step S104: calculating a residual according to the prediction value and a label held by the model initiator for the target task and constructing a loss function, and performing parameter iterative updating on the initiator target model to obtain a target model with the objective of minimizing the loss function.

[0044] In each parameter iterative updating process, the method further synchronously performs the following steps S1-S3:

[0045] Step S1: encrypting the residual by using a second multiplication mask and sending it to the feature provider, so that the feature provider obtains a first mask gradient by encrypting the gradient calculated by the provider target model through a third multiplication mask.

[0046] Step S2: receiving the first mask gradient, removing the second multiplication mask, adding a fourth addition mask, and adding the first multiplication mask of the previous iteration round to obtain a second mask gradient since the second iteration round; and sending the second mask gradient to the feature provider so that the provider target model performs back propagation calculation on the model intermediate parameters containing the mask.

[0047] Step S3: receiving the model intermediate parameters, removing the fourth addition mask, adding the first multiplication mask of the current iteration round, and sending it to the feature provider, so that the feature provider updates the provider target model parameters encrypted by the first multiplication mask after removing the third multiplication mask.

[0048] In steps S101-S103, the participants in the longitudinal logistic regression training include a model initiator and at least one feature provider. In some embodiments, the model initiator is connected to the at least one feature provider through a preset communication protocol, which includes a TCP / IP protocol, a UDP / IP protocol, an HTTP protocol, an HTTPS protocol, a remote procedure call protocol, or a representational state transfer protocol. The model initiator sends a joint training request of a longitudinal logistic regression model to the at least one feature provider. After receiving the iteration construction request, the feature provider processes local training sample data. The local training sample data is output after being processed by a provider target model deployed on the feature provider, and the output result of a local linear function calculated by the local training sample data through a linear function is taken as a second local result. A first multiplication mask of a previous iteration round is added by the model initiator. The second local result of the current round is obtained by processing the local training sample data based on the provider target model of the feature provider. The second local result is calculated based on the updated provider target model parameter of the feature provider target model of the previous round and the local training sample data. An addition mask is used to achieve an encryption effect by adding a mask, and the encryption effect is removed by subtracting the mask. A multiplication mask is used to achieve an encryption effect by multiplying a mask, and the encryption effect is removed by dividing the mask. In the longitudinal logistic regression training, in some embodiments, the local training sample data of the model initiator and the local training sample data of the feature provider include financial transaction data or medical data. The financial transaction data includes transaction records, credit rating information, and card holding status of a user. The medical data includes medical records, genetic history, and allergy history of a user.

[0049] Further, the model initiator aggregates the first local result and the second local result to obtain a global result, which satisfies the following expression: ; wherein, is a parameter updated in the current iteration round of the participant, is local training sample data input by the participant, is the total number of participants.

[0050] In step S104, in order to prevent the model initiator from inversely deducing the parameter and the local training sample data from the global result, a security limit related to the number of iterations is constructed to ensure that the nonlinear equation set constructed from the global result has infinite feasible solutions. In some embodiments, the method further includes parameter iteration update until a preset iteration termination condition is met, and the preset iteration termination condition satisfies the following expression:

[0051] ;

[0052] wherein, denotes the number of times of iterative update, denotes the set of feature providers, i denotes the participating feature provider, denotes the number of continuous features held by the ith participating feature provider. The activation function layer adopts a sigmoid function, a ReLU function or a Tanh function.

[0053] In steps S1-S3, the residual is obtained by calculating the predicted value and the label. In a binary classification task, the residual value ranges from 0 to 1 to avoid the invalidity of the multiplication mask; the model initiator sends all residuals through the same second multiplication mask randomly generated to encrypt, is a non-zero mask; the feature provider generates a third multiplication mask multiplied by the first mask gradient and transmits it to the model initiator; in the non-first iteration round, the second multiplication mask is removed to obtain which needs to be multiplied by the first multiplication mask of the previous iteration round and added with the fourth addition mask, and the second mask gradient is In some embodiments, in the first iteration round, the second mask gradient is sent to the feature provider for the provider target model to perform back propagation calculation on the model intermediate parameters containing the mask, and the calculation satisfies the following expression:

[0054] ;

[0055] wherein, denotes the feature provider, denotes the iteration round, denotes the third random matrix mask, denotes the updated provider target model parameter, denotes the fourth addition mask, denotes the gradient, denotes the provider target model parameter before update.

[0056] Further, the model initiator receives the model intermediate parameters after removing the fourth addition mask by addition and multiplying the first multiplication mask of the current iteration round to obtain and transmits it to the feature provider, and after removing the third multiplication mask the provider target model parameter with the mask is obtained .

[0057] In some embodiments, the method further comprises preprocessing the local training sample data of the model initiator and the local training sample data of the feature provider, the local training sample data of the model initiator containing discrete feature data and continuous feature data, the local training sample data of the feature provider containing discrete feature data and continuous feature data, the discrete feature data being one-hot encoded, and the continuous feature data being max-min normalized. Specifically, the local training sample data is converted into potential features that can better represent the target model through unified feature engineering; for example, the discrete feature data includes gender and color, and the continuous feature data includes age and salary; the one-hot encoding converts the discrete feature data into a binary vector; and the max-min normalization maps the continuous feature data to a specific space.

[0058] In some embodiments, the method further comprises storing the parameters, residuals, loss functions, gradients of the initiator target model and the parameters of the provider target model as a training log and saving them in a cloud storage system, and adding a target task identifier and a timestamp identifier to the training log. Specifically, the target task identifier and the timestamp identifier are used to quickly find and retrieve the content of the training log for training process analysis.

[0059] In another aspect, the present application also provides a longitudinal logistic regression model training system for privacy protection, comprising a processor, a memory, and computer programs / instructions stored in the memory, the processor being configured to execute the computer programs / instructions, and the system implementing the steps of the above method when the computer programs / instructions are executed.

[0060] In another aspect, the present application also provides a computer readable storage medium having computer programs / instructions stored thereon, the programs / instructions being executed by a processor to implement the steps of the above method.

[0061] In another aspect, the present application also provides a computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement the steps of the above method.

[0062] The present application will be described below in conjunction with a specific embodiment:

[0063] This invention proposes a method and system for training a longitudinal logistic regression model with privacy protection, achieving data privacy protection in patient disease research in the medical industry, user credit risk assessment in the financial industry, and user shopping behavior analysis and digital image recognition in the e-commerce industry. The participants jointly executing the logistic regression model training hold longitudinally segmented datasets, meaning each dataset is consistent in sample space, but each dataset only contains partial features of this common set of samples. For example, for the same group of users, hospitals possess users' medical history and health status; banks possess users' financial transaction records and credit rating information; and e-commerce platforms possess users' consumption behavior and preferences. The model initiator sends a request to each feature provider within a pre-defined communication protocol to construct the longitudinal logistic regression model, and each party generates initial model parameters locally corresponding to its held features. All participants collaboratively conduct iterative training of the longitudinal logistic regression model using features, labels, and model parameters as inputs until the maximum number of iterations is reached, resulting in a trained multi-party longitudinal logistic regression model. Each iteration includes two key stages: forward computation and backward update. In the forward computation stage, all participants collaboratively aggregate the linear function output and evaluate the sigmoid function to obtain the model's predicted value. During the reverse update phase, each feature provider interacts with the model initiator to calculate gradients and update model parameters.

[0064] (a) Forward computation stage.

[0065] Assuming the execution of the first... In the next iteration, each participant First, calculate the output of the local linear function locally. ,in These are the model parameters for this round. Participants Input features, The previous iteration was initiated by the model. Add a random mask to the model parameters, and This is sent to the model initiator. Then, the model initiator removes the random mask from the output of all local linear functions. And aggregate them to obtain Next, the model initiator will... The sigmoid function is used as input to evaluate the model's predicted value. The definition of the sigmoid function is as follows:

[0066]

[0067] To ensure that the model initiator cannot pass... Reverse deduction and Construct a safety limit related to the number of iterations to ensure that... The constructed nonlinear equation system has infinitely many feasible solutions. The basic logic of the construction is: first, ... The process is transformed into a system of nonlinear equations. Then, the possible ranges of features obtained by the model initiator and the linear relationships between model parameters in each iteration are converted into equivalent equations and added to the system of nonlinear equations. Finally, a safety constraint related to the number of iterations is obtained by ensuring that the number of variables in the system of equations is greater than the number of equations. Specifically, this safety constraint is... ,in, This represents the set of feature providers. This indicates the number of times an update is performed after iterating through all samples. This represents the number of consecutive features held by the i-th participant. The derivation of the security constraint includes: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Feature Provider The local linear function output is formalized into the following system of nonlinear equations. :

[0068]

[0069] in, yes Input features, These are model parameters. It is the first One sample batch, It is the total number of samples. yes The number of features held It is the number of iteration rounds. It refers to the batch size. This represents the output value of a linear function of each sample. Then, the range of values ​​for the input features is formalized into an equation. To avoid loss of generality, it is assumed that discrete and continuous features exist, and the corresponding unknown variable 𝑥 has a range that is either a set or an interval, respectively. It can be formalized as For an interval, it mainly has four forms, namely: ,in It is a boundary point of an interval. By introducing an artificial variable... The above intervals can be formalized as follows: These equations can be directly added to In this process, a new system of equations is obtained. In particular, for Other variables in the model, such as the model parameters, are assumed to take values ​​in the real number domain by default. Next, the linear relationship between the model parameters in different iterations is formalized, satisfying the following expression:

[0070]

[0071] No. The model parameters of the wheel can be expressed as the first... The model parameters and gradients of the first wheel. And so on, for the second wheel... Wheel model parameters and initializing model parameters The relationship between them can be represented as follows:

[0072]

[0073] in, It is the first Batch samples The Middle The residuals of each sample It's the learning rate. When you add that... In this process, all model parameter variables except for the initialization parameters can be eliminated. Finally, by letting If the number of independent variables exceeds the number of equations, a relationship can be obtained with the number of epochs. The relevant security restrictions and detailed procedures are as follows: [Order / Description of security restrictions / de ... For the sample size, for The number of features held for The number of discrete features held for The continuous set of features held For use in calculation A function of the number of intervals in the middle. for The size, i.e., the number of continuous features. This represents the number of epochs. First, calculate... The number of unknown independent variables and equations. Assume that the hyperparameters related to training, including but not limited to learning rate, batch size, and number of iterations, are public and constant. The variables in the model include input features, initial model parameters, residuals, and artificial variables introduced to account for the range of values ​​of the input features. For the model initiator that receives the output of a locally linear function, since it can obtain the residuals during training, only the residuals are known among the above variables. Therefore, The unknown independent variables include input features, initial model parameters, and artificial variables. The total number of unknown independent variables is... For the equations in , including the equations converted from the linear function output and the equations introduced considering the value range of the input features. The total number of equations is . To ensure that is underdetermined, the number of unknown independent variables must be greater than the number of equations. For this purpose, a constraint needs to be established for , that is, . In practical applications, is usually much larger than , so tends to 0, and is derived. In other words, if the number of epochs is less than the number of continuous features, then is underdetermined.

[0074] (II) (II) Backward update phase.

[0075] The backward update phase uses random masking to protect the residuals and gradients of intermediate calculations; the following homomorphic properties of random masking are used to perform gradient calculation and parameter update in the masked state: 1) addition: if a secret is added to a random mask , an addition operation is performed between and the unmasked data , because can be eliminated from by subtraction; 2) multiplication: if a secret is multiplied by a random mask , a multiplication operation is performed between and the unmasked data , because can be eliminated from by division; 3) inner product: if a secret vector is multiplied by a random mask , an inner product operation is performed between and the unmasked vector , because can be eliminated from by division. The backward update process is as follows:

[0076] 1. Calculate the gradient. The model initiator uses the true label and the model prediction output to calculate the residual, and further calculates the gradient corresponding to the input features. In order to enable the feature provider to perform gradient calculation involving inner product without exposing the residual, the model initiator multiplies all residuals by a same random mask. Specifically, the model initiator first randomly generates a non-zero mask , and multiplies the set of all residuals by . Then, masked residuals to the feature providers. Finally, each feature provider computes the masked gradients with being the set of gradients. In particular, in a binary classification task, the range of residuals is generally in to avoid invalid multiplication masks.

[0077] 2. Update model parameters. To enable the feature providers to update the model parameters , which involves subtraction operations, the feature providers collaborate with the model initiator to convert the multiplication masks in to addition masks without exposing the gradients to the model initiator. Specifically, each feature provider first generates a random matrix mask and multiplies by , then sends to , where is the number of features held by . Then, divides by to obtain and adds a new random vector mask to it. Finally, upon receiving , updates the model parameters as follows:

[0078]

[0079] 3. Post-processing of model parameters. Model parameters not only need to be used for computing the output of local linear functions involving matrix-vector multiplication, but also need to maintain the anonymity of the feature providers. To achieve this goal, the model initiator needs to eliminate the additive masks in the model parameters and re-mask them with multiplication masks. Specifically, each feature provider sends to . Then, eliminates and multiplies by a new random mask and sends it to . Upon eliminating , obtains . In particular, before adding to in the next iteration, needs to be multiplied by Multiply To enable the gradient to update the masked model parameters. .

[0080] Figure 2 This image shows a comparison of the training method for a privacy-preserving longitudinal logistic regression model according to an embodiment of the present invention with other training methods. The training efficiency was tested using the method provided by the present invention. The datasets used included a population dataset (Adult), a banking dataset (Bank), a credit card transaction dataset (Credit-card), and a digit recognition dataset (MNIST). The population dataset had a feature dimension of 104 and a sample size of 45,222; the banking dataset had a feature dimension of 51 and a sample size of 45,211; the credit card transaction dataset had a feature dimension of 91 and a sample size of 30,000; and the digit recognition dataset had a feature dimension of 784 and a sample size of 60,000. Each dataset underwent uniform feature engineering, with one-hot encoding for discrete features and minimax regularization for continuous features. Each dataset was divided into a 60% training set, a 20% validation set, and a 20% test set. This invention is designed for multi-party collaborative scenarios, uniformly dividing features before feature engineering. A subset is created, where each subset is assigned to a participating party that holds the disease-specific label. The parameters involved include batch size (…). Learning rate ), number of epochs ( The batch size is 64, the learning rate is 0.001, the number of epochs is 10, and the bandwidth includes four settings: 20, 50, 70, and 100, in Mbps. Figure 2 (a) A comparison chart showing the effects of training with a population dataset; Figure 2 (b) A comparison chart showing the results of training using the banking business dataset; Figure 2 (c) A comparison chart showing the performance of training using a credit card transaction dataset; Figure 2 (d) is a comparison chart of the training results using the digit recognition dataset.

[0081] The training methods involved in the comparison include the non-privacy protection protocol method NP-LR and the state-of-the-art infinite solution security-based method FedLR and BlindFL, both of which establish security constraints during forward calculation, and use homomorphic encryption technology to realize back propagation, and the evaluation index is running time, in milliseconds. The training effects of the longitudinal logistic regression model training method (ISS-VLR) of the application and the NP-LR, FedLR and BlindFL methods under different bandwidths are compared, and it is observed that the training time of the longitudinal logistic regression model training method is longer than that of the NP-LR, but it has higher training efficiency than the FedLR and BlindFL, and the reason for this result is that the longitudinal logistic regression model training method only uses random masking with smaller computational overhead during back propagation, without relying on any public key cryptography technology. In contrast, FedLR and BlindFL use semi-homomorphic encryption to implement back propagation, introducing huge computational overhead.

[0082] From the software level, the application also provides an infinite solution security-based privacy protection longitudinal logistic regression training system, which includes a plurality of client devices, each client having a communication module to realize the communication connection between the clients. The client devices can communicate with each other using any suitable network protocol, including network protocols that have not been developed as of the filing date of the application. The network protocols include but are not limited to TCP / IP protocol, UDP / IP protocol, HTTP protocol, HTTPS protocol, and Remote Procedure Call Protocol (RPC protocol) and Representational State Transfer (REST protocol) used on top of the above protocols; the system includes:

[0083] (1) A request sending and receiving unit, the model initiator is used to send a training request for a longitudinal logistic regression model participating in privacy protection; the feature provider is used to receive a training request for a longitudinal logistic regression model participating in privacy protection.

[0084] (2) An iterative training unit for jointly training the logistic regression model by all participants until the upper limit of the specified number of iterations is reached, obtaining a trained longitudinal logistic regression model; wherein the iterative training unit specifically performs the following steps for each iteration: each participant locally calculates the local linear function output, and the model initiator collects and aggregates all local linear function outputs to obtain the global function output; the model initiator obtains the model prediction value by evaluating the sigmoid function, and calculates the residual to update the model parameters corresponding to the features held by the model initiator; each feature provider and the model initiator interact under the protection of random masking to complete gradient calculation and parameter update.

[0085] (3) a model output unit configured to output the trained logistic regression model, i.e., a set of model parameters.

[0086] In summary, the present application provides a longitudinal logistic regression model training method and system for privacy protection, wherein a model initiator uses an initiator target model to output a first local linear function output result based on local training sample data, sends a joint training request to a feature provider, and receives a second local linear function output result sent by the feature provider, the model initiator connects at least one feature provider through a preset communication protocol, the second local linear function output result is obtained by processing the local training sample data using a provider target model based on the feature provider, and the provider target model parameters of the provider target model are encrypted by the first multiplication mask of the previous iteration round from the second iteration round; after removing the first multiplication mask of the previous iteration round, the first local result and the second local result are aggregated and input into an activation function layer to obtain a prediction value for the target task; a residual error is calculated according to the prediction value and the label, and a loss function is constructed, and the initiator target model is updated in parameter to obtain the target model with the objective of minimizing the loss function; in each round of parameter iteration and updating process, the residual error is encrypted by a second multiplication mask and sent to the feature provider, and the first mask gradient is obtained by encrypting the gradient calculated by the provider target model through a third multiplication mask; the first mask gradient is received, the second multiplication mask is removed, and the first multiplication mask and a fourth addition mask of the previous iteration round are added to obtain a second mask gradient; the second mask gradient is sent to the feature provider to execute the back propagation calculation of the model intermediate parameters containing the mask by the provider target model; the model intermediate parameters are received, the fourth addition mask is removed, and the first multiplication mask of the current iteration round is added, and the model intermediate parameters are sent to the feature provider to update the provider target model parameters with the mask after removing the third multiplication mask.

[0087] The present application also provides a computer device, which can include a processor, a memory,

[0088] The processor and the memory in the computer device can be connected through a bus or other means.

[0089] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or a combination thereof.

[0090] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the key shielding method of the vehicle display device in the embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory.

[0091] The memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0092] The one or more modules are stored in the memory and, when executed by the processor, perform the method described in the embodiments.

[0093] The embodiments of the present application also provide a computer readable storage medium having a computer program stored thereon, which is executed by the processor to implement the steps of the foregoing edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.

[0094] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, for example, the hardware can comprise an electronic circuit, an Application Specific Integrated Circuit (ASIC), a suitable firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are the program or code segments to perform a specific task. The program or code segments can be stored in a machine-readable medium, or transmitted by a carrier wave as data signals over a transmission medium or communication link.

[0095] It is to be understood that the application is not limited to the particular configurations and processes described herein and shown in the drawings. For simplicity, detailed descriptions of known methods and apparatuses are omitted so as not to obscure the disclosure. In the above-described embodiments, several specific steps are described and illustrated as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and the order of the steps can be changed, or other steps can be added, or replaced, or eliminated, depending on the application.

[0096] In the present application, features described and / or illustrated in relation to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or combined with or instead of features of other embodiments.

[0097] The above description is only preferred embodiments of the present application, and is not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the present application.

Claims

1. A method for privacy-preserving longitudinal logistic regression model training, the method comprising: The method comprises the following steps: The model initiator outputs a first local linear function output result based on local training sample data using an initiator target model; A joint training request is sent to at least one feature provider, and a second local linear function output result sent by the at least one feature provider is received, the model initiator and the at least one feature provider being connected through a preset communication protocol; the second local linear function output result is obtained by processing the local training sample data using a provider target model based on the feature provider, and the provider target model parameters of the provider target model from the second iteration round are encrypted by the first multiplication mask of the previous iteration round; The first multiplication mask in the second local linear function output result is removed, and the first local linear function output result and the second local linear function output result are aggregated, and an activation function layer is input to obtain a prediction value for a target task; A residual error is calculated according to the prediction value and a label held by the model initiator for the target task, and a loss function is constructed, and the initiator target model is iteratively updated in parameters to obtain the target model with the goal of minimizing the loss function; In each parameter iteration update process, the method further synchronously performs the following steps: The residual error is encrypted by a second multiplication mask and sent to the feature provider, so that the feature provider calculates the gradient of the provider target model and obtains a first mask gradient by encrypting the gradient with a third multiplication mask; The first mask gradient is received, the second multiplication mask is removed, a fourth addition mask is added, and the second mask gradient is obtained by simultaneously adding the first multiplication mask of the previous iteration round from the second iteration round; the second mask gradient is sent to the feature provider to enable the provider target model to perform back propagation calculation on the model intermediate parameters containing the mask; The model intermediate parameters are received, the fourth addition mask is removed, and the first multiplication mask of the current iteration round is added, and then sent to the feature provider to enable the feature provider to update the provider target model parameters encrypted by the first multiplication mask after removing the third multiplication mask. 2.The method for training a longitudinal logistic regression model for privacy protection according to claim 1, wherein, In the first iteration round, the second mask gradient is sent to the feature provider to enable the provider target model to perform back propagation calculation on the model intermediate parameters containing the mask, and the following expression is satisfied: ； wherein, denotes a feature provider, denotes an iteration round, denotes a third random matrix mask, denotes updated provider target model parameters, denotes the fourth additive mask, denotes a gradient, denotes provider target model parameters before update. 3.The method for training a longitudinal logistic regression model for privacy preservation according to claim 1, wherein, The local training sample data of the model initiator and the local training sample data of the feature provider comprise financial transaction data or medical data; the financial transaction data comprises transaction records, credit rating information and card holding status of a user; the medical data comprises medical records, genetic history and allergy history of a user. 4.The method for training a longitudinal logistic regression model for privacy protection according to claim 1, wherein, The method further comprises parameter iteration update until a preset iteration termination condition is reached, and the preset iteration termination condition satisfies the following expression: ； wherein, denotes the number of iterations of the iterative update, denotes the set of feature providers, i denotes a participating feature provider, denotes the number of consecutive features held by the i-th participating feature provider. 5.The method for training a longitudinal logistic regression model for privacy preservation according to claim 1, wherein, The method further comprises preprocessing the local training sample data of the model initiator and the local training sample data of the feature provider, the local training sample data of the model initiator containing discrete feature data and continuous feature data, the local training sample data of the feature provider containing discrete feature data and continuous feature data, one-hot encoding the discrete feature data, and max-min normalization of the continuous feature data. 6.The method for training a longitudinal logistic regression model for privacy preservation according to claim 1, wherein, The method further comprises storing the parameters of the initiator target model, the residual, the loss function, the gradient and the parameters of the provider target model as a training log and saving in a cloud storage system, and adding a target task identification and a timestamp identification to the training log. 7.The method for training a longitudinal logistic regression model for privacy preservation according to claim 1, wherein, The model initiator is connected with at least a feature provider through a preset communication protocol, the preset communication protocol comprising a TCP / IP protocol, a UDP / IP protocol, an HTTP protocol, an HTTPS protocol, a remote procedure call protocol or a representational state transfer protocol. 8.A system for privacy-preserving longitudinal logistic regression model training, comprising a processor, a memory and computer programs / instructions stored on the memory, wherein, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

Images