A logistic regression security training method and system for multi-source vertically distributed data

Abstract

The invention belongs to the field of information security technology, and discloses a logistic regression security training method and system for multi-source vertically distributed data. The logistic regression security training method adopts a non-interactive federated learning logistic regression training method under the premise of ensuring the privacy of user local data. The user does not need to ensure that each round of training process is online in real time or consume local computing resources to perform a large number of model update calculations. The user only needs to perform preprocessing calculations and encryption calculations on local training data at the beginning of training, and then hand over the ciphertext data to a cloud service provider, which completes the data aggregation calculation, and hands over the ciphertext global logistic regression training data to a model training initiator, which decrypts the ciphertext global logistic regression training data and uses the global training data to perform logistic regression training, and finally obtains an efficient logistic regression model.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology, and in particular relates to a logistic regression security training method and system for multi-source vertically distributed data. Background Art

[0002] With the advent of the big data era, large-scale data collection has improved the performance of machine learning algorithms and continuously promoted the advancement of artificial intelligence technology. However, behind a large number of artificial intelligence products, the privacy protection of users' personal sensitive data also faces greater risks and challenges. In order to better protect the security of users' personal privacy data, the training mode of federated learning came into being. Compared with the machine learning method of the traditional centralized training mode, federated learning realizes the machine learning training while retaining the original data of all parties locally. It can effectively help multiple institutions build shared high-performance models while meeting data security requirements, and maximize the use of the high computing power of cloud system terminal devices. The emergence of federated learning effectively solves the problem of data islands and helps multiple participants achieve collaborative modeling without sharing original data.

[0003] Traditional federated learning training usually uses the method that users use local data to train models, then send the model gradients to the cloud to aggregate the global model and update the model parameters. Finally, the cloud sends the aggregated results to the participants to update their respective models. The traditional federated learning model requires users to stay online during the training process and consumes a lot of computing resources for model training. At the same time, this model does not encrypt the gradient parameters of the model, and there is also a risk of privacy data leakage to a certain extent.

[0004] Specifically, the patent "Vertical federated learning method, device, system and storage medium with improved security" (application number CN201911128849.6 application publication number CN110704860A) of Shenzhen Qianhai Weizhong Bank Co., Ltd. discloses a secure federated learning logistic regression algorithm. In the scheme of vertical federated logistic regression modeling with a third party, it avoids the possibility of a third party and one of the parties stealing data, and improves the security of the logistic regression model to a certain extent. However, the disadvantage of this method is that the training process requires the participating users to be present at all times, and each round of iteration requires communication between the data owner and the cloud server, which increases the communication overhead.

[0005] Through the above analysis, the problems and defects of the prior art are as follows:

[0006] (1) In the traditional federated learning training method, participants are required to upload the gradient parameters of the local model at each iteration during the training process. However, the gradient parameters of the local training model are not encrypted and protected, which may cause privacy data leakage to a certain extent.

[0007] (2) Traditional federated learning training methods are generally targeted at interactive computing scenarios. During the entire training process, all training participants need to be online in real time, and a large amount of computing resources are required to train the model.

[0008] (3) In traditional federated learning training methods, each iteration of the training model requires a large amount of encrypted communication between the participants and the cloud server, resulting in high communication overhead.

[0009] The difficulty of solving the above problems and defects is:

[0010] (1) In order to prevent the gradient parameters from leaking user privacy during the training process, the gradient parameters need to be encrypted and protected using cryptographic methods. In the multi-round communication between multiple users and the cloud server, the communication overhead caused by the increase in computational overhead and ciphertext size reduces the availability and efficiency of the model.

[0011] (2) In the non-interactive scenario of federated learning training, the existing homomorphic encryption algorithms are difficult to use effectively and cannot provide effective privacy protection for multi-source users.

[0012] (3) In the iterative training process of traditional federated learning, the use of cryptographic primitives to encrypt and protect the model parameters uploaded by users will bring huge communication overhead, and there is a problem of mutual constraints between security and availability.

[0013] The significance of solving the above problems and defects is: for distributed scenarios, designing a solution to the non-interactive federated learning logistic regression problem can effectively solve the problem of personal user privacy information leakage during multi-source data collaborative training, ensure the security of sensitive information during multi-source data user collaborative training, and promote the development of federated learning and big data security related technologies. Summary of the invention

[0014] In view of the problems existing in the prior art, the present invention provides a logistic regression security training method and system for multi-source vertically distributed data.

[0015] The present invention is implemented as follows: a logistic regression safety training method, the logistic regression safety training method comprising:

[0016] System initialization and data normalization: one of the data owners, as the initiator of model training, selects system parameters and generates symmetric homomorphic encryption keys according to system security requirements; uses homomorphic encryption keys to encrypt constants 0 and 1, generates encryption parameters for each data owner, and provides key support for homomorphic operations required for the secure transmission and aggregation of subsequent data; at the same time, the data owner completes data normalization processing locally to provide data support for subsequent logistic regression model training; through this step, the trusted system parameters of the logistic regression training method are selected to ensure the security and privacy of the data during the training process. By performing normalization operations on preprocessed data, the training accuracy of the model is improved, and the speed of the gradient descent algorithm in finding the optimal solution is accelerated;

[0017] Data preprocessing and local data encryption: The model training initiator uses the homomorphic encryption key to perform encryption calculations on the normalized local training data. Other data owners use encryption parameters to perform encryption calculations on the normalized local training data to obtain ciphertext local training data, and send the ciphertext local training data to the cloud server, providing data support for the cloud server to aggregate the ciphertext training data. Through this step, the normalized plaintext data is encrypted to ensure the security and privacy of the data in the subsequent processing process.

[0018] Aggregation of ciphertext training data: The cloud server receives all ciphertext training data from data owners, uses homomorphic encryption properties to securely aggregate the data of any two data owners, and sends the aggregation matrix to one of the data owners as the model training initiator for ciphertext decryption operations. Through this step, the training data in the ciphertext state is aggregated and calculated to provide data support for subsequent logistic regression training.

[0019] Decryption of ciphertext training data: the model training initiator receives the globally aggregated ciphertext training data from the cloud server; then uses the symmetric homomorphic encryption key to perform decryption operations to obtain a globally aggregated training data matrix; the model training initiator responsible for the decryption operation can use the key to decrypt the aggregated ciphertext data; through this step, the aggregated global logistic regression training matrix is ​​decrypted using the symmetric homomorphic encryption key to obtain the global logistic regression training matrix, providing data support for subsequent logistic regression training.

[0020] Logistic regression training and testing, the model training initiator uses the decrypted global logistic regression training data for training, obtains the final trained efficient model, and performs accuracy testing on the obtained model. Through this step, the global logistic regression training matrix is ​​trained with non-interactive logistic regression using the logistic regression gradient descent algorithm, and an efficient logistic regression training model is obtained.

[0021] Further, the system initialization specifically includes:

[0022] (1) The model training initiator PK selects system parameters and generates symmetric homomorphic encryption parameters and distributes them to each participant P k (k≠K);

[0023] a)P K Select system security parameters (k0, k1, k2) and execute the function KeyGen(k0, k1, k2) to generate public parameters PP and symmetric homomorphic encryption key SK;

[0024] b)P K Using the symmetric homomorphic encryption key SK, all participants P are encrypted with constants 0 and 1. k (k≠K) Generate encryption parameter pair γ k ={Enc(1),Enc(0)}, and γ k Distribute to the corresponding participants P k (k≠K);

[0025] (2) PK initializes global model parameters Where D represents the characteristic dimension of data of all participants; P K Select the hyperparameter set for logistic regression model training, which includes learning rate α, batch size β, loop parameter π, and scaling factor κ.

[0026] Furthermore, the data preprocessing and local data encryption specifically include:

[0027] (1) Each parameter square P k First, the local data is aligned and normalized to obtain the preprocessed local data matrix D (k) ;

[0028] a) Each participant P k Delete irrelevant entries in the data set and arrange the remaining data entries accordingly to obtain the local training data set D (k) =[X (k1) ,X (k2) ,...,X (kN) ] T , where X (kn) (n=1,2,...,N) is a D k / (D k +1)-dimensional vector is represented as:

[0029]

[0030] D k Represents user P k The data feature dimensions and meet The complete training data set obtained is D = [D(1) |D (2) |...|D (K) ], each line represents a training data;

[0031] b) Each participant P k Locally count the maximum and minimum values ​​of the attributes of the collected data and generate a local vector:

[0032]

[0033]

[0034] Afterwards, each participant P k By calculation Normalize the local data to obtain the normalized local training data D k ; In addition, the label column [y1,y2,...,y N ] T No normalization is required;

[0035] (2) Each participant P k According to the preprocessed data matrix D (k) , each participant P k Calculate the aggregate subvector V (k) and the aggregate submatrix M (k) , and then use the encryption parameter γ k Or private key SK for training data D (k) , aggregate subvector V (k) and the aggregate submatrix M (k) Perform encryption calculation to obtain ciphertext training data Ciphertext aggregator vector and the ciphertext aggregation matrix And send the obtained ciphertext data to the cloud service provider CSP;

[0036] a) Each participant calculates Expand the local training data to integers and calculate the aggregate submatrix M (k) , aggregate subvector V (k) ;

[0037] Among them, for P K , V (K) is a (D K +1)-dimensional vector, M (K) is a (D K +1)-dimensional symmetric matrices, respectively expressed as:

[0038]

[0039]

[0040] To other data owners P k (k≠K), V (k) It is a D K dimensional vector, M (k) It is a D K dimensional symmetric matrix, expressed as:

[0041]

[0042] b) Each participant uses the encryption parameter γ k calculate Perform encryption calculation on local training data, where n = 1, 2, ..., N, d = 1, 2, ..., D k ; Finally, each participant P k Will and Sent to the cloud service provider CSP.

[0043] Furthermore, the secure aggregation of ciphertext data specifically includes:

[0044] (1) The cloud service provider CSP first trains the data based on the encrypted local data of each participant. Calculate the ciphertext aggregation matrix Then, based on and Construct a global ciphertext aggregation matrix and send it to P K Conduct iterative model training;

[0045] a) After receiving the ciphertext training data provided by all participants, CSP performs aggregation calculation on the local data of any two participants u and v based on the local ciphertext training data to obtain the ciphertext aggregation submatrix

[0046] Among them, when 1≤u<v≤K-1, It is a D u ×D v dimensional matrix, each element of the matrix Expressed as

[0047] When 1≤u<v,v=K, It is a D u ×(D K +1)-dimensional matrix, each element of the matrix It is expressed as:

[0048]

[0049] b) CSP will and Matrices are concatenated to construct a global ciphertext aggregation matrix is a (D+2)-dimensional symmetric matrix:

[0050]

[0051] Finally, CSP will Send to P K .

[0052] Further, the ciphertext data decryption specifically includes:

[0053] (1)P K Receive global ciphertext aggregation matrix The symmetric homomorphic key is used to decrypt the calculation to obtain the logistic regression training matrix, and the matrix A and vector are obtained from it. Used for subsequent iterative update calculation of model parameters.

[0054] a)P K Receive global ciphertext aggregation matrix After that, first delete The last row of the matrix, then P K By calculation For the global ciphertext matrix Decrypt each element of and fill the first row and first column elements with N. The final result matrix in

[0055]

[0056]

[0057] Further, the logistic regression training and testing specifically include:

[0058] (1)P K By using the decrypted matrix A and vector It is used to iteratively update the model parameters. When the model parameter θ is judged to be converged or the maximum number of iterations is reached, the iterative update calculation is stopped. At this time, P K Get the trained logistic regression model.

[0059] a)P K Perform calculations by iteration Update the model parameters. After a certain number of iterations, P K Using matrix A and vector Calculate the loss function:

[0060]

[0061] in L B =θB. The operation represents the Hadamard product of two matrices, while SUM() represents the sum of all elements in the matrix. When the model parameters are judged to be converged or the maximum number of training times is reached, the logical regression training stops, and P K The final logistic regression model was obtained.

[0062] Another object of the present invention is to provide a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps of the logistic regression security training method:

[0063] System initialization and data normalization: one of the data owners, as the initiator of model training, selects system parameters and generates symmetric homomorphic encryption keys according to system security requirements; uses homomorphic encryption keys to encrypt constants 0 and 1, generates encryption parameters for each data owner, and provides key support for homomorphic operations required for subsequent secure data transmission and secure aggregation; at the same time, the data owner completes data normalization processing locally to provide data support for subsequent logistic regression model training;

[0064] Data preprocessing and local data encryption: The model training initiator uses the homomorphic encryption key to perform encryption calculations on the normalized local training data. Other data owners use encryption parameters to perform encryption calculations on the normalized local training data to obtain ciphertext local training data. The ciphertext local training data is then sent to the cloud server, providing data support for the cloud server to aggregate ciphertext training data.

[0065] Aggregation of ciphertext training data: The cloud server receives all ciphertext training data from data owners, uses homomorphic encryption properties to securely aggregate the data of any two data owners, and sends the aggregation matrix to one of the data owners as the model training initiator for ciphertext decryption operations;

[0066] Decryption of ciphertext training data: The model training initiator receives the globally aggregated ciphertext training data from the cloud server. Then, the symmetric homomorphic encryption key is used to perform decryption operations to obtain the globally aggregated training data matrix. The model training initiator responsible for the decryption operation can use the key to decrypt the aggregated ciphertext data.

[0067] Logistic regression training and testing, the model training initiator uses the decrypted global logistic regression training data for training, obtains the final trained efficient model, and performs accuracy testing on the obtained model.

[0068] Another object of the present invention is to provide an information data processing terminal, which is used to implement the logistic regression security training method.

[0069] Another object of the present invention is to provide a logistic regression safety training system for implementing the logistic regression safety training method, the logistic regression safety training system comprising:

[0070] The data owner normalizes the local training data, uses the encryption parameters provided by the model training initiator to perform encryption calculations on the normalized local training data, obtains the ciphertext training data, and sends the ciphertext training data to the cloud service provider;

[0071] The model training initiator initializes the logistic regression security training system, selects appropriate security parameters and generates encryption parameters to distribute to other data owners. At the same time, as one of the data owners, it is also necessary to normalize the local training data, generate ciphertext local training data and send it to the cloud service provider. Finally, it receives the global ciphertext aggregation matrix from the cloud service provider, uses the symmetric homomorphic key to perform decryption calculation to obtain the training matrix, and iteratively updates the logistic regression model to obtain the final logistic regression model.

[0072] The cloud service provider receives the local ciphertext training data provided by all data owners, aggregates the received data, and concatenates the local ciphertext aggregation matrices of all participants to generate global ciphertext training data; and sends the global ciphertext training data to the model training initiator.

[0073] Furthermore, the model training initiator includes:

[0074] System initialization module, including a trusted system parameter selection submodule, a homomorphic encryption key generation submodule, and a homomorphic encryption parameter generation submodule;

[0075] The trusted system parameter selection submodule is used to select the required security parameters according to the system security requirements, including model training hyperparameters and symmetric homomorphic encryption system parameters;

[0076] A homomorphic encryption key generation submodule, used to generate a symmetric homomorphic encryption key according to the selected system security parameters;

[0077] A homomorphic encryption parameter generation submodule is used to generate homomorphic encryption parameters for each data participant according to the generated symmetric homomorphic encryption key, and distribute the symmetric homomorphic encryption parameters to each data owner;

[0078] The data preprocessing module includes a sample alignment submodule, a normalization processing submodule, an aggregation submatrix / vector generation submodule and a data encryption submodule; the sample alignment submodule is used to find the common training samples of each participant and perform an alignment mechanism on the training samples; the normalization processing submodule is used to count the maximum and minimum values ​​of each attribute of the local training data and then perform normalization processing on the local training data; the aggregation submatrix / vector generation submodule is used to perform local data aggregation calculation on the normalized data to generate aggregation subvectors and aggregation submatrices; the data encryption submodule is used to perform encryption calculation on the generated aggregation subvectors, aggregation submatrices and local training data to generate ciphertext aggregation subvectors, ciphertext aggregation submatrices and ciphertext local training data;

[0079] The logistic regression training matrix decryption module is used to perform decryption calculations on the ciphertext logistic regression training data received from the cloud service provider to obtain global logistic regression training data;

[0080] A non-interactive logistic regression training module is used to perform iterative training using the obtained global logistic regression training data to obtain a final logistic regression training model, and to perform an accuracy test on the obtained model;

[0081] The data owner includes: a data preprocessing module, including a sample alignment submodule, a normalization processing submodule, an aggregation submatrix / vector generation submodule and a data encryption submodule; wherein the sample alignment submodule is used to find out the common training samples of each participant and perform an alignment mechanism on the training samples; the normalization processing submodule is used to count the maximum and minimum values ​​of each attribute of the local training data and then perform normalization processing on the local training data; the aggregation submatrix / vector generation submodule is used to perform local data aggregation calculation on the normalized data to generate aggregation subvectors and aggregation submatrices; the data encryption submodule is used to perform encryption calculation on the generated aggregation subvectors, aggregation submatrices and local training data to generate ciphertext aggregation subvectors, ciphertext aggregation submatrices and ciphertext local training data;

[0082] The cloud service provider includes: a ciphertext data processing module, including a ciphertext aggregation sub-matrix calculation sub-module and a logistic regression training matrix splicing sub-module;

[0083] The ciphertext aggregation submatrix calculation submodule uses the ciphertext training data received from each data owner to perform homomorphic data aggregation operations between any two data owners using the ciphertext data to obtain the ciphertext aggregation submatrix; the ciphertext data splicing submodule is used to splice the ciphertext aggregation submatrix received from each participant and the ciphertext aggregation submatrix calculated by the ciphertext data aggregation model to form global logistic regression training data.

[0084] Another object of the present invention is to provide an artificial intelligence data processing terminal, which implements the logistic regression security training method.

[0085] Combined with all the above technical solutions, the advantages and positive effects of the present invention are as follows: The purpose of the present invention is to address the shortcomings of existing federated learning technology and propose a non-interactive multi-source vertically distributed data privacy-preserving logistic regression federated training method. The present invention aims to adopt a non-interactive privacy-preserving logistic regression federated training method under the premise of ensuring the privacy of the user's local data, that is, the user does not have to ensure that each round of training process is online in real time or consume local computing resources for a large number of model update calculations. The user only needs to perform pre-processing calculations and encryption calculations on the local training data at the beginning of training; then the ciphertext data is handed over to the cloud service provider, and the cloud service provider completes the ciphertext data aggregation calculation, and the ciphertext global logistic regression training data is handed over to the model training initiator, and the model training initiator decrypts the ciphertext global logistic regression training data to obtain the global training data matrix, and uses the global training data matrix to perform logistic regression training to obtain the final efficient logistic regression model.

[0086] Compared with the prior art, the present invention has the following advantages:

[0087] (1) To achieve privacy protection of data of multiple data participants, the method adopted by the present invention is that each data participant first normalizes the training data and preprocesses the data set locally, and then uses a symmetric homomorphic encryption algorithm to encrypt the local data, and sends the encrypted ciphertext data to the cloud server. After receiving the ciphertext data sent by all data owners, the server uses the properties of homomorphic encryption to perform matrix multiplication operations between the ciphertext data of each data participant to achieve aggregation calculation of the ciphertext data, and then splices the received local aggregation matrix and the calculated local aggregation matrix to obtain a global ciphertext aggregation matrix. However, since the cloud service does not have a symmetric homomorphic encryption key, it cannot decrypt the training data, and thus cannot use the obtained ciphertext data to infer the local data, thereby ensuring the privacy and security of user data.

[0088] (2) To realize the training mechanism of non-interactive federated learning logistic regression problem, the method adopted by the present invention is that the user uploads the encrypted local training data to the cloud server, and the cloud server aggregates the ciphertext data using the characteristics of homomorphic encryption, and splices the data matrix to finally obtain the global aggregation matrix. Then, the cloud server sends the aggregation matrix to the model training initiator, and the initiator decrypts the ciphertext data and performs subsequent iterative training. Therefore, the user is not required to stay online during the entire training process, nor is the user required to use local computing resources to train the model, which realizes the training of non-interactive federated learning logistic regression problem, greatly reducing the communication overhead and computing overhead of federated learning.

[0089] (3) Improving model training efficiency. In the present invention, a large amount of encrypted data aggregation operations and logistic regression model training are completed by the cloud server. The cloud server has powerful computing power and can quickly complete data processing, thereby improving the efficiency and availability of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a flow chart of the logistic regression safety training method provided by an embodiment of the present invention.

[0091] Figure 2 It is a flow chart of a non-interactive privacy-preserving logistic regression federated training system for multi-source vertically distributed data provided by an embodiment of the present invention.

[0092] Figure 3 This is an architecture diagram of a non-interactive privacy-preserving logistic regression federated training system for multi-source vertically distributed data provided by an embodiment of the present invention.

[0093] Figure 4 It is a sub-flow chart of data processing and logistic regression model generation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0095] In view of the problems existing in the prior art, the present invention provides a logistic regression security training method and system for multi-source vertically distributed data. The present invention is described in detail below in conjunction with the accompanying drawings.

[0096] Example 1

[0097] like Figure 1 As shown, the logistic regression safety training method provided by the present invention comprises the following steps:

[0098] S101: System initialization and data normalization. One of the data owners, as the initiator of model training, selects system parameters and generates symmetric homomorphic encryption keys according to system security requirements, and then uses the homomorphic encryption keys to encrypt constants 0 and 1, generates encryption parameters for each data owner, and provides key support for the homomorphic operations required for the subsequent secure transmission of data and secure aggregation. At the same time, the data owner completes data normalization processing locally to provide data support for subsequent logistic regression model training.

[0099] S102: Data preprocessing and local data encryption. The model training initiator uses the homomorphic encryption key to perform encryption calculations on the normalized local training data. Other data owners use encryption parameters to perform encryption calculations on the normalized local training data to obtain ciphertext local training data. The ciphertext local training data is then sent to the cloud server, providing data support for the cloud server to aggregate ciphertext training data.

[0100] S103: Aggregation of ciphertext training data: The cloud server receives all ciphertext training data from data owners, uses homomorphic encryption properties to securely aggregate the data of any two data owners, and sends the aggregation matrix to one of the data owners as the model training initiator for ciphertext decryption operations. The cloud server can only perform homomorphic operations between ciphertext training data to achieve data aggregation operations. Due to the lack of decryption keys, it cannot decrypt ciphertexts, which ensures the security and privacy of user data.

[0101] S104: Decryption of ciphertext training data. The model training initiator receives the globally aggregated ciphertext training data from the cloud server, and then uses the symmetric homomorphic encryption key to perform decryption operations to obtain a globally aggregated training data matrix. The model training initiator responsible for the decryption operation can use the key to decrypt the aggregated ciphertext data. However, the data owner does not have any information about the plaintext training data corresponding to the aggregated training data, so the local training data of other data owners cannot be obtained.

[0102] S105: Logistic regression training and testing. The model training initiator uses the decrypted global logistic regression training data for training to obtain the final trained efficient model and conducts accuracy testing on the obtained model.

[0103] In step S101 of the present invention, system initialization specifically includes:

[0104] (1) Model training initiator P K Select system parameters and generate symmetric homomorphic encryption parameters to distribute to each participant P k (k≠K).

[0105] a)P KSelect system security parameters (k0, k1, k2) and execute function KeyGen(k0, k1, k2) to generate public parameters PP and symmetric homomorphic encryption key SK.

[0106] b)P K Using the symmetric homomorphic encryption key SK, all participants P are encrypted with constants 0 and 1. k (k≠K) Generate encryption parameter pair γ k ={Enc(1),Enc(0)}, and γ k Distribute to the corresponding participants P k (k≠K).

[0107] (2)P K Initialize global model parameters Where D represents the feature dimension of all participating parties’ data. Then P K Select the hyperparameter set for logistic regression model training, which includes learning rate α, batch size β, loop parameter π, and scaling factor κ.

[0108] In step S102 of the present invention, data preprocessing and local data encryption specifically include:

[0109] (1) Each parameter square P k First, the local data is aligned and normalized to obtain the preprocessed local data matrix D (k) .

[0110] a) Each participant P k Delete irrelevant entries in the data set and arrange the remaining data entries accordingly to obtain the local training data set D (k) =[X (k1) ,X (k2) ,...,X (kN) ] T , where X (kn) (n=1,2,...,N) is a D k / (D k +1)-dimensional vector is represented as

[0111]

[0112] D k Represents user P k The data feature dimensions and meet The final complete training data set is D = [D (1) |D (2) |...|D (K) ], each line represents a piece of training data.

[0113] b) Each participant P kThe local statistics collect the maximum and minimum values ​​of the attributes of the data and generate a local vector

[0114]

[0115]

[0116] Afterwards, each participant P k By calculation Normalize the local data to obtain the normalized local training data D k In addition, the label column [y1,y2,...,y N ] T No normalization is required.

[0117] (2) Each participant P k According to the preprocessed data matrix D (k) , each participant P k Calculate the aggregate subvector V (k) and the aggregate submatrix M (k) , and then use the encryption parameter γ k Or private key SK for training data D (k) , aggregate subvector V (k) and the aggregate submatrix M (k) Perform encryption calculation to obtain ciphertext training data Ciphertext aggregator vector and the ciphertext aggregation matrix The obtained ciphertext data is sent to the cloud service provider CSP.

[0118] a) Each participant calculates Expand the local training data to integers and calculate the aggregate submatrix M (k) , aggregate subvector V (k) .

[0119] Among them, for P K , V (K) is a (D K +1)-dimensional vector, M (K) is a (D K +1)-dimensional symmetric matrices, respectively expressed as:

[0120]

[0121]

[0122] To other data owners P k (k≠K), V (k) It is a D K dimensional vector, M (k) It is a DK dimensional symmetric matrix, expressed as:

[0123]

[0124] b) Each participant uses the encryption parameter γ k calculate Perform encryption calculation on local training data, where n = 1, 2, ..., N, d = 1, 2, ..., D k Finally, each participant P k Will and Sent to the cloud service provider CSP.

[0125] In step S103 of the present invention, the secure aggregation of ciphertext data specifically includes:

[0126] (1) The cloud service provider CSP first trains the data based on the encrypted local data of each participant. Calculate the ciphertext aggregation matrix Then, based on and Construct a global ciphertext aggregation matrix and send it to P K Perform iterative model training.

[0127] a) After receiving the ciphertext training data provided by all participants, CSP performs aggregation calculation on the local data of any two participants u and v based on the ciphertext local training data to obtain the ciphertext aggregation submatrix

[0128] Among them, when 1≤u<v≤K-1, It is a D u ×D v dimensional matrix, each element of the matrix Expressed as

[0129] When 1≤u<v,v=K, It is a D u ×(D K +1)-dimensional matrix, each element of the matrix It is expressed as:

[0130]

[0131] b) CSP will and Matrices are concatenated to construct a global ciphertext aggregation matrix is a (D+2)-dimensional symmetric matrix:

[0132]

[0133] Finally, CSP will Send to P K .

[0134] In step S104 of the present invention, decryption of the ciphertext training data specifically includes:

[0135] (1)P K Receive global ciphertext aggregation matrix The symmetric homomorphic key is used to decrypt the calculation to obtain the logistic regression training matrix, and the matrix A and vector are obtained from it. Used for subsequent iterative update calculation of model parameters.

[0136] a)P K Receive global ciphertext aggregation matrix After that, first delete The last row of the matrix, then P K By calculation For the global ciphertext matrix Decrypt each element of and fill the first row and first column elements with N. The final result matrix in

[0137]

[0138]

[0139] In step S105 of the present invention, the logistic regression training and testing specifically includes:

[0140] (1)P K By using the decrypted matrix A and vector It is used to iteratively update the model parameters. When the model parameter θ is judged to be converged or the maximum number of iterations is reached, the iterative update calculation is stopped. At this time, P K Get the trained logistic regression model.

[0141] a)P K Perform calculations by iteration Update the model parameters. After a certain number of iterations, P K Using matrix A and vector Calculate the loss function:

[0142]

[0143] in L B =θB. The operation represents the Hadamard product of two matrices, while SUM() represents the sum of all elements in the matrix. When the model parameters are judged to be converged or the maximum number of training times is reached, the logical regression training stops, and P K The final logistic regression model was obtained.

[0144] Ordinary technicians in the industry can also use other steps to implement the logistic regression safety training method provided by the present invention. Figure 1 The logistic regression safety training method provided by the present invention is only a specific embodiment.

[0145] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0146] like Figure 3 As shown, the logistic regression security training system for multi-source vertically distributed data provided by the present invention is composed of three subsystems: model training initiator, data owner, and cloud service provider;

[0147] The model training initiator is used to complete system initialization, set the parameters required by the system, generate the symmetric homomorphic keys required in the homomorphic computing process, and distribute the homomorphic encryption parameters obtained by encrypting the constant values ​​0 and 1 with the homomorphic encryption key to all data participants; receive the ciphertext logistic regression training data from the cloud service provider, decrypt it using the key to obtain the global aggregate training data, and perform efficient logistic regression training to obtain the final logistic regression model;

[0148] The data owner normalizes the global training data locally, uses the homomorphic encryption parameters provided by the model training initiator to perform encryption calculations on the normalized local training data, obtains the ciphertext local training data, and sends the local training data to the cloud service provider;

[0149] The cloud service provider receives the encrypted local training data provided by all data owners, performs aggregation calculations on the received data, generates encrypted global training data as logistic regression training data, and sends it to the model training initiator.

[0150] In the present invention, the model training initiator includes:

[0151] The system initialization module includes a trusted system parameter selection submodule, a homomorphic encryption key generation submodule, and a homomorphic encryption parameter generation submodule. Among them, the trusted system parameter selection submodule is used to select the required security parameters according to the system security requirements, including model training hyperparameters and symmetric homomorphic encryption system parameters. The homomorphic encryption key generation submodule is used to generate symmetric homomorphic encryption keys according to the selected system security parameters. The homomorphic encryption parameter generation submodule is used to generate homomorphic encryption parameters for each data participant according to the generated symmetric homomorphic encryption key, and distribute the symmetric homomorphic encryption parameters to each data owner.

[0152] The data preprocessing module includes a sample alignment submodule, a normalization processing submodule, an aggregation submatrix / vector generation submodule and a data encryption submodule. Among them, the sample alignment submodule is used to find the common training samples of each participant and perform an alignment mechanism on the training samples; the normalization processing submodule is used to count the maximum and minimum values ​​of each attribute of the local training data, and then normalize the local training data. The aggregation submatrix / vector generation submodule is used to perform local data aggregation calculation on the normalized data to generate aggregation subvectors and aggregation submatrices. The data encryption submodule is used to perform encryption calculations on the generated aggregation subvectors, aggregation submatrices and local training data to generate ciphertext aggregation subvectors, ciphertext aggregation submatrices and ciphertext local training data.

[0153] The logistic regression training matrix decryption module is used to perform decryption calculations on the ciphertext logistic regression training data received from the cloud service provider to obtain global logistic regression training data.

[0154] The non-interactive logistic regression training module is used to perform iterative training using the obtained global logistic regression training data to obtain a final logistic regression training model, and to perform an accuracy test on the obtained model.

[0155] In the present invention, the data owner includes: a data preprocessing module, including a sample alignment submodule, a normalization processing submodule, an aggregation submatrix / vector generation submodule and a data encryption submodule. Among them, the sample alignment submodule is used to find the common training samples of each participant and perform an alignment mechanism on the training samples; the normalization processing submodule is used to count the maximum and minimum values ​​of each attribute of the local training data, and then normalize the local training data. The aggregation submatrix / vector generation submodule is used to perform local data aggregation calculation on the normalized data to generate aggregation subvectors and aggregation submatrices. The data encryption submodule is used to perform encryption calculations on the generated aggregation subvectors, aggregation submatrices and local training data to generate ciphertext aggregation subvectors, ciphertext aggregation submatrices and ciphertext local training data.

[0156] In the present invention, the cloud service provider includes: a ciphertext data processing module, which includes a ciphertext aggregation submatrix calculation submodule and a logistic regression training matrix splicing submodule. The ciphertext aggregation submatrix calculation submodule uses the ciphertext training data received from each data owner to perform homomorphic data aggregation operations between any two data owners using the ciphertext data to obtain a ciphertext aggregation submatrix; the ciphertext data splicing submodule is used to splice the ciphertext aggregation submatrix received from each participant and the ciphertext aggregation submatrix calculated by the ciphertext data aggregation model to form global logistic regression training data.

[0157] Example 2

[0158] like Figure 3 As shown, the logistic regression safety training system for multi-source vertically distributed data provided by the present invention comprises the following steps:

[0159] Step 1: System initialization and data normalization. In this step, the model training initiator selects appropriate security parameters for the logistic regression security training system, generates symmetric homomorphic encryption keys based on the selected security parameters, uses the homomorphic encryption keys to generate homomorphic encryption parameters for each data owner, and distributes the encryption parameters to each data owner participating in the logistic regression training, providing key support for the homomorphic operations required for the subsequent secure transmission of data and secure aggregation, thereby ensuring the security and privacy of user data in non-interactive scenarios. At the same time, the data owner performs statistical processing on the local training data to obtain the maximum and minimum value information of the training attributes, and then completes the data normalization processing locally to provide data support for the subsequent logistic regression model training.

[0160] Step 2: Data preprocessing and local data encryption. In this step, the data owner uses symmetric homomorphic encryption parameters to perform encryption calculations on the normalized local training data to obtain ciphertext local training data, and sends the ciphertext local training data to the cloud server, providing data support for the cloud server to aggregate ciphertext training data. In this process, the security and privacy of user data are guaranteed through homomorphic encryption technology.

[0161] Step three: Aggregation of ciphertext training data. In this step, the cloud server receives all ciphertext training data from the data owner, and uses the homomorphic encryption property to securely aggregate and concatenate the ciphertext data of the data owner to obtain the ciphertext global logistic regression training data, and sends it to the model training initiator.

[0162] Step 4: Decryption of ciphertext training data. In this step, the model training initiator receives the globally aggregated ciphertext training data from the cloud server, and then uses the symmetric homomorphic encryption key to perform decryption operations to obtain the globally aggregated training data matrix. The model training initiator responsible for the decryption operation can use the key to decrypt the aggregated ciphertext data.

[0163] Step 5: Logistic regression training and testing. In this step, the model training initiator uses the decrypted global logistic regression training data for training to obtain the final trained efficient model and conducts accuracy testing on the obtained model.

[0164] The technical effects of the present invention are described in detail below in conjunction with tests.

[0165] Table 1

[0166]

[0167] As shown in Table 1, the test results of the present invention under different data sets are shown. In the Diabetes Dataset, the data set dimension is 9, the data set size is 768, and the number of iterations is 2*10. 4 The final accuracy is 0.7712. The running time here only includes the computing time of the model training initiator for model iteration training and the computing time of the cloud service provider for aggregating the ciphertext matrix, which is 326ms. In the Breast Cancer Wisconsin Dataset, the dataset dimension is 9, the dataset size is 699, and the number of iterations is 2*10. 4 The final accuracy is 0.9928. The running time here only includes the computing time of the model training initiator for model iteration training and the computing time of the cloud service provider for aggregating the ciphertext matrix, which is 420ms. In the Bank Marketing Dataset, the dataset dimension is 14, the dataset size is 3999, and the number of iterations is 2*10. 4 The final accuracy is 0.8831. The running time here only includes the computing time of the model training initiator for model iteration training and the computing time of the cloud service provider for aggregating the ciphertext matrix, which is 1665ms.

[0168] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0169] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A logistic regression safety training method, characterized in that: The logistic regression safety training method comprises: System initialization and data normalization: one of the data owners, as the initiator of model training, selects system parameters and generates symmetric homomorphic encryption keys according to system security requirements; uses homomorphic encryption keys to encrypt constants 0 and 1, generates encryption parameters for each data owner, and provides key support for homomorphic operations required for subsequent secure data transmission and secure aggregation; at the same time, the data owner completes data normalization processing locally to provide data support for subsequent logistic regression model training; Data preprocessing and local data encryption: The model training initiator uses the homomorphic encryption key to perform encryption calculations on the normalized local training data. Other data owners use encryption parameters to perform encryption calculations on the normalized local training data to obtain ciphertext local training data. The ciphertext local training data is then sent to the cloud server, providing data support for the cloud server to aggregate ciphertext training data. Aggregation of ciphertext training data: The cloud server receives all ciphertext training data from data owners, uses homomorphic encryption properties to securely aggregate the data of any two data owners, and sends the aggregation matrix to one of the data owners as the model training initiator for ciphertext decryption operations; Decryption of ciphertext training data: The model training initiator receives the globally aggregated ciphertext training data from the cloud server. Then, the symmetric homomorphic encryption key is used to perform decryption operations to obtain the globally aggregated training data matrix. The model training initiator responsible for the decryption operation can use the key to decrypt the aggregated ciphertext data. Logistic regression training and testing, the model training initiator uses the decrypted global logistic regression training data for training, obtains the final trained efficient model, and performs accuracy testing on the obtained model.

2. The logistic regression safety training method according to claim 1, characterized in that: The system initialization specifically includes: (1) Model training initiator P K Select system parameters and generate symmetric homomorphic encryption parameters to distribute to each participant P k (k≠K); a)P K Select system security parameters (k0, k1, k2) and execute the function KeyGen(k0, k1, k2) to generate public parameters PP and symmetric homomorphic encryption key SK; b)P K Using the symmetric homomorphic encryption key SK, all participants P are encrypted with constants 0 and 1. k (k≠K) Generate encryption parameter pair γ k ={Enc(1),Enc(0)}, and γ k Distribute to the corresponding participants P k (k≠K); (2)P K Initialize global model parameters Where D represents the characteristic dimension of data of all participants; P K Select the hyperparameter set for logistic regression model training, which includes learning rate α, batch size β, loop parameter π, and scaling factor κ.

3. The logistic regression safety training method according to claim 1, characterized in that: The data preprocessing and local data encryption specifically include: (1) Each parameter square P k First, the local data is aligned and normalized to obtain the preprocessed local data matrix D (k) ; a) Each participant P k Delete irrelevant entries in the data set and arrange the remaining data entries accordingly to obtain the local training data set D (k) =[X (k1) ,X (k2) ,...,X (kN) ] T , where X (kn) (n=1,2,...,N) is a D k / (D k +1)-dimensional vector is represented as: D k Represents user P k The data feature dimensions and meet The complete training data set obtained is D = [D (1) |D (2) |...|D (K) ], each line represents a training data; b) Each participant P k Locally count the maximum and minimum values ​​of the attributes of the collected data and generate a local vector: Afterwards, each participant P k By calculation Normalize the local data to obtain the normalized local training data D k ; In addition, the label column [y1,y2,...,y N ] T No normalization is required; (2) Each participant P k According to the preprocessed data matrix D (k) , each participant P k Calculate the aggregate subvector V (k) and the aggregate submatrix M (k) , and then use the encryption parameter γ k Or private key SK for training data D (k) , aggregate subvector V (k) and the aggregate submatrix M (k) Perform encryption calculation to obtain ciphertext training data Ciphertext Aggregator Vector and the ciphertext aggregation matrix And send the obtained ciphertext data to the cloud service provider CSP; a) Each participant calculates Expand the local training data to integers and calculate the aggregate submatrix M (k) , aggregate subvector V (k) ; Among them, for P K , V (K) is a (D K +1)-dimensional vector, M (K) is a (D K +1)-dimensional symmetric matrices, respectively expressed as: To other data owners P k (k≠K), V (k) It is a D K dimensional vector, M (k) It is a D K dimensional symmetric matrix, expressed as: b) Each participant uses the encryption parameter γ k calculate Perform encryption calculation on local training data, where n = 1, 2, ..., N, d = 1, 2, ..., D k ; Finally, each participant P k Will and Sent to the cloud service provider CSP.

4. The logistic regression safety training method according to claim 1, characterized in that: The secure aggregation of ciphertext data specifically includes: (1) The cloud service provider CSP first trains the data based on the encrypted local data of each participant. Calculate the ciphertext aggregation matrix based on and Construct a global ciphertext aggregation matrix and send it to P K Conduct iterative model training; a) After receiving the ciphertext training data provided by all participants, CSP performs aggregation calculation on the local data of any two participants u and v based on the ciphertext local training data to obtain the ciphertext aggregation submatrix Among them, when 1≤u<v≤K-1, It is a D u ×D v dimensional matrix, each element of the matrix Expressed as When 1≤u<v,v=K, It is a D u ×(D K +1)-dimensional matrix, each element of the matrix It is expressed as: b) CSP will and Matrices are concatenated to construct a global ciphertext aggregation matrix is a (D+2)-dimensional symmetric matrix: Finally, CSP will Send to P K .

5. The logistic regression safety training method according to claim 1, characterized in that: The logistic regression training and testing specifically include: (1)P K Receive global ciphertext aggregation matrix And perform decryption calculation to obtain the global aggregation matrix, and obtain the matrix A and vector from it It is used to iteratively update the model parameters. When the model parameter θ is judged to be converged or the maximum number of iterations is reached, the iterative update calculation is stopped. At this time, P K Get the trained logistic regression model; a)P K Receive global ciphertext aggregation matrix After that, first delete The last row of the matrix, then P K By calculation For the global ciphertext matrix Decrypt each element of and fill the first row and first column elements with N. The final result matrix in: b)P K By performing calculations Update the model parameters; after a certain number of iterations, P K Using matrix A and vector Calculate the loss function: in L B =θB, The operation represents the Hadamard product of two matrices, and SUM() represents the sum of all elements in the matrix; when the model parameters are judged to be converged or the maximum number of training times is reached, the logical regression training stops, and P K The final logistic regression model was obtained.

6. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps of the logistic regression security training method according to claim 1: System initialization and data normalization: one of the data owners, as the initiator of model training, selects system parameters and generates symmetric homomorphic encryption keys according to system security requirements; uses homomorphic encryption keys to encrypt constants 0 and 1, generates encryption parameters for each data owner, and provides key support for homomorphic operations required for subsequent secure data transmission and secure aggregation; at the same time, the data owner completes data normalization processing locally to provide data support for subsequent logistic regression model training; Data preprocessing and local data encryption: The model training initiator uses the homomorphic encryption key to perform encryption calculations on the normalized local training data. Other data owners use encryption parameters to perform encryption calculations on the normalized local training data to obtain ciphertext local training data. The ciphertext local training data is then sent to the cloud server, providing data support for the cloud server to aggregate ciphertext training data. Aggregation of ciphertext training data: The cloud server receives all ciphertext training data from data owners, uses homomorphic encryption properties to securely aggregate the data of any two data owners, and sends the aggregation matrix to one of the data owners as the model training initiator for ciphertext decryption operations; The ciphertext training data is decrypted, and the model training initiator receives the globally aggregated ciphertext training data from the cloud server; Then, the symmetric homomorphic encryption key is used to perform decryption operations to obtain the global aggregated training data matrix; the model training initiator responsible for the decryption operation can use the key to decrypt the aggregated ciphertext data; Logistic regression training and testing, the model training initiator uses the decrypted global logistic regression training data for training, obtains the final trained efficient model, and performs accuracy testing on the obtained model.

7. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the logistic regression safety training method described in any one of claims 1 to 5.

8. A logistic regression safety training system for implementing the logistic regression safety training method according to any one of claims 1 to 5, characterized in that: The logistic regression safety training system comprises: The data owner normalizes the local training data, uses the encryption parameters provided by the model training initiator to perform encryption calculations on the normalized local training data, obtains the ciphertext training data, and sends the ciphertext training data to the cloud service provider; The model training initiator initializes the logistic regression security training system, selects appropriate security parameters and generates encryption parameters to distribute to other data owners. At the same time, as one of the data owners, it is also necessary to normalize the local training data, generate ciphertext local training data and send it to the cloud service provider. Finally, it receives the global ciphertext aggregation matrix from the cloud service provider, uses the symmetric homomorphic key to perform decryption calculation to obtain the training matrix, and iteratively updates the logistic regression model to obtain the final logistic regression model. The cloud service provider receives the local ciphertext training data provided by all data owners, aggregates the received data, and concatenates the local ciphertext aggregation matrices of all participants to generate global ciphertext training data; and sends the global ciphertext training data to the model training initiator.

9. The logistic regression safety training system as claimed in claim 8, characterized in that: The model training initiator includes: System initialization module, including a trusted system parameter selection submodule, a homomorphic encryption key generation submodule, and a homomorphic encryption parameter generation submodule; The trusted system parameter selection submodule is used to select the required security parameters according to the system security requirements, including model training hyperparameters and symmetric homomorphic encryption system parameters; A homomorphic encryption key generation submodule, used to generate a symmetric homomorphic encryption key according to the selected system security parameters; A homomorphic encryption parameter generation submodule is used to generate homomorphic encryption parameters for each data participant according to the generated symmetric homomorphic encryption key, and distribute the symmetric homomorphic encryption parameters to each data owner; The data preprocessing module includes a sample alignment submodule, a normalization processing submodule, an aggregation submatrix / vector generation submodule and a data encryption submodule; the sample alignment submodule is used to find the common training samples of each participant and perform an alignment mechanism on the training samples; the normalization processing submodule is used to count the maximum and minimum values ​​of each attribute of the local training data and then perform normalization processing on the local training data; the aggregation submatrix / vector generation submodule is used to perform local data aggregation calculation on the normalized data to generate aggregation subvectors and aggregation submatrices; the data encryption submodule is used to perform encryption calculation on the generated aggregation subvectors, aggregation submatrices and local training data to generate ciphertext aggregation subvectors, ciphertext aggregation submatrices and ciphertext local training data; The logistic regression training matrix decryption module is used to perform decryption calculations on the ciphertext logistic regression training data received from the cloud service provider to obtain global logistic regression training data; A non-interactive logistic regression training module is used to perform iterative training using the obtained global logistic regression training data to obtain a final logistic regression training model, and to perform an accuracy test on the obtained model; The data owner includes: a data preprocessing module, including a sample alignment submodule, a normalization processing submodule, an aggregation submatrix / vector generation submodule and a data encryption submodule; wherein the sample alignment submodule is used to find out the common training samples of each participant and perform an alignment mechanism on the training samples; the normalization processing submodule is used to count the maximum and minimum values ​​of each attribute of the local training data and then perform normalization processing on the local training data; the aggregation submatrix / vector generation submodule is used to perform local data aggregation calculation on the normalized data to generate aggregation subvectors and aggregation submatrices; the data encryption submodule is used to perform encryption calculation on the generated aggregation subvectors, aggregation submatrices and local training data to generate ciphertext aggregation subvectors, ciphertext aggregation submatrices and ciphertext local training data; The cloud service provider includes: a ciphertext data processing module, including a ciphertext aggregation sub-matrix calculation sub-module and a logistic regression training matrix splicing sub-module; The ciphertext aggregation submatrix calculation submodule uses the ciphertext training data received from each data owner to perform homomorphic data aggregation operations between any two data owners using the ciphertext data to obtain the ciphertext aggregation submatrix; the ciphertext data splicing submodule is used to splice the ciphertext aggregation submatrix received from each participant and the ciphertext aggregation submatrix calculated by the ciphertext data aggregation model to form global logistic regression training data.

10. An artificial intelligence data processing terminal, characterized in that: The artificial intelligence data processing terminal implements the logistic regression security training method described in any one of claims 1 to 5.

Images