Multi-party data model training method and device based on privacy protection

Abstract

The invention belongs to the field of multi-party data sharing encryption, and particularly provides a multi-party data model training method and device based on privacy protection. According to the multi-party data model training method based on privacy protection, model training is distributed to all participants for local training; the method comprises the steps of training a parameter matrix, performing matrix sparsification processing on the trained parameter matrix, and then sending parameters to a server, thereby obtaining a total parameter matrix of the model, reducing the data volume of communication, relieving the problem of overhigh communication cost, increasing the security level by adopting a multi-level matrix encryption mode, and ensuring that even if a part of participants have malicious behaviors, the user experience is improved. The original data cannot be reconstructed through the sub-parameter matrix, the data security problem is effectively solved, meanwhile, each iteration is carried out locally, the pertinence of the data is higher, and therefore it is ensured that the prediction capacity of the finally trained model is more accurate.

Description

Technical Field

[0001] The present invention belongs to the field of multi-party data sharing encryption, and particularly relates to a method and device for training a multi-party data usage model based on privacy protection. Background Art

[0002] In the current context of big data, the data volume has increased sharply, and "data islands" that are isolated from each other have formed among data owners in various industries, seriously hindering the collaborative sharing and efficient computing of data across institutions. Although the traditional centralized data processing mode can effectively converge data for analysis and modeling, the risk of data leakage and privacy infringement is extremely easy to occur during this process.

[0003] In related technologies, the following several methods are adopted for encryption;

[0004] First, by applying an encryption algorithm to the data to ensure the confidentiality of the data during transmission and storage, its algorithm encryption and decryption processes consume computing resources under big data, affecting the system performance; differential privacy technology protects the privacy information of individual data by adding a certain amount of noise to the original data, which can effectively prevent the leakage of personal sensitive information during data processing, but the introduction of noise will affect the accuracy and precision of data analysis; secure multi-party computation (SMC) uses cryptography, by dispersing the data storage in different computing participants and using encryption and protocols to protect data privacy, but the implementation complexity of this method is high, and a large amount of communication and computing costs are required during the computing process.

[0005] Second, based on secure two-party computation, secret sharing, and Yao's circuits, a new privacy protection protocol is implemented for linear regression, logistic regression, and neural networks. Compared with traditional data center training, it does reduce the privacy risk. However, the gradients transmitted between different parties may still leak user privacy.

[0006] Therefore, how to make full use of the value of cross-institutional data on the premise of meeting the requirements of data privacy, security, and supervision has become an urgent problem to be solved.

[0007] It should be noted that the above information disclosed in this background art section is only used to understand the background art of the concept of this application. Therefore, the above description is not considered as information of the prior art. Summary of the Invention

[0008] The embodiments of the present disclosure at least provide a method for training a multi-party data usage model based on privacy protection.

[0009] In a first aspect, the embodiments of the present disclosure provide a method for training a multi-party data usage model based on privacy protection, and the method includes:

[0010] The server distributes the total parameter matrix of the model to be trained to each participating party;

[0011] Each participating party performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server to obtain a partial parameter matrix;

[0012] After all participating parties encrypt the partial parameter matrices, they obtain the encrypted partial parameter matrices and send them to the server;

[0013] The server updates the total parameter matrix according to the encrypted partial parameter matrices sent by all participating parties received, and completes this iteration;

[0014] Iterate a preset number of times to complete the model training.

[0015] In an alternative embodiment, the step that each participating party performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server to obtain a partial parameter matrix includes:

[0016] Each participating party preprocesses the data it stores to obtain the feature matrix X of all samples and the corresponding label y;

[0017] According to the received total parameter matrix w t , initialize the partial parameters of the sub-model, where t represents the number of iterations, and w t represents the total parameter matrix at the t-th iteration;

[0018] Forward propagation, calculate the output of the sub-model according to the feature matrix X of each sample and the corresponding label y The formula is as follows:

[0019]

[0020] In the formula, k represents the k-th participating party, represents the predicted label of the k-th participating party, X k represents the feature matrix of the k-th participating party, represents the updated partial parameter matrix of the k-th participating party at the (t + 1)-th iteration;

[0021] Calculate the loss function L;

[0022] Backward propagation, the formula for updating the local partial parameter matrix is as follows:

[0023]

[0024] In the formula, represents the updated partial parameter matrix of the k-th participating party at the (t + 1)-th iteration, and η represents the learning rate.

[0025] In an alternative embodiment, after all the participating parties encrypt the sub-parameter matrix, they obtain the encrypted sub-parameter matrix and send it to the server, including:

[0026] All participating parties perform matrix decomposition on their own sub-parameter matrix to obtain parameter sub-matrices of p + 1 sub-parameter matrices;

[0027] All participating parties send the parameter sub-matrices to the corresponding p remaining participating parties;

[0028] Each participating party accumulates the parameter sub-matrix it retains itself and all the received parameter sub-matrices to complete encryption, obtains the encrypted sub-parameter matrix after this iteration, and sends it to the server.

[0029] In an alternative embodiment, p represents a random positive integer, and p ∈ [0, n - 1], where n represents the total number of participating parties.

[0030] In an alternative embodiment, the server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all the participating parties to complete this iteration, including:

[0031] Receive the encrypted sub-parameter matrices sent by all the participating parties;

[0032] According to the average aggregation algorithm, calculate the updated total parameter matrix, and the formula is as follows:

[0033]

[0034] In the formula, w t+1 represents the total parameter matrix at the (t + 1)-th iteration, n k represents the number of samples of the k-th participating party, represents the encrypted sub-parameter matrix of the k-th participating party at the (t + 1)-th iteration, and N represents the total number of samples of all the participating parties.

[0035] In an alternative embodiment, after iterating a preset number of times to complete the model training, the method for training a model using multi-party data based on privacy protection further includes:

[0036] Each participating party verifies the accuracy of the model.

[0037] In an alternative embodiment, the verification of the model accuracy by each participating party includes:

[0038] Send the total parameter matrix of the model after training to each participating party;

[0039] Each participating party loads the total parameter matrix to obtain the trained sub-model;

[0040] Each participating party selects a test set based on the data stored by itself and inputs it into the trained sub-model to obtain predicted values;

[0041] Calculate the accuracy rate based on the predicted values and the labels y corresponding to the test set to complete the verification of the sub-model.

[0042] In a second aspect, an embodiment of the present disclosure also provides a multi-party data usage model training device based on privacy protection. The device includes:

[0043] A server and local clients of each participating party communicatively connected to the server;

[0044] The server includes the following modules:

[0045] A distribution module configured to distribute the total parameter matrix of the model to be trained to the local clients of each participating party;

[0046] An update module configured to update the total parameter matrix based on the encrypted partial parameter matrices received from all participating parties to complete this iteration;

[0047] An iteration module configured to iterate a preset number of times to complete the model training;

[0048] The local client includes the following modules:

[0049] A partial parameter matrix acquisition module configured to perform local training on the sub-model based on its own private data and the total parameter matrix obtained from the server to obtain a partial parameter matrix;

[0050] An encryption module configured to encrypt the partial parameter matrix to obtain an encrypted partial parameter matrix;

[0051] A sending module configured to send the encrypted partial parameter matrix to the server.

[0052] In a third aspect, an embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the steps of the above method are implemented.

[0053] In a fourth aspect, an embodiment of the present disclosure also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the above method are implemented.

[0054] The beneficial effects of the present invention are as follows. The method and device for training a multi-party data usage model based on privacy protection perform local training of the model training on each participating party, and after performing matrix sparsification processing on the trained parameter matrix, send the parameters to the server, thereby obtaining the total parameter matrix of the model, reducing the amount of communication data, alleviating the problem of excessive communication costs, adopting a multi-level matrix encryption method to increase the security level, ensuring that even if some participating parties have malicious behaviors, the original data cannot be reconstructed from the partial parameter matrix, effectively solving the data security problem. At the same time, each iteration is performed locally, and the data is more targeted, thereby ensuring that the prediction ability of the finally trained model is more accurate.

[0055] Other features and advantages of the present invention will be described in the following specification, and some of them will become obvious from the specification or be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0056] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically provides preferred embodiments and, in conjunction with the accompanying drawings, details are described as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1 It is a flowchart of the method for training a multi-party data usage model based on privacy protection provided by an embodiment of the present disclosure;

[0059] Figure 2 It is a schematic block diagram of the principle of the device for training a multi-party data usage model based on privacy protection provided by an embodiment of the present disclosure;

[0060] Figure 3 It is a partial structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0062] As used herein, phrases such as "in one embodiment", "according to one embodiment", "in some embodiments", etc. generally refer to the fact that a particular feature, structure, or characteristic after such a phrase can be included in at least one embodiment of the present disclosure. Thus, a particular feature, structure, or characteristic may be included in more than one embodiment of the present disclosure, such that these phrases do not necessarily refer to the same embodiment. As used herein, terms such as "example", "exemplary", etc. are used "as an example, instance, or illustration. Any embodiment, aspect, or design described herein as "example" or "exemplary" is not necessarily to be construed as preferred or superior to other embodiments, aspects, or designs. Instead, the use of terms such as "example", "exemplary", etc. is intended to present concepts in a concrete manner.

[0063] For ease of understanding of this embodiment, first, a model construction method disclosed in the embodiments of the present disclosure will be introduced in detail. The execution subject of the model training method for multi-party data based on privacy protection provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities. Such a computer device may include, for example: a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, this model construction method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0064] The following will, with reference to the accompanying drawings, elaborate on some embodiments of the present invention. Without conflict, the following embodiments and the features in the embodiments may be combined with each other.

[0065] Embodiment 1

[0066] See Figure 1 , in at least one embodiment, the model training method for multi-party data based on privacy protection divides the model training to be performed locally by each participating party, and after performing matrix sparsification processing on the trained parameter matrix, sends the parameters to the server, thereby obtaining the total parameter matrix of the model, reducing the amount of communication data, alleviating the problem of excessive communication costs, adopting a multi-level matrix encryption method to increase the security level, ensuring that even if some participating parties have malicious behaviors, the original data cannot be reconstructed from the sub-parameter matrices, effectively solving the data security problem. At the same time, each iteration is performed locally, and the data is more targeted, thereby ensuring that the prediction ability of the finally trained model is more accurate.

[0067] Specifically, the multi-party data model training method based on privacy protection includes the following steps:

[0068] S110: The server distributes the total parameter matrix of the model to be trained to each participant.

[0069] S120: Each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the sub-parameter matrix.

[0070] Specifically, step S120 includes the following steps:

[0071] S121: Each participant preprocesses the data it stores to obtain the feature matrix X of all samples and the corresponding label y.

[0072] S122: Initialize the sub-parameters of the sub-model according to the received total parameter matrix w t , where t represents the number of iterations, and w t represents the total parameter matrix at the t-th iteration.

[0073] S123: Forward propagation, calculate the output of the sub-model according to the feature matrix X of each sample and the corresponding label y The formula is as follows:

[0074]

[0075] In the formula, k represents the k-th participant, represents the predicted label of the k-th participant, and X k represents the feature matrix of the k-th participant, represents the updated sub-parameter matrix of the k-th participant at the (t + 1)-th iteration.

[0076] Specifically, during forward propagation, 80% of all samples are selected as the training set and input into the sub-model.

[0077] S124: Calculate the loss function L k .

[0078] S125: Backward propagation, the formula for updating the local sub-parameter matrix is as follows:

[0079]

[0080] In the formula, represents the updated sub-parameter matrix of the k-th participant at the (t + 1)-th iteration, and η represents the learning rate.

[0081] S130: All participants encrypt the sub-parameter matrix, obtain the encrypted sub-parameter matrix, and send it to the server.

[0082] Specifically, step S130 includes the following steps:

[0083] S131: All participating parties perform matrix decomposition on their respective sub-parameter matrices to obtain parameter sub-matrices of p + 1 sub-parameter matrices.

[0084] Specifically, the formula for matrix decomposition is as follows:

[0085]

[0086] In the formula, represents the first parameter sub-matrix after decomposition by the k-th participating party, represents the (p + 1)-th parameter sub-matrix after decomposition by the k-th participating party;

[0087] After adding up all the decomposed parameter sub-matrices, the initial sub-parameter matrix can be restored.

[0088] Among them, p represents a random positive integer, and p ∈ [0, n - 1], where n represents the total number of participating parties. Each participating party decomposes the sub-parameter matrix into a random number of parameter sub-matrices and randomly sends them to the corresponding participating parties, thereby improving the confidentiality level and ensuring that even if some participating parties have malicious behavior, the original data cannot be reconstructed through the sub-parameter matrix.

[0089] It should be noted that the specific value of p for each participating party is different, and a random number p needs to be regenerated each time of iteration.

[0090] S132: All participating parties send the parameter sub-matrices to the corresponding p other participating parties.

[0091] S133: Each participating party adds up the parameter sub-matrix it retains and all the received parameter sub-matrices to complete encryption, obtains the encrypted sub-parameter matrix for this iteration, and sends it to the server.

[0092] S140: The server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all participating parties to complete this iteration.

[0093] Among them, step S140 includes the following steps:

[0094] S141: Receive the encrypted sub-parameter matrices sent by all participating parties.

[0095] S142: Calculate the updated total parameter matrix according to the average aggregation algorithm. The formula is as follows:

[0096]

[0097] In the formula, w t+1 represents the total parameter matrix at the (t + 1)-th iteration, n k represents the number of samples of the k-th party, represents the encrypted partial parameter matrix of the k-th party at the (t + 1)-th iteration, and N represents the total number of samples of all parties.

[0098] S150: Iterate a preset number of times to complete model training.

[0099] S160: Each party verifies the accuracy of the model.

[0100] Specifically, S160 includes the following steps:

[0101] S161: Send the total parameter matrix of the model after training completion to each party.

[0102] S162: Each party loads the total parameter matrix to obtain the trained sub-model.

[0103] S163: Each party selects a test set according to the data it stores and inputs it into the trained sub-model to obtain predicted values.

[0104] Specifically, the remaining data not participating in model training forms a test set to test the trained sub-model.

[0105] S164: Calculate the accuracy rate based on the predicted values and the labels y corresponding to the test set to complete the verification of the sub-model.

[0106] Specifically, when evaluating accuracy, this paper adopts a comparative test method. Different models, namely FedAvg, FedProx, and FedAvg-Match, and the model training method proposed in this application (hereinafter referred to as this method), are applied to the dataset, and the differences between their predicted results and the true results are compared. The results are shown in Table 1 below.

[0107] Table 1 Comparison of test accuracies of different algorithms on the LFW dataset

[0108] dataset data distribution FedAvg FedProx FedAvg-Match this method LFW IID 85.23％ 86.05％ 88.92％ 91.55％ LFW non-IID 53.89％ 56.53％ 64.69％ 74.23％

[0109] It can be seen from Table 1 that the performance indicators of this method on both the IID distribution and the non-IID distribution of the dataset are better than those of other baseline models, indicating good prediction performance.

[0110] Embodiment 2

[0111] Please refer to Figure 2, this embodiment also provides a multi - party data usage model training device based on privacy protection. The device includes: a server and local clients of each participant communicatively connected to the server.

[0112] The server includes the following modules:

[0113] A distribution module, which is configured to distribute the total parameter matrix of the model to be trained to the local clients of each participant. It is used to perform the following steps:

[0114] S110: The server distributes the total parameter matrix of the model to be trained to each participant.

[0115] An update module, which is configured to update the total parameter matrix according to the encrypted partial parameter matrices received from all participants, and complete this iteration. It is used to perform the following steps:

[0116] S140: The server updates the total parameter matrix according to the encrypted partial parameter matrices received from all participants, and completes this iteration.

[0117] Among them, step S140 includes the following steps:

[0118] S141: Receive the encrypted partial parameter matrices sent by all participants.

[0119] S142: Calculate the updated total parameter matrix according to the average aggregation algorithm. The formula is as follows:

[0120]

[0121] In the formula, w t+1 represents the total parameter matrix at the (t + 1)-th iteration, n k represents the number of samples of the k-th participant, represents the encrypted partial parameter matrix of the k-th participant at the (t + 1)-th iteration, and N represents the total number of samples of all participants.

[0122] An iteration module, which is configured to iterate a preset number of times to complete the model training. It is used to perform the following steps:

[0123] S150: Iterate a preset number of times to complete the model training.

[0124] The local client includes the following modules:

[0125] A partial parameter matrix acquisition module, which is configured to perform local training on the sub - model according to its own private data and the total parameter matrix obtained from the server, and obtain the partial parameter matrix. It is used to perform the following steps:

[0126] S120: Each participating party performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the partial parameter matrix.

[0127] Specifically, step S120 includes the following steps:

[0128] S121: Each participating party preprocesses the data it stores to obtain the feature matrix X of all samples and the corresponding label y.

[0129] S122: Initialize the partial parameters of the sub-model according to the received total parameter matrix w t , where t represents the number of iterations, and w t represents the total parameter matrix at the t-th iteration.

[0130] S123: Forward propagation, calculate the output of the sub-model according to the feature matrix X of each sample and the corresponding label y The formula is as follows:

[0131]

[0132] In the formula, k represents the k-th participating party, represents the predicted label of the k-th participating party, and X k represents the feature matrix of the k-th participating party, represents the updated partial parameter matrix of the k-th participating party at the (t + 1)-th iteration.

[0133] Specifically, during forward propagation, 80% of all samples are selected as the training set and input into the sub-model.

[0134] S124: Calculate the loss function L k .

[0135] S125: Backward propagation, update the local partial parameter matrix with the following formula:

[0136]

[0137] In the formula, represents the updated partial parameter matrix of the k-th participating party at the (t + 1)-th iteration, and η represents the learning rate.

[0138] Encryption module, which is configured to encrypt the partial parameter matrix to obtain the encrypted partial parameter matrix. It is used to perform the following steps:

[0139] S130: All participating parties encrypt the partial parameter matrix to obtain the encrypted partial parameter matrix and send it to the server.

[0140] Specifically, step S130 includes the following steps:

[0141] S131: All participating parties perform matrix decomposition on their respective sub-parameter matrices to obtain parameter sub-matrices of p + 1 sub-parameter matrices.

[0142] Specifically, the formula for matrix decomposition is as follows:

[0143]

[0144] In the formula, represents the first parameter sub-matrix after decomposition by the k-th participating party, represents the (p + 1)-th parameter sub-matrix after decomposition by the k-th participating party;

[0145] After all the decomposed parameter sub-matrices are accumulated, the initial sub-parameter matrix can be restored.

[0146] Among them, p represents a random positive integer, and p ∈ [0, n - 1], where n represents the total number of participating parties. Each participating party decomposes the sub-parameter matrix into a random number of parameter sub-matrices and randomly sends them to the corresponding participating parties, thereby improving the confidentiality level and ensuring that even if some participating parties have malicious behaviors, the original data cannot be reconstructed through the sub-parameter matrix.

[0147] It should be noted that the specific value of p for each participating party is different, and a random number p needs to be regenerated each time of iteration.

[0148] S132: All participating parties send the parameter sub-matrices to the corresponding p other participating parties.

[0149] S133: Each participating party accumulates the parameter sub-matrix it retains and all the received parameter sub-matrices to complete encryption.

[0150] The sending module is configured to send the encrypted sub-parameter matrix to the server.

[0151] Embodiment 3

[0152] This embodiment provides a computer-readable storage medium, on which a computer program / instructions are stored. It is characterized in that when the computer program / instructions are executed by a processor, the steps of the method provided in Embodiment 1 are implemented.

[0153] By distributing the model training to each participating party for local training, and after performing matrix sparsification on the trained parameter matrices and then sending the parameters to the server, the total parameter matrix of the model is obtained, reducing the amount of communication data and alleviating the problem of excessively high communication costs. The use of multi-level matrix encryption increases the security level, ensuring that even if some participating parties have malicious behavior, the original data cannot be reconstructed from the partial parameter matrices, effectively solving the data security problem. At the same time, each iteration is performed locally, making the data more targeted, thus ensuring that the prediction ability of the finally trained model is more accurate.

[0154] Embodiment 4

[0155] Please refer to Figure 3 , this embodiment provides an electronic device, including: a memory 502 and a processor 501; at least one program instruction is stored in the memory 502; the processor 501, by loading and executing the at least one program instruction, implements the cross-project instant software defect prediction method as provided in Embodiment 1.

[0156] The memory 502 and the processor 501 are connected in a bus manner. The bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors 501 and the memory 502 together. The bus may also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, and therefore, they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on the transmission medium. The data processed by the processor 501 is transmitted over the wireless medium through the antenna. Further, the antenna also receives data and transmits the data to the processor 501.

[0157] The processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 502 can be used to store the data used by the processor 501 when performing operations.

[0158] In summary, the present invention provides a method and apparatus for training a multi-party data usage model based on privacy protection. Among them, the method for training a multi-party data usage model based on privacy protection divides the model training among each participating party for local training. After sparsifying the parameter matrix after training, the parameters are sent to the server, thereby obtaining the total parameter matrix of the model, reducing the amount of communication data, and alleviating the problem of excessive communication costs. The use of a multi-level matrix encryption method increases the security level, ensuring that even if some participating parties have malicious behaviors, the original data cannot be reconstructed from the partial parameter matrix, effectively solving the data security problem. At the same time, each iteration is performed locally, making the data more targeted, thus ensuring that the prediction ability of the finally trained model is more accurate.

[0159] In addition, without departing from the scope of the present disclosure, the discrete or separate technologies, systems, subsystems, and methods described and illustrated in various embodiments may be combined or integrated with other systems, modules, technologies, or methods. Other items shown or discussed as being coupled may be directly connected, or may be indirectly coupled or communicated through some interface, device, or intermediate component in an electrical, mechanical, or other manner. Those skilled in the art can determine other examples of changes, substitutions, and alterations without departing from the spirit and scope disclosed herein.

Claims

1. A method for training a multi-party data usage model based on privacy protection, characterized in that, The method includes: The server distributes the total parameter matrix of the model to be trained to each participant. Each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the partial parameter matrix. All participants encrypt the partial parameter matrix, obtain the encrypted partial parameter matrix, and send it to the server. The server updates the total parameter matrix based on the encrypted partial parameter matrices sent by all participants received, and completes this iteration. Iterate a preset number of times to complete model training.

2. The method for training a multi-party data usage model based on privacy protection according to claim 1, wherein The step where each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server to obtain the partial parameter matrix includes: Each participant preprocesses the data it stores to obtain the feature matrix X of all samples and the corresponding label y. Initialize the sub-parameters of the sub-model according to the received total parameter matrix w t , where t represents the number of iterations, and w t represents the total parameter matrix at the t-th iteration; Forward propagation, calculating the sub-model output based on the feature matrix X of each sample and the corresponding label y The formula is as follows: where k represents the k-th participant, represents the predicted label of the k-th participant, and X k represents the feature matrix of the k-th participant, represents the updated sub-parameter matrix of the k-th participant at the (t + 1)-th iteration; Calculate the loss function L. Backpropagation, and update the local partial parameter matrix with the following formula: In the formula, represents the updated sub-parameter matrix of the k-th participant at the (t + 1)-th iteration, and η represents the learning rate.

3. The method for training a multi-party data usage model based on privacy protection according to claim 1, wherein, The step where all participants encrypt the partial parameter matrix, obtain the encrypted partial parameter matrix, and send it to the server includes: All participants perform matrix decomposition on their own partial parameter matrices to obtain the parameter sub-matrices of p + 1 partial parameter matrices. All participants send the parameter sub-matrices to the corresponding p other participants. Each participant accumulates its own retained parameter sub-matrix and all the received parameter sub-matrices to complete encryption, obtains the encrypted partial parameter matrix for this iteration, and sends it to the server.

4. The method for training a multi-party data usage model based on privacy protection according to claim 3, characterized in that p represents a random positive integer, and p ∈ [0, n - 1], where n represents the total number of participants.

5. The method for training a multi-party data usage model based on privacy protection according to claim 1, characterized in that, The step where the server updates the total parameter matrix based on the encrypted partial parameter matrices sent by all participants received and completes this iteration includes: Receive the encrypted partial parameter matrices sent by all participants. According to the average aggregation algorithm, calculate the updated total parameter matrix with the following formula: where w t+1 represents the total parameter matrix at the (t + 1)-th iteration, n k represents the number of samples of the k-th party, represents the encrypted partial parameter matrix of the k-th party at the (t + 1)-th iteration, and N represents the total number of samples of all parties.

6. The method for training a multi-party data usage model based on privacy protection according to claim 1, wherein, After the step of iterating a preset number of times to complete model training, the multi-party data model training method based on privacy protection further includes: Each participant verifies the accuracy of the model.

7. The method for training a multi-party data usage model based on privacy protection according to claim 6, wherein, The step where each participant verifies the accuracy of the model includes: Distribute the total parameter matrix of the model after training to each participant. Each participant loads the total parameter matrix to obtain the trained sub-model. Each participant selects a test set based on the data it stores and inputs it into the trained sub-model to obtain the predicted value. Calculate the accuracy rate based on the predicted value and the label y corresponding to the test set to complete the verification of the sub-model.

8. A multi-party data usage model training device based on privacy protection, characterized in that, The device includes: The server and the local client of each participant communicatively connected to the server; The server includes the following modules: A distribution module configured to distribute the total parameter matrix of the model to be trained to the local client of each participant; An update module configured to update the total parameter matrix based on the encrypted partial parameter matrices sent by all participants received and complete this iteration; An iteration module configured to iterate a preset number of times to complete model training; The local client includes the following modules: The sub-parameter matrix acquisition module is configured to perform local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and acquire the sub-parameter matrix; The encryption module is configured to perform matrix encryption on the sub-parameter matrix to obtain the encrypted sub-parameter matrix; The sending module is configured to send the encrypted sub-parameter matrix to the server.

9. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method described in claim 1 are implemented.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method described in claim 1 are implemented.

Images