Multi-party data model training method and device based on privacy protection

Abstract

The application belongs to the field of multi-party data sharing encryption, and specifically provides a multi-party data model training method and device based on privacy protection, wherein the multi-party data model training method based on privacy protection performs local training by dividing model training among each participant, performs matrix sparsification processing on the trained parameter matrix, and sends the parameters to the server, thereby obtaining the total parameter matrix of the model, reducing the data volume of communication, alleviating the problem of excessively high communication cost, increasing the security level by adopting a multi-level matrix encryption mode, ensuring that even if some participants have malicious behavior, the original data cannot be reconstructed through the divided parameter matrix, effectively solving the data security problem, and at the same time, each iteration is performed locally, so that the data is more targeted, thereby ensuring that the prediction ability of the finally trained model is more accurate.

Description

Technical Field

[0001] This invention belongs to the field of multi-party data sharing encryption, specifically relating to a method and apparatus for training a model for multi-party data sharing based on privacy protection. Background Technology

[0002] In the current context of big data, the surge in data volume has created isolated "data silos" among data owners across various industries, severely hindering cross-organizational data collaboration, sharing, and efficient computation. While traditional centralized data processing models can effectively aggregate data for analysis and modeling, they are highly susceptible to data leaks and privacy violations.

[0003] The following methods are used for encryption in related technologies;

[0004] The first method ensures data confidentiality during transmission and storage by applying encryption algorithms. However, with large amounts of data, the encryption and decryption processes consume computational resources, impacting system performance. Differential privacy technology protects individual data privacy by adding a certain amount of noise to the original data, effectively preventing the leakage of sensitive personal information during data processing. However, the introduction of noise can affect the precision and accuracy of data analysis. Secure multi-party computation (SMC) uses cryptography to protect data privacy by distributing data among different computational participants and employing encryption and protocols. However, this method is highly complex to implement, and the computation process requires significant communication and computational costs.

[0005] The second approach, based on secure two-way computation, secret sharing, and Yao's circuits, implements a new privacy-preserving protocol for linear regression, logistic regression, and neural networks, which does mitigate privacy risks compared to traditional data center training. However, gradients transmitted between different parties may still potentially leak user privacy.

[0006] Therefore, how to fully utilize the value of cross-agency data while meeting data privacy, security, and regulatory requirements has become an urgent issue to be addressed.

[0007] It should be noted that the information disclosed in this background section is only for understanding the background technology of the present application concept, and therefore, the above description is not considered to constitute prior art information. Summary of the Invention

[0008] This disclosure provides at least one method for training a model using multi-party data with privacy protection.

[0009] In a first aspect, embodiments of this disclosure provide a method for training a model using privacy-preserving multi-party data, the method comprising:

[0010] The server distributes the total parameter matrix of the model to be trained to each participant;

[0011] Each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the sub-parameter matrix.

[0012] All participants encrypt the sub-parameter matrix, obtain the encrypted sub-parameter matrix, and send it to the server;

[0013] The server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all participants, thus completing this iteration;

[0014] The model training is completed after a preset number of iterations.

[0015] In one optional implementation, each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and the step of obtaining the sub-parameter matrix includes:

[0016] Each participant preprocesses its stored data to obtain the feature matrix X and the corresponding label y of all samples;

[0017] Based on the received total parameter matrix w t The sub-parameters of the sub-model are initialized, where t represents the number of iterations and w t This represents the total parameter matrix at the t-th iteration;

[0018] Forward propagation calculates the sub-model output based on the feature matrix X and corresponding label y of each sample. The formula is as follows:

[0019]

[0020] In the formula, k represents the k-th participant. Let X represent the predicted label of the k-th participant. k This represents the feature matrix of the k-th participant. This represents the parameter matrix updated by the k-th participant in the (t+1)-th iteration;

[0021] Calculate the loss function L;

[0022] Backpropagation, the formula for updating the local subparameter matrix is ​​as follows:

[0023]

[0024] In the formula, Let η represent the updated subparameter matrix of the k-th participant at the (t+1)-th iteration, and let η represent the learning rate.

[0025] In one optional implementation, all participants encrypt the sub-parameter matrix, obtain the encrypted sub-parameter matrix, and send it to the server, including:

[0026] All participants perform matrix decomposition on their own parameter matrix to obtain p+1 parameter submatrices of the parameter matrix;

[0027] All participants send the parameter submatrix to the corresponding p other participants;

[0028] Each participant accumulates its own parameter submatrix and all received parameter submatrices, then encrypts the resulting encrypted sub-parameter matrix for this iteration and sends it to the server.

[0029] In one alternative implementation, p represents a random positive integer, and p∈[0,n-1], where n represents the total number of participants.

[0030] In one optional implementation, the server updates the total parameter matrix based on the encrypted sub-parameter matrices received from all participants, completing this iteration, including:

[0031] Receive the encrypted sub-parameter matrix sent by all participants;

[0032] The updated total parameter matrix is ​​calculated using the average aggregation algorithm, as shown in the following formula:

[0033]

[0034] In the formula, w t+1 Let n represent the total parameter matrix at the (t+1)th iteration. k This represents the number of samples from the k-th participant. Let N represent the encrypted parameter matrix of the k-th participant at the (t+1)-th iteration, and let N represent the total number of samples from all participants.

[0035] In one optional implementation, after the model training is completed by iterating a preset number of times, the privacy-preserving multi-party data model training method further includes:

[0036] Each participant validates the accuracy of the model.

[0037] In one alternative implementation, each participant performs accuracy verification on the model, including:

[0038] The total parameter matrix of the trained model is then distributed to each participant.

[0039] Each participant loads the total parameter matrix to obtain a trained sub-model;

[0040] Each participant selects a test set based on its stored data and inputs it into the trained sub-model to obtain the predicted value;

[0041] The accuracy is calculated based on the predicted values ​​and the corresponding labels y in the test set, thus completing the validation of the sub-model.

[0042] Secondly, this disclosure also provides a model training apparatus for multi-party data based on privacy protection, the apparatus comprising:

[0043] The server and the local client of each participant communicating with the server;

[0044] The server includes the following modules:

[0045] The distribution module is configured to distribute the total parameter matrix of the model to be trained to each participant's local client.

[0046] The update module is configured to update the total parameter matrix based on the encrypted sub-parameter matrices received from all participants, thus completing this iteration;

[0047] The iteration module is configured to iterate a preset number of times to complete model training;

[0048] The local client includes the following modules:

[0049] The subparameter matrix acquisition module is configured to perform local training on the sub-model based on its own private data and the total parameter matrix obtained from the server to obtain the subparameter matrix.

[0050] The encryption module is configured to encrypt the subparameter matrix and then obtain the encrypted subparameter matrix.

[0051] The sending module is configured to send the encrypted subparameter matrix to the server.

[0052] Thirdly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0053] Fourthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0054] The beneficial effects of this invention are as follows: This privacy-preserving multi-party data model training method and apparatus distributes model training to each participant for local training, and after performing matrix sparsification on the trained parameter matrix, sends the parameters to the server to obtain the total parameter matrix of the model. This reduces the amount of data communicated and alleviates the problem of excessively high communication costs. The use of multi-level matrix encryption increases the security level, ensuring that even if some participants engage in malicious behavior, the original data cannot be reconstructed from the sub-parameter matrix, effectively solving the data security problem. At the same time, each iteration is performed locally, making the data more targeted, thereby ensuring that the predicted ability of the finally trained model is more accurate.

[0055] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0057] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0058] Figure 1 A flowchart of a privacy-preserving multi-party data model training method provided in this disclosure embodiment;

[0059] Figure 2 A schematic diagram of a privacy-preserving multi-party data model training apparatus provided in this disclosure embodiment;

[0060] Figure 3 A partial structural schematic diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] As used herein, the phrases “in one embodiment,” “according to one embodiment,” “in some embodiments,” etc., generally refer to the fact that a particular feature, structure, or characteristic following the phrase can be included in at least one embodiment of this disclosure. Therefore, a particular feature, structure, or characteristic can be included in more than one embodiment of this disclosure, such that these phrases do not necessarily refer to the same embodiment. As used herein, the terms “example,” “exemplary,” etc., are used to “serve as an example, instance, or illustration.” Any implementation, aspect, or design described herein as “example” or “exemplary” is not necessarily to be construed as preferred or superior to other implementations, aspects, or designs. Rather, the use of the terms “example,” “exemplary,” etc., is intended to present concepts in a specific manner.

[0063] To facilitate understanding of this embodiment, a model building method disclosed in this disclosure will first be described in detail. The execution entity of the privacy-preserving multi-party data model training method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this model building method can be implemented by a processor calling computer-readable instructions stored in memory.

[0064] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0065] Example 1

[0066] See Figure 1 In at least one embodiment, the privacy-preserving multi-party data model training method distributes model training locally among the participating parties, performs matrix sparsification on the trained parameter matrix, and then sends the parameters to the server to obtain the total parameter matrix of the model. This reduces the amount of data communicated and alleviates the problem of excessively high communication costs. The use of multi-level matrix encryption increases the security level, ensuring that even if some participating parties engage in malicious behavior, the original data cannot be reconstructed from the sub-parameter matrix, effectively solving the data security problem. At the same time, each iteration is performed locally, making the data more targeted, thereby ensuring that the predicted ability of the finally trained model is more accurate.

[0067] Specifically, the method for training models using privacy-preserving multi-party data includes the following steps:

[0068] S110: The server distributes the total parameter matrix of the model to be trained to each participant.

[0069] S120: Each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the sub-parameter matrix.

[0070] Specifically, step S120 includes the following steps:

[0071] S121: Each participant preprocesses its stored data to obtain the feature matrix X and the corresponding label y of all samples.

[0072] S122: Based on the received total parameter matrix w t The sub-parameters of the sub-model are initialized, where t represents the number of iterations and w t This represents the total parameter matrix at the t-th iteration.

[0073] S123: Forward propagation, calculate the sub-model output based on the feature matrix X and corresponding label y of each sample. The formula is as follows:

[0074]

[0075] In the formula, k represents the k-th participant. Let X represent the predicted label of the k-th participant. k This represents the feature matrix of the k-th participant. Let represent the parameter matrix updated by the k-th participant in the (t+1)-th iteration.

[0076] Specifically, during forward propagation, 80% of all samples are selected as the training set and input into the sub-model.

[0077] S124: Calculate the loss function L k .

[0078] S125: Backpropagation, the formula for updating the local subparameter matrix is ​​as follows:

[0079]

[0080] In the formula, Let η represent the updated subparameter matrix of the k-th participant at the (t+1)-th iteration, and let η represent the learning rate.

[0081] S130: All participants will perform matrix encryption on the sub-parameter matrix, obtain the encrypted sub-parameter matrix, and send it to the server.

[0082] Specifically, step S130 includes the following steps:

[0083] S131: All participants decompose their own subparameter matrices to obtain p+1 parameter submatrices.

[0084] Specifically, the formula for matrix factorization is as follows:

[0085]

[0086] In the formula, This represents the first parameter submatrix after the decomposition of the k-th participant. This represents the (p+1)th parameter submatrix after the decomposition of the k-th participant;

[0087] The initial subparameter matrix can be reconstructed by summing all the decomposed parameter submatrices.

[0088] Where p represents a random positive integer, and p∈[0,n-1], and n represents the total number of participants. Each participant decomposes the sub-parameter matrix into a random number of parameter sub-matrices and sends them randomly to the corresponding participants, thereby improving the confidentiality level and ensuring that even if some participants engage in malicious behavior, the original data cannot be reconstructed from the sub-parameter matrix.

[0089] It should be noted that the specific value of p is different for each participant, and a new random number p needs to be generated for each iteration.

[0090] S132: All participants send the parameter submatrix to the corresponding p remaining participants.

[0091] S133: Each participant accumulates its own parameter submatrix and all received parameter submatrices to complete the encryption, and sends the encrypted sub-parameter matrix obtained in this iteration to the server.

[0092] S140: The server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all participants, completing this iteration.

[0093] Step S140 includes the following steps:

[0094] S141: Receive the encrypted sub-parameter matrix sent by all participants.

[0095] S142: Calculate the updated total parameter matrix based on the average aggregation algorithm, as shown in the following formula:

[0096]

[0097] In the formula, w t+1 Let n represent the total parameter matrix at the (t+1)th iteration. k This represents the number of samples from the k-th participant. Let N represent the encrypted parameter matrix of the k-th participant at the (t+1)-th iteration, and let N represent the total number of samples from all participants.

[0098] S150: Iterate the preset number of times to complete model training.

[0099] S160: Each participant verifies the accuracy of the model.

[0100] Specifically, S160 includes the following steps:

[0101] S161: Distribute the total parameter matrix of the trained model to each participant.

[0102] S162: Each participant loads the total parameter matrix to obtain the trained sub-model.

[0103] S163: Each participant selects a test set based on its stored data and inputs it into the trained sub-model to obtain the predicted value.

[0104] Specifically, the remaining data that was not used in model training is used to form a test set to test the trained sub-model.

[0105] S164: Calculate the accuracy based on the predicted values ​​and the corresponding labels y in the test set, and complete the validation of the sub-model.

[0106] Specifically, in evaluating accuracy, this paper adopts a comparative testing method, applying different models—FedAvg, FedProx, and FedAvg-Match—as well as the model training method proposed in this application (hereinafter referred to as this method)—on the dataset, and comparing the differences between their prediction results and the actual results. The results are shown in Table 1 below.

[0107] Table 1 Comparison of test accuracy of different algorithms on the LFW dataset.

[0108] Dataset Data distribution FedAvg FedProx FedAvg-Match This method LFW IID 85.23％ 86.05％ 88.92％ 91.55％ LFW non-IID 53.89％ 56.53％ 64.69％ 74.23％

[0109] As shown in Table 1, the performance of this method is better than other baseline models on both the IID and non-IID distributions of the dataset, indicating good predictive performance.

[0110] Example 2

[0111] Please see Figure 2This embodiment also provides a privacy-preserving multi-party data model training device, the device including: a server and a local client of each participant communicating with the server.

[0112] The server includes the following modules:

[0113] The distribution module is configured to distribute the total parameter matrix of the model to be trained to each participant's local client. It is used to perform the following steps:

[0114] S110: The server distributes the total parameter matrix of the model to be trained to each participant.

[0115] The update module is configured to update the total parameter matrix based on the encrypted sub-parameter matrices received from all participants, thus completing the current iteration. It is used to perform the following steps:

[0116] S140: The server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all participants, completing this iteration.

[0117] Step S140 includes the following steps:

[0118] S141: Receive the encrypted sub-parameter matrix sent by all participants.

[0119] S142: Calculate the updated total parameter matrix based on the average aggregation algorithm, as shown in the following formula:

[0120]

[0121] In the formula, w t+1 Let n represent the total parameter matrix at the (t+1)th iteration. k This represents the number of samples from the k-th participant. Let N represent the encrypted parameter matrix of the k-th participant at the (t+1)-th iteration, and let N represent the total number of samples from all participants.

[0122] The iteration module, configured to iterate a preset number of times, completes model training. It is used to perform the following steps:

[0123] S150: Iterate the preset number of times to complete model training.

[0124] The local client includes the following modules:

[0125] The parameter sub-parameter matrix acquisition module is configured to perform local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, thereby acquiring the parameter sub-parameter matrix. It is used to perform the following steps:

[0126] S120: Each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the sub-parameter matrix.

[0127] Specifically, step S120 includes the following steps:

[0128] S121: Each participant preprocesses its stored data to obtain the feature matrix X and the corresponding label y of all samples.

[0129] S122: Based on the received total parameter matrix w t The sub-parameters of the sub-model are initialized, where t represents the number of iterations and w t This represents the total parameter matrix at the t-th iteration.

[0130] S123: Forward propagation, calculate the sub-model output based on the feature matrix X and corresponding label y of each sample. The formula is as follows:

[0131]

[0132] In the formula, k represents the k-th participant. Let X represent the predicted label of the k-th participant. k This represents the feature matrix of the k-th participant. Let represent the parameter matrix updated by the k-th participant in the (t+1)-th iteration.

[0133] Specifically, during forward propagation, 80% of all samples are selected as the training set and input into the sub-model.

[0134] S124: Calculate the loss function L k .

[0135] S125: Backpropagation, the formula for updating the local subparameter matrix is ​​as follows:

[0136]

[0137] In the formula, Let η represent the updated subparameter matrix of the k-th participant at the (t+1)-th iteration, and let η represent the learning rate.

[0138] The encryption module is configured to perform matrix encryption on the subparameter matrix and obtain the encrypted subparameter matrix. It is used to perform the following steps:

[0139] S130: All participants will perform matrix encryption on the sub-parameter matrix, obtain the encrypted sub-parameter matrix, and send it to the server.

[0140] Specifically, step S130 includes the following steps:

[0141] S131: All participants decompose their own subparameter matrices to obtain p+1 parameter submatrices.

[0142] Specifically, the formula for matrix factorization is as follows:

[0143]

[0144] In the formula, This represents the first parameter submatrix after the decomposition of the k-th participant. This represents the (p+1)th parameter submatrix after the decomposition of the k-th participant;

[0145] The initial subparameter matrix can be reconstructed by summing all the decomposed parameter submatrices.

[0146] Where p represents a random positive integer, and p∈[0,n-1], and n represents the total number of participants. Each participant decomposes the sub-parameter matrix into a random number of parameter sub-matrices and sends them randomly to the corresponding participants, thereby improving the confidentiality level and ensuring that even if some participants engage in malicious behavior, the original data cannot be reconstructed from the sub-parameter matrix.

[0147] It should be noted that the specific value of p is different for each participant, and a new random number p needs to be generated for each iteration.

[0148] S132: All participants send the parameter submatrix to the corresponding p remaining participants.

[0149] S133: Each participant completes the encryption by summing its own retained parameter submatrix and all received parameter submatrices.

[0150] The sending module is configured to send the encrypted subparameter matrix to the server.

[0151] Example 3

[0152] This embodiment provides a computer-readable storage medium storing a computer program / instructions thereon, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the method provided in Embodiment 1.

[0153] By distributing model training locally among various participants and then performing matrix sparsification on the trained parameter matrix before sending it to the server, the total parameter matrix of the model is obtained. This reduces the amount of data transmitted and alleviates the problem of excessively high communication costs. The use of multi-level matrix encryption increases the security level, ensuring that even if some participants engage in malicious behavior, the original data cannot be reconstructed from the distributed parameter matrix, effectively solving the data security problem. At the same time, each iteration is performed locally, resulting in more targeted data and ensuring that the final trained model has more accurate predictive capabilities.

[0154] Example 4

[0155] Please see Figure 3 This embodiment provides an electronic device, including: a memory 502 and a processor 501; the memory 502 stores at least one program instruction; the processor 501 loads and executes the at least one program instruction to implement the cross-project real-time software defect prediction method as provided in Embodiment 1.

[0156] The memory 502 and processor 501 are connected via a bus, which may include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 501 and memory 502 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 501 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 501.

[0157] Processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 502 can be used to store data used by processor 501 during operation.

[0158] In summary, this invention provides a privacy-preserving multi-party data model training method and apparatus. The privacy-preserving multi-party data model training method distributes model training locally among the participating parties, performs matrix sparsification on the trained parameter matrix, and then sends the parameters to the server to obtain the model's total parameter matrix. This reduces the amount of data communicated, alleviating the problem of excessively high communication costs. The use of multi-level matrix encryption increases the security layer, ensuring that even if some participating parties engage in malicious behavior, the original data cannot be reconstructed from the sub-parameter matrix, effectively solving the data security problem. Furthermore, each iteration is performed locally, resulting in more targeted data and ensuring that the final trained model has more accurate predictive capabilities.

[0159] Furthermore, without departing from the scope of this disclosure, the discrete or individual technologies, systems, subsystems, and methods described and illustrated in the various embodiments may be combined or integrated with other systems, modules, technologies, or methods. Other items shown or discussed as coupled may be directly connected or indirectly coupled or communicated via some interface, device, or intermediate component in an electrical, mechanical, or other manner. Those skilled in the art can identify other examples of changes, substitutions, and modifications without departing from the spirit and scope of this disclosure.

Claims

1. A method for training a model using privacy-preserving multi-party data, characterized in that, The method includes: The server distributes the total parameter matrix of the model to be trained to each participant; Each participant performs local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, and obtains the sub-parameter matrix. All participants will perform matrix encryption on the sub-parameter matrix, obtain the encrypted sub-parameter matrix, and send it to the server; The server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all participants, thus completing this iteration; The model training is completed by iterating a preset number of times. Each participant performs local training on a sub-model based on its own private data and the total parameter matrix obtained from the server. The steps for obtaining the sub-parameter matrix include: Each participant preprocesses its stored data to obtain the feature matrix X and the corresponding label y of all samples; Based on the received total parameter matrix The sub-parameters of the sub-model are initialized, where t represents the number of iterations. This represents the total parameter matrix at the t-th iteration; Forward propagation calculates the sub-model output based on the feature matrix X and corresponding label y of each sample. The formula is as follows: In the formula, k represents the k-th participant. X represents the predicted label of the k-th participant. k This represents the feature matrix of the k-th participant. This represents the parameter matrix updated by the k-th participant in the (t+1)-th iteration; Calculate the loss function L; Backpropagation, the formula for updating the local subparameter matrix is ​​as follows: ； In the formula, This represents the updated parameter matrix of the k-th participant in the (t+1)-th iteration. The learning rate is represented by the following: After all participants encrypt the subparameter matrix, they obtain the encrypted subparameter matrix and send it to the server, including: All participants perform matrix decomposition on their own parameter matrix to obtain p+1 parameter submatrices of the parameter matrix; All participants send the parameter submatrix to the corresponding p other participants; Each participant accumulates its own parameter submatrix and all received parameter submatrices, then encrypts the resulting encrypted sub-parameter matrix for this iteration and sends it to the server.

2. The method for training a model using privacy-preserving multi-party data as described in claim 1, characterized in that, p represents a random positive integer, and p∈[0,n-1], where n represents the total number of participants.

3. The method for training a model using privacy-preserving multi-party data as described in claim 1, characterized in that, The server updates the total parameter matrix based on the encrypted sub-parameter matrices sent by all participants, completing this iteration, including: Receive the encrypted sub-parameter matrix sent by all participants; The updated total parameter matrix is ​​calculated using the average aggregation algorithm, as shown in the following formula: In the formula, This represents the total parameter matrix at the (t+1)th iteration. n k This represents the number of samples from the k-th participant. Let N represent the encrypted parameter matrix of the k-th participant at the (t+1)-th iteration, and let N represent the total number of samples from all participants.

4. The method for training a model using privacy-preserving multi-party data as described in claim 1, characterized in that, After iterating a preset number of times to complete model training, the privacy-preserving multi-party data model training method further includes: Each participant validates the accuracy of the model.

5. The method for training a model using privacy-preserving multi-party data as described in claim 4, characterized in that, Each participant performs accuracy verification on the model, including: The total parameter matrix of the trained model is then distributed to each participant. Each participant loads the total parameter matrix to obtain a trained sub-model; Each participant selects a test set based on its stored data and inputs it into the trained sub-model to obtain the predicted value; The accuracy is calculated based on the predicted values ​​and the corresponding labels y in the test set, thus completing the validation of the sub-model.

6. A model training device for multi-party data based on privacy protection, characterized in that, The device includes: The server and the local client of each participant communicating with the server; The server includes the following modules: The distribution module is configured to distribute the total parameter matrix of the model to be trained to each participant's local client. The update module is configured to update the total parameter matrix based on the encrypted sub-parameter matrices received from all participants, thus completing this iteration; The iteration module is configured to iterate a preset number of times to complete model training; The local client includes the following modules: The parameter sub-parameter matrix acquisition module is configured to perform local training on the sub-model based on its own private data and the total parameter matrix obtained from the server, thereby acquiring the parameter sub-parameter matrix, i.e.: Each participant preprocesses its stored data to obtain the feature matrix X and the corresponding label y of all samples; Based on the received total parameter matrix The sub-parameters of the sub-model are initialized, where t represents the number of iterations. This represents the total parameter matrix at the t-th iteration; Forward propagation calculates the sub-model output based on the feature matrix X and corresponding label y of each sample. The formula is as follows: In the formula, k represents the k-th participant. X represents the predicted label of the k-th participant. k This represents the feature matrix of the k-th participant. This represents the parameter matrix updated by the k-th participant in the (t+1)-th iteration; Calculate the loss function L; Backpropagation, the formula for updating the local subparameter matrix is ​​as follows: ； In the formula, This represents the updated parameter matrix of the k-th participant in the (t+1)-th iteration. The learning rate is represented by the following: After all participants encrypt the subparameter matrix, they obtain the encrypted subparameter matrix and send it to the server, including: All participants perform matrix decomposition on their own parameter matrix to obtain p+1 parameter submatrices of the parameter matrix; All participants send the parameter submatrix to the corresponding p other participants; Each participant will encrypt their own parameter submatrix and all received parameter submatrices by summing them together, and then send the encrypted sub-parameter matrix obtained in this iteration to the server. The encryption module is configured to perform matrix encryption on the subparameter matrix and then obtain the encrypted subparameter matrix. The sending module is configured to send the encrypted subparameter matrix to the server.

7. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the method of claim 1.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the method of claim 1.

Images