Privacy protection model training method and data interaction system

Abstract

The invention provides a privacy protection model training method and a data interaction system, and the training method is applied to the data interaction system comprising a data owning end and a central server. Calculating by using the decryption equation to obtain a target decoding matrix, and sending the target decoding matrix and a first model parameter of the target model in the current training period to a data owning end; the data owning end calculates a real gradient by using a target model based on the first model parameter, calculates an encryption gradient by using an encryption equation, and sends the encryption gradient to the central server; and the central server performs linear mapping calculation by using the encryption gradient and the target decoding matrix to obtain an estimated gradient, calculates a second model parameter of the current period by using the target model, and sends the second model parameter to the data owning end to complete training of the target model in the current training period. Through the method, the safety and the model performance of training the distributed machine learning model are improved.

Description

Technical Field

[0001] This application relates to the field of distributed machine learning technologies, and in particular, to a method for training a privacy protection model and a data interaction system. Background Art

[0002] When training a distributed machine learning model, it is necessary to transmit data distributed on different terminals to a central server. Although user privacy data is not directly transmitted, hackers or third - party eavesdroppers can use cracking techniques to infer the corresponding original data using the gradient data transmitted during the model training process, thereby stealing user privacy and reducing the security of training the distributed machine learning model.

[0003] Currently, the main idea of the privacy protection model is to perform a certain transformation on the gradient data during the training process to conceal the original data or change its data characteristics, thereby rendering malicious cracking techniques ineffective. At the same time, it ensures that non - malicious normal terminals can still learn experience from the gradient data and update the model. When performing privacy protection on model training through methods including differential privacy, adding noise, and sparsification, if the intensity of privacy protection is increased, the accuracy of the model will be reduced. In addition, since the central server can only passively use the received data unidirectionally, the data owner can arbitrarily modify the transmitted data, reducing the security of training the distributed machine learning model and thus reducing the performance of the model. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method for training a privacy protection model and a data interaction system. In the data interaction system, a privacy protection model using the Stackelberg game framework is used to protect the training process of the target model. The central server determines the target decoding matrix through equilibrium analysis based on its understanding of the data owner. The data owner determines the true gradient based on the model parameters of the current training cycle, encrypts the true gradient using the target decoding matrix and sends it to the central server, and the central server decrypts and estimates the encrypted true gradient using the target decoding matrix, and then updates the model parameters to complete the training of the target model in the current training cycle, improving the security of training the distributed machine learning model and thus improving the performance of the target model.

[0005] An embodiment of this application provides a method for training a privacy protection model. The training method is applied to a data interaction system. Among them, the data interaction system includes a data owner and a central server. The privacy protection model is used to protect the training processes of the target models corresponding to the data owner and the central server respectively. The training method includes: The central server calculates a target decoding matrix using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set by the data owner, and sends the target decoding matrix and the first model parameters of the target model in the current training cycle to the data owner; Based on the first model parameters, the data owner calculates the true gradient of the current cycle using the target model corresponding to the data owner, and calculates an encrypted gradient using a preset encryption equation based on the true gradient and the received target decoding matrix, and sends the encrypted gradient to the central server; The central server performs a linear mapping calculation using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient, and calculates the second model parameters of the current cycle using the target model corresponding to the central server based on the estimated gradient, and sends the second model parameters to the data owner to complete the training of the target model in the current training cycle.

[0006] Further, the training method further includes: In each training cycle, the data owner uses the second model parameters received in the previous training cycle as the first model parameters of the current training cycle, calculates the true gradient, calculates the encrypted gradient through the encryption equation, and sends the encrypted gradient to the central server; In each training cycle, the central server calculates the estimated gradient based on the encrypted gradient and the target decoding matrix, updates the second model parameters of the current training cycle, and sends the second model parameters back to the data owner, and loops through the training cycle until a preset training termination condition is met, and outputs the target model parameters of the target model to complete the training of the target model.

[0007] Further, the central server calculates a target decoding matrix using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set by the data owner, including: The central server determines whether the induction matrix is clear based on the degree of recognition of the induction matrix set by the data owner; If the induction matrix is clear, the induction matrix is determined as the estimated induction matrix, and it is determined whether the estimated induction matrix is an invertible matrix. Based on the judgment result of whether the estimated induction matrix is an invertible matrix, a target decoding matrix is calculated using a preset decryption equation; If the induced matrix is not clear, an estimated induced matrix is determined based on the degree of clarity of the induced matrix, and based on the estimated induced matrix, a target decoding matrix is calculated using a preset decryption equation.

[0008] Further, the calculating the target decoding matrix using the preset decryption equation based on the judgment result of whether the estimated induced matrix is an invertible matrix includes: When the judgment result is that the estimated induced matrix is an invertible matrix, the estimated induced matrix is input into the decryption equation, and the target decoding matrix is calculated by an iterative solution method; When the judgment result is that the estimated induced matrix is a non-invertible matrix, the matrix form of the estimated induced matrix is determined, and the decryption equation is solved using the numerical method corresponding to the matrix form to calculate the target decoding matrix.

[0009] Further, the determining the estimated induced matrix based on the degree of clarity of the induced matrix, and calculating the target decoding matrix based on the estimated induced matrix using the preset decryption equation includes: Based on the degree of clarity of the induced matrix, it is judged whether there is a prior knowledge matrix in the central server; If there is a prior knowledge matrix, the prior knowledge matrix is determined as the estimated induced matrix, and it is judged whether the estimated induced matrix is an invertible matrix. Based on the judgment result of whether the estimated induced matrix is an invertible matrix, the target decoding matrix is calculated using the preset decryption equation; If there is no prior knowledge matrix, the identity matrix is determined as the estimated induced matrix, and the estimated induced matrix is input into the decryption equation to calculate the target decoding matrix.

[0010] Further, the encryption equation and the decryption equation are determined through the following steps: Linearization mapping processing is respectively performed on the target decryption function preset in the central server and the target encryption function preset in the data owner to obtain an optimized decryption function corresponding to the target decryption function and an optimized encryption function corresponding to the target encryption function; The optimized encryption function is solved for the optimization conditions of a differentiable convex optimization problem to obtain the encryption equation; The encryption equation is input into the optimized decryption function to obtain the decryption equation.

[0011] The embodiment of the present application further provides a data interaction system, and the data interaction system includes: A central server, which is used to calculate a target decoding matrix by using a preset decryption equation based on an estimated induction matrix representing the degree of cognition of an induction matrix set by a data owner, and send the target decoding matrix and first model parameters of a target model in the current training cycle to the data owner; The data owner is used to calculate a true gradient of the current cycle by using the target model corresponding to the data owner based on the first model parameters, and calculate an encrypted gradient by using a preset encryption equation based on the true gradient and the received target decoding matrix, and send the encrypted gradient to the central server; When receiving the encrypted gradient, the central server is further used to perform a linear mapping calculation by using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient, and calculate second model parameters of the current cycle by using the target model corresponding to the central server based on the estimated gradient, and send the second model parameters to the data owner to complete the training of the target model in the current training cycle.

[0012] Further, the central server includes: An equilibrium estimation module, which is used to calculate a target decoding matrix by using a preset decryption equation based on an estimated induction matrix representing the degree of cognition of an induction matrix set by a data owner, and send the target decoding matrix and first model parameters of a target model in the current training cycle to the data owner; A decryption encoding module, which is used to perform a linear mapping calculation by using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient, and calculate second model parameters of the current cycle by using the target model corresponding to the central server based on the estimated gradient, and send the second model parameters to the data owner to complete the training of the target model in the current training cycle.

[0013] Further, the central server further includes a parameter update module, and the parameter update module is used for: In each training cycle, calculate the estimated gradient based on the encrypted gradient and the target decoding matrix, update to obtain the second model parameters of the current training cycle, and send the second model parameters back to the data owner, and loop through the training cycle until a preset training termination condition is met, and output the target model parameters of the target model to complete the training of the target model.

[0014] Further, when the equilibrium estimation module is used to calculate the target decoding matrix by using a preset decryption equation based on an estimation induction matrix representing the degree of awareness of the induction matrix set by the data owner, the equilibrium estimation module is used for: Based on the degree of awareness of the induction matrix set by the data owner, determine whether the induction matrix is clear; If the induction matrix is clear, determine the induction matrix as the estimation induction matrix, and determine whether the estimation induction matrix is an invertible matrix. Based on the judgment result of whether the estimation induction matrix is an invertible matrix, calculate the target decoding matrix by using a preset decryption equation; If the induction matrix is not clear, determine the estimation induction matrix based on the degree of clarity of the induction matrix, and calculate the target decoding matrix by using a preset decryption equation based on the estimation induction matrix.

[0015] Further, when the equilibrium estimation module is used to calculate the target decoding matrix by using a preset decryption equation based on the judgment result of whether the estimation induction matrix is an invertible matrix, the equilibrium estimation module is used for: When the judgment result is that the estimation induction matrix is an invertible matrix, input the estimation induction matrix into the decryption equation, and calculate the target decoding matrix by means of iterative solution; When the judgment result is that the estimation induction matrix is a non-invertible matrix, determine the matrix form of the estimation induction matrix, and solve the decryption equation by using the numerical method corresponding to the matrix form to calculate the target decoding matrix.

[0016] Further, when the equilibrium estimation module is used to determine the estimation induction matrix based on the degree of clarity of the induction matrix, and calculate the target decoding matrix by using a preset decryption equation based on the estimation induction matrix, the equilibrium estimation module is used for: Based on the degree of clarity of the induction matrix, determine whether there is a prior knowledge matrix in the central server; If there is a prior knowledge matrix, determine the prior knowledge matrix as the estimation induction matrix, and determine whether the estimation induction matrix is an invertible matrix. Based on the judgment result of whether the estimation induction matrix is an invertible matrix, calculate the target decoding matrix by using a preset decryption equation; If there is no prior knowledge matrix, determine the identity matrix as the estimation induction matrix, and input the estimation induction matrix into the decryption equation to calculate the target decoding matrix.

[0017] Further, the data owner includes: An encryption encoding module, configured to calculate a true gradient of the current period by using a target model corresponding to the data owner based on the first model parameter, calculate an encrypted gradient by using a preset encryption equation based on the true gradient and the received target decoding matrix, and send the encrypted gradient to the central server.

[0018] Further, the data owner further includes a gradient encryption module, and the gradient encryption module is configured to: In each training period, use the second model parameter received in the previous training period as the first model parameter of the current training period, calculate the true gradient, calculate the encrypted gradient through the encryption equation, and send the encrypted gradient to the central server.

[0019] An embodiment of the present application further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the training method of the privacy protection model as described above are executed.

[0020] An embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the training method of the privacy protection model as described above are executed.

[0021] The training method for the privacy protection model and the data interaction system provided by the embodiments of the present application. The training method is applied to the data interaction system. Among them, the data interaction system includes a data owner side and a central server side. The privacy protection model is used to protect the privacy of the training processes of the target models corresponding to the data owner side and the central server side respectively. The training method includes: The central server side calculates a target decoding matrix by using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set for the data owner side, and sends the target decoding matrix and the first model parameters of the target model in the current training cycle to the data owner side; The data owner side calculates the true gradient of the current cycle by using the target model corresponding to the data owner side based on the first model parameters, calculates an encrypted gradient by using a preset encryption equation based on the true gradient and the received target decoding matrix, and sends the encrypted gradient to the central server side; The central server side performs a linear mapping calculation by using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient, calculates the second model parameters of the current cycle by using the target model corresponding to the central server side based on the estimated gradient, and sends the second model parameters to the data owner side to complete the training of the target model in the current training cycle.

[0022] Compared with the existing methods for protecting the privacy of model training, such as differential privacy, adding noise, and sparsification, using the privacy protection model with the Stackelberg game framework to protect the privacy of the training process of the target model in the data interaction system. The central server side determines the target decoding matrix through equilibrium analysis based on the degree of recognition of the data owner side. The data owner side determines the true gradient based on the model parameters of the current training cycle, encrypts the true gradient by using the target decoding matrix and sends it to the central server side, and the central server side decrypts and estimates the encrypted true gradient by using the target decoding matrix, and then updates the model parameters to complete the training of the target model in the current training cycle, improving the security of training the distributed machine learning model and thus improving the performance of the target model.

[0023] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specifically gives preferred embodiments and cooperates with the attached drawings for detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below. It should be understood that the following accompanying drawings only show some embodiments of the present application, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, other related accompanying drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 One of the flowcharts of a method for training a privacy protection model provided by an embodiment of the present application; Figure 2 A data interaction schematic diagram of a privacy protection model provided by an embodiment of the present application; Figure 3 Another flowchart of a method for training a privacy protection model provided by an embodiment of the present application; Figure 4 A schematic structural diagram of a data interaction system provided by an embodiment of the present application; Figure 5 One of the schematic structural diagrams of a central server provided by an embodiment of the present application; Figure 6 Another schematic structural diagram of a central server provided by an embodiment of the present application; Figure 7 One of the schematic structural diagrams of a data owner provided by an embodiment of the present application; Figure 8 Another schematic structural diagram of a data owner provided by an embodiment of the present application; Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some, rather than all, of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, every other embodiment obtained by those skilled in the art without creative efforts belongs to the scope of protection of the present application.

[0027] It has been found that currently, the main idea of privacy protection models is to perform a certain transformation on the gradient data during the training process to mask the original data or change its data characteristics, thereby rendering malicious cracking techniques ineffective, while ensuring that non-malicious normal terminals can still learn experience from the gradient data and update the model. When privacy protection is applied to model training through methods such as differential privacy, adding noise, and sparsification, if the intensity of privacy protection is increased, the accuracy of the model will be reduced.

[0028] For example, in an existing privacy protection model training method, perturbation noise is added to the initial output matrix and sparsification is performed to obtain a perturbed output matrix, and the perturbed output matrix is transmitted to the server so that the server can use the model to process the perturbed output matrix to obtain the model prediction result. The privacy protection implemented by this method is one-sided, that is, the owner of the data sample perturbs the sample with noise and performs sparsification, while the server training end can only passively receive the information given by the data owner end.

[0029] In addition, since the central server can only passively use the received data unidirectionally, the data owner can arbitrarily modify the transmitted data, reducing the security of training a distributed machine learning model and thus reducing the performance of the model.

[0030] In previous research, eavesdroppers could reconstruct private training data from publicly shared gradients, posing a serious privacy challenge to distributed machine learning. Specifically, eavesdroppers could eavesdrop on gradients during transmission and use the eavesdropped gradients for sample reconstruction. Although the reconstructed pictures had more noise than those reconstructed using real gradients, the overall content of the data could still be recognized generally.

[0031] In summary, during the training process of a distributed machine learning model, since the data owner needs to interact data of the calculated gradients with the intermediate server, there is a risk that the local privacy and sensitive information of the data owner and other data are leaked to untrusted third parties, reducing the security of data interaction and model training. Therefore, due to potential cybersecurity issues including gradient leakage and sample inference attacks, data owners are reluctant to transmit the real gradients calculated during model training to the central server.

[0032] Based on this, an embodiment of the present application provides a method for training a privacy protection model, which uses a privacy protection model based on the Stackelberg game framework in a data interaction system to protect the privacy of the training process of the target model. The central server determines the target decoding matrix through equilibrium analysis based on its understanding of the data owner, and the data owner determines the true gradient based on the model parameters of the current training cycle, encrypts the true gradient using the target decoding matrix and sends it to the central server, and the central server decrypts and estimates the encrypted true gradient using the target decoding matrix, and then updates the model parameters to complete the training of the target model in the current training cycle, improving the security of training a distributed machine learning model and thus improving the performance of the target model.

[0033] Please refer to Figure 1 , Figure 1 which is one of the flowcharts of a method for training a privacy protection model provided by an embodiment of the present application. As Figure 1 shown in

[0034] In an embodiment of the present application, the privacy protection model is used to protect the privacy of the training processes of the target models respectively corresponding to the data owner and the central server. The privacy protection model may include a privacy protection model based on the Stackelberg game framework, and a better privacy protection balance is achieved through the dynamic game interaction between the central server and the data owner in the process of training the target model.

[0035] Specifically, please refer to Figure 2 , Figure 2 which is a schematic diagram of data interaction of a privacy protection model provided by an embodiment of the present application. As Figure 2 shown in

[0036] Among them, the goal of the central server is to restore the real gradient by decrypting the blurred gradient as accurately as possible. However, in order to protect its own privacy, the data owner does not want the gradient obtained by the central server to deviate too far from the real gradient, resulting in poor model performance. Therefore, the goal of the data owner is to enable the central server to obtain an estimated gradient near the real gradient. The process of the data owner encrypting and encoding the real gradient and sending it to the central server for decryption and decoding can be constructed as a privacy protection model based on the Stackelberg game framework to protect the data transmitted during the data interaction process from being stolen by an insecure third party to infer the original data.

[0037] Based on this, the privacy protection model based on the Stackelberg game framework can include a distributed model of the non - cooperative game theory of a federated learning framework. Among them, the status of the two parties in the data interaction game is not equal, but there is a sequence. In the embodiment of the present application, the central server acts as the leader and has the advantage of taking the first step, while the data owner acts as the follower and observes the actions of the leader before taking its own actions. In the federated learning framework, the central server usually has a higher status.

[0038] In the embodiment of the present application, the central server generally acts as the data core processor, while the data owner generally acts as the distributed data provider; for example, the company headquarters can act as the central server, and the branch companies can act as the data owners; the core server can act as the central server, and the branch servers can act as the data owners.

[0039] For example, the company headquarters usually acts as the central server and cooperates with the branch companies acting as data owners to train the target model. In this scenario, the company headquarters can first formulate a strategy, and the branch companies can adjust their strategies accordingly to form an iterative process. Among them, the head office can consider the feedback strategies of the branch companies to improve its strategy; it is natural and reasonable to model this scenario as a sequential game rather than a simultaneous game in this scenario, that is, constructing a privacy protection model based on the key features of the Stackelberg game framework can solve the privacy protection problem of model training in distributed machine learning.

[0040] Return to refer Figure 1 , such as Figure 1 shown in, the training method includes: S100. The central server calculates a target decoding matrix by using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set for the data owner, and sends the target decoding matrix and the first model parameters of the target model in the current training cycle to the data owner.

[0041] It should be noted that the target model is a distributed machine learning model. A distributed machine learning model refers to the process of training a machine learning model on multiple computing nodes, which can be servers, workstations, or other types of computing resources. The target model adopts data parallelism and model parallelism, that is, different parts of the target model are trained by the central server and the data owner. The training data is divided into multiple small batches. The central server and the data owner process a part of it and perform forward propagation and backpropagation to calculate gradients using the same model parameters, and then synchronously update the model parameters.

[0042] In the embodiment of the present application, the data owner can set an induction matrix based on the training requirements for the target model input externally. The data owner uses the induction matrix to induce the calculated true gradient to obtain an induced gradient. The induced gradient refers to the gradient that the data owner wants the central server to estimate, so as to prevent the true gradient from being deduced after the gradient data is stolen by a third party.

[0043] In this step, in specific implementation, first, the central server determines an estimated induction matrix based on its understanding of the induction matrix set by the data owner; then, based on the estimated induction matrix, a target decoding matrix is calculated using a preset decryption equation; finally, the first model parameters of the target model in the current training cycle are determined, and the target decoding matrix and the first model parameters of the current training cycle are sent to the data owner.

[0044] Here, the first model parameters and the target decoding matrix are non-critical information, and even if stolen by a third party, the private privacy data of the data owner and the central server cannot be obtained. That is, the privacy protection model in the embodiment of the present application protects the transmission of the true gradient calculated by the data owner using the target model to train the target model.

[0045] In the embodiment of the present application, the expression of the preset decryption equation is as follows.

[0046] 。

[0047] Wherein, represents the target decoding matrix; represents a preset regularization parameter; represents the identity matrix; represents the estimated induction matrix.

[0048] In an implementation manner of the present application, in specific implementation, the step of calculating the target decoding matrix using the preset decryption equation based on the estimated induction matrix representing the understanding degree of the induction matrix set by the data owner in step S100 may include: S110. The central server determines whether the induction matrix is clear based on the degree of awareness of the induction matrix set for the data owner.

[0049] In this step, the central server checks its own degree of awareness of the induction matrix set for the data owner and determines whether the induction matrix is clear. That is, when the central server knows the specific information of the induction matrix, it is determined that the induction matrix is clear; when the central server does not understand the specific information of the induction matrix, it is determined that the induction matrix is not clear.

[0050] Here, when the central server is clear about the induction matrix, the central server can accurately obtain the true gradient of the data owner; when the central server is not clear about the induction matrix, it means that the induction matrix is confidential to the central server, and the central server can make an optimal decision among all feasible options; among them, the induction matrix and the estimated induction matrix reflect the degree of mismatch between the two parties in the game, namely the central server and the data owner.

[0051] S120. If the induction matrix is clear, then determine the induction matrix as the estimated induction matrix, and determine whether the estimated induction matrix is an invertible matrix. Based on the judgment result of whether the estimated induction matrix is an invertible matrix, use a preset decryption equation to calculate the target decoding matrix.

[0052] In the embodiment of the present application, when the central server is clear about the induction matrix, it indicates that at this time, a complete information game is in progress between the central server and the data owner. That is, both the central server and the data owner fully understand each other's characteristics, strategy space, cost function and other information. In this case, the central server is fully trustworthy. The main goal of the game problem in this case is to prevent malicious eavesdroppers from eavesdropping on the gradient during the transmission process, rather than caring that sensitive information is known to the central server.

[0053] In this step, when the central server is clear about the induction matrix, determine the induction matrix as the estimated induction matrix, and determine whether the estimated induction matrix is an invertible matrix. Further, for the case of whether the estimated induction matrix is an invertible matrix, use a preset decryption equation to calculate the target decoding matrix.

[0054] In an implementation manner of the present application, in specific implementation, the step of calculating the target decoding matrix using the preset decryption equation based on the judgment result of whether the estimated induction matrix is an invertible matrix in step S120 may include: S121. When the judgment result is that the estimated induction matrix is an invertible matrix, input the estimated induction matrix into the decryption equation and calculate the target decoding matrix by means of iterative solution.

[0055] In this step, when the estimated induction matrix (induction matrix) is an invertible matrix, since the decryption equation is a non-linear matrix equation, there is an unknown parameter in the decryption equation at this time, that is, the target decoding matrix. In this case, an iterative solution method can be used to obtain the numerical structure solution of the target decoding matrix in the decryption equation. For example, the iterative solution method can include Newton's method, etc.

[0056] Here, when the central server determines the induction matrix and the induction matrix is an invertible matrix, this situation applies when the central server is considered reliable enough by the data owner; for example, assuming that the data owner is a branch company and the central server is the company headquarters, the central server can basically perfectly decrypt and encode the true gradient of the data owner, but any eavesdropper can only obtain the encrypted gradient of the data owner, thus ensuring that the true gradient is not leaked.

[0057] S122. When the judgment result is that the estimated induction matrix is a non-invertible matrix, determine the matrix form of the estimated induction matrix, and use the numerical method corresponding to the matrix form to solve the decryption equation, and calculate the target decoding matrix.

[0058] In this step, when the central server determines the induction matrix and the induction matrix is a non-invertible matrix, based on the matrix form of the estimated induction matrix, use the numerical method corresponding to the matrix form to solve the decryption equation, and calculate the target decoding matrix.

[0059] Here, when the induction matrix is non-invertible, it will pose a challenge when solving the non-linear matrix equation (decryption equation) at this time, and numerical methods can be used to find an approximate solution. For example, when the matrix form of the induction matrix is a diagonal matrix, there are rows with all elements being 0 in the induction matrix. At this time, all elements of the corresponding rows of the target decoding matrix can be set to zero, and then the corresponding values of the non-all-zero rows can be solved one by one. At this time, the decryption equation is simplified to a one-dimensional equation that is easy to solve.

[0060] In the embodiments of the present application, when the data owner hopes to hide certain information, the induction matrix can be set as a non-invertible matrix. Even if the induction matrix is known to the central server, an exact value cannot be estimated. Therefore, the privacy of the data owner can still be protected to a certain extent.

[0061] S130. If the induction matrix is not determined, based on the degree of determination of the induction matrix, determine the estimated induction matrix, and based on the estimated induction matrix, use the preset decryption equation to calculate the target decoding matrix.

[0062] In an embodiment of the present application, when the central server is not clear about the induction matrix, it indicates that a non-complete information game exists between the central server and the data owner at this time. That is, the induction matrix set by the data owner is confidential to the central server. At this time, the degree of the central server's understanding of the induction matrix, that is, the situation of determining the estimated induction matrix, may include that the central server has no information about the induction matrix at all and there is a prior knowledge matrix about the induction matrix in the central server.

[0063] Here, the situation where the central server is not clear about the induction matrix applies to an "honest but curious" central server, that is, the central server will not maliciously use the received information, but will try to learn some information from the received information.

[0064] In an implementation manner of the present application, in specific implementation, the step of determining the estimated induction matrix based on the degree of clarity about the induction matrix in step S130 and calculating the target decoding matrix using a preset decryption equation based on the estimated induction matrix may include: S131. Based on the degree of clarity about the induction matrix, determine whether there is a prior knowledge matrix in the central server.

[0065] In this step, based on the degree of clarity of the central server about the induction matrix, determine whether the data owner has revealed information about the induction matrix to judge whether there is a prior knowledge matrix in the central server.

[0066] S132. If there is a prior knowledge matrix, determine the prior knowledge matrix as the estimated induction matrix, and judge whether the estimated induction matrix is an invertible matrix. Based on the judgment result of whether the estimated induction matrix is an invertible matrix, calculate the target decoding matrix using a preset decryption equation.

[0067] In this step, when there is a prior knowledge matrix in the central server, that is, the data owner has revealed relevant information about the induction matrix to the central server and deduced the prior knowledge matrix, and then determine the prior knowledge matrix as the estimated induction matrix, and judge whether the estimated induction matrix is an invertible matrix. Based on the judgment result of whether the estimated induction matrix is an invertible matrix, calculate the target decoding matrix using the decryption equation.

[0068] Among them, the description of calculating the target decoding matrix in step S132 can refer to the description of S121 to S122 and can achieve the same technical effect, which will not be elaborated here.

[0069] For example, when the data owner is a branch company and the central server is the company headquarters, the branch company is reluctant to disclose the real induction matrix, but can directly tell the company headquarters the deviation from the induction matrix. In this case, the balance of the game is adjusted by this deviation, and then the prior knowledge matrix in the central server (company headquarters) can be deduced.

[0070] S133. If there is no prior knowledge matrix, the identity matrix is determined as the estimated induction matrix, and the estimated induction matrix is input into the decryption equation to calculate the target decoding matrix.

[0071] In this step, when there is no prior knowledge matrix in the central server, that is, the data owner does not disclose the existence of the induction matrix at all. At this time, the central server determines the identity matrix as the estimated induction matrix and inputs the estimated induction matrix into the decryption equation to calculate the target decoding matrix; among them, the unique solution of the target decoding matrix is the identity matrix.

[0072] Here, the encrypted gradient received by the central server is already optimal because the central server does not know additional information. Therefore, the central server cannot do more operations to obtain a more optimal target decoding matrix. In this case, the optimal information that both the eavesdropper and the central server can obtain is the encrypted gradient of the data owner.

[0073] S200. The data owner calculates the real gradient of the current cycle using the target model corresponding to the data owner based on the first model parameter, and calculates the encrypted gradient using the preset encryption equation based on the real gradient and the received target decoding matrix, and sends the encrypted gradient to the central server.

[0074] In the embodiment of the present application, the data owner can calculate the real gradient of the current cycle using the target model corresponding to the data owner based on the first model parameter returned by the central server in the current training cycle; among them, the data owner independently trains the target model based on its own local dataset and calculates the corresponding real gradient. The real gradient is the error change rate of the parameters of the current target model according to the local data, guiding how to adjust the parameters of the target model to reduce the loss function.

[0075] In this step, the data owner inputs the calculated real gradient of the current training cycle and the target decoding matrix sent by the central server into the preset encryption equation to calculate the encrypted gradient, and sends the encrypted gradient to the central server.

[0076] In an implementation manner of the present application, in specific implementation, the encryption equation and the decryption equation are determined through the following steps: Step A: Perform linearization mapping processing on the target decryption function preset in the central server and the target encryption function preset in the data owner side respectively, to obtain the optimized decryption function corresponding to the target decryption function and the optimized encryption function corresponding to the target encryption function.

[0077] It should be noted that the goal of the central server is to obtain an estimated gradient as accurate as possible based on the encrypted gradient, while the data owner side encrypts the true gradient into an encrypted gradient to deliberately induce the estimated gradient to be close to the induced gradient that the data owner side expects the central server to estimate, and at the same time maintain a relatively low encryption and decryption cost.

[0078] Here, regardless of the degree and method of encryption of the true gradient by the data owner side, the goal of the central server is that the closer the estimated gradient estimated from the encrypted gradient is to the true gradient, the better. Then the target encryption function of the central server can be written as a minimization optimization problem, and the expression of the target decryption function is as follows.

[0079] 。

[0080] Among them, represents the true gradient calculated by the data owner side; represents the estimated gradient estimated by the central server from the encrypted gradient.

[0081] Furthermore, the data owner side wants to find a better encryption form so that the estimated gradient estimated by the central server is close to the induced gradient designed by the data owner side, and at the same time maintain a relatively low encryption cost. Then the target encryption function of the data owner side can also be written as a minimization optimization problem, and the expression of the target encryption function is as follows.

[0082] 。

[0083] Among them, represents the induced gradient that the data owner side expects the central server to estimate; represents the encrypted gradient obtained by the data owner side encrypting the true gradient; represents the true gradient calculated by the data owner side; represents the estimated gradient estimated by the central server from the encrypted gradient.

[0084] Here, the target encryption function is a minimization problem with regularization, is the regularization term, used to penalize the deviation between the encrypted gradient and the true gradient. Among them, is the preset regularization parameter, used to control the weight of the regularization term.

[0085] In the embodiments of the present application, when the central server estimates and solves the encrypted gradient sent by the data owner, it needs to use the decoding matrix for calculation. That is, the estimated gradient estimated by the central server for the encrypted gradient can be defined by the linear mapping of the decoding matrix with the corresponding dimension and the encrypted gradient sent by the data owner. The expression of the estimated gradient is as follows.

[0086] 。

[0087] Among them, represents the decoding matrix; represents the estimated gradient estimated by the central server for the encrypted gradient; represents the encrypted gradient obtained by the data owner encrypting the true gradient.

[0088] Furthermore, the data owner needs to induce the true gradient using the induction matrix to determine the induced gradient that it expects the central server to estimate. The expression of the induced gradient is as follows.

[0089] 。

[0090] Among them, represents the induction matrix; represents the induced gradient that the data owner expects the central server to estimate; represents the true gradient calculated by the data owner.

[0091] In this way, based on the expressions of the estimated gradient and the induced gradient, the target decryption function and the target encryption function are converted to be related to the decoding matrix and the induction matrix. Then, the game strategies of the central server and the data owner are converted to solve the optimal target decoding matrix and the encrypted gradient. That is, the interaction between the central server and the data owner can be represented by the following two-layer optimization problem, namely, the optimized decryption function corresponding to the target decryption function and the optimized encryption function corresponding to the target encryption function.

[0092] Here, the expression of the optimized decryption function is as follows.

[0093] 。

[0094] Among them, represents the decoding matrix; represents the encrypted gradient under the target decoding matrix; represents the true gradient.

[0095] Furthermore, the expression of the optimized encryption function is as follows.

[0096] 。

[0097] Among them, denotes the induction matrix; denotes the decoding matrix; denotes the encrypted gradient under the target decoding matrix; denotes the true gradient; denotes the encrypted gradient; denotes a preset regularization parameter.

[0098] Furthermore, the game between the central server and the data owner is shown in the following table. Among them, the central server is the leader, and the data owner is the follower. There is a coupling relationship between the two parties. Specifically, the two-layer optimization problem minimizes its estimated loss from the perspective of the central server (i.e., optimizing the decryption function), and at the same time, it is also necessary to satisfy the constraint condition of the data owner encrypting the true gradient (i.e., optimizing the encryption function).

[0099]

[0100] Among them, denotes the induction matrix; denotes the decoding matrix; denotes the encrypted gradient under the target decoding matrix; denotes the true gradient; denotes the encrypted gradient; denotes a preset regularization parameter; denotes the estimated induction matrix.

[0101] Step B: Solve the optimization conditions of the differentiable convex optimization problem for the optimized encryption function to obtain the encryption equation.

[0102] In this step, solve the optimization conditions of the differentiable convex optimization problem for the optimized encryption function to obtain the optimized coding form of the data owner, that is, determine the encryption equation.

[0103] Here, the expression of the encryption equation is as follows.

[0104] .

[0105] Among them, denotes the induction matrix; denotes the decoding matrix; denotes the encrypted gradient under the target decoding matrix; denotes the true gradient; denotes a preset regularization parameter; is the identity matrix.

[0106] In this way, in the encryption equation, given a target decoding matrix, there is a corresponding encryption gradient under the target decoding matrix. That is, if the optimal target decoding matrix can be obtained and input into the encryption equation, the optimal encryption gradient under the game can be obtained.

[0107] Step C: Input the encryption equation into the optimization decryption function to obtain the decryption equation.

[0108] In this step, when the encryption equation is input into the optimization decryption function to obtain the decryption equation, the expression of the decryption equation is as follows.

[0109] 。

[0110] Where, represents the target decoding matrix; represents the preset regularization parameter; represents the identity matrix; represents the estimated induction matrix.

[0111] Here, the induction matrix is replaced by the estimated induction matrix because although the data owner encrypts using the induction matrix, the central server may not necessarily know the induction matrix clearly. Therefore, the estimated induction matrix is used in the encryption equation instead. The relationship between the estimated induction matrix and the induction matrix can be determined according to the specific actual situation.

[0112] S300: The central server performs a linear mapping calculation using the received encryption gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient. Based on the estimated gradient, the central server calculates the second model parameter of the current cycle using the corresponding target model of the central server, and sends the second model parameter to the data owner to complete the training of the target model in the current training cycle.

[0113] In this step, in specific implementation, first, the central server performs a linear mapping calculation on the received encryption gradient and the target decoding matrix to determine the estimated gradient by decrypting and estimating the true gradient; then, based on the estimated gradient, the central server calculates the second model parameter of the current cycle using the gradient descent method with the corresponding target model of the central server; finally, the second model parameter is sent to the data owner to complete the training of the target model in the current training cycle.

[0114] Optionally, please refer to Figure 3 , Figure 3 which is the second flowchart of a training method for a privacy protection model provided by an embodiment of the present application. As shown in Figure 3As shown, in addition to the training method of the privacy protection model described in steps S100 to S300, the embodiment of the present application further includes steps S400 and S500. Specifically, steps S400 and S500 are used to illustrate the training method for performing multiple training cycles on the target model to improve the performance of the target model.

[0115] S400. In each training cycle, the data owner uses the second model parameters received in the previous training cycle as the first model parameters in the current training cycle, calculates the true gradient, calculates the encrypted gradient through the encryption equation, and sends the encrypted gradient to the central server.

[0116] S500. In each training cycle, the central server calculates the estimated gradient based on the encrypted gradient and the target decoding matrix, updates to obtain the second model parameters in the current training cycle, and sends the second model parameters back to the data owner. The training cycle is repeatedly executed until a preset training termination condition is met, and the target model parameters of the target model are output to complete the training of the target model.

[0117] Among them, the descriptions of S400 to S500 can refer to the descriptions of S200 to S300 and can achieve the same technical effects, which will not be elaborated here.

[0118] In the embodiment of the present application, the preset training termination condition may include but is not limited to that the number of training cycles reaches a preset quantity and the target model parameters show convergence, etc.

[0119] The training method of the privacy protection model provided by the embodiment of the present application uses the privacy protection model of the Stackelberg game framework to protect the privacy of the training process of the target model in the data interaction system. The central server determines the target decoding matrix through equilibrium analysis based on the understanding of the data owner. The data owner determines the true gradient based on the model parameters of the current training cycle, encrypts the true gradient using the target decoding matrix and sends it to the central server, and the central server decrypts and estimates the encrypted true gradient using the target decoding matrix, and then updates the model parameters to complete the training of the target model in the current training cycle, improving the security of training the distributed machine learning model and thus improving the performance of the target model.

[0120] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a data interaction system provided by the embodiment of the present application. As Figure 4 shown, the data interaction system 10 includes: The central server 110 is configured to calculate a target decoding matrix using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set by the data owner 120, and send the target decoding matrix and the first model parameters of the target model in the current training cycle to the data owner 120; The data owner 120 is configured to calculate the true gradient of the current cycle using the target model corresponding to the data owner 120 based on the first model parameters, and calculate an encrypted gradient using a preset encryption equation based on the true gradient and the received target decoding matrix, and send the encrypted gradient to the central server 110; When receiving the encrypted gradient, the central server 110 is further configured to perform a linear mapping calculation using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient, and calculate the second model parameters of the current cycle using the target model corresponding to the central server 110 based on the estimated gradient, and send the second model parameters to the data owner 120 to complete the training of the target model in the current training cycle.

[0121] Further, please refer to Figure 5 、 Figure 6 , Figure 5 which is one of the schematic structural diagrams of a central server provided by an embodiment of the present application; Figure 6 which is another schematic structural diagram of a central server provided by an embodiment of the present application. As Figure 5 shown in The balanced estimation module 111 is configured to calculate a target decoding matrix using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set by the data owner, and send the target decoding matrix and the first model parameters of the target model in the current training cycle to the data owner; The decryption and encoding module 112 is configured to perform a linear mapping calculation using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing the decryption estimation of the true gradient, and calculate the second model parameters of the current cycle using the target model corresponding to the central server based on the estimated gradient, and send the second model parameters to the data owner to complete the training of the target model in the current training cycle.

[0122] Further, as Figure 6 shown in In each training cycle, based on the encrypted gradient and the target decoding matrix, calculate the estimated gradient and update the second model parameters of the current training cycle, and transmit the second model parameters back to the data owner. Loop through the training cycle until a preset training termination condition is met, and output the target model parameters of the target model to complete the training of the target model.

[0123] Further, when the equilibrium estimation module 111 is used to calculate the target decoding matrix by using a preset decryption equation based on the estimation induction matrix representing the degree of awareness of the induction matrix set for the data owner, the equilibrium estimation module 111 is used for: Based on the degree of awareness of the induction matrix set for the data owner, determine whether the induction matrix is clear; If the induction matrix is clear, determine the induction matrix as the estimation induction matrix, and determine whether the estimation induction matrix is an invertible matrix. Based on the judgment result of whether the estimation induction matrix is an invertible matrix, calculate the target decoding matrix by using a preset decryption equation; If the induction matrix is not clear, determine the estimation induction matrix based on the degree of clarity of the induction matrix, and calculate the target decoding matrix by using a preset decryption equation based on the estimation induction matrix.

[0124] Further, when the equilibrium estimation module 111 is used to calculate the target decoding matrix by using a preset decryption equation based on the judgment result of whether the estimation induction matrix is an invertible matrix, the equilibrium estimation module 111 is used for: When the judgment result is that the estimation induction matrix is an invertible matrix, input the estimation induction matrix into the decryption equation, and calculate the target decoding matrix by means of iterative solution; When the judgment result is that the estimation induction matrix is a non-invertible matrix, determine the matrix form of the estimation induction matrix, and solve the decryption equation by using the numerical method corresponding to the matrix form to calculate the target decoding matrix.

[0125] Further, when the equilibrium estimation module 111 is used to determine the estimation induction matrix based on the degree of clarity of the induction matrix, and calculate the target decoding matrix by using a preset decryption equation based on the estimation induction matrix, the equilibrium estimation module 111 is used for: Based on the degree of clarity of the induction matrix, determine whether there is a prior knowledge matrix in the central server; If there is a prior knowledge matrix, determine the prior knowledge matrix as the estimation induction matrix, and determine whether the estimation induction matrix is an invertible matrix. Based on the judgment result of whether the estimation induction matrix is an invertible matrix, calculate the target decoding matrix using a preset decryption equation; If there is no prior knowledge matrix, determine the identity matrix as the estimation induction matrix, and input the estimation induction matrix into the decryption equation to calculate the target decoding matrix.

[0126] Further, please refer to Figure 7 、 Figure 8 , Figure 7 which is one of the schematic structural diagrams of a data owner provided by an embodiment of the present application; Figure 8 which is the second of the schematic structural diagrams of a data owner provided by an embodiment of the present application. As shown in Figure 7 , the data owner 120 includes: An encryption and encoding module 121, configured to calculate the true gradient of the current period using the target model corresponding to the data owner based on the first model parameter, and calculate the encrypted gradient using a preset encryption equation based on the true gradient and the received target decoding matrix, and send the encrypted gradient to the central server.

[0127] Further, as shown in Figure 8 , the data owner 120 further includes a gradient encryption module 122, and the gradient encryption module 122 is configured to: In each training period, use the second model parameter received in the previous training period as the first model parameter of the current training period, calculate the true gradient and calculate the encrypted gradient through the encryption equation, and send the encrypted gradient to the central server.

[0128] The data interaction system provided by the embodiment of the present application uses a privacy protection model of the Stackelberg game framework in the data interaction system to protect the privacy of the training process of the target model. The central server determines the target decoding matrix through equilibrium analysis based on the understanding of the data owner. The data owner determines the true gradient based on the model parameters of the current training period, encrypts the true gradient using the target decoding matrix and sends it to the central server, and the central server decrypts and estimates the encrypted true gradient using the target decoding matrix, and then updates the model parameters to complete the training of the target model in the current training period, improving the security of training the distributed machine learning model, and thus improving the performance of the target model.

[0129] Please refer to Figure 9 , Figure 9 which is the schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown inFigure 9 As shown in Figure 9 , the electronic device 900 includes a processor 910, a memory 920, and a bus 930.

[0130] The memory 920 stores machine-readable instructions executable by the processor 910. When the electronic device 900 runs, the processor 910 communicates with the memory 920 via the bus 930. When the machine-readable instructions are executed by the processor 910, they can perform the steps of the training method of the privacy protection model in the method embodiments as described above Figure 1 and Figure 3 as shown. For the specific implementation manners, reference may be made to the method embodiments and will not be elaborated herein.

[0131] The embodiments of the present application further provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, it can perform the steps of the training method of the privacy protection model in the method embodiments as described above Figure 1 and Figure 3 as shown. For the specific implementation manners, reference may be made to the method embodiments and will not be elaborated herein.

[0132] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0133] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division manners in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed with each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0136] When the above-mentioned functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0137] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of this application, used to illustrate the technical solution of this application, rather than limiting it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed in this application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for training a privacy-preserving model, characterized in that: The training method is applied to a data interaction system; wherein the data interaction system includes a data owner and a central server, the privacy protection model is used to perform privacy protection on the training process of the target model corresponding to the data owner and the central server respectively, and the training method includes: The central service end calculates a target decoding matrix using a preset decryption equation based on an estimated induction matrix representing the degree of cognition of the induction matrix set by the data owner, and sends the target decoding matrix and the first model parameter of the target model in the current training cycle to the data owner; The data owner calculates the real gradient of the current cycle based on the first model parameter and the target model corresponding to the data owner, calculates the encrypted gradient based on the real gradient and the received target decoding matrix using a preset encryption equation, and sends the encrypted gradient to the central server; The central server performs a linear mapping calculation using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing a decrypted estimate of the true gradient, and based on the estimated gradient, calculates the second model parameters of the current cycle using the target model corresponding to the central server, and sends the second model parameters to the data owner to complete the training of the target model in the current training cycle.

2. The method according to claim 1, characterized in that The training method further comprises: In each training cycle, the data owner calculates the true gradient based on the second model parameter received in the previous training cycle as the first model parameter of the current training cycle, calculates the encrypted gradient through the encryption equation, and sends the encrypted gradient to the central server; In each training cycle, the central server calculates the estimated gradient and updates the second model parameters of the current training cycle based on the encrypted gradient and the target decoding matrix, and transmits the second model parameters back to the data owner, and loops the training cycle until a preset training termination condition is met, outputting the target model parameters of the target model to complete the training of the target model.

3. The method according to claim 1, characterized in that The central server calculates a target decoding matrix using a preset decryption equation based on an estimated induction matrix representing the degree of recognition of the induction matrix set by the data owner, including: The central service end determines whether to clarify the induction matrix based on the degree of recognition of the induction matrix set by the data owner end; If the induced matrix is ​​clear, the induced matrix is ​​determined as an estimated induced matrix, and it is determined whether the estimated induced matrix is ​​a reversible matrix. Based on the determination result of whether the estimated induced matrix is ​​a reversible matrix, a target decoding matrix is ​​calculated using a preset decryption equation; If the induced matrix is ​​not clear, an estimated induced matrix is ​​determined based on the clarity of the induced matrix, and a target decoding matrix is ​​calculated using a preset decryption equation based on the estimated induced matrix.

4. The method according to claim 3, characterized in that The method of calculating a target decoding matrix based on a result of judging whether the estimated induced matrix is ​​a reversible matrix by using a preset decryption equation comprises: When the judgment result is that the estimated induced matrix is ​​a reversible matrix, the estimated induced matrix is ​​input into the decryption equation, and a target decoding matrix is ​​calculated by iterative solution; When the judgment result is that the estimated induced matrix is ​​an irreversible matrix, the matrix form of the estimated induced matrix is ​​determined, and the decryption equation is solved using a numerical method corresponding to the matrix form to calculate the target decoding matrix.

5. The method according to claim 3, characterized in that: The step of determining an estimated induced matrix based on the clarity of the induced matrix, and calculating a target decoding matrix based on the estimated induced matrix using a preset decryption equation includes: Based on the clarity of the induced matrix, determining whether there is a priori knowledge matrix in the central server; If there is a priori knowledge matrix, the priori knowledge matrix is ​​determined as the estimated induced matrix, and it is determined whether the estimated induced matrix is ​​a reversible matrix. Based on the determination result of whether the estimated induced matrix is ​​a reversible matrix, a target decoding matrix is ​​calculated using a preset decryption equation; If there is no prior knowledge matrix, the identity matrix is ​​determined as the estimated induced matrix, and the estimated induced matrix is ​​input into the decryption equation to calculate the target decoding matrix.

6. The method according to claim 1, characterized in that The encryption equation and the decryption equation are determined by the following steps: Performing linear mapping processing on the target decryption function preset by the central server and the target encryption function preset by the data owner, respectively, to obtain an optimized decryption function corresponding to the target decryption function and an optimized encryption function corresponding to the target encryption function; Solving the optimization conditions of the differentiable convex optimization problem on the optimized encryption function to obtain the encryption equation; The encryption equation is input into the optimized decryption function to obtain the decryption equation.

7. A data interaction system, characterized in that: The data interaction system comprises: The central service end is used to calculate a target decoding matrix using a preset decryption equation based on an estimated induced matrix representing the degree of cognition of the induced matrix set by the data owner, and send the target decoding matrix and the first model parameter of the target model in the current training cycle to the data owner; The data owner is used to calculate the real gradient of the current cycle based on the first model parameter and the target model corresponding to the data owner, and to calculate the encrypted gradient based on the real gradient and the received target decoding matrix using a preset encryption equation, and send the encrypted gradient to the central server; When the central server receives the encrypted gradient, the central server is further used to perform a linear mapping calculation using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing a decrypted estimate of the true gradient, and based on the estimated gradient, calculate the second model parameters of the current cycle using the target model corresponding to the central server, and send the second model parameters to the data owner to complete the training of the target model in the current training cycle.

8. The system according to claim 7, characterized in that The central server includes: A balanced estimation module, configured to calculate a target decoding matrix using a preset decryption equation based on an estimated induced matrix representing a degree of cognition of the induced matrix set by the data owner, and send the target decoding matrix and a first model parameter of the target model in the current training cycle to the data owner; The decryption encoding module is used to perform a linear mapping calculation using the received encrypted gradient and the target decoding matrix to obtain an estimated gradient representing a decrypted estimate of the true gradient, and based on the estimated gradient, calculate the second model parameters of the current cycle using the target model corresponding to the central server, and send the second model parameters to the data owner to complete the training of the target model in the current training cycle.

9. The system according to claim 7, characterized in that The data owner terminal includes: The encryption coding module is used to calculate the real gradient of the current cycle based on the first model parameters using the target model corresponding to the data owner, and to calculate the encrypted gradient based on the real gradient and the received target decoding matrix using a preset encryption equation, and send the encrypted gradient to the central server.

10. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. When the machine-readable instructions are run by the processor, the steps of the training method of the privacy protection model as described in any one of claims 1 to 6 are executed.

Images