Federated learning method and system based on large model computing power network image privacy protection

Abstract

The present invention discloses a federated learning method and system based on image privacy protection of a large model computing network, obtains node information of an image computing network and node processing image data information, divides nodes into passive nodes and active nodes according to the node information and node processing image data information, and prevents active nodes from inferring feature information of passive nodes through intermediate results of passive nodes and global module gradients of passive nodes by encrypting and protecting the intermediate results of passive nodes and global module gradients of passive nodes; and prevents passive nodes from inferring labels of active nodes through the gradients of total outputs of active nodes by encrypting and protecting the gradients of total outputs of active nodes. The present invention can solve the technical problems of malicious passive nodes stealing labels of active nodes through global modules or gradients and active nodes stealing features of passive nodes through the outputs of global modules of passive nodes.

Description

Technical Field

[0001] The present invention relates to a federated learning method and system for privacy protection of computing network images based on large model distributed training, belonging to the technical field of network information security and data privacy protection. Background Art

[0002] In 2016, Google proposed a distributed learning paradigm, federated learning, which enables multiple nodes and servers to collaboratively train a model. Federated learning provides guidance for solving data and training problems for large models through distributed training. In addition, a new network paradigm, computing power network, has been proposed, which provides a basic platform for solving computing power problems in large model training, fine-tuning and reasoning.

[0003] The computing network connects multiple organizations with cloud servers or edge servers through the network. The computing network uses an efficient computing scheduling algorithm to achieve the sharing of computing resources between independent and cross-domain organizations, while providing users with ubiquitous computing services. Federated learning provides a distributed machine learning method for multiple participants who do not trust each other. Federated learning can be divided into horizontal federated learning (HFL), vertical federated learning (VFL) and federated transfer learning (FTL) according to the way of data division. Horizontal federated learning uses data samples as the division method. Each node has the same feature space but different samples. Common training methods include FedAvg and FedSGD. Vertical federated learning uses data features as the division method. Each node has the same data sample, but each data sample will have different features, and some nodes may not have labels corresponding to the samples. Common training methods include FedBCD and Flex-VFL.

[0004] However, there are some security risks in vertical federated learning. For example, there are attack methods that use gradients to recover labels, which distinguish labels by calculating the cosine distance of the gradients between different samples and the norm of the gradients; direct label inference attacks, passive label inference attacks completed by the model, and active label inference attacks by malicious local optimizers. All three attacks exploit the gradients of the output results and the labels owned by the malicious nodes; and use model inversion attacks to infer feature data privacy. Specifically, this method uses a regularized maximum likelihood estimation method to recover the inference data under white-box data, and also uses the inversion network technology to recover the inference data under the black-box setting, and finally uses a query-free shadow model reconstruction technology to recover the inference data under the query-free setting. Therefore, the security issues of vertical federated learning prevent the application of vertical federated learning in computing power networks. Summary of the invention

[0005] Purpose of the invention: In order to solve the technical problem of node privacy leakage caused by passive nodes stealing active node labels and active nodes stealing passive node features during image processing, the present invention provides a federated learning method for image privacy protection based on large model computing power network, which solves the technical problem of node privacy leakage caused by passive nodes stealing active node labels and active nodes stealing passive node features.

[0006] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:

[0007] A federated learning method based on image privacy protection of a large model computing network obtains node information of an image computing network and information about image data processed by the nodes, wherein the node information includes image features and label information owned by the nodes. The nodes are divided into passive nodes and active nodes according to the node information and the information about image data processed by the nodes. The compact image features and global module parameter gradients extracted from the passive nodes are encrypted and protected to prevent the active nodes from inferring the feature information of the passive nodes through the compact image features of the passive nodes and the global module parameter gradients of the passive nodes. The prediction result gradients of the active nodes are encrypted and protected to prevent the passive nodes from inferring the labels of the active nodes through the prediction result gradients of the active nodes. Specifically, the following steps are included:

[0008] Step S1, obtaining the node information of the image computing network and the node processing image data information.

[0009] Step S2, according to the node information and the node processing image data information, the nodes are divided into passive nodes and active nodes, wherein the active node includes a first global module, a first feature extraction model and a first loss function. The passive node includes a second global module and a second feature extraction model. The passive node randomly initializes the second feature extraction model and part of the second global module parameters, the active node initializes the first feature extraction model, the first global module and the remaining second global module parameters of the passive node, the second global module parameters randomly initialized by the passive node are recorded as the second global module parameters two, and the active node initializes the remaining second global module parameters of the passive node as the second global module parameters one.

[0010] Step S3, the passive node uses the original image features it owns to input into the second feature extraction model to obtain a compact image feature. The passive node uses the compact image feature and the encrypted second global module parameter one to calculate the homomorphic encryption partial prediction result, and then converts the homomorphic encryption partial prediction result into a secret sharing prediction result. The passive node then uses a single mask encryption algorithm to encrypt the secret sharing prediction result to obtain an encrypted secret sharing prediction result. The passive node sends the pseudo-random number vector generated for single mask encryption to a trusted third party for aggregation. The passive node then uses the compact image feature and the second global module parameter two to calculate the partial prediction result, and then adds the partial prediction result to the vector generated when the homomorphic encryption partial prediction result is converted into a secret sharing prediction result to obtain the perturbed partial prediction result. Then, the passive node sends the encrypted secret sharing prediction result and the perturbed partial prediction result to the active node.

[0011] Step S4: The active node uses the original image features to input the first feature extraction model parameters to obtain compact image features. The active node uses the compact image features and the first global module parameters to calculate the image prediction result, and then uses the HE2SS algorithm to decrypt the passive node. The encrypted secret shared prediction results sent are used to obtain the secret shared prediction results. The active node aggregates the image prediction results, secret shared prediction results, perturbed part prediction results and the results sent by the trusted third party to obtain the total prediction results, and then uses the total prediction results to calculate the loss.

[0012] Step S5, the active node calculates the total prediction result The gradient with respect to the loss is then encrypted and sent to all passive nodes.

[0013] In step S6, the passive node uses the encrypted gradient and the compact image feature to calculate the encrypted gradient of the second global module parameter one, and then the passive node converts the encrypted gradient into a secret shared gradient and sends it to the active node.

[0014] Step S7, the passive node encrypts the second global module parameter 2 to obtain the encrypted second global module parameter 2 and sends it to the active node. The active node uses the second global module parameter 1 to add the encrypted second global module parameter 2 to obtain the encrypted global module parameter. The active node uses the total prediction result gradient and the encrypted global module parameter to calculate the encrypted compact image feature gradient of the passive node. The active node sends the encrypted compact image feature gradient to the passive node.

[0015] In step S8, the active node uses the HE2SS algorithm to decrypt the secret shared gradient sent by the passive node and uses the compact image feature and the total prediction result gradient to calculate the gradient of the global module parameter. Then the active node uses the total prediction result gradient and the global module parameter to obtain the gradient of the compact image feature.

[0016] Step S9, the passive node uses the vector generated by calling the HE2SS algorithm in step S8 to update the second global module parameter 2 to obtain a new second global module parameter 2. The active node uses the decrypted secret shared gradient to update the second global module parameter 1 to obtain a new second global module parameter 1. At the same time, the active node uses the global module parameter gradient to update the global module parameter. Then the active node uses the compact image feature gradient to back propagate and update the first feature extraction model parameters. Finally, the active node encrypts the new second global module parameter 1 to obtain the encrypted new second global module parameter 1 and sends it to the passive node. The passive node uses the compact image feature gradient to back propagate and update the second feature extraction model parameters.

[0017] Step S10, iterate the above steps S3-S9 until the feature extraction model of each node and the global module converge or reach the maximum training round.

[0018] Preferred: The method for calculating the homomorphic encryption partial prediction result in step S3 is:

[0019]

[0020] in, Represents the prediction result of the homomorphic encryption part, represents the active node, Indicates Active nodes, , Indicates the number of passive nodes, represents compact image features, Represents the second global module parameter one, represents the Paillier homomorphic encryption homomorphic multiplication algorithm, Indicates the active node public key.

[0021] The method for obtaining the encrypted secret sharing prediction result in step S3 is:

[0022]

[0023] in, are vectors, For plain text, is the ciphertext, is a vector generated by a pseudo-random number generator, is the modulus, Represents a node.

[0024] Preferably: the secret sharing prediction result in step S4 is:

[0025]

[0026] in: represents the secret sharing prediction result, represents the HE2SS algorithm, Indicates a passive node The encrypted secret shared prediction results are sent.

[0027] Preferably, the encrypted gradient of the second global module parameter 1 in step S6 is:

[0028]

[0029] in, Indicates the second global module parameter one The encrypted gradient of Indicates the second global module parameter one The gradient of Indicates a passive node Compact image features The matrix transpose of Represents the total prediction result gradient.

[0030] Preferably, the encrypted compact image feature gradient in step S7 is:

[0031]

[0032] in, represents the encrypted compact image feature gradient, Representing compact image features The gradient of Represents the second global module parameter 2, Represents the second global module parameter one, represents the total prediction result gradient, Indicates encryption of the second global module parameter 2.

[0033] Preferably, in step S9, the active node is calculated according to the following formula:

[0034]

[0035] Passive Node Calculated using the following formula:

[0036]

[0037] in, Indicates active node Passive nodes owned No. The second global module parameter of the round is one, Indicates active node Passive nodes owned No. The second global module parameter of the round is one, represents the learning rate, Represents the vector generated by calling the HE2SS algorithm, Indicates a passive node Owned by The second global module parameter of the round is two, Indicates a passive node Owned by The second global module parameter of the round is two, Represents the gradient of the second global module block parameter one.

[0038] Another object of the present invention is to provide a federated learning system based on privacy protection of large model computing power network images, which is used to implement the federated learning method based on privacy protection of large model computing power network images, including an information acquisition unit, a node division unit, an active node unit, and a passive node unit, wherein:

[0039] The information acquisition unit is used to obtain node information of the image computing network and node processing image data information.

[0040] The node division unit is used to divide the nodes into passive nodes and active nodes according to the node information and the node processing image data information.

[0041] The active node unit is used for the passive node to use the original image features it owns to input into the second feature extraction model to obtain a compact image feature. The passive node uses the compact image feature and the encrypted second global module parameter one to calculate the homomorphic encryption partial prediction result, and then converts the homomorphic encryption partial prediction result into a secret sharing prediction result. The passive node then uses a single mask encryption algorithm to encrypt the secret sharing prediction result to obtain an encrypted secret sharing prediction result. The passive node sends the pseudo-random number vector generated for single mask encryption to a trusted third party for aggregation. The passive node then uses the compact image feature and the second global module parameter two to calculate the partial prediction result, and then adds the partial prediction result to the vector generated when the homomorphic encryption partial prediction result is converted into a secret sharing prediction result to obtain the perturbed partial prediction result. Then, the passive node sends the encrypted secret sharing prediction result and the perturbed partial prediction result to the active node. The active node calculates the total prediction result The gradient relative to the loss is then encrypted and sent to all passive nodes. The active node uses the second global module parameter one and the encrypted second global module parameter two to obtain the encrypted global module parameter. The active node uses the total prediction result gradient and the encrypted global module parameter to calculate the encrypted compact image feature gradient of the passive node. The active node sends the encrypted compact image feature gradient to the passive node. The active node uses the HE2SS algorithm to decrypt the secret shared gradient sent by the passive node and uses the compact image feature and the total prediction result gradient to calculate the gradient of the global module parameter. Then the active node uses the total prediction result gradient and the global module parameter to obtain the gradient of the compact image feature.

[0042] The passive node unit is used for the active node to use the original image features to input the first feature extraction model parameters to obtain compact image features. The active node uses the compact image features and the first global module parameters to calculate the image prediction result, and then uses the HE2SS algorithm to decrypt the passive node The encrypted secret shared prediction result sent is used to obtain the secret shared prediction result. The active node aggregates the image prediction result, the secret shared prediction result, the perturbation part prediction result and the trusted third party to obtain the total prediction result, and then uses the total prediction result to calculate the loss. The passive node uses the encrypted gradient and compact image feature to calculate the encrypted gradient of the second global module parameter one, and then the passive node converts the encrypted gradient into a secret shared gradient and sends it to the active node. The passive node encrypts the second global module parameter two to obtain the encrypted second global module parameter two and sends it to the active node. The passive node uses the vector generated by calling the HE2SS algorithm in step S8 to update the second global module parameter two to obtain a new second global module parameter two. The active node uses the decrypted secret shared gradient to update the second global module parameter one to obtain a new second global module parameter one, and at the same time, the active node uses the global module parameter gradient to update the global module parameter, and then the active node uses the compact image feature gradient to back propagate and update the first feature extraction model parameters, and finally the active node encrypts the new second global module parameter one to obtain the encrypted new second global module parameter one, and sends it to the passive node. The passive node uses the compact image feature gradient to back propagate and update the second feature extraction model parameters.

[0043] Another object of the present invention is to provide a computer system, comprising a memory and a processor, wherein the memory is used to store a computer program / instruction. The processor is used to execute the computer program / instruction to implement the federated learning method based on large model computing power network image privacy protection.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention encrypts the total output of the active node when implementing label protection of the active node of the image computing network, preventing the passive node from inferring the label information of the active node through the total output gradient transmitted by the active node, and ensuring that the vertical federated learning will not leak the label of the active node during back propagation.

[0046] 2. When implementing feature protection of passive nodes, the present invention encrypts the two parts of the global module output of the passive node and the global module gradient calculated by the passive node itself, preventing the active node from inferring the feature information of the passive node through the output of the global module or the gradient of the global module of the passive node, thereby realizing the protection of the feature information of the passive node in the process of vertical federated learning.

[0047] 3. The present invention is applied in an image computing network environment, and can efficiently schedule computing resources through the image computing network to realize distributed training of large models. The proposed privacy protection method protects the privacy information of each node, and vertical federated learning can enable some unlabeled nodes to make full use of their own data to train ideal models. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The flowchart of the federated learning method based on large model computing power network image privacy protection. DETAILED DESCRIPTION

[0049] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0050] A federated learning method based on image privacy protection of a large model computing network obtains node information of an image computing network and information about image data processed by the nodes, wherein the node information includes image features and label information owned by the nodes. The nodes are divided into passive nodes and active nodes according to the node information and information about image data processed by the nodes. The compact image features and global module parameter gradients extracted from the passive nodes are encrypted and protected to prevent the active nodes from inferring the feature information of the passive nodes through the compact image features of the passive nodes and the global module parameter gradients of the passive nodes. The gradients of the prediction results of the active nodes are encrypted and protected to prevent the passive nodes from inferring the labels of the active nodes through the gradients of the prediction results of the active nodes. Figure 1 As shown, the following steps are included:

[0051] Step S1, obtaining node information of an image computing network and information about image data processed by the nodes. The computing network includes more than two nodes, wherein the nodes are computing units, computers or servers with a neural network structure, and the neural network structure includes a global module, a feature extraction model and a loss function.

[0052] Step S2, according to the node information and the node processing image data information, the nodes are divided into passive nodes and active nodes, wherein the active node includes a first global module, a first feature extraction model and a first loss function. The passive node includes a second global module and a second feature extraction model. The passive node randomly initializes the second feature extraction model and part of the second global module parameters, the active node initializes the first feature extraction model, the first global module and the remaining second global module parameters of the passive node, the second global module parameters randomly initialized by the passive node are recorded as the second global module parameters two, and the active node initializes the remaining second global module parameters of the passive node as the second global module parameters one.

[0053] In this embodiment, nodes that only have image features are marked as passive nodes. , , Indicates the number of passive nodes, marking nodes that have both image features and labels as active nodes . Passive Node and active nodes Generate public and private keys for homomorphic encryption. Active node Initialize the first feature extraction model parameters , passive node Initialize the second feature extraction model parameters .

[0054] When the number of passive nodes When the passive node Initialize the second global module parameter two Active Node Initialize all passive nodes The second global module parameter is , making the passive node The first global module parameter satisfy Active Node Set the second global module parameter to After encryption, the second global module parameter after encryption is obtained. Finally, the active node The encrypted second global module parameter Send to passive node .

[0055] When the number of passive nodes When the passive node Initialize the second global module parameter two Active Node Initialize the passive node The second global module parameter is , making the passive node The first global module parameter satisfy Active Node Set the second global module parameter to After encryption, the second global module parameter after encryption is obtained. , then the active node The encrypted second global module parameter Send to passive node . Passive Node Initialize the active node The second global module parameter two and the second global module parameter two , and set the second global module parameter Send to active node .

[0056] Passive Node and active nodes Method and active node for generating public and private keys for homomorphic encryption Encrypt some global module parameters and The algorithm used is the Paillier homomorphic encryption algorithm, and its method is as follows:

[0057] Paillier homomorphic encryption key generation algorithm ,in represents the length of the public parameter, i.e. the key. The algorithm randomly selects two Large prime numbers , then calculate and The product of Then, the algorithm randomly selects a ,in yes The union of The order is belongs to The set of elements of , represents Carmichael's function, that is . The algorithm then checks whether:

[0058]

[0059] in, If it is satisfied, we will is represented as a public parameter (i.e., a public key), is represented as a private parameter (i.e., private key). Therefore, we can get .

[0060] Paillier homomorphic encryption algorithm ,in represents the public key, Represents plaintext. The algorithm inputs plaintext , choose a random number , the algorithm outputs the ciphertext .

[0061] Paillier homomorphic encryption and decryption algorithm ,in Represents the private key, Represents ciphertext. The algorithm inputs ciphertext , the algorithm outputs plaintext .

[0062] Paillier homomorphic encryption homomorphic addition algorithm ,in represents the public key, and The input of this algorithm is two plaintexts and the public key , the additive homomorphism of the algorithm can be expressed as: .

[0063] Paillier homomorphic encryption homomorphic multiplication algorithm ,in represents the public key, and The input of this algorithm is two plaintexts and the public key , the multiplicative homomorphism of the algorithm can be expressed as: .

[0064] Step S3, the passive node uses the original image features it owns to input into the second feature extraction model to obtain a compact image feature. The passive node uses the compact image feature and the encrypted second global module parameter one to calculate the homomorphic encryption partial prediction result, and then converts the homomorphic encryption partial prediction result into a secret sharing prediction result. The passive node then uses a single mask encryption algorithm to encrypt the secret sharing prediction result to obtain an encrypted secret sharing prediction result. The passive node sends the pseudo-random number vector generated for single mask encryption to a trusted third party for aggregation. The passive node then uses the compact image feature and the second global module parameter two to calculate the partial prediction result, and then adds the partial prediction result to the vector generated when the homomorphic encryption partial prediction result is converted into a secret sharing prediction result to obtain the perturbed partial prediction result. Then, the passive node sends the encrypted secret sharing prediction result and the perturbed partial prediction result to the active node.

[0065] like Figure 1 As shown, when the number of passive nodes When the passive node Use the original features of the image you have Input to the second feature extraction model parameters Compact image features are obtained from . Passive Node Using compact image features The second global module parameter after encryption Calculate the homomorphic encryption part of the prediction result , and then use the homomorphic encryption part to predict the result Convert to secret shared prediction results Then the passive node Encrypt the secret shared prediction results using a single mask encryption algorithm Get the encrypted secret shared prediction result , represents the encrypted secret sharing prediction result, represents a pseudo-random number vector, Represents the secret sharing prediction result. The passive node will use the pseudo-random number vector generated for single-mask encryption Send to a trusted third party for aggregation. Passive nodes then use compact image features With the second global module parameter two Calculate some prediction results , and then use some of the prediction results The vector generated when the homomorphically encrypted partial prediction result is converted into a secret shared prediction result Add together to get the partial prediction results of the disturbance ,in, represents the partial prediction results of the disturbance, Represents some prediction results, Represents some prediction results The vector generated when the homomorphically encrypted partial prediction result is converted into a secret shared prediction result. Then, the passive node Share the prediction results with encrypted secret And the prediction results of the disturbance part Send to active node .

[0066] When the number of passive nodes When the active node Randomly generated and compact image features Perturbation vectors of the same dimension , and then add them together to get the perturbed compact image features , then Send to passive node , and set the second global module parameter and the perturbation vector Sent to a trusted third party for aggregation. Passive nodes Using original image features Input to the second feature extraction model parameters Compact image features are obtained from Then, the passive node Using compact image features The second global module parameter after encryption Calculate the homomorphic encryption part of the prediction result , then the passive node Homomorphically encrypt part of the prediction results Convert to secret shared partial prediction results . Passive Node Encrypt the secret shared prediction results using a single mask encryption algorithm , and get the encrypted secret sharing prediction result At the same time, the vector generated by the single mask encryption algorithm Sent to a trusted third party. Then, the passive node Using compact image features With the second global module parameter two Calculate some prediction results , and then compare the result with the result generated when converting the homomorphic encryption part prediction result to the secret sharing part prediction result Add together to get the partial prediction results of the disturbance Passive nodes use perturbed compact image features With the second global module parameter Calculate the partial prediction results of the disturbance , add the partial prediction results of the two disturbances to get the partial prediction result of the disturbance , and finally the passive node Share the prediction results with encrypted secret And some prediction results of disturbance Send to active node .

[0067] Passive Node Using compact image features Global modules with encrypted parts Calculate the homomorphic encryption part of the prediction result and passive nodes Using compact image features Some global module parameters related to homomorphic encryption Calculate the homomorphic encryption part of the prediction result Method: Using Paillier homomorphic encryption homomorphic multiplication algorithm and the forward propagation algorithm, where the encrypted partial global module parameters are expressed as , meaning that the active node Encrypted nodes belong to passive nodes Some global module parameters , passive node Some of the global module parameters are , these two parts of the global module parameters satisfy , Indicates belonging to Complete global module parameters. Passive nodes The compact image feature representation is , the method to calculate the prediction results of the homomorphic encryption part:

[0068]

[0069] in, Represents the prediction result of the homomorphic encryption part, represents the active node, Indicates Active nodes, , Indicates the number of passive nodes, represents compact image features, Represents the second global module parameter one, represents the Paillier homomorphic encryption homomorphic multiplication algorithm, Indicates the active node public key.

[0070] Homomorphically encrypt part of the prediction results Convert to secret shared prediction results And homomorphically encrypt the prediction results Convert to secret sharing partial prediction results Method: Use HE2SS algorithm to homomorphically encrypt vector and nodes As input, Respectively represent the node IDs. First, the HE2SS algorithm determines whether Is it equal to , if they are equal, then call the Paillier homomorphic encryption decryption algorithm Decrypted vector If they are not equal, the node Randomly generate and Random vectors of the same shape , then calculate get , and finally return or .

[0071] Encrypt the secret shared prediction results using a single mask encryption algorithm , and get the encrypted secret sharing prediction result and encrypt the secret shared prediction results using a single mask encryption algorithm , the method to obtain the encrypted secret sharing prediction result is:

[0072]

[0073] in, are vectors, For plain text, is the ciphertext, is a vector generated by a pseudo-random number generator, is the modulus, Represents a node.

[0074] node Will Sent to the aggregator (server), then all The nodes in the cluster tell the aggregation party in some secret way. , and then the aggregator calculates:

[0075]

[0076]

[0077] in, is a node set, is the aggregated vector.

[0078] Here, passive nodes are randomly generated and Same shape As the above , and then add them together to get .

[0079] The trusted third-party aggregation includes:

[0080] When the number of passive nodes When the trusted third party receives all the passive nodes’ , calculate their sum ,Right now:

[0081]

[0082] When the number of passive nodes When the trusted third party receives the and the active node sends and After that, we calculate ,Right now:

[0083]

[0084] Step S4: The active node uses the original image features to input the first feature extraction model parameters to obtain compact image features. The active node uses the compact image features and the first global module parameters to calculate the image prediction result, and then uses the HE2SS algorithm to decrypt the passive node. The encrypted secret shared prediction results sent are used to obtain the secret shared prediction results. The active node aggregates the image prediction results, secret shared prediction results, perturbed part prediction results and the results sent by the trusted third party to obtain the total prediction results, and then uses the total prediction results to calculate the loss.

[0085] When the number of passive nodes When the active node Using original image features Input to the first feature extraction model parameters Get compact image features Active Node Using compact image features With the first global module parameter Calculate the image prediction result , and then use the HE2SS algorithm to decrypt the passive node Send encrypted secret shared prediction results Get secret sharing prediction results Active Node Aggregate image prediction results , Secret Sharing Prediction Results , the prediction results of the disturbance part and sent by a trusted third party Get the total prediction results , and then use the total prediction results Calculating Losses .

[0086] When the number of passive nodes When the active node Using original image features Input to the first feature extraction model parameters Get compact image features Active Node Using compact image features With the second global module parameter two Calculate the prediction results of the image part , and then use the HE2SS algorithm to decrypt the passive node Send encrypted secret shared prediction results Get secret sharing prediction results Active Node Aggregate image partial prediction results , Secret Sharing Prediction Results , the prediction results of the disturbance part and sent by a trusted third party Get the total prediction results , and then use the total prediction results Calculating Losses .

[0087] Active Node Aggregate image prediction results , Secret Sharing Prediction Results , the prediction results of the disturbance part and sent by a trusted third party Total prediction results The method is:

[0088]

[0089] in, Represents the total prediction result, It is aggregated from all passive nodes by a trusted third party The mask, represents the secret sharing prediction result, Represents the prediction result of the disturbance part.

[0090] Active Node Aggregate image partial prediction results , Secret Sharing Prediction Results , the prediction results of the disturbance part and sent by a trusted third party Overall prediction results The method is:

[0091]

[0092] Decryption of passive nodes using HE2SS algorithm Send encrypted secret shared prediction results , and obtain the secret sharing prediction result and decrypt passive nodes using the HE2SS algorithm Send encrypted secret shared prediction results , the secret sharing prediction result is:

[0093]

[0094] in: represents the secret sharing prediction result, represents the HE2SS algorithm, Indicates a passive node The encrypted secret shared prediction results are sent.

[0095] Step S4 includes the following steps:

[0096] Step S5, the active node calculates the total prediction result The gradient with respect to the loss is then encrypted and sent to all passive nodes.

[0097] The gradient After encryption, it is sent to all passive nodes, including:

[0098] Active Node Call the encryption algorithm of Paillier encryption Calculated .in, Indicates loss Relative to the total forecast results gradient.

[0099] In step S6, the passive node uses the encrypted gradient and the compact image feature to calculate the encrypted gradient of the second global module parameter one, and then the passive node converts the encrypted gradient into a secret shared gradient and sends it to the active node.

[0100] When the number of passive nodes When the passive node Using encrypted gradients With compact image features Calculate the second global module parameter Encrypted gradient , then the passive node Encrypting the gradient Convert to secret shared gradients And send it to the active node .

[0101] When the number of passive nodes When the passive node Using encrypted gradients With compact image features Calculate the second global module parameter Encrypted gradient , then the passive node Encrypting the gradient Convert to secret shared gradients And send it to the active node .

[0102] Active Node Generation and Gradient Same dimensions , and then with the gradient Add the perturbation gradient Then, the active node Will Send to ,Will Sent to a trusted third party for aggregation. Passive nodes receive Afterwards, through the second global module parameter one Computing compact image features Partial gradient . Passive Node Produce and Same dimensions and compare it with Add together to get the perturbed gradient . Passive Node Perturb part of the gradient Send to active node , and Sent to a trusted third party for aggregation.

[0103] The encrypted gradient of the second global module parameter one is:

[0104]

[0105] in, Indicates the second global module parameter one The encrypted gradient of Indicates the second global module parameter one The gradient of Indicates a passive node Compact image features The matrix transpose of Represents the total prediction result gradient.

[0106] Passive Node Call HE2SS to get the secret shared gradient, that is:

[0107]

[0108] Passive Node Will Send to active node .

[0109] The passive node receive Afterwards, through the second global module parameter one Computing compact image features Partial gradient , passive node Calculate as follows:

[0110]

[0111] in, Represents the total prediction results The gradient of Indicates active node The generated vector, Represents some global module parameters The transpose of .

[0112] Step S7, the passive node encrypts the second global module parameter 2 to obtain the encrypted second global module parameter 2 and sends it to the active node. The active node uses the second global module parameter 1 to add the encrypted second global module parameter 2 to obtain the encrypted global module parameter. The active node uses the total prediction result gradient and the encrypted global module parameter to calculate the encrypted compact image feature gradient of the passive node. The active node sends the encrypted compact image feature gradient to the passive node.

[0113] When the number of passive nodes When the passive node Encrypt the second global module parameter 2 Get the second global module parameter of encryption and sends it to the active node Active Node Use the second global module parameter one Encryption 2nd global module parameter 2 Add together to get the encrypted global module parameters Active Node Using the total prediction gradient and encrypt global module parameters Compute passive nodes Encrypted compact image feature gradient Active Node Encrypting the compact image feature gradient Send to passive node .

[0114] When the number of passive nodes When the passive node Encrypt the second global module parameter 2 Get the second global module parameter of encryption and sends it to the active node Active Node Use the second global module parameter one Encryption 2nd global module parameter 2 Add together to get the encrypted global module parameters Active Node Using the total prediction gradient and encrypt global module parameters Compute passive nodes Encrypted compact image feature gradient Active Node Encrypting the compact image feature gradient Send to passive node .

[0115] Passive Node Call the Paillier homomorphic encryption algorithm Encrypt some of your own global module parameters ,Right now:

[0116]

[0117] The encrypted compact image feature gradient is:

[0118]

[0119] in, represents the encrypted compact image feature gradient, Representing compact image features The gradient of Represents the second global module parameter 2, Represents the second global module parameter one, represents the total prediction result gradient, Indicates encryption of the second global module parameter 2.

[0120] In step S8, the active node uses the HE2SS algorithm to decrypt the secret shared gradient sent by the passive node and uses the compact image feature and the total prediction result gradient to calculate the gradient of the global module parameter. Then the active node uses the total prediction result gradient and the global module parameter to obtain the gradient of the compact image feature.

[0121] When the number of passive nodes When the active node Decrypting Passive Nodes Sent secret shared gradient Using compact image features Gradient with the total prediction result Calculate the global module parameters Gradient , then the active node Using the total prediction gradient With global module parameters Get compact image features Gradient .

[0122] When the number of passive nodes When the active node Decrypting Passive Nodes Sent secret shared gradient , and then use some global module parameters And the total prediction result gradient Computing compact image features The partial gradient of Vectors aggregated by trusted third parties Compute compact image features Gradient Then the active node Using compact image features Gradient with the total prediction result Calculate the global module parameters Gradient .

[0123] Active Node Decrypting Passive Nodes Sent secret shared gradient and active nodes Decrypting Passive Nodes Sent secret shared gradient Calculated using the following formula:

[0124]

[0125] in, Indicates the second global module parameter one The gradient of Represents a vector generated using the HE2SS algorithm.

[0126] Using compact image features Gradient with the total prediction result Calculate the global module parameters Gradient Calculated using the following formula:

[0127]

[0128] in, Representing compact image features The transpose of Represents the total prediction result gradient.

[0129] Active Node Using the total prediction gradient With global module parameters Get compact image features Gradient Calculated using the following formula:

[0130]

[0131] Use the second global module parameter one And the total prediction result gradient Computing compact image features The partial gradient of Vectors aggregated by trusted third parties Compute compact image features Gradient Calculated using the following formula:

[0132]

[0133] Among them, the vector According to the number of passive nodes in the trusted third party The formula for calculation is .

[0134] Step S9, the passive node uses the vector generated by calling the HE2SS algorithm in step S8 to update the second global module parameter 2 to obtain a new second global module parameter 2. The active node uses the decrypted secret shared gradient to update the second global module parameter 1 to obtain a new second global module parameter 1. At the same time, the active node uses the global module parameter gradient to update the global module parameter. Then the active node uses the compact image feature gradient to back propagate and update the first feature extraction model parameters. Finally, the active node encrypts the new second global module parameter 1 to obtain the encrypted new second global module parameter 1 and sends it to the passive node. The passive node uses the compact image feature gradient to back propagate and update the second feature extraction model parameters.

[0135] When the number of passive nodes When the passive node Use the HE2SS algorithm generated in step S8 Update the second global module parameter 2 get Active Node Use the decrypted secret to share the gradient Update the second global module parameter one get , while the active node Using global module parameter gradients Update global module parameters , then the active node Using compact image feature gradients Back propagate and update the first feature extraction model parameters , the last active node Encrypt new second global module parameter one get and sends it to the passive node . Passive Node Using compact image feature gradients Back propagate and update the second feature extraction model parameters .

[0136] When the number of passive nodes When the passive node Use the HE2SS algorithm generated in step S8 Update the second global module parameter 2 get Active Node Use the decrypted secret to share the gradient Update the second global module parameter one get , while the active node Using global module parameter gradients Update the second global module parameter 2 , then the active node Using compact image feature gradients Back propagate and update the first feature extraction model parameters , the last active node Encrypt new second global module parameter one get and sends it to the passive node . Passive Node Using compact image feature gradients Back propagate and update the second feature extraction model parameters .

[0137] In step S9, the active node decrypts the second secret sharing gradient of the passive node and updates the second global module 2 of the passive node with the second secret sharing gradient. The passive node updates the second global module 1 by using the vector generated when converting the second encrypted gradient into the second secret sharing gradient:

[0138] The active node is calculated according to the following formula:

[0139]

[0140] Passive Node Calculated using the following formula:

[0141]

[0142] in, Indicates active node Passive nodes owned No. The second global module parameter of the round is one, Indicates active node Passive nodes owned No. The second global module parameter of the round is one, represents the learning rate, Represents the vector generated by calling the HE2SS algorithm, Indicates a passive node Owned by The second global module parameter of the round is two, Indicates a passive node Owned by The second global module parameter of the round is two, Represents the gradient of the second global module block parameter one.

[0143] Active Node Using global module parameter gradients Update global module parameters Calculated using the following formula:

[0144]

[0145] Active Node Using global module parameter gradients Update the second global module parameter 2 Calculated using the following formula:

[0146]

[0147] Active Node Encrypt new second global module parameter one get and active nodes Encrypt new second global module parameter one get The encryption algorithm of Paillier homomorphic encryption is called ,Right now:

[0148]

[0149] Step S10, iterate the above steps S3-S9 until the feature extraction model of each node and the global module converge or reach the maximum training round.

[0150] In the computing power network environment, the present invention proposes to use the HE2SS algorithm, the Paillier algorithm and the single-mask encryption algorithm to protect the compact image features and partial global module gradients of passive nodes for honest but curious active nodes in vertical federated learning, under the premise of achieving stable distributed training model effects, and prevents active nodes from inferring the original image feature information of passive nodes through the compact image features of passive nodes and partial global module gradients of passive nodes. For honest but curious passive nodes in vertical federated learning, the present invention proposes to use the Paillier algorithm to protect the gradient of the total prediction results of active nodes, and avoids passive nodes from inferring the labels of active nodes through the gradient of the total prediction results of active nodes. The present invention applies the HE2SS algorithm, the Paillier algorithm and the single-mask encryption algorithm to vertical federated learning, and realizes a new solution for computing power network privacy protection federated learning for large model distributed training.

[0151] The present invention uses the Paillier algorithm to encrypt the total prediction result of the active node when implementing label protection of the active node, preventing the passive node from inferring the label information of the active node through the total prediction result gradient transmitted by the active node, and ensuring that the label of the active node will not be leaked in the vertical federated learning during back propagation.

[0152] The present invention uses the HE2SS algorithm, the Paillier algorithm and the single mask encryption algorithm to encrypt the two partial prediction results of the passive node and the global module parameter gradient calculated by the passive node when realizing the feature protection of the passive node, thereby preventing the active node from inferring the original image feature information of the passive node through the prediction results of the global module for the compact image features or the gradient of the global module parameters of the passive node, and realizing the protection of the original image feature information of the passive node in the process of vertical federated learning.

[0153] In another embodiment, a federated learning system based on privacy protection of large model computing power network images is provided, which is used to implement the federated learning method based on privacy protection of large model computing power network images, including an information acquisition unit, a node division unit, an active node unit, and a passive node unit, wherein:

[0154] The information acquisition unit is used to obtain node information of the image computing network and node processing image data information.

[0155] The node division unit is used to divide the nodes into passive nodes and active nodes according to the node information and the node processing image data information.

[0156] The active node unit is used for the passive node to use the original image features it owns to input into the second feature extraction model to obtain a compact image feature. The passive node uses the compact image feature and the encrypted second global module parameter one to calculate the homomorphic encryption partial prediction result, and then converts the homomorphic encryption partial prediction result into a secret sharing prediction result. The passive node then uses a single mask encryption algorithm to encrypt the secret sharing prediction result to obtain an encrypted secret sharing prediction result. The passive node sends the pseudo-random number vector generated for single mask encryption to a trusted third party for aggregation. The passive node then uses the compact image feature and the second global module parameter two to calculate the partial prediction result, and then adds the partial prediction result to the vector generated when the homomorphic encryption partial prediction result is converted into a secret sharing prediction result to obtain the perturbed partial prediction result. Then, the passive node sends the encrypted secret sharing prediction result and the perturbed partial prediction result to the active node. The active node calculates the total prediction result The gradient relative to the loss is then encrypted and sent to all passive nodes. The active node uses the second global module parameter one and the encrypted second global module parameter two to obtain the encrypted global module parameter. The active node uses the total prediction result gradient and the encrypted global module parameter to calculate the encrypted compact image feature gradient of the passive node. The active node sends the encrypted compact image feature gradient to the passive node. The active node uses the HE2SS algorithm to decrypt the secret shared gradient sent by the passive node and uses the compact image feature and the total prediction result gradient to calculate the gradient of the global module parameter. Then the active node uses the total prediction result gradient and the global module parameter to obtain the gradient of the compact image feature.

[0157] The passive node unit is used for the active node to use the original image features to input the first feature extraction model parameters to obtain compact image features. The active node uses the compact image features and the first global module parameters to calculate the image prediction result, and then uses the HE2SS algorithm to decrypt the passive node The encrypted secret shared prediction result sent is used to obtain the secret shared prediction result. The active node aggregates the image prediction result, the secret shared prediction result, the perturbation part prediction result and the trusted third party to obtain the total prediction result, and then uses the total prediction result to calculate the loss. The passive node uses the encrypted gradient and compact image feature to calculate the encrypted gradient of the second global module parameter one, and then the passive node converts the encrypted gradient into a secret shared gradient and sends it to the active node. The passive node encrypts the second global module parameter two to obtain the encrypted second global module parameter two and sends it to the active node. The passive node uses the vector generated by calling the HE2SS algorithm in step S8 to update the second global module parameter two to obtain a new second global module parameter two. The active node uses the decrypted secret shared gradient to update the second global module parameter one to obtain a new second global module parameter one, and at the same time, the active node uses the global module parameter gradient to update the global module parameter, and then the active node uses the compact image feature gradient to back propagate and update the first feature extraction model parameters, and finally the active node encrypts the new second global module parameter one to obtain the encrypted new second global module parameter one, and sends it to the passive node. The passive node uses the compact image feature gradient to back propagate and update the second feature extraction model parameters.

[0158] In another embodiment, a computer system is provided, characterized in that it includes a memory and a processor, wherein the memory is used to store computer programs / instructions. The processor is used to execute the computer programs / instructions to implement a federated learning method based on large model computing power network image privacy protection.

[0159] Specifically, the present invention uses image datasets such as MNIST, cifar and EMNIST for image classification tasks, and uses convolutional neural network (CNN) as the basic neural network, where the convolutional neural network can be replaced by any large language model. This method can safely implement large language model training. Each institution or enterprise with different computing power is regarded as a computing power node, and each computing power node has different training data characteristics. Assume that there are The computing power nodes only have image data features, which are expressed as ,in Indicates the node index, Indicates the number of images, express The size of the feature vector of the image data it possesses. This type of node is represented as a passive node. In this system, only one computing node has the label corresponding to each image data, that is, the data owned by the computing node is represented as ,in Representation Node The image features, Representation Node The computing node with the label corresponding to each image is represented as an active node. Finally, a trusted third party is set up In each iteration, the specific steps are as follows:

[0160] 1) System initialization

[0161] Set the public parameters for generating the public and private key pairs for each node , and then set the parameters for federated learning, for example: , , wait.

[0162] Passive Node :use Initialize the public and private key pair The model used for feature extraction is represented as , then Random initialization. Represents CNN, that is, A vector of size is taken as input, through The vector dimension is The feature output of , which represents a more compact feature obtained after a series of operations on the original feature . Then, randomly initialize some global modules , this module is initialized with the global module of the active node Can form a complete global module ,Right now .

[0163] Active Node : Initialize the public and private key pair . Using CNN as the feature extraction model, it is expressed as And initialize randomly. Then for each Random initialization belongs to Some global modules Then, save Locally, and encrypt using your own public key get Finally, Send to .

[0164] 2) Forward propagation

[0165] Passive Node :Based on the original image features As input, through the feature extraction model Get a more compact feature vector for the image Using this feature vector Calculate the partial prediction value of the global module on the batch training samples Then, the node uses the HE2SS algorithm to convert the partial prediction vector Converted into a partial secret sharing prediction vector and the vector generated when calling the HE2SS algorithm, that is, . Set the secret sharing prediction value to Nodes randomly generate and secret-share prediction vectors Mask vectors of the same shape , and the mask vector Sent to a trusted third party. The node calculates the encrypted part of the secret shared prediction vector to obtain . The last node passes through some global modules Calculate another partial global module pair feature vector The prediction vector and Add, that is ,Will and Send to active node .

[0166] Trusted Third Party: Aggregates the mask vectors generated by all passive nodes ,Right now , then Send to active node .

[0167] Active Node :The original feature vector of the image As input, through the feature extraction model Get a more compact feature vector Then calculate the prediction result of the global module for the batch training data For each , decrypt the encrypted secret shared prediction vector sent by the passive node Aggregate and decrypt the predicted output vector of all node global modules Using the labels of the batch training data and the aggregated output, the loss function is used to calculate the loss value of the training. .

[0168] 3) Back Propagation

[0169] Active Node :implement The back propagation process obtains the gradient of the prediction vector for this training . For the gradient Encrypted . Encrypt the gradient Send to all .

[0170] Upon receiving all the messages from the passive node Partial global module gradients sent After that, it is decrypted by the HE2SS algorithm to obtain . Compact image features obtained using feature extraction models Calculate your own global module parameters Gradient . Then use its own global module parameters to calculate the compact image features Gradient .

[0171] Received passive node Partially encrypted global module parameters sent Then, connect it to the passive node Another part of the global module parameters Add together to get the encrypted complete global module parameters .Then, Gradient calculation using prediction vector Compact image features Gradient , and the gradient Send to .

[0172] Passive Node : Receive active node The encrypted prediction vector gradient sent After that, use its own feature vector to calculate some global module parameters Gradient Then, the HE2SS algorithm is used to convert the gradient Converted to secret shared gradient and vector generated in HE2SS algorithm, namely . Set some global module parameters The secret shared gradient Send to active node .

[0173] Encrypt some of your own global module parameters . Then the encrypted global module parameters Send to .

[0174] Receive the encrypted gradient of the compact image features After that, decryption is performed to obtain the gradient of the compact image feature .

[0175] 4) Model update

[0176] Passive Node : Vector generated using HE2SS Update some global modules , and utilize the gradient of compact image features Back propagate and update the parameters of the feature extraction model CNN .

[0177] Active Node : Use some global modules The gradient updates the passive nodes Some global modules , and then encrypt the new partial global module parameters , the encrypted global module parameters Send to .

[0178] Using global module parameters The gradient of the global module parameters is updated . Utilize the gradient of compact image features Back propagate and update the parameters of the feature extraction model CNN .

[0179] After all iterations are completed, the image computing network privacy-preserving federated learning method for large-model distributed training adopted by the present invention can achieve secure training and stable model training effects in the image computing network environment, while protecting active node labels and passive node features. Therefore, the present invention can solve the technical problems of malicious passive nodes stealing active node labels through global modules or gradients and active nodes stealing passive node features through the output of the global modules of passive nodes.

[0180] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A federated learning method based on large model computing power network image privacy protection, characterized by: The node information of the image computing network and the node processing image data information are obtained, wherein the node information includes the image features and label information owned by the node; the nodes are divided into passive nodes and active nodes according to the node information and the node processing image data information, and the compact image features and global module parameter gradients extracted from the passive nodes are encrypted and protected to prevent the active nodes from inferring the feature information of the passive nodes through the compact image features of the passive nodes and the global module parameter gradients of the passive nodes; the prediction result gradients of the active nodes are encrypted and protected to prevent the passive nodes from inferring the labels of the active nodes through the prediction result gradients of the active nodes; the steps include: Step S1, obtaining node information of the image computing network and node processing image data information; Step S2, dividing the nodes into passive nodes and active nodes according to the node information and the node processing image data information, wherein the active node includes a first global module, a first feature extraction model and a first loss function; the passive node includes a second global module and a second feature extraction model; the passive node randomly initializes the second feature extraction model and part of the second global module parameters, the active node initializes the first feature extraction model, the first global module and the remaining part of the second global module parameters of the passive node, the part of the second global module parameters randomly initialized by the passive node is recorded as the second global module parameter two, and the active node initializes the remaining part of the second global module parameters of the passive node as the second global module parameter one; Step S3, the passive node uses the original image features it owns to input into the second feature extraction model to obtain a compact image feature; the passive node uses the compact image feature and the encrypted second global module parameter one to calculate the homomorphic encryption partial prediction result, and then converts the homomorphic encryption partial prediction result into a secret sharing prediction result; then the passive node uses the single mask encryption algorithm to encrypt the secret sharing prediction result to obtain an encrypted secret sharing prediction result; the passive node sends the pseudo-random number vector generated for single mask encryption to a trusted third party for aggregation; the passive node then uses the compact image feature and the second global module parameter two to calculate the partial prediction result, and then adds the partial prediction result to the vector generated when the homomorphic encryption partial prediction result is converted into the secret sharing prediction result to obtain the perturbed partial prediction result, and then the passive node sends the encrypted secret sharing prediction result and the perturbed partial prediction result to the active node; Method to calculate the prediction results of the homomorphic encryption part: in, Represents the prediction result of the homomorphic encryption part, represents the active node, Indicates Active nodes, , Indicates the number of passive nodes, represents compact image features, Represents the second global module parameter one, represents the Paillier homomorphic encryption homomorphic multiplication algorithm, Represents the active node public key; The method for obtaining the encrypted secret sharing prediction result in step S3 is: in, For plain text, is the ciphertext, is a vector generated by a pseudo-random number generator, is the modulus, Represents a node; Step S4, the active node uses the original image features to input the first feature extraction model parameters to obtain compact image features; the active node uses the compact image features and the first global module parameters to calculate the image prediction result, and then uses the HE2SS algorithm to decrypt the passive node The encrypted secret shared prediction result sent is used to obtain the secret shared prediction result; the active node aggregates the image prediction result, the secret shared prediction result, the perturbed part prediction result and the prediction result sent by the trusted third party to obtain the total prediction result, and then uses the total prediction result to calculate the loss; The secret sharing prediction result is: in: represents the secret sharing prediction result, represents the HE2SS algorithm, Indicates a passive node The encrypted secret shared prediction results sent; Step S5, the active node calculates the total prediction result The gradient relative to the loss is then encrypted and sent to all passive nodes; Step S6, the passive node uses the encrypted gradient and the compact image feature to calculate the encrypted gradient of the second global module parameter one, and then the passive node converts the encrypted gradient into a secret shared gradient and sends it to the active node; Step S7, the passive node encrypts the second global module parameter 2 to obtain the encrypted second global module parameter 2 and sends it to the active node; the active node uses the second global module parameter 1 to add the encrypted second global module parameter 2 to obtain the encrypted global module parameter; the active node uses the total prediction result gradient and the encrypted global module parameter to calculate the encrypted compact image feature gradient of the passive node; the active node sends the encrypted compact image feature gradient to the passive node; Step S8, the active node uses the HE2SS algorithm to decrypt the secret shared gradient sent by the passive node and uses the compact image feature and the total prediction result gradient to calculate the gradient of the global module parameter. Then the active node uses the total prediction result gradient and the global module parameter to obtain the gradient of the compact image feature. Step S9, the passive node uses the vector generated by calling the HE2SS algorithm in step S8 to update the second global module parameter 2 to obtain a new second global module parameter 2; the active node uses the decrypted secret shared gradient to update the second global module parameter 1 to obtain a new second global module parameter 1, and at the same time, the active node uses the global module parameter gradient to update the global module parameter, and then the active node uses the compact image feature gradient to back propagate and update the first feature extraction model parameter, and finally the active node encrypts the new second global module parameter 1 to obtain the encrypted new second global module parameter 1, and sends it to the passive node; the passive node uses the compact image feature gradient to back propagate and update the second feature extraction model parameter; Step S10, iterate the above steps S3-S9 until the feature extraction model of each node and the global module converge or reach the maximum training round.

2. The method for protecting privacy of images in a federated learning environment based on large-model computing network according to claim 1, characterized in that: The encrypted gradient of the second global module parameter 1 in step S6 is: in, Indicates the second global module parameter one The encrypted gradient of Indicates the second global module parameter one The gradient of Indicates a passive node Compact image features The matrix transpose of Represents the total prediction result gradient.

3. The method for protecting privacy of images in a federated learning environment based on large-model computing network according to claim 2, characterized in that: The encrypted compact image feature gradient in step S7 is: in, represents the encrypted compact image feature gradient, Representing compact image features The gradient of Represents the second global module parameter 2, Represents the second global module parameter one, represents the total prediction result gradient, Indicates encryption of the second global module parameter 2.

4. The method for protecting privacy of images in a federated learning environment based on large-model computing network according to claim 3 is characterized by: In step S9, the active node is calculated according to the following formula: Passive Node Calculated using the following formula: in, Indicates active node Passive nodes owned No. The second global module parameter of the round is one, Indicates active node Passive nodes owned No. The second global module parameter of the round is one, represents the learning rate, Represents the vector generated by calling the HE2SS algorithm, Indicates a passive node Owned by The second global module parameter of the round is two, Indicates a passive node Owned by The second global module parameter of the round is two, Represents the gradient of the second global module block parameter one.

5. The method for protecting privacy of network images based on large-model computing power federated learning according to claim 4 is characterized by: In step S2, nodes that only have image features are marked as passive nodes. , , Indicates the number of passive nodes, marking nodes that have both image features and labels as active nodes ; Passive node and active nodes Generate public and private keys for homomorphic encryption; Active node Initialize the first feature extraction model parameters , passive node Initialize the second feature extraction model parameters .

6. A federated learning system based on large model computing power network image privacy protection, characterized in that: A federated learning method for implementing the privacy protection of a large model computing network image according to any one of claims 1 to 5, comprising an information acquisition unit, a node division unit, an active node unit, and a passive node unit, wherein: The information acquisition unit is used to acquire node information of the image computing network and node processing image data information; The node division unit is used to divide the nodes into passive nodes and active nodes according to the node information and the node processing image data information; The active node unit is used for the passive node to use the original image features it owns to input into the second feature extraction model to obtain a compact image feature; the passive node uses the compact image feature and the encrypted second global module parameter one to calculate the homomorphic encryption partial prediction result, and then converts the homomorphic encryption partial prediction result into a secret sharing prediction result; then the passive node uses the single mask encryption algorithm to encrypt the secret sharing prediction result to obtain an encrypted secret sharing prediction result; the passive node sends the pseudo-random number vector generated for single mask encryption to a trusted third party for aggregation; the passive node then uses the compact image feature and the second global module parameter two to calculate the partial prediction result, and then adds the partial prediction result to the vector generated when converting the homomorphic encryption partial prediction result into the secret sharing prediction result to obtain the perturbed partial prediction result, and then the passive node sends the encrypted secret sharing prediction result and the perturbed partial prediction result to the active node; the active node calculates the total prediction result The gradient relative to the loss is then encrypted and sent to all passive nodes; the active node uses the second global module parameter one and the encrypted second global module parameter two to obtain the encrypted global module parameter; the active node uses the total prediction result gradient and the encrypted global module parameter to calculate the encrypted compact image feature gradient of the passive node; the active node sends the encrypted compact image feature gradient to the passive node; the active node uses the HE2SS algorithm to decrypt the secret shared gradient sent by the passive node and uses the compact image feature and the total prediction result gradient to calculate the gradient of the global module parameter, and then the active node uses the total prediction result gradient and the global module parameter to obtain the gradient of the compact image feature; The passive node unit is used for the active node to use the original image feature to input the first feature extraction model parameter to obtain the compact image feature; the active node uses the compact image feature and the first global module parameter to calculate the image prediction result, and then uses the HE2SS algorithm to decrypt the passive node The encrypted secret shared prediction result sent is obtained to obtain the secret shared prediction result; the active node aggregates the image prediction result, the secret shared prediction result, the perturbation part prediction result and the trusted third party to obtain the total prediction result, and then uses the total prediction result to calculate the loss; the passive node uses the encrypted gradient and compact image feature to calculate the encrypted gradient of the second global module parameter one, and then the passive node converts the encrypted gradient into a secret shared gradient and sends it to the active node; the passive node encrypts the second global module parameter two to obtain the encrypted second global module parameter two and sends it to the active node; the passive node uses the vector generated by calling the HE2SS algorithm in step S8 to update the second global module parameter two to obtain a new second global module parameter two; the active node uses the decrypted secret shared gradient to update the second global module parameter one to obtain a new second global module parameter one, and at the same time the active node uses the global module parameter gradient to update the global module parameter, and then the active node uses the compact image feature gradient to back propagate and update the first feature extraction model parameters, and finally the active node encrypts the new second global module parameter one to obtain the encrypted new second global module parameter one, and sends it to the passive node; the passive node uses the compact image feature gradient to back propagate and update the second feature extraction model parameters.

7. A computer system, characterized in that: It includes a memory and a processor, the memory is used to store computer programs / instructions; the processor is used to execute the computer programs / instructions to implement any of claims 1-5 as described in the federated learning method for protecting image privacy based on large model computing power network.

Images