A Federated Learning Method and System for Decentralized Orthogonal Encryption Based on Symmetric Noise

Abstract

The present invention relates to the field of artificial intelligence security technology, and discloses a federated learning method and system for decentralized orthogonal encryption based on symmetric noise, which realizes efficient and secure communication and computing in federated learning; performs SVD decomposition on the initial model to generate a row representation matrix and a column representation matrix, determines the row and column sampling index values through the Nystrom method and distributes them to the clients; the clients generate local private orthogonal encryption matrices and global shared orthogonal encryption matrices to encrypt the local model matrices; divides the private encryption matrix into N sub-shares through the threshold secret sharing mechanism, constructs a t-1 degree polynomial to realize secret distribution, and forms a distribution matrix; the central server performs weight sorting rather than aggregation on the encrypted matrices uploaded by the clients to generate a set of sorted encrypted matrices; the clients gradually remove the encrypted matrices through the iterative decryption process, using the transpose of the orthogonal matrix and secret sharing reconstruction, and restore the global aggregation result through the noise cancellation mechanism.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security technology, and particularly relates to a federated learning method and system based on decentralized orthogonal encryption with symmetric noise. Background Art

[0002] Preventing privacy leakage is a key issue in federated learning. Commonly used methods include gradient obfuscation, compression, and encrypted transmission, etc. For gradient obfuscation, differential privacy is an effective means to protect privacy by adding Gaussian noise to the gradient. However, compared with the fixed parameter strategy, although dynamically adjusting the noise variance and injection amount can improve the privacy protection effect, it often leads to a loss of model accuracy and increases the optimization difficulty.

[0003] On the other hand, exploring the possibility of privacy protection from the perspective of gradient compression has also received attention. For example, methods such as Top-K and QSGD were initially proposed to reduce the communication overhead in federated learning, but it was found that they also have a certain privacy protection ability. Nevertheless, regarding privacy enhancement as a by-product of gradient compression is not appropriate, and it is necessary to re-examine and design the client data privacy protection mechanism in existing algorithms. Based on this, some new methods such as PEFL (Privacy-Enhanced Federated Learning) have been proposed to resist specific types of attacks without compromising the model performance. However, the security assumptions of these methods are usually that the server is completely trustworthy, which may not hold in practice.

[0004] Gradient encryption, as a fundamental data leakage protection measure, allows encrypted transmission of gradients and performs arithmetic operations in the encrypted state. Homomorphic encryption (HE), as a technology to defend against DLG (Deep Leakage from Gradients) attacks, supports gradient aggregation without decryption, while providing privacy protection and security. However, homomorphic encryption usually comes with high computational and communication costs, so it is necessary to seek a balance between efficiency and security.

[0005] To solve the high overhead problem brought by homomorphic encryption, efficient schemes such as lossless matrix masked federated SVD have been proposed, which significantly improve the computational efficiency. However, these methods are mainly applicable to specific scenarios and do not fully consider how to defend against gradient leakage attacks. In addition, most gradient encryption methods rely on a trusted third party to generate masks or keys, increasing the complexity of key management and trust risks in practical applications.

[0006] In summary, the current research work faces challenges in the balance between efficiency and security and the limitations of security assumptions. Summary of the Invention

[0007] The object of the present invention is to solve the above problems, and a federated learning method and system based on symmetric noise decentralized orthogonal encryption are designed.

[0008] The first aspect of the present invention provides a federated learning method based on symmetric noise decentralized orthogonal encryption, including the following steps:

[0009] S1. The central server performs SVD decomposition on the initial model to generate a row representation matrix and a column representation matrix, and determines the row and column sampling index values through the Nystrom method and distributes them to the clients;

[0010] S2. The client generates a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix. After using Gram-Schmidt orthogonalization processing, a double orthogonal transformation is used to encrypt the local model matrix;

[0011] S3. The private encryption matrix is divided into N sub-shares through a threshold secret sharing mechanism, and a t-1 degree polynomial is constructed to achieve secret distribution, forming a distribution matrix;

[0012] S4. The central server performs weight sorting instead of aggregation on the encrypted matrices uploaded by the clients to generate a set of sorted encrypted matrices;

[0013] S5. The client uses the transpose of the orthogonal matrix and secret sharing reconstruction in the iterative decryption process to gradually remove the encrypted matrix, and finally restores the global aggregation result through a noise cancellation mechanism.

[0014] Optionally, in the first implementation manner of the first aspect of the present invention, the generation method of the global shared orthogonal matrix in step S2 is: The generation method is as follows:

[0015] , ;

[0016] Among them, , The matrices are generated distributively through secure multi-party computation.

[0017] Optionally, in the second implementation manner of the first aspect of the present invention, the specific implementation of the secret sharing in step 3 includes:

[0018] Construct polynomials independently for each element of the matrix, and use the polynomial calculation results as sub-secrets for distribution;

[0019] Construct a distribution matrix, and the non-diagonal elements in the distribution matrix contain the shared sub-secrets among the clients.

[0020] Optionally, in the third implementation manner of the first aspect of the present invention, the polynomial construction includes a dynamic threshold adjustment mechanism:

[0021] Calculate the effective threshold value in real time according to the online status of the client, and dynamically adjust the number of Lagrange interpolation calculation nodes.

[0022] Optionally, in the fourth implementation manner of the first aspect of the present invention, the server sorting process in step 4 includes:

[0023] Perform lexicographical sorting on the encrypted client matrix according to the client number to ensure the discrete state of the encrypted data.

[0024] Optionally, in the fifth implementation manner of the first aspect of the present invention, the server sorting process in step 4 includes:

[0025] Establish an encrypted data verification structure based on the Merkle tree, generate a root hash value for the sorted encrypted matrix set, and the client confirms the data integrity and order consistency by verifying the hash chain.

[0026] Optionally, in the sixth implementation manner of the first aspect of the present invention, the iterative decryption process in step 5 includes:

[0027] Perform preliminary decryption by right multiplying the transpose of the global orthogonal matrix;

[0028] Reconstruct the private encrypted matrix by Lagrange interpolation method;

[0029] Execute the left multiplication decryption operation in rounds, and finally obtain the unencrypted global matrix through the noise cancellation mechanism.

[0030] The second aspect of the present invention provides a federated learning system for decentralized orthogonal encryption based on symmetric noise, and the system includes:

[0031] An SVD decomposition module for the central server to perform SVD decomposition on the initial model, generate a row representation matrix and a column representation matrix, and determine the row and column sampling index values through the Nystrom method and send them to the client;

[0032] An encryption module for the client to generate a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix, and after using Gram-Schmidt orthogonalization processing, encrypt the local model matrix by using double orthogonal transformation;

[0033] A construction module for dividing the private encrypted matrix into N sub-shares through a threshold secret sharing mechanism, constructing a t-1 degree polynomial to realize secret distribution, and forming a distribution matrix;

[0034] A sorting module for the central server to perform weight sorting rather than aggregation on the encrypted matrix uploaded by the client, and generate a sorted encrypted matrix set;

[0035] A decryption module, which is used for the client to gradually remove the encryption matrix through an iterative decryption process by using the transpose of the orthogonal matrix and secret sharing reconstruction, and finally restore the global aggregation result through a noise cancellation mechanism.

[0036] Optionally, in the first implementation manner of the second aspect of the present invention, the construction module includes:

[0037] A construction sub-module, which is used to independently construct polynomials for each element of the matrix and distribute the polynomial calculation results as sub-secrets;

[0038] A building sub-module, which is used to build a distribution matrix, and the non-diagonal elements in the distribution matrix contain the shared sub-secrets between clients.

[0039] Optionally, in the second implementation manner of the second aspect of the present invention, the decryption module includes:

[0040] A preliminary decryption sub-module, which is used to perform preliminary decryption by right-multiplying the transpose of the global orthogonal matrix;

[0041] A reconstruction sub-module, which is used to reconstruct the private encryption matrix by Lagrange interpolation method;

[0042] A cancellation sub-module, which is used to perform left-multiplication decryption operations in rounds, and finally obtain the unencrypted global matrix through a noise cancellation mechanism.

[0043] In the technical solution provided by the present invention, S1: The central server performs SVD decomposition on the initial model to generate a row representation matrix and a column representation matrix, and determines row and column sampling index values through the Nystrom method and sends them to the client; the client generates a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix, and after using Gram-Schmidt orthogonalization processing, uses double orthogonal transformation to encrypt the local model matrix; the private encryption matrix is divided into N sub-shares through a threshold secret sharing mechanism, and a t-1 degree polynomial is constructed to achieve secret distribution to form a distribution matrix; the central server performs weight sorting rather than aggregation on the encrypted matrices uploaded by the client to generate a sorted set of encrypted matrices; the client gradually removes the encrypted matrices through an iterative decryption process, using the transpose of the orthogonal matrix and secret sharing reconstruction, and finally restores the global aggregation result through a noise cancellation mechanism; the present invention designs a decentralized orthogonal encryption framework based on symmetric noise, realizes efficient encryption and lossless decryption of gradients, and effectively resists inference attacks from semi-honest servers, clients, and malicious third parties; innovatively combines symmetric noise with the secret sharing mechanism. By introducing symmetric noise, the privacy of intermediate values in the gradient decryption process is protected, and at the same time, the secret sharing mechanism is used to ensure the security of the private orthogonal matrix and prevent any single party from reconstructing private information; a lightweight gradient privacy encryption and decryption scheme, while ensuring privacy protection, only has a minimal impact on accuracy, and does not rely on a third-party trusted institution, realizing efficient and secure communication and computing in federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0045] Figure 1 It is a flowchart of the federated learning method for decentralized orthogonal encryption based on symmetric noise provided by an embodiment of the present invention;

[0046] Figure 2 It is a visualization schematic diagram of the DLG attack on the MNIST dataset provided by an embodiment of the present invention;

[0047] Figure 3 It is a visualization schematic diagram of the iDLG attack on the MNIST dataset provided by an embodiment of the present invention;

[0048] Figure 4 It is a visualization schematic diagram of the DLG attack on the CIFAR10 dataset provided by an embodiment of the present invention;

[0049] Figure 5 It is a visualization schematic diagram of the iDLG attack on the CIFAR10 dataset provided by an embodiment of the present invention;

[0050] Figure 6 It is a schematic diagram of accuracy comparison on the MNIST dataset provided by an embodiment of the present invention;

[0051] Figure 7 It is a schematic diagram of accuracy comparison on the CIFAR10 dataset provided by an embodiment of the present invention;

[0052] Figure 8 It is a schematic diagram of the influence of different Rank values on the accuracy of the embodiment of the present invention on the MNIST dataset;

[0053] Figure 9 It is a schematic diagram of the influence of different Rank values on the accuracy of the embodiment of the present invention on the CIFAR10 dataset;

[0054] Figure 10 It is a schematic diagram of the structure of the federated learning system for decentralized orthogonal encryption based on symmetric noise provided by an embodiment of the present invention. Specific embodiments

[0055] Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned drawings of the present invention are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order different from that shown or described here. In addition, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0056] Embodiment 1

[0057] Please refer to Figure 1 , this embodiment provides a federated learning method for decentralized orthogonal encryption based on symmetric noise, including the following steps:

[0058] S1. The central server performs SVD decomposition on the initial model to generate a row representation matrix and a column representation matrix, and determines row and column sampling index values through the Nystrom method and distributes them to the clients;

[0059] S2. The client generates a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix. After using Gram-Schmidt orthogonalization processing, a double orthogonal transformation is used to encrypt the local model matrix;

[0060] S3. Split the private encryption matrix into N sub-shares through the threshold secret sharing mechanism, construct a polynomial of degree t - 1 to achieve secret distribution, and form a distribution matrix;

[0061] S4. The central server performs weight sorting rather than aggregation on the encryption matrices uploaded by the clients, and generates a set of sorted encryption matrices;

[0062] S5. The client uses the transpose of the orthogonal matrix and secret sharing reconstruction through an iterative decryption process to gradually remove the encryption matrices, and finally recovers the global aggregation result through the noise cancellation mechanism.

[0063] In this embodiment, S1: SVD initialization: The CS decomposes the initialized model layer by layer into two sub-matrix sets and , which represent the row representation matrix and the column representation matrix respectively, and obtain fixed row and column sampling index values and through random sampling of the gradients by the Nystrom method. The generated sub-matrices and row and column sampling index values are sent to all clients for training and subsequent encryption operations.

[0064] In this embodiment, S2: Generation of the encryption matrix: Theorem 1 proves that the product of any orthogonal matrices is still an orthogonal matrix, and data perturbation can be performed through orthogonal matrix transformation, just like adding noise in differential privacy (DP) for privacy protection. Theorem 2 proves that the perturbed and have the same singular values, and the singular vectors of and can be orthogonally transformed with each other, so that the singular vectors of and can be obtained by removing matrix perturbation from the singular vectors of . Therefore, through random orthogonal matrix multiplication, effective data perturbation can be achieved without changing the data characteristics, so as to achieve the purpose of privacy protection. However, if both matrices and are publicly shared, the encryption is in vain. At the same time, if both are privatized, lossless decryption is very difficult and costly. We hope that the encryption of the client can be both public and private like public and private keys. Therefore, the client will generate a locally private orthogonal transformation encryption matrix and a globally shared orthogonal transformation encryption matrix during initialization.

[0065] Theorem 1: For any ​The product of orthogonal matrices is still an orthogonal matrix;

[0066] Proof. Assume that and are both orthogonal matrices, that is, they satisfy and . Verify whether their product is still an orthogonal matrix. First, calculate :

[0067] ;

[0068] Next, verify whether is equal to :

[0069] ;

[0070] Using the associative law of matrix multiplication, the above formula can be expanded:

[0071] ;

[0072] Therefore, satisfies the definition of an orthogonal matrix.

[0073] Theorem 2: For any matrix , its SVD is . If two random orthogonal matrices and are used to perturb to form the encrypted matrix , the encrypted matrix shares the same singular values as the original matrix and . Further, through the known orthogonal matrices , the singular vectors of the encrypted matrix can be converted back to the singular vectors of the original matrix

[0074] Proof. Substitute the decomposition of and use the associative law of matrix multiplication:

[0075] ;

[0076] It can be verified by Theorem 1 that and are also orthogonal matrices. Let :

[0077] ;

[0078] Therefore, the encrypted matrix has the same singular values as the original matrix ​ Decompose under the condition of , indicating that the singular values of the matrix remain unchanged after orthogonal transformation.

[0079] In this embodiment, S3: Secret sharing of the private encryption matrix: Although we can encrypt data through orthogonal matrix transformation, the multiplication operation of the orthogonal matrix does not retain the additive homomorphic property. Therefore, after the CS end aggregates the encryption matrix, it is often difficult for the client to decrypt it losslessly. Therefore, in order to achieve secure and trusted aggregation, we design an iterative solution secret sharing method. The client receives unaggregated ciphertext data and decrypts the data step by step through its own private key and the globally shared key.

[0080] In this stage, our goal is to perform threshold secret sharing on the client's private encryption matrix so that even if the server or some clients obtain part of the encrypted matrix data, they cannot recover the original matrix. Only when enough clients work together can the original private matrix be recovered. Specifically, we divide each client's private encryption matrix (such as or ), and divide it through the secret sharing protocol and share these divided shares with other clients.

[0081] The threshold secret sharing scheme consists of a sharing operation Share() and a recovery operation Rec(). In the sharing operation Share(), the secret distributor randomly generates a -degree polynomial:

[0082] ;

[0083] where is the secret of the secret distributor , , , …, are randomly selected coefficients, is the required minimum sharing quantity, is a large prime number used to define the finite field , and its value must be large enough to ensure that both the secret and the coefficients of the polynomial can be represented in this field. The secret distributor selects different values , calculates the polynomial , and distributes the calculation result of the polynomial as a shared sub-secret to the corresponding participants . Because is a matrix rather than a coefficient, the shape of needs to be the same as the shape of . And the sharing method of each element of the matrix is independent, so in this article is not affected by The influence of the shape, that is, each matrix element is mapped to the constant term of a polynomial, and the coefficients of the polynomial generate sub-secrets for each matrix element:

[0084] ;

[0085] where is the matrix with the number of rows and columns being respectively, corresponding to the coefficients of in the polynomial. For each participant, the shared result is the secret matrix , :

[0086] ;

[0087] Finally, the distribution matrix is formed:

[0088] ;

[0089] In addition to the private matrices , and the shared matrices , , each client participating in the training also has other secret sub-shares , shared by other participating clients, but any single client cannot know the complete of other clients.

[0090] In this embodiment, S4: Client Encryption: In this stage, each client will generate two random real matrices through two integer random seeds and in the initial stage. is used to generate the private random matrix , then generates the public random matrix . To ensure that the matrices have orthogonal properties, we use the Gram - Schmidt scheme to orthogonalize them, generating the private encrypted orthogonal matrix and the shared encrypted matrix to be collaboratively computed. Then, through secure multi - party computation (SMPC), the global shared orthogonal matrix , is generated. The CS will perform a cumulative multiplication calculation on all participating in client training to obtain the globally shared orthogonal transformation encrypted matrix , Therefore, the encryption of sub - matrices can be achieved by multiplying the encryption matrices on the left and right respectively, and the process can be expressed as:

[0091] ;

[0092] To enhance privacy protection, we introduce an encryption method based on symmetric noise. Specifically, before encrypting the matrix, a noise term is added to the plaintext data to increase privacy protection during the decryption process. This method can ensure that even if an attacker obtains partial information during the iterative decryption process, it is difficult to recover the original data.

[0093] In this embodiment, each client adds noise to the original matrix during encryption, and the formula is as follows:

[0094] ;

[0095] where is the introduced symmetric noise term. Even if the client secret-shares and gradually decrypts and , the intermediate results obtained are also noisy interference values. Only when the secret-sharing process ends can the property that the noise mean is zero be utilized to obtain the denoised decryption result in a convergent manner.

[0096] In this paper, the noise is part of the encrypted data and is finally processed together through threshold decryption. Therefore, it does not introduce additional communication overhead due to the introduction of a new encryption strategy and avoids the leakage of sensitive information during the intermediate process.

[0097] In this embodiment, S5: Server sorting: The aggregation process of the traditional SVD-based FL model is that the CS side calculates the sum of the products of the matrices and weights of all clients respectively:

[0098] ;

[0099] where the weight , represents the proportion of the dataset size of each client in the total dataset. Since this framework uses a non-homomorphic encryption strategy, directly performing encrypted aggregation in the presence of the private matrix will make it impossible for the client to recover valid information losslessly from the aggregation result, and may instead increase the complexity of decryption, thus affecting the performance of the model. To decrypt the aggregated matrix and such hybrid encryption security models losslessly, in this framework, the CS does not perform actual addition operations on the uploaded data, but sorts the encrypted data according to the client number . Therefore, the encrypted information received by the client is as follows:

[0100] ;

[0101] And there is a characteristic that the noise mean is zero:

[0102] .

[0103] S6: Client decryption: After the client receives the and sent by CS, it decrypts by multiplying the right side by the transpose of the globally shared orthogonal transformation matrix to obtain , :

[0104] ;

[0105] ;

[0106] Since is an orthogonal matrix and satisfies , so can be easily removed during decryption. For the privately encrypted matrix that is secretly shared, we recover the secret through polynomial interpolation. During the secret reconstruction operation Rec(), it is necessary to or more than participant members jointly reconstruct the secret before jointly reconstructing . Specifically, the matrix elements recover the secret by solving the following Lagrange interpolation:

[0107] ;

[0108] When the in the polynomial takes the value of 0, the reconstructed secret element is obtained, and the final result of the inverse mapping reconstruction is the secret matrix .

[0109] After reconstructing the encrypted matrix, we can gradually left-multiply the transpose of the encrypted matrix to decrypt the global left and right sub-matrices losslessly, as shown in formula (19):

[0110] ;

[0111] where represents the result of the client secretly sharing under the secret sub-share , and is a decryption function. Each client uses to partially decrypt the left encrypted matrix. Taking the th client's as an example, the iterative decryption process of the secret sharing is as follows:

[0112] ;

[0113] ;

[0114] …

[0115] ;

[0116] And so on until all clients jointly reconstruct the global matrix .

[0117] Although in the step-by-step decryption process, the secretly transposed matrix obtained by recursive multiplication will expose the left and right sub-matrices of each client, due to the introduction of the noise term, the intermediate results are only the disturbed values after adding noise. Only after the secret sharing process ends can the property that the mean value of the noise is zero be used to obtain the denoised decryption result in a convergent manner.

[0118] Analysis of experimental results:

[0119] The performance of the experimental platform adopted in the embodiments of the present invention: It is configured with 1 Intel(R) Xeon(R) Gold 6330 CPU with a main frequency of 2.00 GHz, equipped with 32 × 4 GB of memory and an NVIDIA GeForce RTX 4090 GPU, running the Ubuntu 20.04.4 LTS operating system, the programming language is Python3.10, and PyTorch2.0.0 is used for model training. Set the number of FL iterations T = 300, the number of users N = 20, and all clients participate in each iteration.

[0120] Experiments were conducted on 2 datasets to evaluate the effectiveness of the proposed method:

[0121] MNIST: It is a computer vision dataset widely used in machine learning. It consists of 70,000 grayscale images of handwritten digits from 0 to 9, of which 60,000 are used for training and 10,000 are used for testing. Each image is a 28×28 pixel grayscale image representing the digits from 0 to 9. On the MNIST dataset, we used the classic LeNet-5 model for experiments. LeNet-5 is a lightweight convolutional neural network suitable for processing simple grayscale image classification tasks.

[0122] CIFAR10: It is also commonly used in image classification tasks in machine learning. It consists of 60,000 color images in 10 categories, with 50,000 images for training and 10,000 images for testing. Each image is a 32×32 pixel square and is divided into 10 categories such as birds, trucks, airplanes, dogs, deer, cats, boats, horses, frogs, and cars. Due to the larger image size and the diversity of objects in the images, the CIFAR10 dataset is more challenging than SVHN. On the CIFAR-10 dataset, we adopted the ResNet-34 model, whose deep residual network structure can effectively handle the larger image size and diversity issues in the CIFAR-10 dataset while providing high classification accuracy.

[0123] Figure 2 Shows the image reconstruction iteration process of the DLG attack on the MNIST dataset. On simple datasets, the attacker can restore a reconstruction result close to the original image. After increasing the confusion value and sparsity of the gradient, the image reconstruction quality decreases, but the category of the reconstructed image can still be distinguished on simple datasets.

[0124] Figure 3 Shows the image reconstruction iteration process of the iDLG attack on the MNIST dataset. iDLG replaces the SGD algorithm of the original DLG by introducing the adaptive momentum optimizer (Adam). The gradient backpropagation mechanism can better capture local detail features, and visually it is closer to the continuity and smoothness of real brushstrokes.

[0125] Figure 4 、 Figure 5 Represent the image reconstruction iteration processes of the DLG attack and the iDLG attack on the CIFAR10 dataset respectively. Compared with the MNIST dataset, the reconstructed images on CIFAR10 are more difficult. After increasing the confusion value and sparsity of the gradient, the image reconstruction quality decreases, and it is almost impossible to reconstruct the original image.

[0126] Figure 6 、 Figure 7 Show the model accuracies of different defense methods on the MNIST and CIFAR10 datasets respectively. The accuracy of our method on each dataset is not much different from that of FedAvg. In contrast, for the traditional defense methods DP and GC, the accuracy on the simple dataset MNIST may not be affected, but when on more complex datasets, the accuracies of DP and GC decrease significantly. This shows that our lossless decryption method can better maintain the accuracy when dealing with more complex datasets and does not significantly affect the model performance while protecting privacy.

[0127] Figure 8 、 Figure 9It respectively shows the influence of different Rank values on the accuracy on the MNIST and CIFAR10 datasets. On relatively simple datasets such as MNIST, as the Rank value decreases, the accuracy changes little, only decreasing by 0.36%, indicating that the framework can maintain a high model performance at a lower Rank. While on more complex datasets, the decrease in the Rank value leads to a slightly more obvious decrease in accuracy, which decreases by 2.61% on CIFAR-10.

[0128] Security analysis:

[0129] To prove that the encrypted mask matrix is secure in the FedSND framework, we use formulas and mathematical derivations to demonstrate how the mask matrix ensures data privacy and anti-attack ability. The following are the relevant formulas and proofs for the encrypted mask matrix and its security.

[0130] A. Privacy analysis against reconstruction attacks

[0131] An attacker attempts to recover the central matrix through the known sampling indices and :

[0132] ;

[0133] When the private matrices of all clients are the same, i.e., , then , . However, when the private matrices are different, the aggregated global sub-matrix is a matrix that depends on and cannot be simply extracted, so , . For ease of representation, we use the symbol to represent the abstract complex relationship matrix conventioned from and . When the attacker recovers the central matrix, they will reconstruct the original gradient matrix and try to solve the following equation:

[0134] ;

[0135] We can see that after the reconstructed original gradient matrix is decrypted using the shared matrix , it still has the with abstract complex relationships, which is mathematically equivalent to solving a high-dimensional linear equation system:

[0136] ;

[0137] Therefore, when the orthogonal matrices of the clients are random, it ensures that the server cannot restore the secret without stealing enough private matrices.

[0138] B. Threshold Secret Sharing Mechanism Based on Symmetric Noise

[0139] In FedSND, the system uses the Shamir threshold secret sharing scheme with the threshold set to , and at least shares of the secret are required to reconstruct the original secret . Suppose there are attackers who jointly attempt to recover the secret . The attackers can obtain shares and the corresponding points . This allows the attackers to list the following linear equations:

[0140] ;

[0141] These systems of equations contain unknowns: . However, the attackers only have equations, so this system of equations is under - determined, and the solution set contains infinitely many possible polynomials and cannot be uniquely determined by linear algebra methods or . When , the attackers obtain enough shares to be able to completely recover the polynomial through interpolation, thus leaking the secret .

[0142] Since the values sent by the server are sorted rather than aggregated, the intermediate values during the step - by - step decryption by the client will be stolen and inferred by the attackers. However, FedSNDOE adds symmetric noise during the encryption phase, so the intermediate values after decrypting the recovered secret are fuzzy values that differ greatly from the initial values, and the attackers cannot accurately infer the original data. It is not until all the intermediate values are decrypted that the positive and negative noises can cancel each other out in an aggregated manner.

[0143] Embodiment 2

[0144] Please refer to Figure 10 , which is the structural schematic diagram of the federated learning system for decentralized orthogonal encryption based on symmetric noise provided by the embodiments of the present invention. The method includes:

[0145] The SVD decomposition module is used for the central server to perform SVD decomposition on the initial model, generate a row representation matrix and a column representation matrix, and determine the row - column sampling index values through the Nystrom method and send them to the client;

[0146] An encryption module, which is used for the client to generate a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix. After using the Gram-Schmidt orthogonalization process, a double orthogonal transformation is adopted to encrypt the local model matrix;

[0147] A construction module, which is used to divide the private encryption matrix into N sub-shares through a threshold secret sharing mechanism, construct a polynomial of degree t - 1 to achieve secret distribution, and form a distribution matrix;

[0148] A sorting module, which is used for the central server to perform weight sorting rather than aggregation on the encryption matrices uploaded by the clients, and generate a sorted set of encryption matrices;

[0149] A decryption module, which is used for the client to gradually remove the encryption matrix through an iterative decryption process, using the transpose of the orthogonal matrix and secret sharing reconstruction, and finally recover the global aggregation result through a noise cancellation mechanism.

[0150] In this embodiment, the construction module includes:

[0151] A construction sub-module, which is used to independently construct polynomials for each element of the matrix, and use the polynomial calculation results as sub-secrets for distribution;

[0152] A building sub-module, which is used to build a distribution matrix, and the non-diagonal elements in the distribution matrix contain the shared sub-secrets between clients.

[0153] In this embodiment, the decryption module includes:

[0154] A preliminary decryption sub-module, which is used to perform preliminary decryption by right-multiplying the transpose of the global orthogonal matrix;

[0155] A reconstruction sub-module, which is used to reconstruct the private encryption matrix by Lagrange interpolation method;

[0156] A cancellation sub-module, which is used to perform left-multiplication decryption operations in rounds, and finally obtain the unencrypted global matrix through a noise cancellation mechanism.

[0157] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification are only preferred examples of the present invention, and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A federated learning method for decentralized orthogonal encryption based on symmetric noise, characterized in that It includes the following steps: S1. The central server performs SVD decomposition on the initial model to generate a row representation matrix and a column representation matrix, and determines the row and column sampling index values through the Nystrom method and distributes them to the clients; S2. The client generates a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix. After using the Gram-Schmidt orthogonalization process, a double orthogonal transformation is used to encrypt the local model matrix; S3. The private encryption matrix is divided into N sub-shares through a threshold secret sharing mechanism, and a t-1 degree polynomial is constructed to realize secret distribution, forming a distribution matrix; S4. The central server performs weight sorting rather than aggregation on the encrypted matrices uploaded by the clients, generating a sorted set of encrypted matrices; S5. The client gradually removes the encrypted matrices through an iterative decryption process, using the transpose of the orthogonal matrix and secret sharing reconstruction, and finally restores the global aggregation result through a noise cancellation mechanism.

2. The federated learning method for decentralized orthogonal encryption based on symmetric noise according to claim 1, wherein The global shared orthogonal matrix in step S2 is generated as follows: , ； Among them, and the matrix is distributively generated through secure multi-party computation.

3. The federated learning method for decentralized orthogonal encryption based on symmetric noise according to claim 1, wherein, The specific implementation of the secret sharing in step 3 includes: Independently construct a polynomial for each element of the matrix, and use the polynomial calculation result as the sub-secret for distribution; Construct a distribution matrix, where the non-diagonal elements in the distribution matrix contain the shared sub-secrets among the clients.

4. The federated learning method for decentralized orthogonal encryption based on symmetric noise according to claim 3, characterized in that, The polynomial construction includes a dynamic threshold adjustment mechanism: Calculate the effective threshold value in real time according to the online status of the clients, and dynamically adjust the number of Lagrange interpolation calculation nodes.

5. The federated learning method for decentralized orthogonal encryption based on symmetric noise according to claim 1, characterized in that, The server sorting process in step 4 includes: Perform a lexicographical sort on the encrypted client matrices according to the client numbers to ensure the discrete state of the encrypted data.

6. The federated learning method for decentralized orthogonal encryption based on symmetric noise according to claim 1, wherein The server sorting process in step 4 includes: Establish an encrypted data verification structure based on the Merkle tree, generate a root hash value for the sorted set of encrypted matrices, and the client confirms the data integrity and order consistency by verifying the hash chain.

7. A federated learning method for decentralized orthogonal encryption based on symmetric noise as claimed in claim 1, characterized in that, The iterative decryption process in step 5 includes: Perform preliminary decryption by right-multiplying the transpose of the global orthogonal matrix; Reconstruct the private encryption matrix through the Lagrange interpolation method; Execute the left-multiplying decryption operation in rounds, and finally obtain the unencrypted global matrix through the noise cancellation mechanism.

8. A federated learning system for decentralized orthogonal encryption based on symmetric noise, characterized in that, The system includes: An SVD decomposition module for the central server to perform SVD decomposition on the initial model, generate a row representation matrix and a column representation matrix, and determine the row and column sampling index values through the Nystrom method and distribute them to the clients; An encryption module for the client to generate a local private orthogonal encryption matrix and a global shared orthogonal encryption matrix. After using the Gram-Schmidt orthogonalization process, a double orthogonal transformation is used to encrypt the local model matrix; A construction module for dividing the private encryption matrix into N sub-shares through a threshold secret sharing mechanism, constructing a t-1 degree polynomial to realize secret distribution, and forming a distribution matrix; A sorting module for the central server to perform weight sorting rather than aggregation on the encrypted matrices uploaded by the clients, generating a sorted set of encrypted matrices; A decryption module for the client to gradually remove the encrypted matrices through an iterative decryption process, using the transpose of the orthogonal matrix and secret sharing reconstruction, and finally restore the global aggregation result through a noise cancellation mechanism.

9. A federated learning system for decentralized orthogonal encryption based on symmetric noise as claimed in claim 8, wherein The construction module includes: Constructor sub-module, which is used to independently construct polynomials for each element of the matrix, and distribute the polynomial calculation results as sub-secrets; Builder sub-module, which is used to build a distribution matrix, and the non-diagonal elements in the distribution matrix contain the shared sub-secrets among the clients.

10. A federated learning system for decentralized orthogonal encryption based on symmetric noise as claimed in claim 8, wherein The decryption module includes: Initial decryption sub-module, which is used to perform initial decryption by right-multiplying the transpose of the global orthogonal matrix; Reconstruction sub-module, which is used to reconstruct the private encryption matrix by Lagrange interpolation method; Cancellation sub-module, which is used to perform left-multiplication decryption operations in rounds, and finally obtain the unencrypted global matrix through the noise cancellation mechanism.

Images