Personalized federal learning method based on homomorphic encryption

Abstract

The invention discloses a personalized federal learning method based on homomorphic encryption, which comprises the following steps of: firstly, introducing constraint between a global model and a personalized model through a regularization method, adopting a separated updating strategy, performing multi-round updating on the personalized model by a client by utilizing local data, and performing multi-round updating on the personalized model by utilizing local data; and the personalized model parameters are uploaded to a server. Meanwhile, an optimization strategy is provided, namely, the strategy of dynamically adjusting the aggregation weight by utilizing the precision and frequency of each client on the public verification set is used, so that the global aggregation weight is dynamically adjusted, and the global model is updated more fairly and accurately. Besides, a model pool is introduced to realize preferential updating of a local model of the client, dynamic weight adjustment and model pool selection are both lightweight operations, a fully homomorphic encryption technology is adopted to realize ciphertext model aggregation, original precision / frequency data and model parameters of the client are ensured to be encrypted in the whole process in transmission and calculation, and gradient inversion attacks can be effectively defended.

Description

Technical Field

[0001] The present invention relates to the field of federated learning, and in particular to a personalized federated learning method based on homomorphic encryption. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, federated learning, as an emerging distributed machine learning framework, has received widespread attention. However, due to the challenge of data heterogeneity, the global model cannot be well generalized to the local data distribution of different clients. Therefore, personalized federated learning is a current research hotspot. On the other hand, although the personalized federated learning mechanism can collaboratively complete model training without leaving the local data sets of multiple participants, the problem of privacy data leakage still exists. Therefore, how to balance the personalized learning effect and privacy data security of personalized federated learning is the starting point of this invention.

[0003] The basic theory of federated learning technology is based on the independent and identically distributed assumption, which requires that the feature variables of each participant must satisfy statistical independence and obey the same probability distribution. However, in actual industrial application scenarios, this theoretical assumption faces severe challenges, mainly reflected in the problem of data heterogeneity: due to factors such as geographical restrictions and differences in business scale among participants, the local data sets they hold generally have limited sample sizes and uneven annotation quality; in addition, cross-regional collaboration may generate heterogeneity in the distribution of multimodal data; even more complicated, when using active learning mechanisms for sample screening, manual intervention may lead to selection bias and destroy the statistical independence between features. Therefore, in actual scenarios, the data of each participant often exhibits a highly non-independent and identically distributed (Non-IID) characteristic, resulting in a significant decrease in the generalization performance of the global model on local data.

[0004] The core idea of personalized federated learning is to provide customized models that adapt to the data characteristics of different clients by balancing the universality of the global model with the personalized needs of the local model. Existing methods can be divided into three categories:

[0005] The first category involves regularization-based collaborative optimization of global and local models. For example, the FedProx algorithm proposed by Li et al. introduces a regularization term in the local objective function to constrain the model from deviating from the global direction. However, static regularization weights are difficult to adapt to dynamic changes in client data quality, leading to significant interference from low-quality or malicious clients on the global model. The second category involves model fine-tuning, such as the local adaptation strategy proposed by Yu et al., in which the client fine-tunes a small amount of local data based on the global model to generate a personalized model. However, since the global model is the sole initialization source, if the global model and local data distribution differ significantly, the fine-tuned model may still deviate from the optimal solution. Furthermore, the client fails to leverage the historically optimal model, relearning previously acquired knowledge and reducing training efficiency. Furthermore, frequent fine-tuning may cause the model to forget global common features, impairing generalization and leading to catastrophic forgetting. The third category involves meta-learning-based frameworks that generate personalized initialization parameters through meta-optimization. For example, the Per-FedAvg algorithm proposed by Jiang et al. utilizes meta-gradient updates for fast local adaptation. However, this approach is complex in calculating meta-gradients, placing a heavy computational burden on the client, making it unsuitable for resource-constrained devices such as IoT sensors.

[0006] These methods mitigate the impact of non-IID data through different mechanisms, but still have different shortcomings. At the same time, the further development of personalized federated learning is also constrained by data privacy and security issues. Model parameters can be easily exploited by malicious attackers during the transmission process, leading to the leakage of data privacy of participants. It is known that the relinearization technology of the BGV (Brakerski-Gentry-Vaikuntanathan) homomorphic encryption algorithm can effectively control ciphertext expansion. After the server performs multiple rounds of multiplication operations, the ciphertext size remains relatively stable. The BGV algorithm has been widely used in practice. Therefore, BGV is combined to protect the data privacy of this solution and achieve efficient and secure model aggregation in an encrypted environment. Summary of the Invention

[0007] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0008] The present invention proposes a personalized federated learning method based on homomorphic encryption. First, a constraint is introduced between the global model and the personalized model through a regularization method. A separate update strategy is adopted. The client first uses local data to perform multiple rounds of updates on the personalized model, and then uploads the personalized model parameters to the server. At the same time, an optimization strategy is proposed, which is a strategy that dynamically adjusts the aggregation weight based on the accuracy and frequency of each client on the public verification set, thereby dynamically adjusting its global aggregation weight and updating the global model more fairly and accurately. In addition, a model pool is introduced to realize the preferential update of the client's local model. Both dynamic weight adjustment and model pool selection are lightweight operations. Fully homomorphic encryption technology is used to realize ciphertext model aggregation, ensuring that the client's original accuracy / frequency data and model parameters are fully encrypted during transmission and calculation, which can effectively defend against gradient inversion attacks.

[0009] To achieve the above objectives, the present invention proposes a personalized federated learning method based on homomorphic encryption, comprising:

[0010] Personalized federated learning: The server initializes the global model parameters and the parameters of each client's personalized model, and distributes the global model parameters to each client. The client generates a local model based on training accuracy, participation frequency, and training using local data, and performs weighted aggregation based on the client's training accuracy and the amount of information about the number of training participations. The client's local model is updated preferentially by introducing a model pool, and the initial model is selected from the global model, the historical optimal model, and the current local model for fine-tuning to select the starting point for the current round of updates. A regularization term is introduced to output the regularized optimized local personalized model parameters. The local personalized model parameters are updated multiple times using local data. Each round of optimization follows an iterative formula based on gradient descent. After multiple rounds of local updates, dynamic weights are calculated based on the amount of information about accuracy and frequency, and the models of all clients are aggregated using the dynamic weights to obtain a global model. The global model is updated based on the aggregated global model parameters.

[0011] Privacy data is protected based on the BGV homomorphic encryption algorithm: first, the model parameters, precision data, and frequency data generated based on personalized federated learning are input into the BGV homomorphic encryption algorithm on the client to generate ciphertext data; then the server uses the homomorphic addition operation supported by BGV to aggregate the precision ciphertext data and the frequency ciphertext data, and returns it to the client, thereby distributing the calculation of dynamic weights to each client; then the client sends the weight ciphertext calculated by itself to the server to participate in the global model secret calculation; combined with the homomorphic addition and multiplication characteristics of BGV, the server aggregates and generates secret global model parameters; finally, the client downloads the global model ciphertext, decrypts it using the BGV private key, and obtains the updated global model parameters.

[0012] This personalized federated learning method, based on homomorphic encryption, improves the model's robustness to heterogeneous data by introducing a regularization term. By separating the global model update from the optimization process of the client's personalized model, training efficiency is improved and communication costs are reduced. By dynamically adjusting the aggregation weight based on the accuracy and frequency of each client's information on the public validation set, the global model is updated more fairly and accurately. By enabling preferential updates of client-side local models through a model pool, personalized learning results are further improved and the model's generalization ability for non-IID data is enhanced. Secondly, considering the confidentiality of model-related data and combining the characteristics of the solution, BGV homomorphic encryption technology is used to protect the data security of personalized federated learning.

[0013] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 2 is an architectural diagram of a personalized federated learning method based on homomorphic encryption according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0018] The following describes a flowchart of a personalized federated learning method based on homomorphic encryption proposed according to an embodiment of the present invention with reference to the accompanying drawings.

[0019] like Figure 1As shown, the present invention introduces a regularization term to jointly optimize the global model and the client's personalized model, enabling each client's personalized model to fully adapt to local data while maintaining consistency with the global model. For federated learning scenarios with limited communication resources, this approach reduces communication overhead by separating the global model update from the client's personalized model optimization process. Furthermore, the design utilizes a separate update strategy, allowing clients to update their personalized models locally over multiple rounds without requiring frequent synchronization with the server, further improving training efficiency. To enhance personalization, the solution introduces a precision / frequency weighted aggregation algorithm. This algorithm dynamically adjusts the aggregation weight based on each client's accuracy and frequency on a public validation set, thereby dynamically adjusting the global aggregation weight and providing a fairer and more accurate update of the global model. Furthermore, a model pool is introduced to enable optimal updates of client-side local models, allowing initial models to be fine-tuned from the global model, the historically optimal model, and the current local model. This further enhances personalized learning and strengthens the model's generalization capabilities for non-IID data. This includes enhancing the model's generalization capabilities for non-IID data. In the face of data distribution drift or poor client data quality, the model pool mechanism can improve model robustness by selecting a more stable initial model (such as the historically optimal model). At the same time, fully homomorphic encryption technology is used to protect the accuracy data, frequency data, and gradient parameters of each client, and secure encrypted aggregation is performed on the server, which can effectively resist gradient inversion attacks and protect the data security of each client.

[0020] The personalized federated learning method based on homomorphic encryption in an embodiment of the present invention includes:

[0021] Personalized federated learning: The server initializes the global model parameters and the parameters of each client's personalized model, and distributes the global model parameters to each client. The client generates a local model based on training accuracy, participation frequency, and training using local data, and performs weighted aggregation based on the client's training accuracy and the amount of information about the number of training participations. The client's local model is updated preferentially by introducing a model pool, and the initial model is selected from the global model, the historical optimal model, and the current local model for fine-tuning to select the starting point for the current round of updates. A regularization term is introduced to output the regularized optimized local personalized model parameters. The local personalized model parameters are updated multiple times using local data. Each round of optimization follows an iterative formula based on gradient descent. After multiple rounds of local updates, dynamic weights are calculated based on the amount of information about accuracy and frequency, and the models of all clients are aggregated using the dynamic weights to obtain a global model. The global model is updated based on the aggregated global model parameters.

[0022] Privacy data is protected based on the BGV homomorphic encryption algorithm: first, the model parameters, precision data, and frequency data generated based on personalized federated learning are input into the BGV homomorphic encryption algorithm on the client to generate ciphertext data; then the server uses the homomorphic addition operation supported by BGV to aggregate the precision ciphertext data and the frequency ciphertext data, and returns it to the client, thereby distributing the calculation of dynamic weights to each client; then the client sends the weight ciphertext calculated by itself to the server to participate in the global model secret calculation; combined with the homomorphic addition and multiplication characteristics of BGV, the server aggregates and generates secret global model parameters; finally, the client downloads the global model ciphertext, decrypts it using the BGV private key, and obtains the updated global model parameters.

[0023] The embodiment of the present invention is implemented through the following scheme:

[0024] S1, first initialize the global model parameters w g , and initialize the parameter w for the corresponding personalized model on each client k , where k represents the kth client. On the server side, the initialization of the global model is usually based on random distribution, while the initial value of the client personalized model can directly inherit the initial value of the global model to ensure the consistency of all client models at the beginning. Afterwards, the server will use the global model w g Distributed to each client to provide a reference baseline for the local training process.

[0025] S2, introduces the public validation set on the client and uses the training accuracy Acc k and participation frequency f k As a weighting factor, in each round of training, the client trains the model using local data and sends the training accuracy and number of training sessions along with the gradient to the server. The server then performs a weighted aggregation based on the client's training accuracy and number of training sessions. This includes:

[0026] S201, after the client sends information to the server in each round, the server responds to the client's Acc k and f k Normalization is performed as follows:

[0027]

[0028] S202. Using only the weighted method of the client's data volume cannot guarantee the fairness of federated learning during the training process. The amount of information of the client will vary depending on the training accuracy and the frequency of participation. Therefore, the amount of information is used to measure the weight. Since each client's local data is different, the training accuracy between clients will vary greatly. Generally speaking, the lower the training accuracy, the more information the client needs to learn. This is also to solve the preference for certain clients generated during the federated learning process and achieve a fairer result. The amount of information of the client training accuracy is defined as follows:

[0029]

[0030] Where c is a very small constant close to 0, which is used to avoid the true value of the logarithm being 0.

[0031] S203: The more rounds a client participates in training, the more information it has. Conversely, the less information it has. The amount of information about the client's participation frequency is defined as follows:

[0032]

[0033] S204, Acc k _inf and f k The amount of information in _inf can be normalized as follows:

[0034]

[0035] S3 introduces the concept of a model pool to optimize local model updates. This strategy uses a selective update strategy to select the optimal model, enhancing the personalized effectiveness of federated learning. The model pool includes three main model sources: the current client's local model, the global model sent by the server in this round, and the optimal model selected in the previous update round. This mechanism dynamically adjusts the direction of local updates by comprehensively considering historical information and the current global model. This includes:

[0036] S301, let the model pool of client k in round t be Denote the model pool as It includes the following three models:

[0037] S3011, local model The model finally selected by the client in the previous round serves as the starting point for the current round of updates.

[0038] S3012, global model The global model distributed by the server in this round represents the global optimization direction.

[0039] S3013, the best model in the previous round The client selects the best model from the model pool in the previous update round.

[0040] S302, model optimization update strategy. When the client updates locally, it first selects the model with the best performance from the model pool as the initial model for training. On the local validation set The loss on The optimal strategy selects the model with the smallest loss As the initial model for the current update:

[0041]

[0042] S303, local update process: using the selected initial model Based on the local data D, through multiple rounds of gradient descent or other optimization methods, k Update the model to get the local model of the client in this round where η is the learning rate.

[0043] S304, update of model pool. After completing local training, the client selects the best model from the model pool again as the final model based on the validation set loss. Participate in the next round of model pool construction:

[0044]

[0045] Here, the model pool contains not only the model after the current round of training It also includes historical and global models to enable more robust selection.

[0046] S4 introduces a regularization term to penalize large changes in the current global model, reduce weight differences, and improve the robustness of the model to heterogeneous data.

[0047] S401, in each round of training, each client independently performs local optimization of the personalized model. The client optimization objective function is defined as:

[0048]

[0049] Among them, f k (w k ) represents the loss function of the k-th client, usually based on its w k Local data calculation; regularization term ||w k -w g || 2 Strengthened personalized model k With the global model w g, and λ represents the regularization parameter that controls the strength of the global model on the personalized model. Through this regularization mechanism, the personalized model can fully adapt to local data without deviating from the global learning goal.

[0050] S5, in order to achieve efficient model optimization in as few communication rounds as possible, adopts a separated update approach. Specifically, the client first uses local data to perform multiple rounds of updates on the personalized model. Each round of optimization follows an iterative formula based on gradient descent:

[0051]

[0052] Among them, η is the learning rate, which controls the step size of each update.

[0053] S6, after multiple rounds of local updates, the client uploads the personalized model parameters to the server instead of directly uploading local gradients or data, further reducing communication costs and protecting privacy. In the above process, the client obtains the local personalized model parameters, the accuracy information Acc k _inf and frequency information f k _inf, send these three parameters to the server.

[0054] S7, the server calculates the dynamic weight based on the learning frequency and training accuracy In this way, we participate in the aggregation process of the model.

[0055]

[0056] Where β + χ = 1. Based on the client's model accuracy and the frequency of learning participation, a formula for model aggregation is proposed. β and χ are the selected proportional effects on the aggregated model.

[0057] S8, in round t, the server uses dynamic weight Aggregate the models of all clients to get the global model

[0058]

[0059] The dynamic weight adjustment in this formula can effectively reflect the performance differences of each client model on the validation set, thereby incorporating more high-quality individual updates into the global model.

[0060] S9, the server starts the process of updating the global model. The update formula of the global model is:

[0061]

[0062] Here, γ represents the learning rate for global updates, and K is the total number of clients. Through this update formula, the global model absorbs information uploaded by all clients and incorporates it into its own weights, gradually approaching the optimal global model.

[0063] S10: After the global model is updated, the server redistributes the latest global model to the client, which uses it as the reference model for the next round of local optimization. This iterative process continues until the changes in the global model meet the set convergence criteria or the predetermined number of iterations is reached.

[0064] Specifically, the personalized federated learning solution is as follows:

[0065]

[0066] Furthermore, gradient data privacy protection is performed based on the BGV homomorphic encryption algorithm.

[0067] Understandably, a public-private key pair is generated by a trusted key management node, which then publishes the public key information and distributes the private key to each client via a secure channel. On the one hand, there are currently no mature symmetric encryption algorithms that support homomorphic computation, so public key encryption is the only option. On the other hand, the BGV algorithm is a typical, mature, fully homomorphic encryption scheme that has been widely used. For example, Martins et al. used the BGV scheme to achieve strong privacy and practical performance in the cloud, and it has been deployed in Microsoft browser software for password monitoring.

[0068] The BGV homomorphic encryption algorithm consists of the following four parts: key generation (KeyGen), encryption (Enc), decryption (Dec) and homomorphic operation (Eval), where homomorphic operations include addition homomorphism and multiplication homomorphism.

[0069] When generating keys in the initialization phase (i.e., executing the KeyGen algorithm), an encryption key pair is constructed by generating a random polynomial and adding noise, where the public key pk is used for encryption and the private key sk is used for decryption. Executing the encryption (Enc) algorithm means converting the original gradient w k , the amount of information Acc of training accuracy k _inf and the amount of information f of the participating frequencies k_inf is embedded in a polynomial ring, and ciphertext is generated using pk and random polynomial r. The encryption of personalized models, accuracy, and frequency prevents attackers from inferring the client data distribution through the original data, thereby ensuring privacy and security. In addition, by introducing the modular chain design, the noise is gradually reduced with the calculation depth, avoiding the risk of single-round noise overflow. The BGV homomorphic operation (Eval) during the aggregation phase includes addition and multiplication. The server performs these operations directly in the ciphertext space to ensure that the results after multiple calculations can still be decrypted into the correct original gradient information. During the decryption phase, the client executes the decryption algorithm (Dec), and the global gradient after the server-side homomorphic calculation is calculated through sk. The specific process is as follows:

[0070] Initialization phase: It is agreed that each client participating in federated learning holds an identical public and private key pair. The public and private key pairs are generated by a trusted key management node, which then publishes the public key information and distributes the private key to each client preparing to participate in federated learning over a secure channel.

[0071] Each participant performs iterative optimization of the personalized model parameters and then adjusts its local gradient w k , training accuracy Acc k and the number of training sessions f k Encrypt the data and then upload the encrypted parameters to the central server.

[0072] Personalized model encryption: C k In local data D k On training model w k , first w k Zoom in 10 n Multiply and round to get w k = "w k 10 n ", where "" is the rounding symbol; then encrypted as Enc(w k );

[0073] Accuracy encryption: The client is in the public verification set D val Calculate the model accuracy Acc k , first of all, Acc k Zoom in 10 m Acc is obtained by multiplying and rounding k = "Acc k 10 m ", and then encrypt it to Enc(Acc k );

[0074] Frequency encryption: The client is in the public verification set D val The calculation frequency f k , encrypted as Enc(f k );

[0075] Client upload {Enc(w k ),Enc(Acc k ),Enc(f k )} to the server;

[0076] In the aggregation phase, the central server receives the encryption parameters uploaded by all participants and performs aggregation calculations on these parameters in a secret state. The homomorphic characteristics of BGV allow homomorphic addition and multiplication calculations to be performed directly in the ciphertext state.

[0077] Aggregation stage 1:

[0078] The server aggregates the accuracy and training frequency in the encrypted state:

[0079]

[0080] The server sends the aggregated two secret parameters to the K clients.

[0081] Accuracy of client-side decryption aggregation:

[0082] The training frequency after client decryption aggregation:

[0083] Client to Acc k and f k Normalization is performed as follows:

[0084]

[0085] The amount of information the client uses to calculate training accuracy:

[0086]

[0087] The client calculates the amount of information for participation frequency:

[0088]

[0089] Client to Acc k _inf and f k _inf is normalized as follows:

[0090]

[0091]

[0092] The present invention balances the weights based on the frequency of learning and the accuracy of training to participate in the model aggregation process. The client calculates the weights of participating in the model aggregation process:

[0093] α k =βAcc k _inf+χf k _inf

[0094] The client first sets α k Zoom in 10 ξ Multiply and round to get α k = "α k 10 ξ ", then encrypt the value Enc(α k ), and finally Enc(α k ) is returned to the server.

[0095] Aggregation stage 2:

[0096] The server passes the secret dynamic weight Enc(α k ) Aggregate the secret model parameters of all clients to obtain the global model:

[0097]

[0098] The entire aggregation process is completed in ciphertext, and the server cannot obtain the plaintext model information of any participant;

[0099] The central server sends the aggregated encrypted global model parameters to each participant, and each participant decrypts them using their private key to obtain the updated global model parameters.

[0100] The central server sends the encrypted global model parameter Eval(w g ) is sent to all parties;

[0101] Each participant uses the private key to decrypt Eval(w g ):

[0102] w g =Dec sk (Eval(w g ))·10 -nξ

[0103] The client gets the updated global model parameter w g , and put it into the model pool as the reference model for the next round of local optimization. This parameter represents the global optimization direction.

[0104] According to the personalized federated learning method based on homomorphic encryption, a regularization method is first used to introduce constraints between the global model and the personalized model. This ensures that each client's personalized model is fully adapted to local data while maintaining consistency with the global model. A separate update strategy is employed: the client first updates the personalized model using local data for multiple rounds before uploading the personalized model parameters to the server, rather than directly uploading local gradient data. This reduces communication costs. An optimization strategy is also proposed, allowing the client to update the personalized model locally over multiple rounds without requiring frequent synchronization with the server, further improving training efficiency. Furthermore, a precision / frequency weighted aggregation algorithm is applied on the client side. This strategy dynamically adjusts the aggregation weight based on the accuracy and frequency of each client on a public validation set, thereby dynamically adjusting the global aggregation weight and providing a fairer and more accurate update of the global model. Furthermore, a model pool is introduced to enable optimal updates of the client's local model. This allows the client to select an initial model from the global model, the historically optimal model, and the current local model for fine-tuning, further improving personalized learning performance. Both dynamic weight adjustment and model pool selection are lightweight operations, eliminating the need for complex meta-optimization processes and adapting to edge device computing resources. Fully homomorphic encryption technology is used to achieve ciphertext model aggregation, ensuring that the client's original precision / frequency data and model parameters are encrypted throughout transmission and calculation, effectively defending against gradient inversion attacks. The BGV homomorphic encryption algorithm is used for data privacy protection because it can effectively control ciphertext expansion through relinearization technology. After executing multiple rounds of multiplication on the server, the ciphertext size remains relatively stable. Modulus switching technology is used to control noise growth by switching the ciphertext from one modulus to a smaller modulus, which is highly consistent with the proposed personalized federated learning framework.

[0105] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0106] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

Claims

1. A personalized federated learning method based on homomorphic encryption, characterized in that: include: Personalized federated learning: The server initializes global model parameters, initializes parameters for each client's personalized model, and distributes the global model parameters to each client; The client generates a local model based on training accuracy, participation frequency, and training using local data, and performs weighted aggregation based on the client's training accuracy and the amount of information on training participation times; By introducing a model pool to selectively update the client's local model, the initial model is selected from the global model, the historical optimal model, and the current local model for fine-tuning to select the optimal starting point for the current round of updates. A regularization term is introduced to output the regularized and optimized local personalized model parameters. The local personalized model parameters are updated multiple times using local data. Each round of optimization follows an iterative formula based on gradient descent. After multiple rounds of local updates, dynamic weights are calculated based on the amount of information about precision and frequency. The dynamic weights are then used to aggregate the models of all clients to obtain a global model. The global model is then updated based on the aggregated global model parameters. Protecting private data based on the BGV homomorphic encryption algorithm: First, on the client side, the model parameters, accuracy data, and frequency data generated based on personalized federated learning are input into the BGV homomorphic encryption algorithm to generate ciphertext data. The server then uses the homomorphic addition operation supported by BGV to aggregate the precision ciphertext data and frequency ciphertext data and returns it to the client, thereby distributing the calculation of dynamic weights to each client; the client then sends the weight ciphertext calculated by itself to the server to participate in the global model secret state calculation; combined with the homomorphic addition and multiplication characteristics of BGV, the server aggregates and generates secret global model parameters; finally, the client downloads the global model ciphertext, decrypts it using the BGV private key, and obtains the updated global model parameters.

2. The method according to claim 1, characterized in that After the client sends information to the server in each round, the server responds to the client's Acc k and f k Normalization is performed as follows: The amount of information about client training accuracy is defined as follows: Where c is a constant close to 0; The amount of information about client participation frequency is defined as follows: Acc k _inf and f k The information content of _inf is normalized as follows:

3. The method according to claim 2, characterized in that Assume that the model pool of client k in round t is Denote the model pool as Contains three models: local model The model finally selected by the client in the previous round is used as the starting point for the current round of updates; the global model The global model distributed by the server in this round represents the global optimization direction; Best model from the previous round The best model selected by the client from the model pool in the previous update round; Model optimization update strategy: When the client updates locally, it first selects the model with the best performance from the model pool as the initial model for training; On the local validation set The loss on The optimal strategy selects the model with the smallest loss As the initial model for the current update: Local update process: with the selected initial model Based on the local data D, through multiple rounds of gradient descent or other optimization methods, k Update the model to get the local model of the client in this round Where η is the learning rate; Update of the model pool: After completing local training, the client selects the best model from the model pool as the final model based on the validation set loss. Participate in the next round of model pool construction: The model pool contains not only the model trained in the current round Also includes historical models and global models.

4. The method according to claim 3, characterized in that In each round of training, each client independently performs local optimization of the personalized model; the client optimization objective function is defined as: Among them, f k (w k ) represents the loss function of the kth client; the regularization term ||w k -w g || Strengthen the personalized model w k With the global model w g ; λ represents the regularization parameter that controls the strength of the global model to the personalized model.

5. The method according to claim 4, characterized in that The personalized model is updated multiple times using local data. Each round of optimization follows an iterative formula based on gradient descent: Among them, η is the learning rate, which controls the step size of each update.

6. The method according to claim 5, characterized in that Calculate dynamic weights based on learning frequency and training accuracy Where β + χ = 1; a formula for model aggregation is proposed based on the client's model accuracy and the frequency of participation in learning; β and χ are the selected proportional effects on the aggregated model.

7. The method according to claim 6, characterized in that In round t, through dynamic weight Aggregate the models of all clients to get the global model 8. The method according to claim 7, characterized in that The server starts the process of updating the global model. The update formula of the global model is: Where γ represents the learning rate of global update and K is the total number of clients.

9. The method according to claim 1, characterized in that The BGV homomorphic encryption algorithm consists of four parts: key generation, encryption, decryption, addition homomorphic operation, and multiplication homomorphic operation.

10. The method according to claim 9, characterized in that When generating the key in the initialization phase, the key generation KeyGen algorithm is executed to construct an encryption key pair by generating a random polynomial and adding noise, where the public key pk is used for encryption and the private key sk is used for decryption; executing the encryption algorithm means converting the original gradient w k , the amount of information Acc of training accuracy k _inf and the amount of information f of the participating frequencies k _inf is embedded in a polynomial ring, and ciphertext is generated using pk and random polynomial r; the BGV homomorphic operations during the aggregation phase include addition and multiplication. During the decryption phase, the client executes the decryption algorithm and calculates the global gradient after the server-side homomorphic calculation through sk.

11. The method according to claim 10, characterized in that The method further comprises: Initialization phase: It is agreed that each client participating in federated learning holds an identical public and private key pair. The public and private key pairs are generated by a trusted key management node, which then publishes the public key information and distributes the private key to each client preparing to participate in federated learning through a secure channel. Each client participating in federated learning holds an identical public / private key pair. Each participant performs iterative optimization of the personalized model parameters and then adjusts its local gradient w k , training accuracy Acc k and the number of training sessions f k Encrypt and then upload the encrypted parameters to the central server; Personalized model encryption: C k In local data D k On training model w k , first w k Zoom in 10 n Multiply and round to get w k =「w k 10 n ", where "" is the rounding symbol; then encrypted as Enc(w k ); Accuracy encryption: The client is in the public verification set D val Calculate the model accuracy Acc k , first of all, Acc k Zoom in 10 m Acc is obtained by multiplying and rounding k = "Acc k 10 m ", and then encrypt it to Enc(Acc k ); Frequency encryption: The client is in the public verification set D val The calculation frequency f k , encrypted as Enc(f k ); Client upload {Enc(w k ),Enc(Acc k ),Enc(f k )} to the server; In the aggregation phase, the central server receives the encrypted parameters uploaded by all participants and performs aggregate calculations on these parameters in a secret state. The homomorphic characteristics of BGV allow homomorphic addition and multiplication calculations to be performed directly in the ciphertext state: Aggregation stage 1: The server aggregates the accuracy and training frequency in the encrypted state: The server sends the aggregated two secret parameters to the K clients; Accuracy of client-side decryption aggregation: The training frequency after client decryption aggregation: Client to Acc k and f k Normalization is performed as follows: The amount of information the client uses to calculate training accuracy: The client calculates the amount of information for participation frequency: Client to Acc k _inf and f k _inf is normalized as follows: The client calculates the weight of participating in the model aggregation process: a k =βAcc k _inf+χf k _inf The client first sets α k Zoom in 10 ξ Multiply and round to get α k = "α k 10 ξ ", and then encrypt to get Enc(α k ), and finally Enc(α k ) is returned to the server; Aggregation stage 2: The server passes the secret dynamic weight Enc(α k ) Aggregate the secret model parameters of all clients to obtain the global model: The entire aggregation process is completed in ciphertext, and the server cannot obtain the plaintext model information of any participant; The central server sends the aggregated encrypted global model parameters to each participant, and each participant uses the private key to decrypt and obtain the updated global model parameters; The central server sends the encrypted global model parameter Eval(w g ) is sent to all parties; Each participant uses the private key to decrypt Eval(w g ): w g =Dec sk (Eval(w g ))·10 -nξ The client gets the updated global model parameter w g , and put it into the model pool as the reference model for the next round of local optimization, and the parameter w g Represents the global optimization direction.

Images