Privacy-preserving personalized federated learning methods and apparatus for heterogeneous scenarios

Abstract

This invention discloses an efficient personalized privacy-preserving federated learning method for heterogeneous scenarios. The specific implementation steps include: 1. System initialization and key distribution; 2. Global model distribution; 3. Client base layer and personalization layer updates; 4. Encrypted upload and local update; 5. Weighted aggregation of the global model; 6. Global model update. This invention overcomes the limitations of heterogeneous data scenarios based on a "base + personalization layers" model, alleviating the problems of low global model accuracy and poor generalization caused by data heterogeneity. This invention designs a privacy-preserving framework based on CKKS fully homomorphic encryption, achieving client privacy protection during the federated learning process. Simultaneously, it significantly reduces computational and communication overhead and improves encryption efficiency under large-scale vectors and model parameters.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and more specifically, to a privacy-preserving personalized federated learning method and apparatus for heterogeneous scenarios. Background Technology

[0002] Data security is a critical factor in the application and development of federated learning, attracting widespread attention from governments, industries, and academia both domestically and internationally. However, due to the distributed nature of federated learning, training scenarios involve numerous discrete users with uneven computing power, data distribution, and network stability. Training in such heterogeneous environments can negatively impact the progress of cloud server aggregation tasks and the accuracy of the final global model. Furthermore, federated learning is currently applied in many sensitive information domains, such as finance, healthcare, and smart cities. Much of the local training data used by participating clients is private data, such as medical data and financial transaction records. Therefore, fine-grained information about individual data sources must not be disclosed during the entire training process. The emergence of various privacy attack schemes indicates that federated learning still faces a severe threat environment.

[0003] In their paper "Personalized Federated Learning using Hypernetworks (PMLR, 2021)," Shamsian et al. proposed a centralized personalized federated hypernetwork (pFedHN) structure. This structure generates unique and diverse local models for each user based on specific circumstances. Because pFedHN does not require transmitting hypernetwork parameters, the computational communication overhead is not affected by model size. The main technical description of this method is as follows: We denote the hypernetwork as h(·; φ) and the target network (i.e., the classification network) as f(·; θ). The hypernetwork is deployed on the server side. Each client obtains its corresponding parameters by sending an embedding vector v to the server. After training, the client sends the gradient of its network's parameters θ back to the server. The limitations of this method are: it primarily addresses the high communication overhead during the communication process, without designing privacy protection measures for the actual untrusted scenarios of federated learning; and the form of the hyperparameters is not applicable in some practical scenarios.

[0004] In their paper "Personalized Federated Few-Shot Learning" (TNNLS, 2022), Zhao et al. proposed a personalized federated learning framework, pFedFSL, which aims to learn a personalized and differentiated feature space for each client by encouraging more collaboration among clients with similar distributions. Unlike typical federated learning, pFedFSL performs model fusion in a distributed manner across multiple clients. First, each client receives a series of models sent by the server. The client-side computation calculates aggregate weights based on the model's fit to the local training data, performs personalized model aggregation locally, and updates the model using the updated model parameters and aggregate weights. The method sends the data to the server. Its drawbacks are: using a global model locally for personalized aggregation increases the computational overhead for each client; and the parameters passed between the client and server are high-dimensional gradients and model parameters, significantly increasing computational costs. Furthermore, the communication between the client and server lacks privacy protection, making it impossible to guarantee the security of gradients and models during transmission. Summary of the Invention

[0005] The purpose of this invention is to address the problem that, in existing heterogeneous scenarios, the accuracy of the global model jointly trained in a federated learning framework is even lower than that of the local models trained independently by each client, and it cannot meet the personalized needs of different clients. This invention proposes a personalized privacy-preserving federated learning method for heterogeneous scenarios. This method is used to train a high-precision, high-generalization, and high-availability personalized local model in a heterogeneous environment, ensuring that data from other clients is effectively utilized and that statistical differences in data do not lead to low accuracy or poor availability of the final model.

[0006] The technical approach to achieving the objectives of this invention is as follows: all user devices share a set of base layers with identical parameters, and each device has different personalization layers. These layers are suitable for the different training data of each client. During federated learning, the base layers are shared with the parameter server, while the personalization layers are kept private by each client. Simultaneously, we use CKKS homomorphic encryption technology to encrypt the model parameters exchanged during client-server communication, ensuring that the client's privacy information hidden in the model parameters is not leaked. Compared to other encryption algorithms, the CKKS encryption algorithm significantly reduces computational and communication overhead and improves encryption efficiency under large-scale vectors and model parameters.

[0007] To achieve the above objectives, the embodiments of this application provide the following technical solutions:

[0008] On one hand, this application provides a privacy-preserving personalized federated learning method for heterogeneous scenarios. The method includes: Step S1, constructing a first global model and initializing it, the initialization including setting the number of iteration rounds, learning rate, and aggregation update algorithm of the global model; Step S2, generating a public key for the local server and a private key corresponding to each client, and distributing the private key to the corresponding client, the public key and private key are used for encryption or decryption of global model parameters between the server and multiple clients; Step S3, encrypting the first global model using the public key, and distributing the encrypted first global model to multiple clients in sequence, so that the clients can decrypt it according to their local private key to obtain the plaintext first global model, the plaintext first global model is used to trigger the clients to update the base layer parameters to the first global model within the local training rounds, and perform gradient descent calculation to obtain the local base layer and personalized layer. The update process is as follows: First, the client encrypts the base layer model parameters using the public key as the update for this round, and sends the encrypted base layer model parameters and the amount of local training data to the server. Second, the client receives the base layer model parameters and the amount of local training data from each client, and calculates the corresponding aggregation weight based on these parameters. Third, the client performs weighted aggregation on the base layer model parameters from each client according to the aggregation weight to obtain a second global model. The second global model is then encrypted using the public key, and distributed to each client. Steps S3 to S5 are repeated until the preset number of iterations for the global model is reached. The second global model is the first global model after weighted aggregation. Fourth, the client outputs the second global model corresponding to the last iteration.

[0009] Optionally, in step S1, before constructing and initializing the first global model, the method further includes: initializing the local personalization layer and the base layer for each client.

[0010] Optionally, in step S3, the client updates the base layer parameters to the first global model within the local training epochs and performs gradient descent calculation, including:

[0011] The client counts the amount of local training data, determines the learning rate, number of local training rounds, and batch size based on its own needs, and uses the stochastic gradient descent algorithm for local updates to obtain the local updates of the base layer and the personalized layer.

[0012] Optionally, in step S3, encrypting the first global model using a public key includes:

[0013] Based on the properties of the CKKS homomorphic encryption algorithm, the model parameters can be fragmented with a length of k. The server node will update the global model in the t-th round. Divided into A fragment

[0014] Each shard result is encrypted using the CKKS encryption algorithm to obtain the ciphertext of the global model in round t. in This indicates the rounding up operation. Indicates the rounding down operation, [] pk This represents the CKKS encryption algorithm. Optionally, in step S3, the client uses the public key to encrypt the base layer model parameters and uses this as the update for the current round, including:

[0015] Client C j Statistical analysis of local training data D j The quantity is denoted as n j Determine the learning rate β, the number of local training rounds r, and the batch size b for local training, and use the SGD algorithm for local updates;

[0016] The local update using the SGD algorithm includes:

[0017] Client C j Received global model distributed by server Use private key sk Decryption is performed to obtain the plaintext global model.

[0018] Within the local training rounds k∈[1,r], the base layer parameters Update to global model Gradient descent is calculated according to the following formula to obtain the local updates of the base layer and the personalization layer in the t-th round;

[0019]

[0020] In a particular round of local training, for sample x, the forward computation operations of the base layer and the personalized layer are as follows:

[0021]

[0022] Where, positive integer K B and K p This represents the base layer and the personalization layer on each client, with the base layer weight parameter being... The weight parameters of the personalization layer are The corresponding vector activation function is and The empirical loss function used in training is shown below:

[0023]

[0024] Secondly, embodiments of this application provide a privacy-preserving personalized federated learning device for heterogeneous scenarios, the device comprising:

[0025] An initialization module is used to construct and initialize the first global model, which includes setting the number of iteration rounds, learning rate, and aggregation update algorithm of the global model;

[0026] The key module is used to generate a public key for the local server and a private key for each client, and to distribute the private key to the corresponding client. The public key and private key are used for encryption or decryption of global model parameters between the server and multiple clients.

[0027] The distribution module is used to encrypt the first global model using a public key and distribute the encrypted first global model to multiple clients in sequence, so that the clients can decrypt it using their local private keys to obtain the plaintext first global model. The plaintext first global model is used to trigger the clients to update the base layer parameters to the first global model within the local training rounds and perform gradient descent calculations to obtain the local updates of the base layer and personalized layers. Then, the clients use the public key to encrypt the base layer model parameters as the update for the current round and send the encrypted base layer model parameters and the amount of local training data to the server.

[0028] The receiving module is used to receive the base layer model parameters and the amount of local training data fed back by each client, and to calculate the corresponding aggregate weight based on the base layer model parameters and the amount of local training data.

[0029] The calculation module is used to perform weighted aggregation on the basic layer model parameters fed back by the client according to the aggregation weight to obtain a second global model, and to encrypt the second global model with the public key, distribute the encrypted second global model to each client, and repeat steps S3 to S5 until the preset number of iterations of the global model is reached. The second global model is the first global model after weighted aggregation.

[0030] The output module is used to output the second global model corresponding to the last iteration round number.

[0031] Thirdly, embodiments of this application provide a privacy-preserving personalized federated learning device for heterogeneous scenarios, the device including a memory and a processor.

[0032] The memory is used to store the computer program; the processor is used to execute the computer program to implement the steps of the above-described privacy-preserving personalized federated learning method for heterogeneous scenarios.

[0033] Fourthly, embodiments of this application provide a medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described privacy-preserving personalized federated learning method for heterogeneous scenarios.

[0034] The beneficial effects of this invention are as follows:

[0035] First, efficient and secure encryption protection of model parameters. Existing personalized federated learning schemes either operate in plaintext scenarios or employ perturbation-based privacy protection encryption techniques such as differential privacy to ensure security. These either fail to provide sufficient security guarantees for users or the introduced privacy protection techniques cause a loss of accuracy in the global model. Therefore, this invention uses a fully homomorphic encryption algorithm based on CKKS to encrypt model parameters containing hidden privacy data, achieving an efficient and secure federated learning process.

[0036] Second, resistance to heterogeneous environments. We use a "base + personalization layers" model for local training. Unlike traditional federated learning frameworks, we use local training data for personalization layer training. Furthermore, a global model trained on data from other clients is used to improve the accuracy of the personalization layers. This shields the global model's accuracy from the impact of heterogeneous data environments, allowing individual clients to obtain local personalized models with higher accuracy than the global model during joint training with other clients. -- The server calculates corresponding aggregation weights based on the model updates received from each client and the amount of data. For example, on one hand, the server performs weighted aggregation of client updates based on the weights to obtain the new round of global model parameters; on the other hand, the server encrypts the global model and distributes it to each client for the next round of local updates.

[0037] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of a privacy-preserving personalized federated learning method for heterogeneous scenarios as described in an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of a privacy-preserving personalized federated learning device for heterogeneous scenarios as described in an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of a privacy-preserving personalized federated learning device for heterogeneous scenarios as described in an embodiment of the present invention;

[0042] Figure 4 This is a behavioral interaction diagram of a privacy-preserving personalized federated learning method for heterogeneous scenarios, as described in this embodiment of the invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0044] It should be noted that similar reference numerals or letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0045] Before providing examples, a brief description of the distribution of data acquisition devices and power equipment is necessary. A fixed number of test samples are randomly selected from power equipment within a given area, and corresponding data acquisition devices are set on these samples to enable real-time detection of the equipment. Secondly, the specific types of power equipment to be monitored must be the same, such as transformers, cables, and substations. This embodiment will use high-voltage transformers as the subject of this description. Other power equipment can be identified based on the relevant principles disclosed in the embodiment to achieve the same partial discharge type identification, which will not be elaborated upon in this specification.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a privacy-preserving personalized federated learning method for heterogeneous scenarios, the method including steps S1, S2, S3, S4, S5 and S6.

[0048] Step S1: Construct the first global model and initialize it. The initialization includes setting the number of iteration rounds, learning rate and aggregation update algorithm of the global model.

[0049] Before constructing and initializing the first global model as described in step S1, the process may further include: initializing the local personalization layer and the base layer for each client.

[0050] Step S2: Generate the public key of the local server and the private key corresponding to each client, and distribute the private key to the corresponding client. The public key and private key are used for encryption or decryption of global model parameters between the server and multiple clients.

[0051] Step S3: Encrypt the first global model using the public key, and distribute the encrypted first global model to multiple clients in sequence, so that each client can decrypt it using its local private key to obtain the plaintext first global model. The plaintext first global model is used to trigger the client to update the base layer parameters to the first global model within the local training rounds, and perform gradient descent calculation to obtain the local updates of the base layer and personalized layer. Then, the client uses the public key to encrypt the base layer model parameters as the update for the current round, and sends the encrypted base layer model parameters and the amount of local training data to the server.

[0052] The specific operation of encrypting the first global model using a public key in step S3 can be as follows:

[0053] Step S31: Based on the properties of the CKKS homomorphic encryption algorithm, the encryption length k is used to shard the model parameters, and the server node updates the global model in the t-th round. Divided into A fragment

[0054] Step S32: Encrypt each shard result using the CKKS encryption algorithm to obtain the ciphertext of the global model in round t.

[0055]

[0056] in This indicates the rounding up operation. Indicates the rounding down operation, [] pk This indicates the CKKS encryption algorithm.

[0057] Step S4: Receive the base layer model parameters and the amount of local training data fed back by each client, and calculate the corresponding aggregate weights based on the base layer model parameters and the amount of local training data;

[0058] Step S5: Perform weighted aggregation on the basic layer model parameters fed back by the client according to the aggregation weight to obtain the second global model, encrypt the second global model with the public key, distribute the encrypted second global model to each client, and repeat steps S3 to S5 until the preset number of iterations of the global model is reached. The second global model is the first global model after weighted aggregation.

[0059] Step S6: Output the second global model corresponding to the last iteration round number.

[0060] Example 2

[0061] This implementation, based on Example 1, further illustrates the specific operation process in step S3 where the client uses a public key to encrypt the base layer model parameters as the update procedure for this round. Here, this implementation uses a single client C in the t-th round as an example. j Taking the training process as an example, the relevant explanation is as follows:

[0062] Client C j Statistical analysis of local training data D j The quantity is denoted as n j Determine the learning rate β, the number of local training rounds r, and the batch size b for local training, and use the SGD algorithm for local updates;

[0063] The local update using the SGD algorithm includes:

[0064] Client C j Received global model distributed by server Use private key sk Decryption is performed to obtain the plaintext global model.

[0065] Within the local training rounds k∈[1,r], the base layer parameters Update to global model Gradient descent is calculated according to the following formula to obtain the local updates of the base layer and the personalization layer in the t-th round;

[0066]

[0067] In a particular round of local training, for sample x, the forward computation operations of the base layer and the personalized layer are as follows:

[0068]

[0069] Where, positive integer K B and K p This represents the base layer and the personalization layer on each client, with the base layer weight parameter being... The weight parameters of the personalization layer are The corresponding vector activation function is and The empirical loss function used in training is shown below:

[0070]

[0071] Example 3

[0072] This embodiment is based on, as follows Figure 4 The interaction between the server and the client, as shown, illustrates the concept of this invention from multiple perspectives.

[0073] Step 1, Initialization;

[0074] For the server's global model and all clients (C = {C1, C2, ..., C...}), n The personalized model is randomly initialized, while the global model is initialized to... The j-th client C j The base layer is initialized as follows The personalization layer is initialized as follows:

[0075] Set the server's global iteration round number e and learning rate α. The server performs aggregation updates based on the FedAvg algorithm.

[0076] Generate a public key pk and a private key sk, which are used by the server and client to encrypt and decrypt model parameters.

[0077] Step 2: Encryption and distribution of the global model;

[0078] Since the federated learning process requires multiple rounds of training, this section will use the t-th round as an example for explanation.

[0079] The server uses the public key pk to access the global model. Encrypt to obtain

[0080] The server will encrypt the global model. Distributed to clients C1, C2, ..., C n .

[0081] Step 3, Local training;

[0082] Client C j Statistical analysis of local training data D j The quantity is denoted as n j Determine the local training learning rate β, the number of local training rounds r, and the batch size b, and use the SGD algorithm for local updates.

[0083] With a single client C in round t j Taking the training process as an example, C j Received global model distributed by Server Use private key sk Decryption is performed to obtain the plaintext global model.

[0084] Within the local training rounds k∈[1,r], the base layer parameters Update to global model Gradient descent is calculated according to the following formula to obtain the local updates of the base layer and personalization layer in the t-th round.

[0085]

[0086] For a given round of local training, the forward computation operations of the base layer and the personalization layer for sample x are as follows.

[0087]

[0088] Where, positive integer K B and K p This represents the number of base layers and personalization layers on each client, with the base layer weight parameter being... The weight parameters of the personalization layer are: The corresponding vector activation function is and The empirical loss function used in training is shown below. (See attached image) Figure 3 As shown.

[0089]

[0090] Step 4, upload the local update;

[0091] Client C j Use public key pk to access base layer parameters Encryption is performed to obtain This is an update for this round.

[0092] The client will encrypt the update model and data volume n j Return it to the server.

[0093] Step 4, global model aggregation update;

[0094] The server receives data from each client C. j ,j∈[1,n] receive the t-th round update and the amount of data n j The server calculates the corresponding aggregation weight γ. j The weight calculation method is as follows:

[0095]

[0096] The server performs weighted aggregation based on the weights to obtain the global model parameters for round t. The global model update method is as follows:

[0097]

[0098] The server will After encryption, Distribute to C1, C2, ..., C n Proceed to the next round of local updates. Global model updates will stop after reaching the set number of rounds e.

[0099] Example 4

[0100] like Figure 2 As shown, this implementation provides a privacy-preserving personalized federated learning device for heterogeneous scenarios, the device comprising:

[0101] Initialization module 71 is used to construct and initialize the first global model, wherein the initialization includes setting the number of iteration rounds, learning rate and aggregation update algorithm of the global model;

[0102] The key module 72 is used to generate a public key for the local server and a private key for each client, and to distribute the private key to the corresponding client. The public key and private key are used for encryption or decryption of global model parameters between the server and multiple clients.

[0103] The distribution module 73 is used to encrypt the first global model using a public key and distribute the encrypted first global model to multiple clients in sequence, so that the clients can decrypt it using their local private keys to obtain the plaintext first global model. The plaintext first global model is used to trigger the clients to update the base layer parameters to the first global model within the local training rounds and perform gradient descent calculations to obtain the local updates of the base layer and the personalized layer. Then, the clients use the public key to encrypt the base layer model parameters as the update for the current round and send the encrypted base layer model parameters and the amount of local training data to the server.

[0104] The receiving module 74 is used to receive the base layer model parameters and the amount of local training data fed back by each client, and calculate the corresponding aggregate weight based on the base layer model parameters and the amount of local training data.

[0105] The calculation module 75 is used to perform weighted aggregation on the basic layer model parameters fed back by the client according to the aggregation weight to obtain a second global model, and to encrypt the second global model with the public key, distribute the encrypted second global model to each client, and repeat steps S3 to S5 until the preset number of iterations of the global model is reached. The second global model is the first global model after weighted aggregation.

[0106] Output module 76 is used to output the second global model corresponding to the last iteration round number.

[0107] It should be noted that the specific manner in which each module performs its operation in the apparatus described in the above embodiments has been described in detail in the embodiments of the method, and will not be elaborated here.

[0108] Example 3

[0109] Corresponding to the above method embodiments, this disclosure also provides a privacy-preserving personalized federated learning device for heterogeneous scenarios. The privacy-preserving personalized federated learning device for heterogeneous scenarios described below and the privacy-preserving personalized federated learning method for heterogeneous scenarios described above can be referred to in correspondence.

[0110] Figure 3This is a block diagram illustrating a privacy-preserving personalized federated learning device 800 for heterogeneous scenarios, according to an exemplary embodiment. Figure 3 As shown, the electronic device 800 may include a processor 801 and a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0111] The processor 801 controls the overall operation of the electronic device 800 to complete all or part of the steps in the privacy-preserving personalized federated learning method for heterogeneous scenarios described above. The memory 802 stores various types of data to support the operation of the electronic device 800. This data may include, for example, instructions for any application or method operating on the electronic device 800, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 802 or transmitted via communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0112] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the privacy-preserving personalized federated learning method for heterogeneous scenarios described above.

[0113] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the privacy-preserving personalized federated learning method for heterogeneous scenarios described above. For example, the computer-readable storage medium may be the memory 802 including the program instructions described above, which may be executed by the processor 801 of the electronic device 800 to complete the privacy-preserving personalized federated learning method for heterogeneous scenarios described above.

[0114] Example 4

[0115] Corresponding to the above method embodiments, this disclosure also provides a readable storage medium. The readable storage medium described below corresponds to and can be referred to in relation to the privacy-preserving personalized federated learning method for heterogeneous scenarios described above.

[0116] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the privacy-preserving personalized federated learning method for heterogeneous scenarios described in the above method embodiments.

[0117] Specifically, the readable storage medium can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.

[0118] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A privacy-preserving personalized federated learning method for heterogeneous scenarios, characterized in that, The method includes: Step S1: Construct the first global model and initialize it. The initialization includes setting the number of iteration rounds, learning rate and aggregation update algorithm of the global model. Step S2: Generate the public key of the local server and the private key corresponding to each client, and distribute the private key to the corresponding client. The public key and private key are used for encryption or decryption of global model parameters between the server and multiple clients. Step S3: Encrypt the first global model using the public key, and distribute the encrypted first global model to multiple clients in sequence, so that each client can decrypt it using its local private key to obtain the plaintext first global model. The plaintext first global model is used to trigger the client to update the base layer parameters to the first global model within the local training rounds, and perform gradient descent calculation to obtain the local updates of the base layer and personalized layer. Then, the client uses the public key to encrypt the base layer model parameters as the update for the current round, and sends the encrypted base layer model parameters and the amount of local training data to the server. Step S4: Receive the base layer model parameters and the amount of local training data fed back by each client, and calculate the corresponding aggregate weights based on the base layer model parameters and the amount of local training data; Step S5: Perform weighted aggregation on the basic layer model parameters fed back by the client according to the aggregation weight to obtain the second global model, encrypt the second global model with the public key, distribute the encrypted second global model to each client, and repeat steps S3 to S5 until the preset number of iterations of the global model is reached. The second global model is the first global model after weighted aggregation. Step S6: Output the second global model corresponding to the last iteration round number.

2. The privacy-preserving personalized federated learning method for heterogeneous scenarios according to claim 1, characterized in that, In step S1, before constructing and initializing the first global model, the method further includes: initializing the local personalization layer and the base layer for each client.

3. The privacy-preserving personalized federated learning method for heterogeneous scenarios according to claim 1, characterized in that, In step S3, the client updates the base layer parameters to the first global model within the local training epochs and performs gradient descent calculation, including: The client counts the amount of local training data, determines the learning rate, number of local training rounds, and batch size based on its own needs, and uses the stochastic gradient descent algorithm for local updates to obtain the local updates of the base layer and the personalized layer.

4. The privacy-preserving personalized federated learning method for heterogeneous scenarios according to claim 1, characterized in that, In step S3, encrypting the first global model using a public key includes: Based on the properties of the CKKS homomorphic encryption algorithm, it is possible to encrypt length The model parameters are sharded, and the server node will shard the first... Round global model update Divided into A fragment ; Each fragment is encrypted using the CKKS encryption algorithm to obtain the first fragment. Global model ciphertext ; in This indicates the rounding up operation. This indicates a round-down operation. This indicates the CKKS encryption algorithm.

5. A privacy-preserving personalized federated learning device for heterogeneous scenarios, characterized in that, The device includes: An initialization module is used to construct and initialize the first global model, which includes setting the number of iteration rounds, learning rate, and aggregation update algorithm of the global model; The key module is used to generate a public key for the local server and a private key for each client, and to distribute the private key to the corresponding client. The public key and private key are used for encryption or decryption of global model parameters between the server and multiple clients. The distribution module is used to encrypt the first global model using a public key and distribute the encrypted first global model to multiple clients in sequence, so that the clients can decrypt it using their local private keys to obtain the plaintext first global model. The plaintext first global model is used to trigger the clients to update the base layer parameters to the first global model within the local training rounds and perform gradient descent calculations to obtain the local updates of the base layer and personalized layers. Then, the clients use the public key to encrypt the base layer model parameters as the update for the current round and send the encrypted base layer model parameters and the amount of local training data to the server. The receiving module is used to receive the base layer model parameters and the amount of local training data fed back by each client, and to calculate the corresponding aggregate weight based on the base layer model parameters and the amount of local training data. The calculation module is used to perform weighted aggregation of the basic layer model parameters fed back by the client according to the aggregation weight to obtain a second global model, and to encrypt the second global model using the public key. The encrypted second global model is then distributed to each client, and the work of the distribution module, the receiving module, and the calculation module is repeated until the preset number of iterations of the global model is reached. The second global model is the first global model after weighted aggregation. The output module is used to output the second global model corresponding to the last iteration round number.

Images