Federated learning method, apparatus, device, storage medium, and program product
By calculating the matching degree of update directions between the local model and the global model and performing differentiated encryption processing, the balance between data privacy and security and data value mining in federated learning is solved, and efficient cross-domain data sharing is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2026-02-04
- Publication Date
- 2026-06-19
AI Technical Summary
Existing federated learning technologies struggle to efficiently extract data value while ensuring data privacy and security when achieving cross-domain data sharing, and also suffer from issues such as degraded model performance and low processing efficiency.
By calculating the matching degree of update directions between the local model and the global model in the federated learning framework, encrypting the model parameters in a targeted manner, and performing differentiated processing at the central node, cross-domain data sharing is achieved.
While ensuring data privacy and security, it efficiently mines the value of cross-domain data, improving model performance and processing efficiency.
Smart Images
Figure CN122242811A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data, and more particularly to a federated learning method, apparatus, device, storage medium, and program product. Background Technology
[0002] In the digital economy era, cross-domain data sharing has become a key way to unlock the value of data, and its realization currently mainly relies on federated learning technology. Federated learning (FL) is a distributed machine learning technology that enables cross-domain data sharing by exchanging model parameters or intermediate results without sharing the original data, thus fully exploring the value of cross-domain data while ensuring data privacy and security.
[0003] However, to ensure the security of cross-domain data, federated learning techniques require privacy-preserving mechanisms to protect data when transmitting and processing model parameters or intermediate results. While these mechanisms protect data privacy, they often lead to a decrease in model performance. Furthermore, using complex processing methods for encrypted data, such as secure multi-party computation, while mitigating the impact on model performance, often results in reduced processing efficiency. Therefore, existing federated learning techniques struggle to find a suitable balance between efficiently extracting value from cross-domain data and ensuring data security. Summary of the Invention
[0004] This application provides a federated learning method, apparatus, device, storage medium, and program product. By encrypting the transmitted model parameters based on the matching degree between the update direction of the local model and the update direction of the global model before transmitting model parameters in the federated learning framework, a balance is achieved between efficiently mining the value of cross-domain data and ensuring data security.
[0005] Firstly, this application provides a federated learning method applied to multiple local nodes in a federated learning framework, which also includes a central node. The method includes:
[0006] Retrieve shared samples from local nodes; shared samples include data from shared data objects across multiple local nodes.
[0007] Based on the shared samples, the local model deployed on the local node is updated to obtain the local model parameters;
[0008] Based on the local model parameters, calculate the matching degree between the updated local model and the global model deployed on the central node in terms of update direction;
[0009] Based on the matching degree of the update direction, the local model parameters are encrypted, and the encrypted local model parameters and the matching degree of the update direction are uploaded to the central node, so that the central node can obtain the global model parameters of the global model based on the matching degree of the encrypted local model parameters and the update direction.
[0010] Secondly, this application provides a federated learning method applied to a central node in a federated learning framework, which also includes multiple local nodes. The method includes:
[0011] Based on the matching degree between the encrypted local model parameters and update direction uploaded by each local node, the fusion method of the local model parameters is determined;
[0012] Based on the matching degree of the update direction and the fusion method, the encrypted local model parameters and the global model parameters of the global model deployed in the central node are fused to obtain the fused global model parameters.
[0013] The merged global model parameters are sent to each local node to replace the local model parameters of the local model deployed on each local node.
[0014] Thirdly, this application provides a federated learning device applied to multiple local nodes in a federated learning framework, the federated learning framework also including a central node, the device comprising:
[0015] The acquisition module is used to acquire shared samples from local nodes; shared samples include data from shared data objects across multiple local nodes.
[0016] The parameter update module is used to update the local model deployed on the local node based on the shared samples to obtain the local model parameters;
[0017] The matching degree calculation module is used to calculate the matching degree between the updated local model and the global model deployed in the central node in terms of update direction, based on the local model parameters.
[0018] The encrypted upload module is used to encrypt the local model parameters based on the matching degree of the update direction, and upload the encrypted local model parameters and the matching degree of the update direction to the central node, so that the central node can obtain the global model parameters of the global model based on the matching degree of the encrypted local model parameters and the update direction.
[0019] Fourthly, this application provides a federated learning device for use as a central node in a federated learning framework, which also includes multiple local nodes. The device comprises:
[0020] The fusion method determination module is used to determine the fusion method of local model parameters in response to the matching degree of the encrypted local model parameters and update direction uploaded by each local node.
[0021] The global model parameter determination module is used to fuse the encrypted local model parameters and the global model parameters of the global model deployed on the central node based on the matching degree and fusion method of the update direction, so as to obtain the fused global model parameters.
[0022] The communication module is used to send the merged global model parameters to each local node to replace the local model parameters of the local model deployed on each local node.
[0023] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;
[0024] The memory stores instructions that the computer executes;
[0025] The processor executes computer execution instructions stored in memory to implement the methods provided in the first or second aspect above.
[0026] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the first or second aspect above.
[0027] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the first or second aspect above.
[0028] The federated learning method, apparatus, device, storage medium, and program products provided in this application achieve cross-domain data sharing based on federated learning technology. Specifically, in the local nodes of the federated learning architecture, the local model is trained using shared samples from the local nodes, updating the model parameters of the local model. Shared samples are only visible to the local nodes, and the original data does not leave the domain, ensuring the security of the original data. After updating the local model, the consistency of the update direction between the local model update and the global model deployed on the central node in the federated learning architecture is calculated, i.e., the matching degree of the update direction. This matching degree can represent the contribution of the local model parameters to the global model. Finally, based on the consistency of the update direction, model parameters with different contributions are encrypted and uploaded differently. By comprehensively considering the complexity of processing model parameters and the contribution of model parameters, the model parameters are processed differently, achieving efficient cross-domain data sharing. While ensuring data privacy and security, it can also fully and efficiently explore the value of cross-domain data. Attached Figure Description
[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0030] Figure 1 A schematic diagram of the structure of a federated learning framework provided in an embodiment of this application;
[0031] Figure 2 A flowchart illustrating a federated learning method provided in an embodiment of this application;
[0032] Figure 3 A flowchart illustrating another federated learning method provided in this application embodiment;
[0033] Figure 4 A flowchart illustrating yet another federated learning method provided in this application embodiment;
[0034] Figure 5 A flowchart illustrating another federated learning method provided in an embodiment of this application;
[0035] Figure 6 A schematic diagram of the structure of a federated learning device provided in an embodiment of this application;
[0036] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0037] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0038] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.
[0040] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.
[0041] It should be noted that the federated learning methods, devices, equipment, storage media, and program products provided in this application can be used in the field of big data, or in any field other than big data. The application fields of the federated learning methods, devices, equipment, storage media, and program products in this application are not limited.
[0042] This application applies to cross-domain data sharing scenarios in fields such as finance. Cross-domain data refers to heterogeneous data distributed across different organizations, regions, or systems. In the digital economy era, cross-domain data sharing has become a key way to unlock the value of data. For example, in the financial sector, banks and other financial institutions need to jointly analyze user behavior data to identify behavioral risks and improve the stability and security of financial institution operations. However, cross-domain data is scattered across different systems and often involves sensitive information. Due to limitations in protecting sensitive data, the original data cannot be directly shared. Therefore, how to achieve cross-domain data sharing without leaking the original data is of great significance.
[0043] Federated Learning (FL) is a distributed machine learning technique that allows participants to share data by exchanging model parameters or intermediate results without sharing the original data. This achieves the goal of making data available but not visible, and keeping the model active while the data remains stationary.
[0044] Figure 1 This is a schematic diagram illustrating the structure of a federated learning framework provided in an embodiment of this application. Figure 1 As shown, the federated learning framework provided in this application includes multiple local nodes and a central node, with each local node communicating with the central node. In this federated learning architecture, each local node deploys its own local model. Local nodes train their local models using local data, resulting in local models that rely solely on that local data. Each local node then shares its model parameters with the central node. The central node fuses these model parameters to obtain model parameters that incorporate cross-domain data. Finally, the model parameters incorporating cross-domain data are distributed to each local node, replacing the model parameters of the local models. The resulting local model thus achieves cross-domain sharing.
[0045] In federated learning, the transfer and processing of data, such as model parameters, between local nodes and the central node is achieved through a privacy-preserving computation mechanism. Specifically, local nodes upload encrypted model parameters to the central node, which then processes these parameters using the privacy-preserving mechanism. This ensures that while the original data is only visible to local nodes, it can be used by the central node, achieving a balance between privacy protection and data utilization.
[0046] However, the introduction of privacy-preserving computation mechanisms can reduce the value of processed data, ultimately impacting model performance and increasing the complexity of data processing. For example, if privacy protection is achieved by adding noise, the noise will reduce data quality, ultimately affecting the accuracy of the model's output. While using more complex encryption methods to directly compute model parameters in an encrypted state has a lower impact on data quality, the processing method is complex, affecting the efficiency of obtaining the final result. Therefore, existing federated learning techniques struggle to find a suitable balance between efficiently mining the value of cross-domain data and ensuring data security.
[0047] The federated learning method provided in this application aims to address the aforementioned technical problems of existing technologies. Specifically, after obtaining the local model parameters at the local node, the consistency of the update direction between the local model update and the global model update is calculated, i.e., the matching degree of the update direction. Furthermore, based on the matching degree of the update direction, the encryption or upload method of the local model parameters is selected differently, effectively achieving a balance between efficiently mining the value of cross-domain data and achieving data security.
[0048] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0049] Figure 2 This is a flowchart illustrating a federated learning method provided in an embodiment of this application. The federated learning method provided in this application is applied to multiple local nodes in a federated learning framework, such as... Figure 1 In addition to local nodes, the federated learning framework also includes a central node. For example... Figure 2 As shown, the method provided in this embodiment includes:
[0050] Step S201: Obtain the common samples of the local nodes.
[0051] This application can be applied to the financial sector. Specifically, in the financial sector, the local nodes in this application can be the systems or branch systems of various financial institutions, such as the financial systems corresponding to each branch. The central node is a pre-configured server that communicates with each local node.
[0052] The shared samples include data from shared data objects across multiple local nodes.
[0053] The data objects are the objects corresponding to each piece of data in the local node's database. These objects include users, products, etc.
[0054] For example, users included in multiple local nodes can be regarded as shared data objects, and the data corresponding to the shared data objects, such as user transaction data and user basic attribute data, can be used as shared samples.
[0055] For example, products included in multiple local nodes, such as fund investment products, can be regarded as common data objects, and the data corresponding to the common data objects, such as product basic data, product return information, and product transaction information, can be used as common samples.
[0056] In this step, shared data objects can be predefined. Local nodes retrieve data from the shared data objects from their local data, using this data as shared samples.
[0057] In some embodiments, the central node can be used to identify shared data objects. Local nodes obtain shared data objects from the central node and acquire their data as shared samples.
[0058] In this embodiment, local nodes and the central node can identify shared data objects based on the Private Set Intersection (PSI) technique. The PSI technique involves encrypting the identification information of data objects at the local node using encryption techniques (such as RSA encryption, hash functions, etc.) and uploading it to the central node. The central node then determines the shared data objects among the multiple local nodes based on the encrypted identification information uploaded by the local nodes.
[0059] By outputting only the encrypted identifier information of shared data objects, sample alignment is achieved without exposing the data objects owned by local nodes. This avoids using different samples to train each local model separately, as the parameters of each local model do not have the conditions for fusion, which ultimately affects the convergence of the global model.
[0060] Step S202: Update the local model deployed on the local node based on the shared samples to obtain the local model parameters.
[0061] A local model is a model deployed on various local nodes to implement specific functions. A local model can be a statistical model or a machine learning model. For example, a local model can be a user behavior risk identification model, a product information statistical model, or a product purchase probability prediction model. This application aims to integrate cross-domain data to update the model parameters of local models, without specifically limiting the type and function of the local model.
[0062] A global model is a model deployed at a central node used to fuse the parameters of various local models. Similarly, this application does not specifically limit the type and function of the global model.
[0063] Local model parameters are the variables that the local model needs to learn and optimize. They determine how the local model maps and transforms input data to accomplish a specific task. For example, in a local model based on linear regression, local model parameters include weight coefficients and bias coefficients. Different types of local models include different local model parameters, which are not limited here.
[0064] In this step, the local model deployed on the local node is updated using shared samples to obtain the updated local model parameters.
[0065] Specifically, the local model deployed on the local node is trained using shared samples, and the model parameters of the local model are updated.
[0066] For example, labels are added to the shared samples, representing the expected output of the local model. For instance, if the local model is a risk prediction model, the label for the shared samples is determined based on the actual situation corresponding to the sample, indicating whether an anomaly exists. If an anomaly exists, the label is set to "risk exists"; if no anomaly exists, the label is set to "no risk exists." The shared samples are then input into the local model built based on the existing model architecture, such as a machine learning model. The local model obtains its output based on its initial local model parameters. It should be noted that if the local model has no local model parameters, initial values need to be assigned to it, which can be random values or values set based on experience.
[0067] Based on the output and labels, calculate the performance metrics of the local model, such as loss value, precision, and recall. If the performance metrics do not meet the preset convergence conditions, such as the loss value being greater than or equal to the preset loss threshold, update the local model parameters and repeat the relevant steps of inputting common samples into the local model built based on the existing model architecture. If the performance metrics meet the preset convergence conditions, such as the loss value being less than the preset loss threshold, the local model update is considered complete, and the local model parameters are obtained.
[0068] Step S203: Based on the local model parameters, calculate the matching degree between the updated local model and the global model deployed on the central node in terms of update direction.
[0069] The update direction indicates the trend of model parameter changes during model updates. It shows how each parameter changes, such as increasing or decreasing. The consistency of the update direction reflects the contribution of the updated local model parameters to the convergence of the global model; generally, higher consistency indicates a greater contribution.
[0070] In this step, based on the local model parameters, the local changes in the updated local model parameters are calculated to determine the update direction of the local model. The global changes in the global model parameters deployed on the central node are obtained to determine the update direction of the global model. The consistency between the update directions of the local model and the global model is used to determine the matching degree of the update directions.
[0071] In some embodiments, when the central node updates the global model, it calculates the global changes in the global model parameters, i.e., the update direction of the global model, and sends this information to the local node. The local node can directly read the update direction of the global model.
[0072] In some embodiments, the update direction of the global model can be obtained by obtaining the model parameters of the local model before the update, and the global model parameters issued by the central node in the previous iteration or cycle. The changes in the model parameters of the local model before the update and the global model parameters issued by the central node in the previous iteration or cycle are calculated to determine the update direction of the global model.
[0073] Step S204: Based on the matching degree of the update direction, encrypt the local model parameters and upload the encrypted local model parameters and the matching degree of the update direction to the central node, so that the central node can obtain the global model parameters of the global model based on the encrypted local model parameters and the matching degree of the update direction.
[0074] In this step, the contribution of a local model parameter to the global model update can be determined based on the matching degree of the update direction. Then, differentiated processing is applied to local model parameters with different contributions.
[0075] In some embodiments, the local model parameters can be encrypted using a corresponding encryption method based on preset conditions that the matching degree of the update direction meets, and the encrypted local model parameters and the matching degree of the update direction can be uploaded to the central node.
[0076] The preset conditions include: if the matching degree is less than a preset threshold, then a more efficient encryption method is used to encrypt the local model parameters; if the matching degree is greater than or equal to the preset threshold, then a less noisy encryption method is used to encrypt the local model parameters.
[0077] In federated learning, after encrypting parameters, computation is required on the encrypted parameters without decryption. Processing parameters in the encrypted state introduces noise into the results. To avoid the impact of noise on the processing results, more complex techniques are needed, naturally leading to additional computational overhead and reduced processing efficiency. This application achieves a balance between accuracy and processing efficiency by selectively choosing encryption methods based on matching degree.
[0078] In some embodiments, local model parameters may be selectively encrypted and uploaded based on the degree of matching of the update direction.
[0079] Specifically, if the matching degree of the update direction is greater than or equal to the preset upload threshold, the corresponding local model parameters are encrypted, and the encrypted local model parameters and the matching degree of the update direction are uploaded to the central node. If the matching degree of the update direction is less than the preset upload threshold, the local model parameters are not encrypted or uploaded.
[0080] This embodiment only processes and uploads local model parameters that contribute significantly, without uploading local model parameters that contribute less, thus improving the processing efficiency of local nodes and the central node.
[0081] After the central node receives the uploaded encrypted local model parameters and the matching degree of the update direction, it merges the local model parameters and update directions of multiple local nodes to obtain the global model parameters. These global model parameters are then passed to each local node. Each local node can use the global model parameters, which incorporate cross-domain data, to replace its local model parameters, thus achieving cross-domain data sharing in the resulting local model.
[0082] In some embodiments, after replacing the local model parameters with global model parameters, a test set can be used to verify whether the local model has been trained successfully. Specifically, the test set is input into the local model, and the performance metric of the local model is calculated based on the output and the labels of the test set. If the performance metric does not meet the preset convergence condition, the method is repeated, i.e., the next round of training is performed, until the performance metric meets the preset convergence condition. The test set includes, but is not limited to, some shared samples.
[0083] In other words, the shared samples can be divided into a training set and a test set. The training set is used to perform step S202, and the test set is used to verify whether the local model has been successfully trained.
[0084] The federated learning method provided in this embodiment achieves cross-domain data sharing based on federated learning technology. Specifically, in the local nodes of the federated learning architecture, the local model is trained using the shared samples of the local nodes, updating the model parameters of the local model. The shared samples are only visible to the local nodes, and the original data does not leave the domain, ensuring the security of the original data. After updating the local model, the consistency of the update direction of the local model with the update direction of the global model deployed on the central node in the federated learning architecture is calculated, i.e., the matching degree of the update direction. This matching degree can represent the contribution of the local model's parameters to the global model. Finally, based on the consistency of the update direction, model parameters with different contributions are encrypted and uploaded in a differentiated manner. By comprehensively considering the complexity of processing model parameters and the contribution of model parameters, the model parameters are processed differently, achieving efficient cross-domain data sharing. While ensuring data privacy and security, it can also fully and efficiently explore the value of cross-domain data.
[0085] Figure 3 This is a flowchart illustrating another federated learning method provided in an embodiment of this application. This embodiment is... Figure 2 Based on the illustrated embodiment, a more detailed explanation of steps S203 and S204 is provided. For example... Figure 3 As shown, the method provided in this embodiment includes:
[0086] Step S301: Obtain the common samples of the local nodes.
[0087] The shared samples include data from shared data objects across multiple local nodes.
[0088] Step S302: Update the local model deployed on the local node based on the shared samples to obtain the local model parameters.
[0089] Step S303: Calculate the update direction of the local model parameters based on the local model parameters and the model parameters of the local model before the update.
[0090] In this step, the model parameters of the local model before the update are obtained. These local model parameters and the original local model parameters are then substituted into the gradient calculation formula or the increment calculation formula to obtain the update direction of the local model parameters.
[0091] For example, the local model parameters and the model parameters of the local model before the update are concatenated into vectors, and the difference between the vectors is used to determine the update direction of the local model parameters.
[0092] For example, the local model parameters and the model parameters of the local model before the update are concatenated into vectors, and the gradient between the two vectors is used to determine the update direction of the local model parameters.
[0093] Step S304: Obtain the update direction of the global model parameters of the global model deployed on the central node.
[0094] In this step, after updating the global model parameters of the global model deployed on the central node, the central node calculates the update direction of the global model parameters according to the method in step S303. The local node communicates with the central node to obtain the update direction of the global model parameters.
[0095] Step S305: Calculate the cosine similarity between the update direction of the local model parameters and the update direction of the global model parameters to obtain the matching degree of the update direction.
[0096] In this step, the local node calculates the cosine similarity between the update directions of the local model parameters and the global model parameters to obtain the matching degree of the update directions. Specifically, this can be represented as:
[0097]
[0098] Where cos(θ) is the matching degree of the update direction; ω local This indicates the direction of local model parameter updates; ω global This indicates the direction for updating the global model parameters.
[0099] Step S306: Based on the comparison result of the matching degree of the update direction and the preset threshold, determine the target encryption method from multiple encryption methods.
[0100] In this embodiment, various encryption methods can be used to further encrypt the local model parameters.
[0101] Specifically, based on the comparison result of the matching degree of the update direction with the preset threshold, if the comparison result shows that the matching degree of the update direction is greater than or equal to the preset threshold, the first encryption method is selected from multiple encryption methods; if the comparison result shows that the matching degree of the update direction is less than the preset threshold, the second encryption method is selected from multiple encryption methods.
[0102] The first encryption method is more secure, preventing high-value data from being easily decrypted and leaked. Furthermore, its corresponding encrypted data processing method is more accurate, avoiding significant errors in the processing of encrypted data. The second encryption method is simpler, avoiding excessive computational overhead during encryption and subsequent processing of encrypted data, thus improving processing efficiency.
[0103] Optionally, the encryption method includes homomorphic encryption and differential privacy encryption; if the matching degree of the update direction is less than the first threshold, the target encryption method is differential privacy encryption; if the matching degree of the update direction is greater than or equal to the first threshold, the target encryption method is homomorphic encryption.
[0104] Homomorphic encryption is a method of encryption that allows direct computation on encrypted data, with the decrypted result being identical to the plaintext. While homomorphic encryption allows direct computation on encrypted data, it involves significant computational complexity and time-consuming data processing.
[0105] Differential privacy encryption is a method that uses random noise (such as Laplace noise or Gaussian noise) to protect privacy and ensure that the output does not reveal the original data. Privacy encryption methods have low computational cost and short data processing time, but they can easily lead to insufficient accuracy of the results, which in turn can degrade model performance.
[0106] In this embodiment, if the matching degree of the update direction is less than the first threshold, it indicates that the local model parameter contributes little to the global model, and the differential privacy encryption method is determined as the target encryption method. If the matching degree of the update direction is greater than or equal to the first threshold, it indicates that the local model parameter contributes significantly to the global model, and the homomorphic encryption method is determined as the target encryption method.
[0107] In this embodiment, differential privacy encryption is used to encrypt local model data with low contribution to improve subsequent processing efficiency; homomorphic encryption is used to encrypt local model data with high contribution to ensure the accuracy of the processed data and improve model performance.
[0108] Optionally, if the matching degree of the update direction is less than the second threshold, the step of uploading the encrypted local model parameters and the matching degree of the update direction to the central node will not be performed. The second threshold is less than the first threshold.
[0109] In some embodiments, local model parameters that contribute less can be discarded directly, further improving the efficiency of subsequent processing.
[0110] Specifically, if the matching degree of the update direction is less than the second threshold, the local model parameter is discarded, that is, the subsequent steps are not executed.
[0111] If the matching degree of the update direction is greater than or equal to the second threshold and less than the first threshold, then the differential privacy encryption method is determined as the target encryption method; if the matching degree of the update direction is greater than or equal to the first threshold, then the homomorphic encryption method is determined as the target encryption method.
[0112] Step S307: Encrypt the local model parameters using the target encryption method.
[0113] In this step, the local node further encrypts the local model parameters using the target encryption method, such as homomorphic encryption or differential privacy encryption.
[0114] Step S308: Upload the encrypted local model parameters and the matching degree of the update direction to the central node.
[0115] In this step, the local node uploads the encrypted local model parameters and the matching degree of the update direction to the central node so that the central node can continue subsequent processing.
[0116] In some embodiments, the encrypted local model parameters, the matching degree of the update direction, and the target encryption method can also be uploaded to the central node.
[0117] The target encryption method is determined from multiple encryption methods by comparing the matching degree of the update direction with the preset threshold. This method is simple and easy to implement. At the same time, the matching degree of the update direction is obtained by calculating the cosine similarity of the update direction, thus realizing the effective quantification of the matching degree of the update direction.
[0118] Figure 4 This is a flowchart illustrating another federated learning method provided in an embodiment of this application. The method provided in this embodiment is applied to the central node in a federated learning framework, such as... Figure 1 In addition to the central node, the federated learning framework also includes multiple local nodes. For example... Figure 4 As shown, the method provided in this embodiment includes:
[0119] Step S401: In response to the matching degree of the encrypted local model parameters and update direction uploaded by each local node, select the fusion method of the local model parameters.
[0120] In this step, in response to the matching degree of the encrypted local model parameters and update direction uploaded by multiple local nodes, the fusion method of the local model parameters is selected based on the matching degree of the update direction.
[0121] The fusion method corresponds to the encryption method of the local model parameters, which is the selected method for processing the encrypted local model parameters.
[0122] Specifically, based on the comparison between the matching degree of the update direction and the preset threshold, the fusion method for the local model parameters is determined from multiple fusion methods. This fusion method corresponds to the target encryption method for the local model parameters.
[0123] Optionally, the fusion method of the local model parameters can be selected, including: the target encryption method for obtaining the encrypted local model parameters; the target encryption method includes homomorphic encryption and differential privacy encryption; if the target encryption method is differential privacy encryption, the fusion method of the local model parameters is selected as direct fusion; if the target encryption method is homomorphic encryption, the fusion method of the local model parameters is selected as fusion after decryption through secure multi-party computation.
[0124] In this embodiment, the local node uploads the encrypted local model parameters, the matching degree of the update direction, and the target encryption method to the central node. The central node obtains the target encryption method of the received encrypted local model parameters. If the target encryption method is differential privacy encryption, the fusion method corresponding to differential privacy encryption is selected to directly fuse the local model parameters; if the target encryption method is homomorphic encryption, the fusion method corresponding to homomorphic encryption is selected to fuse the local model parameters after secure multi-party computation and decryption.
[0125] Direct fusion can attenuate noise in local model data encrypted using differential privacy encryption; that is, noise can be reduced through weighted averaging, introducing only low noise and significantly improving fusion efficiency. Secure multi-party computation decryption followed by fusion is for homomorphic encryption methods, using multi-party collaboration to complete the processing task and ensure data integrity. This method allows for the fusion of encrypted data through secure multi-party computation, resulting in higher accuracy.
[0126] By selecting a fusion method corresponding to the target encryption method, targeted processing of the local model parameters is achieved, ensuring that the target encryption method can achieve the effects provided by the above embodiments, that is, while ensuring data privacy and security, it can also fully and efficiently explore the value of cross-domain data.
[0127] Step S402: Based on the matching degree of the update direction and the fusion method, the encrypted local model parameters and the global model parameters of the global model deployed in the central node are fused to obtain the fused global model parameters.
[0128] In this step, the central node, based on the matching degree of the update direction, uses the selected fusion method to fuse the encrypted local model parameters uploaded by multiple local nodes and the current global model parameters of the global model deployed by the central node, to obtain the fused global model parameters.
[0129] For example, the weight of each local node can be determined based on the matching degree of the update direction. Using the selected fusion method, the encrypted local model parameters and global model parameters are weighted and fused based on the weights to obtain the fused global model parameters.
[0130] Step S403: Send the merged global model parameters to each local node to replace the local model parameters of the local model deployed on each local node.
[0131] In this step, after obtaining the merged global model parameters, the central node sends the merged global model parameters to each local node. After receiving the global model parameters, each local node replaces its local model parameters with the global model parameters, thus updating the local model based on shared cross-domain data.
[0132] Optionally, the method further includes: responding to the encrypted identification information of the data object uploaded by multiple local nodes, determining the encrypted identification information common to the multiple local nodes based on the encrypted identification information; sending the encrypted identification information common to the multiple local nodes to the multiple local nodes, so that each local node can obtain the common sample of the local nodes based on the encrypted identification information common to the multiple local nodes.
[0133] In existing federated learning techniques, local nodes do not communicate with each other, and each local node uses its own data samples to train its deployed local model. However, due to the large differences in data distribution and model structure among multiple local nodes, existing federated learning frameworks are difficult to adapt to heterogeneous scenarios, ultimately leading to difficulties in global model convergence.
[0134] In this embodiment, multiple local nodes encrypt the identification information of their data objects, such as unique identifiers, and upload them to the central node.
[0135] For example, multiple local nodes use a collision-resistant hash function, such as SHA-256, to generate a hash value, which is then used as encrypted identification information.
[0136] After receiving the encrypted identifier information of the data objects uploaded by multiple local nodes, the central node calculates the intersection of the encrypted identifier information to determine the encrypted identifier information common to the multiple local nodes, and sends the common encrypted identifier information to each local node.
[0137] The local node receives the shared encrypted identification information, decrypts it, and identifies the data object corresponding to the identification information. The data in the data object corresponding to the identification information is then identified as the shared sample.
[0138] This embodiment achieves cross-domain sample alignment. While preventing the leakage of privacy data, it only outputs the identification information of common samples. Common samples can be used as the basis for model updates, ensuring that each local model is trained or updated based on a common dataset. It can adapt to heterogeneous data of different local nodes and improve the availability of model data after training or updating.
[0139] The federated learning method provided in this embodiment achieves cross-domain data sharing by selectively fusing encrypted local model data at the central node, thus greatly unlocking the value of the data.
[0140] Figure 5 This is a flowchart illustrating another federated learning method provided in this application embodiment. To more clearly illustrate the interaction process between the central node and multiple local nodes in the federated learning framework, this embodiment provides a detailed description of the federated learning method. For example... Figure 5 As shown, the method provided in this embodiment includes:
[0141] Step S501: Multiple local nodes obtain the identification information of the data objects in the local nodes and encrypt the identification information.
[0142] In step S502, the local node uploads the encrypted identifier information to the central node.
[0143] In step S503, the central node responds to the encrypted identification information of the data objects uploaded by multiple local nodes, and determines the encrypted identification information common to the multiple local nodes based on the encrypted identification information.
[0144] In step S504, the central node sends the encrypted identification information shared by multiple local nodes to multiple local nodes.
[0145] Step S505: The local node obtains the common samples of the local node. The common samples include the data of the common data objects in multiple local nodes.
[0146] Step S506: The local node updates the local model deployed on the local node based on the shared samples to obtain the local model parameters.
[0147] Step S507: The local node calculates the matching degree between the updated local model and the global model deployed by the central node in terms of update direction based on the local model parameters.
[0148] In step S508, the local node encrypts the local model parameters based on the matching degree of the update direction.
[0149] In step S509, the local node uploads the encrypted local model parameters and the matching degree of the update direction to the central node.
[0150] In step S510, the central node responds to the matching degree of the encrypted local model parameters and update direction uploaded by each local node, and selects the fusion method of the local model parameters.
[0151] Step S511: Based on the matching degree of the update direction and the fusion method, the central node fuses the encrypted local model parameters and the global model parameters of the global model deployed by the central node to obtain the fused global model parameters.
[0152] In step S512, the central node sends the fused global model parameters to each local node.
[0153] Step S513: The central node replaces the local model parameters of the local models deployed on each local node with the global model parameters.
[0154] The federated learning method provided in this embodiment is similar in principle and technical effect to the method provided in the above embodiments, and will not be described again here.
[0155] Figure 6 This is a schematic diagram of a federated learning device provided in an embodiment of this application. The federated learning device provided in this embodiment is applied to multiple local nodes in a federated learning framework, which also includes a central node. As shown in Figure 6, the federated learning device provided in this embodiment includes an acquisition module 601, a parameter update module 602, a matching degree calculation module 603, and an encrypted upload module 604.
[0156] The acquisition module 601 is used to acquire the common samples of the local nodes; the parameter update module 602 is used to update the local model deployed on the local nodes according to the common samples to obtain the local model parameters; the matching degree calculation module 603 is used to calculate the matching degree of the updated local model and the global model deployed on the central node according to the local model parameters; the encrypted upload module 604 is used to encrypt the local model parameters according to the matching degree of the update direction, and upload the encrypted local model parameters and the matching degree of the update direction to the central node, so that the central node can obtain the global model parameters of the global model based on the encrypted local model parameters and the matching degree of the update direction.
[0157] Optional, encrypted upload module 604, specifically used for:
[0158] Based on the comparison results between the matching degree of the update direction and the preset threshold, the target encryption method is determined from multiple encryption methods; the local model parameters are encrypted using the target encryption method; and the matching degree of the encrypted local model parameters and the update direction is uploaded to the central node.
[0159] Optionally, the encryption method includes homomorphic encryption and differential privacy encryption; if the matching degree of the update direction is less than the first threshold, the target encryption method is differential privacy encryption; if the matching degree of the update direction is greater than or equal to the first threshold, the target encryption method is homomorphic encryption.
[0160] Optionally, the encrypted upload module 604 also includes an upload cancellation unit, used for:
[0161] If the matching degree of the update direction is less than the second threshold, the step of uploading the encrypted local model parameters and the matching degree of the update direction to the central node will not be executed, where the second threshold is less than the first threshold.
[0162] Optional, the matching degree calculation module 603 is specifically used for:
[0163] Based on the local model parameters and the model parameters of the local model before the update, calculate the update direction of the local model parameters; obtain the update direction of the global model parameters of the global model deployed on the central node; calculate the cosine similarity between the update directions of the local model parameters and the update directions of the global model parameters to obtain the matching degree of the update directions.
[0164] The federated learning apparatus provided in this application embodiment can be used to execute the technical solution of the federated learning method provided in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again here.
[0165] This application also provides another federated learning device. The federated learning device provided in this embodiment is applied to the central node in a federated learning framework, which also includes multiple local nodes. The federated learning device provided in this embodiment includes a fusion method determination module, a global model parameter determination module, and a communication module.
[0166] The fusion method determination module is used to determine the fusion method of local model parameters in response to the matching degree of encrypted local model parameters and update direction uploaded by each local node; the global model parameter determination module is used to fuse the encrypted local model parameters and the global model parameters of the global model deployed by the central node based on the matching degree of update direction and fusion method to obtain the fused global model parameters; the communication module is used to send the fused global model parameters to each local node to replace the local model parameters of the local model deployed by each local node.
[0167] Optionally, this fusion method determines the module, specifically for:
[0168] Obtain the target encryption method for the encrypted local model parameters; the target encryption method includes homomorphic encryption and differential privacy encryption; if the target encryption method is differential privacy encryption, then the fusion method for the local model parameters is direct fusion; if the target encryption method is homomorphic encryption, then the fusion method for the local model parameters is fusion after decryption through secure multi-party computation.
[0169] Optionally, the federated learning device also includes a shared sample determination module for:
[0170] In response to the encrypted identification information of data objects uploaded by multiple local nodes, the encrypted identification information shared by the multiple local nodes is determined based on the encrypted identification information; the encrypted identification information shared by the multiple local nodes is sent to the multiple local nodes so that each local node can obtain the common sample of the local nodes based on the encrypted identification information shared by the multiple local nodes.
[0171] The federated learning apparatus provided in this application embodiment can be used to execute the technical solution of the federated learning method provided in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again here.
[0172] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. As shown in the figure, the electronic device of this embodiment may include: at least one processor 701; and a memory 702 communicatively connected to at least one processor 701; wherein, the memory 702 stores instructions that can be executed by at least one processor 701, and the instructions are executed by at least one processor 701 to cause the electronic device to perform the method as described in any of the above embodiments.
[0173] Optionally, the memory 702 can be either standalone or integrated with the processor 701. When the memory 702 is set up independently, the device also includes a bus for connecting the memory 702 and the processor 701.
[0174] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.
[0175] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the methods provided in any of the foregoing embodiments can be implemented.
[0176] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in any of the foregoing embodiments.
[0177] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0178] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0179] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.
[0180] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.
[0181] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.
[0182] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0183] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.
[0184] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0185] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A federated learning method, characterized in that, The method, applied to multiple local nodes in a federated learning framework, which also includes a central node, comprises: Obtain shared samples from local nodes; the shared samples include data of shared data objects among the multiple local nodes; Based on the shared samples, the local model deployed on the local node is updated to obtain the local model parameters; Based on the local model parameters, calculate the matching degree between the updated local model and the global model deployed at the central node in terms of update direction; Based on the matching degree of the update direction, the local model parameters are encrypted, and the encrypted local model parameters and the matching degree of the update direction are uploaded to the central node, so that the central node can obtain the global model parameters of the global model based on the matching degree of the encrypted local model parameters and the update direction.
2. The method of claim 1, wherein, The step of encrypting the local model parameters based on the matching degree of the update direction includes: Based on the comparison result between the matching degree of the update direction and the preset threshold, the target encryption method is determined from multiple encryption methods; The local model parameters are encrypted using the target encryption method.
3. The method of claim 2, wherein, The encryption methods include homomorphic encryption and differential privacy encryption. If the matching degree of the update direction is less than the first threshold, the target encryption method is the differential privacy encryption method; if the matching degree of the update direction is greater than or equal to the first threshold, the target encryption method is the homomorphic encryption method.
4. The method of claim 3, wherein, The method further includes: If the matching degree of the update direction is less than the second threshold, the step of uploading the encrypted local model parameters and the matching degree of the update direction to the central node will not be performed, wherein the second threshold is less than the first threshold.
5. The method according to any one of claims 1 to 4, characterized in that, The step of calculating the matching degree between the updated local model and the update direction of the global model deployed at the central node based on the local model parameters includes: Calculate the update direction of the local model parameters based on the local model parameters and the model parameters of the local model before the update; Obtain the update direction of the global model parameters of the global model deployed on the central node; Calculate the cosine similarity between the update direction of the local model parameters and the update direction of the global model parameters to obtain the matching degree of the update direction.
6. A federated learning method, characterized in that, The method, applied to a central node in a federated learning framework, which also includes multiple local nodes, comprises: Based on the matching degree between the encrypted local model parameters and the update direction uploaded by each local node, a fusion method for the local model parameters is selected. Based on the matching degree of the update direction and the fusion method, the encrypted local model parameters and the global model parameters of the global model deployed by the central node are fused to obtain the fused global model parameters. The fused global model parameters are sent to each of the local nodes to replace the local model parameters of the local models deployed on each of the local nodes.
7. The method according to claim 6, characterized in that, The method for selecting the fusion of the local model parameters includes: Obtain the target encryption method for the encrypted local model parameters; the target encryption method includes homomorphic encryption and differential privacy encryption. If the target encryption method is differential privacy encryption, then the fusion method for the local model parameters is selected as direct fusion; If the target encryption method is homomorphic encryption, then the fusion method for the local model parameters is to fuse them after decryption through secure multi-party computation.
8. The method according to claim 6 or 7, characterized in that, The method further includes: In response to the encrypted identification information of the data objects uploaded by the multiple local nodes, the encrypted identification information common to the multiple local nodes is determined based on the encrypted identification information; The encrypted identifier information shared by the multiple local nodes is sent to the multiple local nodes so that each local node can obtain the common sample of the local node based on the encrypted identifier information shared by the multiple local nodes.
9. A federated learning device, characterized in that, The apparatus comprises multiple local nodes applied in a federated learning framework, the federated learning framework also including a central node, and includes: The acquisition module is used to acquire common samples of local nodes; the common samples include data of common data objects among the multiple local nodes; The parameter update module is used to update the local model deployed on the local node based on the shared samples to obtain the local model parameters; The matching degree calculation module is used to calculate the matching degree between the updated local model and the update direction of the global model deployed by the central node based on the local model parameters. An encrypted upload module is used to encrypt the local model parameters according to the matching degree of the update direction, and upload the encrypted local model parameters and the matching degree of the update direction to the central node, so that the central node can obtain the global model parameters of the global model based on the matching degree of the encrypted local model parameters and the update direction.
10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.