Privacy-protected distributed machine learning client clustering method, device and product
By employing multi-valued homomorphic encryption, cross-key ring aggregation, and adaptive re-clustering strategies, this approach addresses the issues of decreased model accuracy and privacy leakage in federated learning clustering, achieving efficient client-side clustering applicable to fields such as finance, healthcare, and the Internet of Things.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-04-03
AI Technical Summary
Existing clustering federated learning methods suffer from decreased model accuracy in non-independent and identically distributed data environments, and face privacy risks during client-side clustering, low computational efficiency, insecure cross-key aggregation, and complex dynamic client participation.
Employing multi-valued homomorphic encryption, cross-key ring aggregation protocol, and adaptive re-clustering strategy, multiple label distribution components are encoded into a single ciphertext through shift operations. Communication is optimized using logical ring topology, and the shift of cluster centers is monitored to dynamically adjust the clustering structure.
It significantly improves computational efficiency, reduces communication complexity, intelligently adapts to dynamic environments, enables efficient privacy-preserving client clustering, and is suitable for large-scale federated learning deployments.
Smart Images
Figure CN121786522A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of information security and machine learning technology, and more specifically, to a privacy-preserving distributed machine learning client clustering method, apparatus, and product. Background Technology
[0002] Federated learning is a distributed machine learning paradigm that allows multiple clients to collaboratively train a global model while maintaining data locality, exchanging only model update information. In standard federated learning, the training process involves iterative rounds, each consisting of three phases: first, the server distributes the current global model to the clients; then, each client performs local training on its private dataset and computes model updates; finally, the server aggregates these updates to improve the global model. Federated learning has been successfully applied in fields such as banking, healthcare, and e-commerce, enabling cross-institutional machine learning collaboration without sharing raw data.
[0003] However, in real-world scenarios, client data often exhibits non-independent and identically distributed characteristics. This data heterogeneity leads to a significant decrease in the accuracy of the global model. Specifically, when the data held by clients varies considerably, the traditional federated learning algorithm's simple averaging of model updates across all clients introduces model bias, making it difficult for the global model to adapt to various data distributions and ultimately impacting model performance.
[0004] Cluster federated learning addresses this problem by clustering clients with similar data distributions and training a specialized model for each cluster. The core idea of cluster federated learning is to identify groups of clients with similar data distributions and allow these clients to collaboratively train a specialized model suited to their data characteristics, rather than forcing all clients to participate in training a single global model. This approach leverages data homogeneity within clusters while isolating the adverse effects of data heterogeneity between different clusters.
[0005] To achieve effective client clustering, existing clustering federated learning methods mainly rely on two strategies: The first is gradient-based clustering. These methods cluster by analyzing the similarity of client gradient updates, based on the theory that clients with similar data distributions tend to produce similar gradient updates during model training. However, gradient-based clustering requires frequent exchange of high-dimensional gradient vectors, causing communication overhead to increase linearly with model size. For deep neural networks, the dimension of gradient vectors can reach millions or even billions, making communication cost a system bottleneck. The second is label distribution-based clustering. These methods utilize client label distributions for clustering because label distributions directly capture the root cause of data heterogeneity and have a lower dimensionality (equal to the number of categories, not the number of model parameters). Compared to gradient vectors, the dimension of label distributions is typically only tens to hundreds, significantly reducing communication overhead. However, sharing the original label distribution still poses privacy risks; attackers can use this information to accurately estimate the relative frequency of different categories in the client dataset, potentially revealing sensitive statistical characteristics about the client group.
[0006] To protect label distribution privacy during clustering, existing methods typically integrate privacy-preserving techniques, primarily including: Homomorphic encryption: allows addition and multiplication operations to be performed directly on the ciphertext without decryption or multi-round communication. Homomorphic encryption maintains the exact numerical relationships of the data without introducing any noise, thus protecting privacy without compromising model accuracy. Secure multi-party computation: achieves secure computation through multi-party protocols, but typically has high communication overhead, and its multi-round interaction protocols scale poorly with the number of clients. Differential privacy: protects label distribution by adding noise, but noise can distort the true similarity relationships between clients, leading to reduced clustering accuracy. Despite the advantages of homomorphic encryption in terms of privacy protection, existing privacy-preserving clustering methods based on homomorphic encryption face three key technical challenges: First, there is the problem of low computational efficiency. Traditional Paillier homomorphic encryption methods require encrypting each label distribution component separately, performing a separate encryption operation for each category. To ensure security under the decision composite residues assumption, Paillier encryption typically requires a large ciphertext modulus of 2048 bits, while client label distributions usually only occupy less than 256 bits of space, resulting in a serious waste of computational and communication resources. Although existing multi-value packing techniques can merge multiple values into a single ciphertext to reduce encryption operations, they only support pure addition operations and cannot meet the dual requirements of cluster similarity calculation, which simultaneously requires homomorphic addition (for aggregating cross terms) and scalar multiplication (for operations between client data and cluster centers).
[0007] Second, there is a trade-off between security and efficiency in cross-key aggregation. Traditional single-key homomorphic encryption schemes share the same public key across all clients, posing a serious security vulnerability: if the server colludes with any client or the shared key is leaked, all clients' private data will be exposed simultaneously. Multi-key homomorphic encryption schemes address this security issue by assigning independent key pairs to each client; however, traditional homomorphic encryption operations only support ciphertext operations under the same public key, making direct aggregation of clients with different keys computationally infeasible. Existing cross-key aggregation methods employ a distributed-encryption strategy, requiring each client to decompose its data into N additive shares and encrypt data shares for all other N-1 participants. This results in each client needing to perform N encryption operations and transmit N ciphertexts, leading to double communication complexity. This has become a serious bottleneck in large-scale deployments.
[0008] Third, the complexity of handling dynamic client participation. In actual deployments, frequent client joining or leaving can disrupt established clustering structures: departing clients reduce the representativeness of cluster centers, while newly joined clients introduce unseen label distributions, potentially conflicting with existing assignments and leading to suboptimal clustering quality. Existing methods for handling dynamic client changes involve a trade-off between efficiency and quality: global re-clustering performs a full K-Means re-clustering after each client change to ensure optimal clustering assignments, but incurs significant computational and communication overhead, especially when client changes are frequent; local adjustment methods simply assign new clients to the nearest existing cluster and remove departing clients, maintaining computational efficiency but potentially causing the clustering structure to gradually deteriorate over time, ultimately compromising the effectiveness of specialized models.
[0009] Therefore, there is an urgent need for a privacy-preserving client clustering scheme that can resist various privacy attacks, provide good scalability and computational efficiency, and effectively handle dynamic client participation to meet the needs of large-scale federated learning deployments. Summary of the Invention
[0010] To address the issues of decreased model accuracy and privacy leakage during client-side clustering in existing federated learning models under non-independent and identically distributed data environments, this invention provides a privacy-preserving distributed machine learning client-side clustering method, apparatus, and product.
[0011] Specifically, the first aspect of the present invention provides a privacy-preserving distributed machine learning client clustering method, involving N clients and a coordinating server, including: Generate a unique homomorphic encryption key pair for each client and coordinate server initialization. Cluster centers; Each client encodes the label distribution vector obtained during training into a single ciphertext using multi-value packing technology, calculates the similarity with each cluster center using homomorphic encryption, and selects the most similar cluster. The label distribution vector is used to characterize the sample distribution of all categories of the client. N clients are arranged in a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server so that the server can obtain the aggregation result based on the encrypted results. Changes to the client are detected, and the impact of these changes on the clustering structure is monitored. A decision on whether to perform global re-clustering is made based on the cluster center displacement threshold.
[0012] In one implementation, each client encodes the label distribution vector obtained during training into a single ciphertext using a multi-value packing technique, including: Perform dynamic range analysis on the label distribution vector, and calculate the maximum value, minimum value and required bit width for each label distribution component, where the label distribution component is an element in the label distribution vector; Prioritize the tag distribution components according to frequency or amplitude, and assign shift offsets to the tag distribution components according to the required bit width; Each tag component is shifted based on the shift offset and packaged into a single ciphertext to generate a plaintext message.
[0013] In one implementation, homomorphic encryption is used to calculate the similarity with each cluster center and select the most similar cluster, including: Ciphertext is generated based on the plaintext message, the public key in the homomorphic encryption key pair, and the Paillier encryption algorithm. Based on the generated ciphertext, shift offset, and cluster center, the tag component and cluster center component are multiplied in the encrypted state, and then multiplied with the accumulated ciphertext to obtain the decrypted dot product value. The accumulated ciphertext is the accumulated product value of the encrypted points of the client tag distribution and the cluster center. Calculate the complete cosine similarity based on the decrypted dot product values and select the most similar cluster.
[0014] In one implementation, N clients are arranged in a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server, so that the server obtains the final aggregation result based on the encrypted results, including: Will The clients are arranged in a logical ring structure, and each client determines its first successor and second successor; Each client splits its label distribution into two additive shared components, where the label distribution is a vector representation of all label distribution components; Each client encrypts its first and second successors based on two addition shares before uploading them to the server; The server calculates the aggregated ciphertext for each client. The client uses its private key to decrypt the aggregated ciphertext and obtains a partial decrypted result. The server reconstructs the decrypted portion of the result from each client to obtain the final aggregated result.
[0015] In one implementation, changes to the client are detected, the impact of these changes on the clustering structure is monitored, and a decision on whether to perform global re-clustering is made based on a cluster center displacement threshold, including: Set up a client join set and a client leave set, and monitor both sets to determine the affected clusters. The client join set is used to record clients that join, and the client leave set is used to record clients that leave. For each client added to the set, calculate its similarity to each cluster center and assign it to the most similar cluster. For each client added to the set, remove it from its current cluster. Calculate the cluster center displacement for the affected clusters; Determine if the cluster center position exceeds the cluster center displacement threshold. If it does, perform global re-clustering.
[0016] In one implementation, for each client added to the set, the similarity between the client and each cluster center is calculated and the client is assigned to the most similar cluster, including: For each client added to the set, calculate the similarity with the cluster centers and obtain the optimal cluster index:
[0017] in, Represented as client The best clustering index to choose Represents the total number of clusters. Indicates the range of clustering index. This indicates that the function returns the parameter that minimizes the value. Indicates the first Round The center of each cluster; The cluster corresponding to the best clustering index is taken as the most similar cluster and assigned to the most similar cluster.
[0018] In one implementation, calculating the cluster center displacement for the affected clusters includes: For each affected cluster, calculate the original centers and the updated centers, where the original centers are calculated as follows: The updated method for calculating cluster centers is as follows: ; Calculate the cluster center displacement based on the original cluster centers and the updated cluster centers. :
[0019] in, Let i be the original cluster center, and i be the i-th client. Indicates client The label distribution vector, For the first Round clustering A collection of clients, This is the set of clients after a partial update.
[0020] Based on the same inventive concept, a second aspect of the present invention provides a privacy-preserving distributed machine learning client clustering device, involving N clients and a coordinating server, comprising: The system initialization module is used to generate independent homomorphic encryption key pairs for each client and coordinate server initialization. Cluster centers; The privacy-preserving similarity calculation module is used by each client to encode the label distribution vector obtained during the training process into a single ciphertext through multi-value packing technology, and to calculate the similarity with each cluster center using homomorphic encryption and select the most similar cluster. The label distribution vector is used to represent the sample distribution of all categories of the client. The secure aggregation update module is used to arrange N clients into a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server so that the server can obtain the aggregation result based on the encrypted results. The adaptive re-clustering management module is used to detect changes in the client, monitor the impact of these changes on the clustering structure, and decide whether to perform global re-clustering based on the cluster center displacement threshold.
[0021] Based on the same inventive concept, a third aspect of the present invention provides a computer program product, including computer program instructions, which, when executed on a computer, implement the privacy-preserving distributed machine learning client clustering method described in the first aspect. Based on the same inventive concept, a fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the privacy-preserving distributed machine learning client clustering method described in the first aspect.
[0022] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows: (1) Significantly improved computational efficiency. The multi-valued packing homomorphic encryption technology proposed in this invention encodes multiple label distribution components into a single ciphertext through shift operations, achieving encryption operation optimization from linear to constant level. It also supports component-level addition and position-specific scalar multiplication, directly meeting the dual computational requirements of similarity calculation in K-Means clustering, and eliminating the limitation of existing multi-valued packing schemes that only support pure addition.
[0023] (2) Fundamental optimization of communication complexity. The cross-key ring aggregation protocol designed in this invention restricts each client to encrypting data only to a fixed number (2) of successor nodes through a logical ring topology, achieving efficient aggregation with aggregation cost independent of the number of clients. This architectural optimization makes multi-key homomorphic encryption practical in large-scale federated learning deployments.
[0024] (3) Intelligent adaptation to dynamic environment. The adaptive re-clustering strategy proposed in this invention achieves an intelligent balance between clustering quality and computational efficiency by monitoring the displacement of cluster centers. This strategy makes dynamic decisions based on the actual impact of changes in the client on the clustering structure, avoiding the fixed trade-off between global re-clustering (high computational cost but high quality) and local adjustment (high efficiency but potential quality degradation) in existing methods.
[0025] (4) Practical Performance Verification. Through extensive experimental verification on different domain datasets, this invention achieves a reduction of over 95% in communication overhead and a 25x computational speedup compared to existing state-of-the-art methods, while maintaining high model accuracy. This significant performance improvement makes large-scale federated learning deployment with privacy protection a reality, providing a practical technical solution for industries with stringent privacy requirements such as finance, healthcare, and the Internet of Things. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is an overall flowchart of the privacy-preserving distributed machine learning client clustering method in this embodiment of the invention; Figure 2 This is a detailed flowchart of the privacy-preserving distributed machine learning client clustering method in this embodiment of the invention; Figure 3 This is a structural block diagram of a privacy-preserving distributed machine learning client clustering device in an embodiment of the present invention. Detailed Implementation
[0028] Federated learning allows multiple clients to collaboratively train a global model while maintaining the locality of their private data. However, it faces the challenge of decreased model accuracy in environments with non-independent and identically distributed data. Clustered federated learning addresses this issue by clustering clients with similar data distributions and training specialized models. However, the sharing of label distributions during client clustering poses a privacy risk, and existing privacy protection schemes suffer from technical bottlenecks in terms of computational efficiency, cross-key aggregation, and dynamic client management.
[0029] This invention addresses the shortcomings of the aforementioned technologies by providing a privacy-preserving distributed machine learning client clustering method. The core innovation of this method lies in three collaborative technical components: a multi-valued homomorphic encryption protocol, a cross-key ring aggregation protocol, and an adaptive re-clustering strategy. Specifically, firstly, the multi-valued homomorphic encryption protocol encodes multiple label distribution components into a single ciphertext using shift operations, and combines position-specific scaling factors to achieve component-level addition and scalar multiplication operations, reducing the encryption operation complexity from linear to constant complexity. Secondly, the cross-key ring aggregation protocol employs a logical ring topology structure so that each client encrypts data only for a fixed number of successor nodes, optimizing the aggregation communication complexity from quadratic to linear. Finally, the adaptive re-clustering strategy achieves an intelligent balance between clustering quality and computational efficiency based on cluster center displacement monitoring.
[0030] Please see Figure 1 The system architecture provided in this embodiment includes four coordinated phases. This embodiment also provides a privacy-preserving distributed machine learning client clustering method, involving entities including... The method involves one client and one coordination server, and the specific steps are as follows: S1: Generate an independent homomorphic encryption key pair for each client and coordinate server initialization. Cluster centers; Specifically, S1 is the system initialization step, which generates an independent homomorphic encryption key pair for each client, used for subsequent homomorphic encryption. The number of clients... With the number of cluster centers It is a positive integer and can be adjusted according to the actual situation.
[0031] S2: Each client encodes the label distribution vector obtained during training into a single ciphertext using multi-value packing technology, calculates the similarity with each cluster center using homomorphic encryption, and selects the most similar cluster. The label distribution vector is used to represent the sample distribution of all categories of the client.
[0032] Specifically, S2 is the system initialization step, which is a privacy-preserving similarity calculation. This step uses a value-packed homomorphic encryption protocol to encode multiple label distribution components into a single ciphertext using shift operations, and combines a position-specific scaling factor to implement component-level addition and scalar multiplication operations, reducing the encryption operation from linear complexity to constant complexity.
[0033] Please see Figure 2 This is a detailed process framework for the privacy-preserving distributed machine learning client clustering method in this embodiment of the invention.
[0034] Specifically, each client encodes the label distribution vector obtained during training into a single ciphertext using a multi-value packing technique, including: S2.1: Perform dynamic range analysis on the label distribution vector, calculate the maximum value, minimum value and required bit width for each label distribution component, where the label distribution component is an element in the label distribution vector; S2.2: Prioritize the tag distribution components according to frequency or amplitude, and allocate shift offsets to the tag distribution components according to the required bit width; S2.3: Shift each tag component based on the shift offset and pack it into a single ciphertext to generate a plaintext message.
[0035] In the specific implementation process, the client In the The label distribution vector obtained in the training round is ,in Indicates client Data categories The sample proportion , Total number of clients , The total number of categories, express 3D real vector space; label distribution vector Indicates client In all The distribution of samples across each category.
[0036] S2.1 When performing dynamic range analysis on the label distribution vector, the label distribution vector... The first in Each element is denoted as ,Right now Tag distribution component This represents the proportion of samples from a single category in the label distribution vector. It is the basic unit that constitutes the complete label distribution vector. For each label distribution component... Calculate: Minimum value maximum value Required bit width: ,in Indicates the component index.
[0037] In S2.2, select the shift offset. Make ,in The total plaintext width is reserved, with high bits reserved to prevent overflow.
[0038] S2.3 uses multi-valued packet encoding to construct a packetized message and generate a plaintext message. ,in Indicates a left shift operation. The above formula represents the efficient packaging of multiple tag distribution components into a single plaintext message.
[0039] Homomorphic encryption is used to calculate the similarity with each cluster center and select the most similar cluster, including: S2.4: Generate ciphertext based on the plaintext message, the public key in the homomorphic encryption key pair, and the Paillier encryption algorithm; S2.5: Based on the generated ciphertext, shift offset, and cluster center, perform multiplication on the tag component and cluster center component in the encrypted state, and then multiply it with the accumulated ciphertext to obtain the decrypted dot product value, where the accumulated ciphertext is the accumulated product value of the encrypted points of the client tag distribution and the cluster center. S2.6: Calculate the complete cosine similarity based on the decrypted dot product value and select the most similar cluster.
[0040] In its implementation, the Paillier homomorphic encryption scheme includes the following algorithms: Key generation Based on security parameters Choose two different prime number and ,calculate: Choose a random integer ,satisfy ,calculate ,in Output public key and private key ; Encryption Algorithm : For plaintext Select random number Calculate the ciphertext: ; Decryption algorithm : For ciphertext Calculate plaintext: .
[0041] The cryptographic operations supported include: homomorphic addition. Scalar multiplication: ,in The plaintext corresponding to the ciphertext. It is a plaintext scalar.
[0042] In S2.4, the ciphertext is generated as follows: ,in This is the client's public key.
[0043] For client i, and cluster center Perform cosine similarity The calculation, where Indicates client The label distribution vector, Indicates the first Cluster centers, Indicates client In category Sample proportion Representing cluster centers In category The proportion value above.
[0044] To achieve privacy-preserving similarity calculation, we first initialize... Then execute: For all ,in For the cluster center 1 component, of which This represents the cumulative result in encrypted form, indicating the cumulative product of the client label distribution and the encrypted points of the cluster centers. Indicates the index of the label distribution component. This indicates that the ciphertext is encrypted using the scalar multiplication property of homomorphic encryption. Upgraded to The exponentiation, achieving the first power in the encrypted state. Components With cluster center components Multiplication operation; then combined with the cumulative ciphertext Multiplication (homogeneous addition) gradually accumulates the cross terms of all components, and the final output is the decrypted dot product value. .
[0045] The client calculates the full cosine similarity based on the above process and selects the most similar cluster.
[0046] S3: Arrange N clients into a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server so that the server can obtain the aggregation result based on the encrypted results.
[0047] Specifically, S2 is the secure aggregation update step. This step uses a cross-key ring aggregation protocol and a logical ring topology to ensure that each client encrypts data for only a fixed number of successor nodes, thus optimizing the aggregation communication complexity from quadratic to linear.
[0048] Specifically, S3 can be achieved through the following steps: S3.1: Will The clients are arranged in a logical ring structure, and each client determines its first successor and second successor; S3.2: Each client divides its label distribution into two additive shared components, where the label distribution is represented in vector form by all label distribution components; S3.3: Each client encrypts its first and second successors based on two addition sharing methods before uploading them to the server; S3.4: The server calculates the aggregated ciphertext for each client; S3.5: The client uses its private key to decrypt the aggregated ciphertext and obtains a partial decrypted result; S3.6: The server reconstructs the decrypted partial results from each client to obtain the final aggregated result.
[0049] In the specific implementation process, for the client Its first successor is The second successor is .
[0050] S3.2 performs data segmentation, dividing the client... Label distribution Divide into two shared addition operations: ,in This is a randomly generated addition sharing mechanism. Regarding label distribution and the label distribution component, label distribution... It is dimensional vector Each element This is called a tag distribution component, representing the client. In category The proportion of samples on the label distribution. The label distribution is a vector representation of all label distribution components, which are the basic elements that make up the label distribution vector.
[0051] S3.3 performs ring encryption on the client side. Encryption is only applied to its successor nodes: and will Uploaded to the server, where Indicates client The first successor node's shared ciphertext for addition. Client The second successor node shares the ciphertext for addition. Indicates client The public key of the first successor node, Indicates client The public key of the second successor node; S3.4 performs homomorphic aggregation, with the server performing aggregation on each client. calculate: ,in Indicates targeting the client The aggregated ciphertext, Indicates that all first successor nodes are client , Indicates that all second successor nodes are client , This indicates that for all first successor nodes... client Summation, This indicates that for all second successor nodes... client Sum.
[0052] Step S3.5 is sequential decryption and reconstruction, for each client. Decryption The result is then returned to the server, which reconstructs the final aggregated result. ,in For the client private key, Indicates client Partial results after decryption.
[0053] The complexity optimization achieved by the S3 cross-key aggregation protocol includes: optimized encryption operations: the number of encryption attempts per client is reduced from traditional schemes. The number of communication operations has been reduced from 2 to 1, regardless of the total number of clients; communication complexity has been optimized: the total communication complexity has decreased from [previous value]. Reduce to Total communication volume from Reduce to Aggregation round number optimization: Homomorphic addition round number from Reduce to ;in Total number of clients This refers to the data size for a single client.
[0054] S4: Detect changes in the client and monitor their impact on the clustering structure. Decide whether to perform global re-clustering based on the cluster center displacement threshold.
[0055] Specifically, S4 is adaptive re-clustering management. This step achieves an intelligent balance between clustering quality and computational efficiency through an adaptive re-clustering strategy based on cluster center displacement monitoring.
[0056] Specifically, S4 includes: S4.1: Set up a client join set and a client leave set, and monitor the two sets to determine the affected clusters. The client join set is used to record clients that join, and the client leave set is used to record clients that leave. S4.2: For clients added to the set, calculate their similarity to each cluster center and assign them to the most similar cluster; for clients added to the set, remove them from their current cluster. S4.3: Calculate the cluster center displacement for the affected clusters; S4.4: Determine whether the cluster center position exceeds the cluster center displacement threshold. If it does, perform global re-clustering.
[0057] In the specific implementation process, S4.1: Set the client to join the collection. Leave the collection with the client The system monitors two sets: the client joining set, which represents the set of clients newly joining the system at the current moment, and the client leaving set, which represents the set of clients leaving the system at the current moment. These two sets are used to record client joining and leaving.
[0058] The K-Means clustering objective function for the adaptive re-clustering strategy is: ,in: Indicates the first A cluster of client sets, For clustering The center, For the client The squared Euclidean distance between the label distribution and the cluster centers; Clustering assignment rules: .
[0059] S4.2 local clustering update, for clients added Calculate its similarity to each cluster center and assign it to the most similar cluster: For those leaving the client, they are directly removed from their current cluster, where Represented as client The best clustering index to choose Represents the total number of clusters. Indicates the range of clustering index. This indicates that the function returns the parameter that minimizes the value. Indicates the first Round The center of each cluster is determined by taking the cluster corresponding to the best cluster index as the nearest cluster and assigning it to that cluster.
[0060] It should be noted that the similarity calculated in S4.2 and the similarity calculated in step 2.6 represent the application of the same calculation process at different stages. Specifically, the privacy-preserving similarity calculation in step 2.6 refers to the initial clustering process, where the client and... Calculate the similarity of the initial cluster centers and select the most similar cluster; the local update similarity calculation in step S4.2 refers to the calculation of similarity when new clients are added in a dynamic environment. With the present Cluster centers Calculate similarity and assign to the most similar cluster. The selection of the most similar cluster is determined in both steps by calculating cosine similarity to determine client affiliation.
[0061] Step S4.3: Calculate the cluster center displacement for each affected cluster. Calculate the original centers and the updated centers: Original centers: Update Center: Displacement: ,in For the first Round clustering A collection of clients, For the client set after partial updates; Step S4.4: Threshold decision, for each cluster Compare displacements With predefined threshold If it exists If the result is positive, then global re-clustering is triggered; otherwise, the local update result is accepted to avoid the overhead of global re-clustering.
[0062] Based on the same inventive concept, this embodiment discloses a privacy-preserving distributed machine learning client clustering device, involving N clients and a coordination server. Please refer to [link to relevant documentation]. Figure 3 The device includes: System initialization module 101 is used to generate independent homomorphic encryption key pairs for each client and coordinate server initialization. Cluster centers; The privacy-preserving similarity calculation module 102 is used to encode the label distribution vector obtained during the training process into a single ciphertext through multi-value packing technology for each client, calculate the similarity with each cluster center using homomorphic encryption and select the most similar cluster. The label distribution vector is used to represent the sample distribution of all categories of the client. The secure aggregation update module 103 is used to arrange N clients into a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server so that the server can obtain the aggregation result based on the encrypted results. The adaptive re-clustering management module 104 is used to detect changes in the client, monitor the impact of changes in the client on the clustering structure, and decide whether to perform global re-clustering based on the cluster center displacement threshold.
[0063] Based on the same inventive concept, the present invention also provides a computer program product that, when the computer program instructions are run on a computer, implements the privacy-preserving distributed machine learning client clustering method of Embodiment 1.
[0064] Based on the same inventive concept, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in Embodiment 1.
[0065] The present invention will be further illustrated by specific experiments below. Please see Table 1 for the experimental results of computational overhead (s) of this embodiment and other embodiments. Table 2 for the experimental results of communication overhead (MB) of this embodiment and other embodiments.
[0066] Table 1. Comparison of computational costs (unit: seconds)
[0067] Table 2: Comparison of Communication Overhead (Unit: MB)
[0068] This experiment compares our proposed implementation with other solutions in terms of both computational and communication overhead. Five datasets were selected (MNIST, CIFAR-10, Tiny-ImageNet, Speech Commands, and AGNews), covering different domains such as image classification, speech recognition, and text classification. As shown in Tables 1 and 2, our invention achieves significant performance improvements on all datasets. The speedup in computational overhead increased from 11.2× to 35.3×, and the reduction in communication overhead increased from 81.3% to 96.2%.
[0069] This invention presents a privacy-preserving distributed machine learning client clustering method. Through the synergistic operation of three core technologies—multi-valued homomorphic encryption, cross-key ring aggregation, and adaptive re-clustering—this method effectively addresses the technical challenges of existing privacy-preserving clustering schemes in terms of computational efficiency, cross-key aggregation, and dynamic client management. Experimental results demonstrate that this invention significantly improves computational and communication efficiency while maintaining high model accuracy and privacy protection, providing a practical technical solution for large-scale privacy-preserving federated learning.
[0070] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0071] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0072] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the embodiments of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and variations.
Claims
1. A privacy-preserving distributed machine learning client clustering method, characterized in that, It involves N clients and one coordination server, including: Generate a unique homomorphic encryption key pair for each client and coordinate server initialization. Cluster centers; Each client encodes the label distribution vector obtained during training into a single ciphertext using multi-value packing technology, calculates the similarity with each cluster center using homomorphic encryption, and selects the most similar cluster. The label distribution vector is used to characterize the sample distribution of all categories of the client. N clients are arranged in a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server so that the server can obtain the aggregation result based on the encrypted results. Changes to the client are detected, and the impact of these changes on the clustering structure is monitored. A decision on whether to perform global re-clustering is made based on the cluster center displacement threshold.
2. The privacy-preserving distributed machine learning client clustering method as described in claim 1, characterized in that, Each client encodes the label distribution vector obtained during training into a single ciphertext using a multi-value packing technique, including: Perform dynamic range analysis on the label distribution vector, and calculate the maximum value, minimum value and required bit width for each label distribution component, where the label distribution component is an element in the label distribution vector; Prioritize the tag distribution components according to frequency or amplitude, and assign shift offsets to the tag distribution components according to the required bit width; Each tag component is shifted based on the shift offset and packaged into a single ciphertext to generate a plaintext message.
3. The privacy-preserving distributed machine learning client clustering method as described in claim 2, characterized in that, Homomorphic encryption is used to calculate the similarity with each cluster center and select the most similar cluster, including: Ciphertext is generated based on the plaintext message, the public key in the homomorphic encryption key pair, and the Paillier encryption algorithm. Based on the generated ciphertext, shift offset, and cluster center, the tag component and cluster center component are multiplied in the encrypted state, and then multiplied with the accumulated ciphertext to obtain the decrypted dot product value. The accumulated ciphertext is the accumulated product value of the encrypted points of the client tag distribution and the cluster center. Calculate the complete cosine similarity based on the decrypted dot product values and select the most similar cluster.
4. The privacy-preserving distributed machine learning client clustering method as described in claim 1, characterized in that, N clients are arranged in a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server. The server then obtains the final aggregation result based on the encrypted results, including: Will The clients are arranged in a logical ring structure, and each client determines its first successor and second successor; Each client splits its label distribution into two additive shared components, where the label distribution is a vector representation of all label distribution components; Each client encrypts its first and second successors based on two addition sharing methods before uploading them to the server; The server calculates the aggregated ciphertext for each client. The client uses its private key to decrypt the aggregated ciphertext and obtains a partial decrypted result. The server reconstructs the decrypted portion of the result from each client to obtain the final aggregated result.
5. The privacy-preserving distributed machine learning client clustering method as described in claim 4, characterized in that, Detect changes in the client side and monitor their impact on the clustering structure. Based on cluster center displacement thresholds, decide whether to perform global re-clustering, including: Set up a client join set and a client leave set, and monitor both sets to determine the affected clusters. The client join set is used to record clients that join, and the client leave set is used to record clients that leave. For each client added to the set, calculate its similarity to each cluster center and assign it to the most similar cluster. For each client added to the set, remove it from its current cluster. Calculate the cluster center displacement for the affected clusters; Determine if the cluster center position exceeds the cluster center displacement threshold. If it does, perform global re-clustering.
6. The privacy-preserving distributed machine learning client clustering method as described in claim 5, characterized in that, For each client added to the set, its similarity to each cluster center is calculated and it is assigned to the most similar cluster, including: For each client added to the set, calculate the similarity with the cluster centers and obtain the optimal cluster index: in, Represented as client The best clustering index to choose Indicates the total number of clusters. Indicates the range of clustering index. This indicates that the function returns the parameter that minimizes the value. Indicates the first Round The center of each cluster; The cluster corresponding to the best clustering index is taken as the most similar cluster and assigned to the most similar cluster.
7. The privacy-preserving distributed machine learning client clustering method as described in claim 5, characterized in that, Calculate the cluster center displacement for the affected clusters, including: For each affected cluster, calculate the original centers and the updated centers, where the original centers are calculated as follows: The updated method for calculating cluster centers is as follows: ; Calculate the cluster center displacement based on the original cluster centers and the updated cluster centers. : in, Let i be the original cluster center, and i be the i-th client. Indicates client The label distribution vector, For the first Wheel clustering A collection of clients, This is the set of clients after a partial update.
8. A privacy-preserving distributed machine learning client clustering device, characterized in that, It involves N clients and one coordination server, including: The system initialization module is used to generate independent homomorphic encryption key pairs for each client and coordinate server initialization. Cluster centers; The privacy-preserving similarity calculation module is used by each client to encode the label distribution vector obtained during the training process into a single ciphertext through multi-value packing technology, and to calculate the similarity with each cluster center using homomorphic encryption and select the most similar cluster. The label distribution vector is used to represent the sample distribution of all categories of the client. The secure aggregation update module is used to arrange N clients into a logical ring structure. Each client encrypts a fixed number of successor nodes and sends the encrypted results to the server so that the server can obtain the aggregation result based on the encrypted results. The adaptive re-clustering management module is used to detect changes in the client, monitor the impact of these changes on the clustering structure, and decide whether to perform global re-clustering based on the cluster center displacement threshold.
9. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed on a computer, they implement the privacy-preserving distributed machine learning client clustering method as described in any one of claims 1 to 7.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the privacy-preserving distributed machine learning client clustering method as described in any one of claims 1 to 7.