A privacy protection K-means clustering method for unbalanced data distribution

Abstract

The application relates to the technical field of data processing, and provides a privacy protection K-means clustering method for unbalanced data distribution, which comprises the following steps: a server selects a participating client, and sets a mean clustering input parameter; the client performs Bayesian Gaussian mixture clustering on local data, and obtains local center points and weights; the server performs weighted clustering on the local center points of all clients, and obtains initialized global center points; the client performs local iterative training on the initialized global center points, and obtains a local summation term; and the server performs summation on the local summation term, and updates the global center points. According to the privacy protection K-means clustering method for unbalanced data distribution, the data samples unevenly distributed in different clients can be effectively globally grouped under the premise of privacy protection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a privacy-preserving K-means clustering method for imbalanced data distribution. Background Technology

[0002] For unlabeled data, unsupervised federated learning aims to discover overall patterns from scattered and privacy-sensitive data. For example, representation learning on diagnostic case images across medical institutions helps to more comprehensively assess the pathogenesis mechanisms; federated clustering of trajectory data from different vehicle users can identify risk areas on urban roads without revealing individual location privacy. Federated clustering is an important area of ​​research in unsupervised federated learning, aiming to group globally similar (local) data points together. This involves clustering data points distributed across multiple clients based on a global similarity metric, with all data remaining available but not visible on the client devices.

[0003] Although federated clustering is a type of distributed clustering, it differs from traditional distributed clustering methods. Traditional fully distributed clustering methods do not prioritize privacy protection, and the data distribution is relatively balanced. For horizontal federated clustering with imbalanced client data, extending partitioning clustering, represented by K-Means, to federated learning scenarios has become a key research focus.

[0004] As is well known, centroid initialization is one of the key factors determining clustering results. Centralized clustering can generate good initial centroids by sampling from the dataset; however, without dataset constraints on a trusted server, sampling can lead to privacy issues. Random initialization is a viable method, but it ignores the distribution of client-side data, resulting in ill-conditioned groupings.

[0005] Furthermore, imbalanced data distribution can result in client-side data containing fewer categories than the total number of categories, and the categories included by different clients may vary. In this case, the client's forced division based on the total number of categories using a global centroid actually adds "noise" to centroids that do not belong to any of the categories. This leads to significant deviations in the centroids obtained by each client during local training, affecting the consistency of the global aggregation centroids and causing insufficient convergence in federated training.

[0006] Therefore, how to provide a privacy-preserving K-means clustering method to address privacy sensitivities and imbalanced data distribution in clustering problems has become an urgent technical issue to be solved. Summary of the Invention

[0007] In view of this, the present invention provides an improved privacy-preserving K-means clustering method, which obtains better global centroids through federated initialization polling than random initialization, and further improves the clustering effect through multiple rounds of joint training between the server and the client.

[0008] This invention provides a privacy-preserving K-means clustering method for imbalanced data distributions, comprising:

[0009] Step S1: The server selects participating clients and sets the input parameters for mean clustering;

[0010] Step S2: The client performs Bayesian-Gaussian mixture clustering on its local data to obtain local centroids and weights;

[0011] Step S3: The server performs weighted clustering of the local centroids of all clients to obtain the initialized global centroids;

[0012] Step S4: The client iteratively trains the initialized global center point locally to obtain the local summation term;

[0013] Step S5: The server sums up the local summation items and updates the global center point.

[0014] Furthermore, the mean clustering input parameters in step S1 of the method of the present invention include: the number of clustering target categories K, the maximum number of polling times, and the global center iteration termination threshold.

[0015] Further, step S2 of the method of the present invention includes:

[0016] Step S21: The server broadcasts the number of clustering target categories K to N clients;

[0017] Step S22: The client receives the number of clustering target categories K, uses the Bayesian-Gaussian mixture algorithm to perform local clustering on the local dataset until convergence, and returns the resulting mixture tuple consisting of the mean vector of N pairs of K centroids and the weights corresponding to the centroids to the server.

[0018] Further, step S3 of the method of the present invention includes:

[0019] Step S31: The server receives the mixed tuples from all clients, concatenates the mean vectors of the center points into a mean vector matrix, and concatenates the corresponding weights into a weight vector.

[0020] Step S32: The server uses the weighted K-means algorithm to cluster the mean vector matrix and the weight vector, and outputs K new mean vectors, which are the initialized global centroids.

[0021] Further, step S4 of the method of the present invention includes:

[0022] Step S41: The server broadcasts the initialized global center point obtained in step S32 to all clients;

[0023] Step S42: The client initializes the K-means model based on the initialized global centroid and performs local iterative training on the dataset to obtain new local centroids;

[0024] Step S43: The client statistically analyzes the number of samples belonging to each class and the sum of samples in the data set to obtain a local summation term.

[0025] Furthermore, in step S42 of the method of the present invention, local iterative training of the dataset is performed, including: calculating the distance from each point in the client dataset to the K global center points and assigning the point to the class corresponding to the index with the smallest distance.

[0026] Further, step S5 of the method of the present invention includes: the server collecting the summation items of all clients, summing the summation items of each class, dividing the global sample sum by the number of global samples to obtain a new global center point vector, and updating the global center point.

[0027] Furthermore, in the method of the present invention, after each round of local iterative training, the client updates the global center point of that round through the server using the method in step S5.

[0028] Furthermore, in the method of the present invention, after the client performs no more than 5 local iterations of training, the server calculates the mean of the corresponding global centroid by category index.

[0029] Finally, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method.

[0030] The privacy-preserving K-means clustering method for imbalanced data distributions of the present invention has the following beneficial effects:

[0031] 1) Throughout the entire process, all client data was never shared or circulated and remained locally, effectively preventing the leakage of sensitive information;

[0032] 2) It solves the problem of class imbalance among samples from different clients, which helps to obtain a better global clustering model. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of the privacy-preserving K-means clustering method for imbalanced data distribution according to the first embodiment of the present invention.

[0035] Figure 2 This is a flowchart of step S2 of the privacy-preserving K-means clustering method for imbalanced data distribution according to the second embodiment of the present invention.

[0036] Figure 3 This is a flowchart of step S3 of the privacy-preserving K-means clustering method for imbalanced data distribution according to the second embodiment of the present invention.

[0037] Figure 4 This is a flowchart of the privacy-preserving K-means clustering method for imbalanced data distribution according to the third embodiment of the present invention. Detailed Implementation

[0038] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0040] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0041] Figure 1 This is a flowchart of a privacy-preserving K-means clustering method for imbalanced data distribution according to an exemplary first embodiment of the present invention, as shown below. Figure 1 As shown, the method described in this embodiment includes:

[0042] Step S1: The server selects participating clients and sets the input parameters for mean clustering;

[0043] Step S2: The client performs Bayesian-Gaussian mixture clustering on its local data to obtain local centroids and weights;

[0044] Step S3: The server performs weighted clustering of the local centroids of all clients to obtain the initialized global centroids;

[0045] Step S4: The client iteratively trains the initialized global center point locally to obtain the local summation term;

[0046] Step S5: The server sums up the local summation items and updates the global center point.

[0047] The mean clustering input parameters in step S1 of this embodiment include: the number of target clusters K, the maximum number of polling rounds, and the global center iteration termination threshold.

[0048] In the method of this embodiment, repeating Figure 1 Steps S2 to S5 of the method shown continue until the training round t is greater than the maximum number of polling rounds T, or the following condition is met:

[0049]

[0050] Where ε is the global center iteration termination threshold, μ t-1 and μ t These are the global centers in rounds t-1 and t, respectively;

[0051] The result will be output as follows:

[0052]

[0053] Where k represents the k-th category, μ * This represents the optimal center of the final global K-means model, consisting of K center point vectors.

[0054] A second exemplary embodiment of the present invention provides a privacy-preserving K-means clustering method for imbalanced data distribution. This embodiment is... Figure 1 Preferred embodiments of the method shown are as follows: Figure 2 As shown, step S2 of the method described in this embodiment includes:

[0055] Step S21: The server broadcasts the number of clustering target categories K to N clients;

[0056] Step S22: The client receives the number of clustering target categories K, uses the Bayesian-Gaussian mixture algorithm to perform local clustering on the local dataset until convergence, and returns the resulting mixture tuple consisting of the mean vector of N pairs of K centroids and the weights corresponding to the centroids to the server.

[0057] like Figure 3 As shown, step S3 of the method described in this embodiment includes:

[0058] Step S31: The server receives the mixed tuples from all clients, concatenates the mean vectors of the center points into a mean vector matrix, and concatenates the corresponding weights into a weight vector.

[0059] Step S32: The server uses the weighted K-means algorithm to cluster the mean vector matrix and the weight vector, and outputs K new mean vectors, which are the initialized global centroids;

[0060] In this embodiment, the initialized global center point can be represented as follows:

[0061]

[0062] Where the superscript 0 indicates the result of initialization, This represents the center point vector of the k-th class.

[0063] In this embodiment, for the locally obtained center vectors, an activation weight vector π is defined to evaluate the effectiveness of these K center vectors. For example, assuming K is 3, given a client i, its k (i) The value is 1. If the weight vector is [1,0,0], it means that the first mean vector is valid. In this embodiment, Bayesian Gaussian Mixture (BGM) is used to determine this weight.

[0064] In this embodiment, the local dataset D of client i i Local data point x (x∈D) i () is a weighted mixture of K Gaussian distributions, expressed by the following formula:

[0065]

[0066] Wherein, N(x|μ k , Σ k The distribution is Gaussian;

[0067] Θ={π k ,μ k ,Σ k} k＝1,...,K These are the unknown parameters estimated by the expectation minimization algorithm, including the mixed weight π. k∈[0,1], and Σ k π k =1, and also includes the mean μ of the kth component. k Sum of variance Σ k ;

[0068] This embodiment employs an approximate Dirichlet process inference algorithm to estimate the weight distribution. Due to the Bayesian nature of the BGM, this process requires a prior parameter for weight concentration. Specifying a low value for the concentration prior parameter will cause the model to place most of the weights on a few principal components (centroids), while setting the weights of the remaining principal components very close to zero. This achieves the purpose of evaluating the effectiveness of local centers.

[0069] In this embodiment, each client performs Bayesian-Gaussian mixture clustering and outputs a mixture tuple consisting of the mean vectors of N pairs of K centroids and the corresponding weights (activation weights) of the centroids. The server collects the Bayesian-Gaussian mixture results from all clients, concatenates the centroid mean vectors into a mean matrix of size (N×K)×d, and concatenates the corresponding weights into an N×K dimensional vector. The weighted K-means algorithm is then used to output K new mean vectors. The weighted K-means algorithm supports sample weighting, which allows samples with higher weights to be considered when calculating cluster centers.

[0070] According to the method of this embodiment, the stronger the cluster heterogeneity, the better the quality of the initial points of the federated clusters obtained by local Bayesian Gaussian mixture clustering and global weighted K-Means. If there are at least K clients participating in the initialization process, each client actually has only one cluster, and the cluster distributions of different clients are different, then in the activation weights obtained by local Bayesian inference for each client, only one bit is 1, and the rest are 0. When the server performs weighted K-Means clustering, it discards the centroids with a weight of 0, thus obtaining exactly the K initial centroids corresponding to these different categories.

[0071] Figure 4 This is a flowchart of a privacy-preserving K-means clustering method for imbalanced data distribution according to an exemplary third embodiment of the present invention. This embodiment is... Figure 1 Preferred embodiments of the method shown are as follows: Figure 4 As shown, step S4 of the method described in this embodiment includes:

[0072] Step S41: The server broadcasts the initialized global center point obtained in step S32 to all clients.

[0073] Step S42: The client initializes the K-means model based on the initialized global centroid and performs local iterative training on the dataset to obtain new local centroids;

[0074] Step S43: The client statistically analyzes the number of samples belonging to each class and the sum of samples in the data set to obtain a local summation term.

[0075] In step S42 of this embodiment, local iterative training of the dataset is performed, including: calculating the distance from each point in the client dataset to the K global centroids and assigning the point to the class corresponding to the index with the smallest distance. In practical applications, the distance from each point in the dataset to the K global centroids can be calculated using the following formula:

[0076]

[0077] Among them, C k Let x represent the set of samples of category k. Suppose that among the K global centroids, sample x is related to the k-th global centroid. If the distance is the smallest, then x belongs to C. k .

[0078] In step S43 of this embodiment, the number of samples of the k-th class for the z-th client is represented as: Samples are represented as follows:

[0079]

[0080] Where z represents the client, k t Let k represent category k in round t, and this formula represents all categories belonging to C. k The samples are summed bitwise to obtain the sample and vector.

[0081] In this embodiment, the local iterative training is initiated by the server. When the server initiates the t-th (t>0) round of local iterative training, it sends the global center point of the previous round to the client. The global center point of the previous round is represented as follows:

[0082]

[0083] Here, the formula represents the set of K global center point vectors in the (t-1)th round.

[0084] The present invention provides a privacy-preserving K-means clustering method for imbalanced data distribution in an exemplary fourth embodiment. This embodiment is... Figure 1 In a preferred embodiment of the method shown, step S5 of the method in this embodiment includes: the server collecting summation items from all clients, summing the summation items for each class, dividing the global sample sum by the number of global samples to obtain a new global center point vector, and updating the global center point.

[0085] The new global center point vector in this embodiment is represented as follows:

[0086]

[0087] Where k represents the k-th category, K represents the number of clustering target categories, and this formula represents the set of K global centroid vectors in the t-th round.

[0088] In this embodiment, after each round of local iterative training, the client updates the global centroid of that round via the server using the method in step S5. In practical applications, the k-th global centroid can be calculated using the following formula:

[0089]

[0090] in, This represents the global centroid of class k in round t. It is obtained by first summing the local samples of all N clients belonging to class k, resulting in a global summation vector (i.e., Then, sum the number of local samples belonging to the k-th class for all N clients to obtain the global sample count (i.e., Then, divide the two by a vector to get the result.

[0091] In this embodiment, to avoid the problem of inconsistent local centers, the client can first perform only a small number of local iterations, such as selecting the number of local iterations from 1 to 5, to avoid excessive deviation of the center across clients; then, the server calculates the mean of the corresponding global center point according to the category index, so that the mean of the global center point in the two rounds does not produce a large update difference, thereby ensuring the convergence of the local iteration training process.

[0092] The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method.

[0093] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A privacy-preserving K-means clustering method for imbalanced data distribution, characterized in that, The method includes: Step S1: The server selects participating clients and sets the input parameters for mean clustering; Step S2: The client performs Bayesian-Gaussian mixture clustering on its local data to obtain local centroids and weights; Step S3: The server performs weighted clustering of the local centroids of all clients to obtain the initialized global centroids; Step S4: The client iteratively trains the initialized global center point locally to obtain the local summation term; Step S5: The server sums up the local summation items and updates the global center point; The input parameters for mean clustering in step S1 include: the number of target clusters K, the maximum number of rounds, and the global center iteration termination threshold; Step S2 includes: Step S21: The server broadcasts the number of clustering target categories K to N clients; Step S22: The client receives the number of clustering target categories K, uses the Bayesian-Gaussian mixture algorithm to perform local clustering on the local dataset until convergence, and returns the resulting mixture tuple consisting of the mean vector of N pairs of K centroids and the weights corresponding to the centroids to the server. Step S3 includes: Step S31: The server receives the mixed tuples from all clients, concatenates the mean vectors of the center points into a mean vector matrix, and concatenates the corresponding weights into a weight vector. Step S32: The server uses the weighted K-means algorithm to cluster the mean vector matrix and the weight vector, and outputs K new mean vectors, which are the initialized global centroids.

2. The privacy-preserving K-means clustering method for imbalanced data distribution according to claim 1, characterized in that, Step S4 includes: Step S41: The server broadcasts the initialized global center point obtained in step S32 to all clients; Step S42: The client initializes the K-means model based on the initialized global centroid and performs local iterative training on the dataset to obtain new local centroids; Step S43: The client statistically analyzes the number of samples belonging to each class and the sum of samples in the data set to obtain a local summation term.

3. The privacy-preserving K-means clustering method for imbalanced data distribution according to claim 1, characterized in that, In step S42, local iterative training is performed on the dataset, including: calculating the distance from each point in the client dataset to the K global center points and assigning the point to the class corresponding to the index with the smallest distance.

4. The privacy-preserving K-means clustering method for imbalanced data distribution according to claim 1, characterized in that, Step S5 includes: the server collects the summation items from all clients, sums up the summation items for each class, divides the global sample sum by the number of global samples to obtain a new global centroid vector, and updates the global centroid.

5. The privacy-preserving K-means clustering method for imbalanced data distribution according to claim 1, characterized in that, After each round of local iteration training, the client updates the global center point of that round through the server using the method in step S5.

6. The privacy-preserving K-means clustering method for imbalanced data distribution according to claim 1, characterized in that, After the client performs no more than 5 local iterations of training, the server calculates the mean of the corresponding global centroid by category index.

7. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any one of claims 1-6.

Images