A federated secure clustering method and system for processing unbalanced data

Abstract

The present invention discloses a method and system for processing unbalanced data in a federated secure clustering method, wherein the method comprises: performing local clustering on the unbalanced data set of each client based on a competitive learning algorithm to obtain a plurality of subclusters; calculating the cluster center, radius, size and standard deviation of each subcluster, and transmitting the cluster center, radius, size and standard deviation of each subcluster to a central server; the central server performs global clustering based on the cluster center, radius, size and standard deviation of each subcluster using a subcluster merging method to obtain a global clustering result; transmitting the global clustering result to each client, and each client updates the cluster label of each data point based on the global clustering result. The present invention can avoid missing the detection of small clusters, and also alleviates the cluster center offset problem in the partition-based clustering method used in the existing method.

Description

Technical Field

[0001] The present invention mainly relates to the field of data processing technology, and in particular to a federal secure clustering method and system for processing unbalanced data. Background Art

[0002] Federated learning has been widely studied in recent years. It is a distributed learning method that ensures common learning among clients while avoiding the problem of privacy leakage. Federated learning can be divided into supervised learning and unsupervised learning, and federated clustering is one of the unsupervised learning tasks. Existing federated clustering methods mainly focus on extending the k-means algorithm to the federated learning scenario, which inevitably makes clustering inherit the limitations brought by the k-means algorithm, that is, each learning client needs to give the number of real clusters in advance, but in reality it is often difficult for us to determine the number of real clusters on the client. At the same time, this type of partitioning clustering method easily causes the cluster center to shift towards the direction of the dense cluster during the clustering process, so it cannot handle unbalanced data well, but the data in the real world is often unbalanced. Therefore, the existing methods perform poorly in handling unbalanced data; the cluster value needs to be known on the client, but we can only know the total number of clusters in the entire data set on the server, but cannot determine how many clusters these data points on the client belong to. In this regard, how to better handle unbalanced data in a federated learning environment is worth exploring. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a federal secure clustering method and system for processing unbalanced data, which can avoid missing the detection of small clusters and alleviate the cluster center offset problem that occurs in the partition-based clustering method used by the existing method.

[0004] In order to solve the above technical problems, the present invention provides a federated secure clustering method for processing unbalanced data, the method comprising:

[0005] Based on the competitive learning algorithm, the unbalanced data set of each client is locally clustered to obtain several sub-clusters;

[0006] Calculate the cluster center, radius, size and standard deviation of each sub-cluster, and transmit the cluster center, radius, size and standard deviation of each sub-cluster to the central server;

[0007] The central server performs global clustering based on the cluster center, radius, size and standard deviation of each subcluster using a subcluster merging method to obtain a global clustering result;

[0008] The global clustering result is transmitted to each client, and each client updates the clustering label of each data point based on the global clustering result.

[0009] Optionally, the competitive learning algorithm is used to perform local clustering on the unbalanced data set of each client to obtain a plurality of subclusters, including:

[0010] Initializing the attribution degree matrix and the attraction degree matrix for each unbalanced data set to obtain the initial attribution degree matrix and the initial attraction degree matrix of each unbalanced data set;

[0011] Based on the initial attribution matrix and the initial attraction matrix, an iterative update process is performed using a message passing mechanism to a preset number of iterations to obtain a target attribution matrix and a target attraction matrix;

[0012] The target attribution matrix and the target attraction matrix are added together, and corresponding calculations are performed to obtain the cluster center of the data set on each client. Based on the cluster center, the unbalanced data set of each client is locally clustered to obtain several sub-clusters.

[0013] Optionally, the calculating of the cluster center, radius, size and standard deviation of each sub-cluster includes:

[0014] Calculate the mean of all data points in each sub-cluster and use the mean as the cluster center of the sub-cluster;

[0015] Calculate the average distance from all data points in each sub-cluster to the cluster center, and use the average distance as the radius of the sub-cluster;

[0016] Calculate the number of all data points in each subcluster and use the number of all data points as the size of the subcluster;

[0017] Calculate the standard deviation of all data points in each subcluster to obtain the standard deviation of each subcluster.

[0018] Optionally, performing global clustering based on the cluster center, radius, size and standard deviation of each subcluster using a subcluster merging method to obtain a global clustering result includes:

[0019] Calculate the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster;

[0020] Select the two subclusters with the smallest pairwise distance coefficient to merge, obtain a new cluster, and calculate the relevant parameters of the new cluster;

[0021] After calculating the relevant parameters of the new cluster, the pairwise distance coefficients of all remaining subclusters are recalculated, subclusters are merged, and the relevant parameters of the new cluster are calculated to a preset number of iterations to obtain the corresponding new clusters and the corresponding cluster center sets.

[0022] Optionally, the calculating of the pairwise distance coefficients between sub-clusters based on the cluster center, radius, size and standard deviation of each sub-cluster includes:

[0023] The degree of overlap between sub-clusters is calculated based on the radius of each sub-cluster and the distance between the cluster centers of the sub-clusters.

[0024] Based on the standard deviation of each sub-cluster, calculate the degree of difference in the standard deviation between sub-clusters;

[0025] The pairwise distance coefficients between subclusters are calculated based on the location of the cluster center of each subcluster, the degree of overlap between subclusters, and the degree of difference in standard deviations between subclusters.

[0026] Optionally, the calculation expression for the overlap degree between the subclusters is:

[0027]

[0028] Among them, o(C i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between.

[0029] Optionally, the calculation expression of the pairwise distance coefficient between the subclusters is:

[0030]

[0031] Among them, d(C i ,C j ) is a subcluster C i and subcluster C j The pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The degree of difference in standard deviation between them, β is a hyperparameter.

[0032] Optionally, the relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster.

[0033] Optionally, the calculation expression of the cluster center of the new cluster is:

[0034]

[0035] Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster;

[0036] The calculation expression of the radius of the new cluster is:

[0037]

[0038] Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster.

[0039] In addition, the present invention also provides a federated secure clustering system for processing unbalanced data, the system comprising:

[0040] Local clustering module: used to perform local clustering on the unbalanced data set of each client based on the competitive learning algorithm to obtain several sub-clusters;

[0041] Subcluster parameter calculation and transmission module: used to calculate the cluster center, radius, size and standard deviation of each subcluster, and transmit the cluster center, radius, size and standard deviation of each subcluster to the central server;

[0042] Global clustering module: used for the central server to perform global clustering based on the cluster center, radius, size and standard deviation of each sub-cluster using a sub-cluster merging method to obtain a global clustering result;

[0043] Clustering label updating module: used to transmit the global clustering result to each client, and each client updates the clustering label of each data point based on the global clustering result.

[0044] In the embodiment of the present invention, the competitive learning algorithm can be used to adaptively cluster data into several sub-clusters of similar size, and the number of clusters does not need to be known in advance on the client. By calculating the cluster center, radius, size and standard deviation of the sub-cluster obtained on the client, and transmitting the cluster center, radius, size and standard deviation of the sub-cluster to the central server for global clustering, data privacy leakage can be avoided in the process of transmitting data from the client to the central server. In the central server, the pairwise distance coefficient between sub-clusters is calculated by the cluster center, radius, size and standard deviation of the sub-cluster to merge sub-clusters from different clients, and global clustering is completed, so that the sub-clusters are merged from fine granularity to coarse granularity, which can avoid missing the detection of small clusters and alleviate the cluster center offset problem in the partition-based clustering method used in the existing method, thereby having better clustering performance for unbalanced data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flow chart of a federated secure clustering method for processing unbalanced data in an embodiment of the present invention;

[0047] Figure 2 It is a schematic diagram of the structural composition of a federated secure clustering system for processing unbalanced data in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Embodiment 1

[0050] See also Figure 1 , Figure 1 It is a flowchart of a federal secure clustering method for processing unbalanced data in an embodiment of the present invention.

[0051] like Figure 1 As shown, a federated secure clustering method for processing unbalanced data includes:

[0052] S11: Perform local clustering on the unbalanced data set of each client based on the competitive learning algorithm to obtain several subclusters;

[0053] In the specific implementation process of the present invention, the competitive learning algorithm is used to locally cluster the unbalanced data set of each client to obtain a number of sub-clusters, including: initializing the attribution matrix and the attraction matrix of each unbalanced data set to obtain the initial attribution matrix and the initial attraction matrix of each unbalanced data set; based on the initial attribution matrix and the initial attraction matrix, iteratively updating the process using the message passing mechanism to a preset number of iterations to obtain the target attribution matrix and the target attraction matrix; adding the target attribution matrix and the target attraction matrix, and performing corresponding calculations to obtain the clustering center of the data set on each client, and locally clustering the unbalanced data set of each client based on the clustering center to obtain a number of sub-clusters.

[0054] Specifically, the unbalanced data comes from multiple clients, each client may contain a different number of samples, and each sample is represented by a set of attributes. The attribution matrix and the attraction matrix are initialized for each unbalanced data set, and the similarity matrix between the data points in the unbalanced data set can be constructed. The attribution matrix and the attraction matrix are initialized according to the similarity matrix to obtain the initial attribution matrix and the initial attraction matrix of each unbalanced data set. Based on the initial attribution matrix and the initial attraction matrix, the message passing mechanism is used to perform iterative update processing to a preset number of iterations to obtain the target attribution matrix and the target attraction matrix. The target attribution matrix and the target attraction matrix are added, and relevant calculations are performed to obtain an evaluation matrix. The data corresponding to the maximum value of each row in the evaluation matrix is ​​selected as the cluster center to obtain the cluster center of the data set on each client. Based on the cluster center, the unbalanced data set of each client is locally clustered, that is, the data with the same cluster center is aggregated into a cluster to obtain several subclusters. Some existing methods that do not require knowing the actual number of clusters of the data can be used on the client to perform local clustering. It is only necessary to obtain relatively small sub-clusters. For this purpose, mean shift clustering algorithms and density-based clustering algorithms can also be used. A competitive learning algorithm is selected on the client. This method can adaptively cluster data into several sub-clusters of similar size, and the client does not need to know the number of clusters in advance.

[0055] S12: Calculate the cluster center, radius, size and standard deviation of each sub-cluster, and transmit the cluster center, radius, size and standard deviation of each sub-cluster to the central server;

[0056] In the specific implementation process of the present invention, the calculation of the cluster center, radius, size and standard deviation of each subcluster includes: calculating the mean of all data points in each subcluster, and taking the mean as the cluster center of the subcluster; calculating the average distance from all data points in each subcluster to the cluster center, and taking the average distance as the radius of the subcluster; calculating the number of all data points in each subcluster, and taking the number of all data points as the size of the subcluster; calculating the standard deviation of all data points in each subcluster, and obtaining the standard deviation of each subcluster.

[0057] Specifically, in order to ensure that data privacy is not leaked during the data transmission from the client to the server, a transmission strategy is designed to calculate the cluster center, radius, size and standard deviation of the sub-cluster obtained on the client, and transmit these parameters to the central server for the next stage of global clustering. Calculate the mean of all data points in each sub-cluster and use the mean as the cluster center of the sub-cluster; calculate the average distance from all data points in each sub-cluster to the cluster center and use the average distance as the radius of the sub-cluster; calculate the number of all data points in each sub-cluster and use the number of all data points as the size of the sub-cluster; calculate the standard deviation of all data points in each sub-cluster and obtain the standard deviation of each sub-cluster. If the data is multidimensional, calculate the standard deviation of each dimension and form it into a vector to represent the standard deviation vector of the sub-cluster. After the calculation is completed, upload it to the central server.

[0058] S13: the central server performs global clustering based on the cluster center, radius, size and standard deviation of each sub-cluster using a sub-cluster merging method to obtain a global clustering result;

[0059] In the specific implementation process of the present invention, the subcluster merging method is used to perform global clustering based on the cluster center, radius, size and standard deviation of each subcluster to obtain a global clustering result, including: calculating the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster; selecting two subclusters with the smallest pairwise distance coefficient to merge to obtain a new cluster, and calculating the relevant parameters of the new cluster; after calculating the relevant parameters of the new cluster, recalculating the pairwise distance coefficients, merging the subclusters and calculating the relevant parameters of the new cluster for all remaining subclusters until a preset number of iterations are performed to obtain a corresponding number of new clusters and a corresponding set of cluster centers.

[0060] Furthermore, the method of calculating the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster includes: calculating the degree of overlap between subclusters based on the radius of each subcluster and the cluster center distance between subclusters; calculating the degree of difference in standard deviations between subclusters based on the standard deviation of each subcluster; and calculating the pairwise distance coefficient between subclusters based on the cluster center position of each subcluster, the degree of overlap between subclusters and the degree of difference in standard deviations between subclusters.

[0061] Furthermore, the calculation expression of the overlap degree between the sub-clusters is:

[0062]

[0063] Among them, o(C i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between.

[0064] Furthermore, the calculation expression of the pairwise distance coefficient between the subclusters is:

[0065]

[0066] Among them, d(C i ,C j ) is a subcluster C i and subcluster C j The pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The degree of difference in standard deviation between them, β is a hyperparameter.

[0067] Furthermore, the relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster.

[0068] Furthermore, the calculation expression of the cluster center of the new cluster is:

[0069]

[0070] Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster;

[0071] The calculation expression of the radius of the new cluster is:

[0072]

[0073] Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster.

[0074] Specifically, the total number of subclusters from all clients is calculated, and the cluster center set is initialized. A loop process is entered to merge subclusters. In each round of the loop process, the pairwise distance coefficients between all subclusters are calculated, and the overlap degree between subclusters is calculated based on the radius of each subcluster and the cluster center distance between subclusters. The calculation expression of the overlap degree between subclusters is:

[0075]

[0076] Among them, o(C i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c jThe distance between them. The difference degree of the standard deviation between subclusters is calculated based on the standard deviation of each subcluster, mainly to calculate the absolute value of the difference between the two standard deviations, and the pairwise distance coefficient between subclusters is calculated based on the cluster center position of each subcluster, the overlap degree between subclusters and the difference degree of the standard deviation between subclusters. The calculation expression of the pairwise distance coefficient between subclusters is:

[0077]

[0078] Among them, d(C i ,C j ) is a subcluster C i and subcluster C j The pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The difference in standard deviations between them, β is a hyperparameter, and the distance between the cluster centers of each sub-cluster is calculated using the Euclidean distance. The hyperparameter is used to control the degree of influence of the standard deviation difference on the value of the entire distance coefficient. When the value of the pairwise distance coefficient is smaller, it means that the distance coefficient between the corresponding two sub-clusters is also smaller, which means that the similarity is greater. Select the two sub-clusters with the smallest pairwise distance coefficient to merge, obtain a new cluster, and calculate the relevant parameters of the new cluster. The relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster. The size of the new cluster is also the number of data points calculated for the new cluster, and the standard deviation of the new cluster is also the standard deviation of all data points. Because there is no specific value of the data points of the sub-cluster on the central server, the radius and cluster center of the new cluster are estimated by weighted method. The calculation expression of the cluster center of the new cluster is:

[0079]

[0080] Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster;

[0081] The calculation expression of the radius of the new cluster is:

[0082]

[0083] Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster. After calculating the relevant parameters of the new cluster, execute the next cycle, recalculate the pairwise distance coefficients, merge the subclusters, and calculate the relevant parameters of the new cluster for all the remaining subclusters until the preset number of iterations. The preset number of iterations can be the total number of subclusters of all clients minus the specified value. According to the similarity between the subclusters, the subclusters from different clients are continuously merged on the central server until a specified number of clusters remain, and a number of new clusters are obtained. The new clusters are formed into corresponding cluster center sets, and the corresponding new clusters and the corresponding cluster center sets are obtained, which are the global clustering results. A method for merging subclusters is adopted. The global clustering results of all data are obtained on the server. This method merges the subclusters from fine granularity to coarse granularity, which can avoid missing the detection of small clusters, and also alleviates the cluster center offset problem in the partition-based clustering method used in the existing method, thereby having better clustering performance for unbalanced data.

[0084] S14: Transmitting the global clustering result to each client, and each client updates the clustering label of each data point based on the global clustering result.

[0085] In the specific implementation process of the present invention, the corresponding cluster center set is transmitted back to each corresponding client, so that the client updates the clustering label of each data point to complete the global clustering process.

[0086] In the embodiment of the present invention, the competitive learning algorithm can be used to adaptively cluster data into several sub-clusters of similar size, and the number of clusters does not need to be known in advance on the client. By calculating the cluster center, radius, size and standard deviation of the sub-cluster obtained on the client, and transmitting the cluster center, radius, size and standard deviation of the sub-cluster to the central server for global clustering, data privacy leakage can be avoided in the process of transmitting data from the client to the central server. In the central server, the pairwise distance coefficient between sub-clusters is calculated by the cluster center, radius, size and standard deviation of the sub-cluster to merge sub-clusters from different clients, and global clustering is completed, so that the sub-clusters are merged from fine granularity to coarse granularity, which can avoid missing the detection of small clusters and alleviate the cluster center offset problem in the partition-based clustering method used in the existing method, thereby having better clustering performance for unbalanced data.

[0087] Embodiment 2

[0088] See also Figure 2 , Figure 2 It is a schematic diagram of the structural composition of a federated secure clustering system for processing unbalanced data in an embodiment of the present invention.

[0089] like Figure 2 As shown, a federated secure clustering system for processing unbalanced data, the system comprising:

[0090] Local clustering module 21: used to perform local clustering on the unbalanced data set of each client based on a competitive learning algorithm to obtain a number of sub-clusters;

[0091] In the specific implementation process of the present invention, the competitive learning algorithm is used to perform local clustering on the unbalanced data set of each client to obtain a number of sub-clusters, including: initializing the attribution matrix and the attraction matrix for each unbalanced data set to obtain the initial attribution matrix and the initial attraction matrix of each unbalanced data set; based on the initial attribution matrix and the initial attraction matrix, iteratively updating the process using the message passing mechanism to a preset number of iterations to obtain the target attribution matrix and the target attraction matrix; adding the target attribution matrix and the target attraction matrix, and performing relevant calculations to obtain the clustering center of the data set on each client, and locally clustering the unbalanced data set of each client based on the clustering center to obtain a number of sub-clusters.

[0092] Specifically, the unbalanced data comes from multiple clients, each client may contain a different number of samples, and each sample is represented by a set of attributes. The attribution matrix and the attraction matrix are initialized for each unbalanced data set, and the similarity matrix between the data points in the unbalanced data set can be constructed. The attribution matrix and the attraction matrix are initialized according to the similarity matrix to obtain the initial attribution matrix and the initial attraction matrix of each unbalanced data set. Based on the initial attribution matrix and the initial attraction matrix, the message passing mechanism is used to perform iterative update processing to a preset number of iterations to obtain the target attribution matrix and the target attraction matrix. The target attribution matrix and the target attraction matrix are added, and relevant calculations are performed to obtain an evaluation matrix. The unbalanced data corresponding to the maximum value of each row in the evaluation matrix are selected as the cluster center to obtain the cluster center of the data set on each client. Based on the cluster center, the unbalanced data set of each client is locally clustered, that is, the data with the same cluster center is aggregated into a cluster to obtain several subclusters. Some existing methods that do not require knowing the actual number of clusters of the data can be used on the client to perform local clustering. It is only necessary to obtain relatively small sub-clusters. For this purpose, mean shift clustering algorithms and density-based clustering algorithms can also be used. A competitive learning algorithm is selected on the client. This method can adaptively cluster data into several sub-clusters of similar size, and the client does not need to know the number of clusters in advance.

[0093] Subcluster parameter calculation and transmission module 22: used to calculate the cluster center, radius, size and standard deviation of each subcluster, and transmit the cluster center, radius, size and standard deviation of each subcluster to the central server;

[0094] In the specific implementation process of the present invention, the calculation of the cluster center, radius, size and standard deviation of each subcluster includes: calculating the mean of all data points in each subcluster, and taking the mean as the cluster center of the subcluster; calculating the average distance from all data points in each subcluster to the cluster center, and taking the average distance as the radius of the subcluster; calculating the number of all data points in each subcluster, and taking the number of all data points as the size of the subcluster; calculating the standard deviation of all data points in each subcluster, and obtaining the standard deviation of each subcluster.

[0095] Specifically, in order to ensure that data privacy is not leaked during the data transmission from the client to the server, a transmission strategy is designed to calculate the cluster center, radius, size and standard deviation of the sub-cluster obtained on the client, and transmit these parameters to the central server for the next stage of global clustering. Calculate the mean of all data points in each sub-cluster and use the mean as the cluster center of the sub-cluster; calculate the average distance from all data points in each sub-cluster to the cluster center and use the average distance as the radius of the sub-cluster; calculate the number of all data points in each sub-cluster and use the number of all data points as the size of the sub-cluster; calculate the standard deviation of all data points in each sub-cluster and obtain the standard deviation of each sub-cluster. If the data is multidimensional, calculate the standard deviation of each dimension and form it into a vector to represent the standard deviation vector of the sub-cluster. After the calculation is completed, upload it to the central server.

[0096] Global clustering module 23: used for the central server to perform global clustering based on the cluster center, radius, size and standard deviation of each sub-cluster using a sub-cluster merging method to obtain a global clustering result;

[0097] In the specific implementation process of the present invention, the subcluster merging method is used to perform global clustering based on the cluster center, radius, size and standard deviation of each subcluster to obtain a global clustering result, including: calculating the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster; selecting two subclusters with the smallest pairwise distance coefficient to merge to obtain a new cluster, and calculating the relevant parameters of the new cluster; after calculating the relevant parameters of the new cluster, recalculating the pairwise distance coefficients, merging the subclusters and calculating the relevant parameters of the new cluster for all remaining subclusters until a preset number of iterations are performed to obtain a corresponding number of new clusters and a corresponding set of cluster centers.

[0098] Furthermore, the method of calculating the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster includes: calculating the degree of overlap between subclusters based on the radius of each subcluster and the cluster center distance between subclusters; calculating the degree of difference in standard deviations between subclusters based on the standard deviation of each subcluster; and calculating the pairwise distance coefficient between subclusters based on the cluster center position of each subcluster, the degree of overlap between subclusters and the degree of difference in standard deviations between subclusters.

[0099] Furthermore, the calculation expression of the overlap degree between the sub-clusters is:

[0100]

[0101] Among them, o(C i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between.

[0102] Furthermore, the calculation expression of the pairwise distance coefficient between the subclusters is:

[0103]

[0104] Among them, d(C i ,C j ) is a subcluster C i and subcluster C j The pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The degree of difference in standard deviation between them, β is a hyperparameter.

[0105] Furthermore, the relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster.

[0106] Furthermore, the calculation expression of the cluster center of the new cluster is:

[0107]

[0108] Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster;

[0109] The calculation expression of the radius of the new cluster is:

[0110]

[0111] Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster.

[0112] Specifically, the total number of subclusters from all clients is calculated, and the cluster center set is initialized. A loop process is entered to merge subclusters. In each round of the loop process, the pairwise distance coefficients between all subclusters are calculated, and the overlap degree between subclusters is calculated based on the radius of each subcluster and the cluster center distance between subclusters. The calculation expression of the overlap degree between subclusters is:

[0113]

[0114] Among them, o(C i ,C j ) is a subcluster C i and subcluster C f The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between them. The difference degree of the standard deviation between subclusters is calculated based on the standard deviation of each subcluster, mainly to calculate the absolute value of the difference between the two standard deviations, and the pairwise distance coefficient between subclusters is calculated based on the cluster center position of each subcluster, the overlap degree between subclusters and the difference degree of the standard deviation between subclusters. The calculation expression of the pairwise distance coefficient between subclusters is:

[0115]

[0116] Among them, d(C i ,C j ) is a subcluster C i and subcluster C jThe pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The difference in standard deviations between them, β is a hyperparameter, and the distance between the cluster centers of each sub-cluster is calculated using the Euclidean distance. The hyperparameter is used to control the degree of influence of the standard deviation difference on the value of the entire distance coefficient. When the value of the pairwise distance coefficient is smaller, it means that the distance coefficient between the corresponding two sub-clusters is also smaller, which means that the similarity is greater. Select the two sub-clusters with the smallest pairwise distance coefficient to merge, obtain a new cluster, and calculate the relevant parameters of the new cluster. The relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster. The size of the new cluster is also the number of data points calculated for the new cluster, and the standard deviation of the new cluster is also the standard deviation of all data points. Because there is no specific value of the data points of the sub-cluster on the central server, the radius and cluster center of the new cluster are estimated by weighted method. The calculation expression of the cluster center of the new cluster is:

[0117]

[0118] Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster;

[0119] The calculation expression of the radius of the new cluster is:

[0120]

[0121] Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster. After calculating the relevant parameters of the new cluster, execute the next cycle, recalculate the pairwise distance coefficients, merge the subclusters, and calculate the relevant parameters of the new cluster for all the remaining subclusters until the preset number of iterations is reached. The preset number of iterations can be the total number of subclusters of all clients minus the specified value. According to the similarity between the subclusters, the subclusters from different clients are continuously merged on the central server until a specified number of clusters remain, and a number of new clusters are obtained. The new clusters are formed into corresponding cluster center sets, and the corresponding new clusters and the corresponding cluster center sets are obtained, which are the global clustering results. A method for merging subclusters is adopted. The global clustering results of all data are obtained on the server. This method merges the subclusters from fine granularity to coarse granularity, which can avoid missing the detection of small clusters, and also alleviates the cluster center offset problem in the partition-based clustering method used in the existing method, thereby having better clustering performance for unbalanced data.

[0122] Clustering label updating module 24: used to transmit the global clustering result to each client, and each client updates the clustering label of each data point based on the global clustering result.

[0123] In the specific implementation process of the present invention, the corresponding cluster center set is transmitted back to each corresponding client, so that the client updates the clustering label of each data point to complete the global clustering process.

[0124] In the embodiment of the present invention, the competitive learning algorithm can be used to adaptively cluster data into several sub-clusters of similar size, and the number of clusters does not need to be known in advance on the client. By calculating the cluster center, radius, size and standard deviation of the sub-cluster obtained on the client, and transmitting the cluster center, radius, size and standard deviation of the sub-cluster to the central server for global clustering, data privacy leakage can be avoided in the process of transmitting data from the client to the central server. In the central server, the pairwise distance coefficient between sub-clusters is calculated by the cluster center, radius, size and standard deviation of the sub-cluster to merge sub-clusters from different clients, and global clustering is completed, so that the sub-clusters are merged from fine granularity to coarse granularity, which can avoid missing the detection of small clusters and alleviate the cluster center offset problem in the partition-based clustering method used in the existing method, thereby having better clustering performance for unbalanced data.

[0125] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0126] In addition, the above is a detailed introduction to a federal secure clustering method and system for processing unbalanced data provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A federated secure clustering method for processing unbalanced data, characterized in that: The method comprises: Based on the competitive learning algorithm, the unbalanced data set of each client is locally clustered to obtain several sub-clusters; Calculate the cluster center, radius, size and standard deviation of each sub-cluster, and transmit the cluster center, radius, size and standard deviation of each sub-cluster to the central server; The central server performs global clustering based on the cluster center, radius, size and standard deviation of each subcluster using a subcluster merging method to obtain a global clustering result; Transmitting the global clustering result to each client, and each client updating the clustering label of each data point based on the global clustering result; The method of performing global clustering based on the cluster center, radius, size and standard deviation of each subcluster by using a subcluster merging method to obtain a global clustering result includes: calculating the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster; selecting two subclusters with the smallest pairwise distance coefficients to merge to obtain a new cluster, and calculating the relevant parameters of the new cluster; after calculating the relevant parameters of the new cluster, recalculating the pairwise distance coefficients, merging the subclusters and calculating the relevant parameters of the new cluster for all remaining subclusters until a preset number of iterations are performed to obtain a corresponding number of new clusters and a corresponding cluster center set; The step of calculating the pairwise distance coefficients between subclusters based on the cluster center, radius, size and standard deviation of each subcluster comprises: calculating the degree of overlap between subclusters based on the radius of each subcluster and the distance between the cluster centers of the subclusters; calculating the degree of difference in the standard deviations between the subclusters based on the standard deviation of each subcluster; and calculating the pairwise distance coefficients between the subclusters based on the cluster center position of each subcluster, the degree of overlap between the subclusters and the degree of difference in the standard deviations between the subclusters. The calculation expression of the overlap degree between the subclusters is: Among them, o(C i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between The calculation expression of the pairwise distance coefficient between the subclusters is: Among them, d(C i ,C j ) is a subcluster C i and subcluster C j The pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The degree of difference in standard deviation between them, β is a hyperparameter; The relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster; The calculation expression of the cluster center of the new cluster is: Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster; The calculation expression of the radius of the new cluster is: Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster.

2. The method for processing unbalanced data using a federated secure clustering algorithm according to claim 1, wherein: The competitive learning algorithm is used to perform local clustering on the unbalanced data set of each client to obtain several sub-clusters, including: Initializing the attribution degree matrix and the attraction degree matrix for each unbalanced data set to obtain the initial attribution degree matrix and the initial attraction degree matrix of each unbalanced data set; Based on the initial attribution matrix and the initial attraction matrix, an iterative update process is performed using a message passing mechanism to a preset number of iterations to obtain a target attribution matrix and a target attraction matrix; The target attribution matrix and the target attraction matrix are added together, and corresponding calculations are performed to obtain the cluster center of the data set on each client. Based on the cluster center, local clustering is performed on the unbalanced data set of each client to obtain several sub-clusters.

3. The method for processing unbalanced data using a federated secure clustering algorithm according to claim 1, characterized in that: The calculation of the cluster center, radius, size and standard deviation of each sub-cluster includes: Calculate the mean of all data points in each sub-cluster and use the mean as the cluster center of the sub-cluster; Calculate the average distance from all data points in each sub-cluster to the cluster center, and use the average distance as the radius of the sub-cluster; Calculate the number of all data points in each subcluster and use the number of all data points as the size of the subcluster; Calculate the standard deviation of all data points in each subcluster to obtain the standard deviation of each subcluster.

4. A federated secure clustering system for processing unbalanced data, characterized in that: The system comprises: Local clustering module: used to perform local clustering on the unbalanced data set of each client based on the competitive learning algorithm to obtain several sub-clusters; Subcluster parameter calculation and transmission module: used to calculate the cluster center, radius, size and standard deviation of each subcluster, and transmit the cluster center, radius, size and standard deviation of each subcluster to the central server; Global clustering module: used for the central server to perform global clustering based on the cluster center, radius, size and standard deviation of each sub-cluster using a sub-cluster merging method to obtain a global clustering result; Clustering label updating module: used for transmitting the global clustering result to each client, and each client updates the clustering label of each data point based on the global clustering result; The method of performing global clustering based on the cluster center, radius, size and standard deviation of each subcluster by using a subcluster merging method to obtain a global clustering result includes: calculating the pairwise distance coefficient between subclusters based on the cluster center, radius, size and standard deviation of each subcluster; selecting two subclusters with the smallest pairwise distance coefficients to merge to obtain a new cluster, and calculating the relevant parameters of the new cluster; after calculating the relevant parameters of the new cluster, recalculating the pairwise distance coefficients, merging the subclusters and calculating the relevant parameters of the new cluster for all remaining subclusters until a preset number of iterations are performed to obtain a corresponding number of new clusters and a corresponding cluster center set; The step of calculating the pairwise distance coefficients between subclusters based on the cluster center, radius, size and standard deviation of each subcluster comprises: calculating the degree of overlap between subclusters based on the radius of each subcluster and the distance between the cluster centers of the subclusters; calculating the degree of difference in the standard deviations between the subclusters based on the standard deviation of each subcluster; and calculating the pairwise distance coefficients between the subclusters based on the cluster center position of each subcluster, the degree of overlap between the subclusters and the degree of difference in the standard deviations between the subclusters. The calculation expression of the overlap degree between the subclusters is: Among them, o(C i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i The radius of is a subcluster C j The radius, dist(c i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between The calculation expression of the pairwise distance coefficient between the subclusters is: Among them, d(C i ,C j ) is a subcluster C i and subcluster C j The pairwise distance coefficient between i ,c j ) is a subcluster C i The cluster center c i and subcluster C j The cluster center c j The distance between i ,C j ) is a subcluster C i and subcluster C j The degree of overlap between is a subcluster C i and subcluster C j The degree of difference in standard deviation between them, β is a hyperparameter; The relevant parameters of the new cluster include the cluster center, radius, size and standard deviation of the new cluster; The calculation expression of the cluster center of the new cluster is: Among them, c ′ The center of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, and c2 is the cluster center of the second subcluster C2 in the new cluster; The calculation expression of the radius of the new cluster is: Among them, r ′ is the radius of the new cluster, is the size of the first subcluster C1 in the new cluster, is the size of the second subcluster C2 in the new cluster, c1 is the cluster center of the first subcluster C1 in the new cluster, c2 is the cluster center of the second subcluster C2 in the new cluster, c ′ is the cluster center of the new cluster, dist(c ′ ,c1) is the cluster center c of the new cluster ′ The distance between the cluster center c1 of the first subcluster in the new cluster, is the radius of the first subcluster C1 in the new cluster, dist(c ′ ,c2) is the cluster center c of the new cluster ′ The distance between the cluster center c2 of the second subcluster in the new cluster, is the radius of the second subcluster C2 in the new cluster.

Images