Federal deep clustering method and system based on differential privacy, processing equipment and storage medium

Abstract

The invention relates to a federal deep clustering method and system based on differential privacy, processing equipment and a storage medium, and is characterized in that the method comprises the steps that a central server trains a global model based on a public data set and a local clustering model of each client, and a trained global model is obtained; and each client adopts the trained global model to predict the local data set to obtain a corresponding clustering distribution result, so that the method can effectively relieve adverse effects caused by isomerism, improves the clustering performance of the model, and can be widely applied to the fields of data analysis and network security.

Description

Technical Field

[0001] The present invention relates to the fields of data analysis and network security, and in particular to a federated deep clustering method, system, processing device and storage medium based on differential privacy. Background Art

[0002] As one of the branches of federated learning, federated clustering is responsible for the important task of clustering distributed data. As an emerging distributed learning framework, federated learning provides a new solution to the distributed data clustering problem. Federated clustering can cluster data distributed among different participants based on global similarity while protecting data privacy. This method allows participants to keep their data locally, thereby reducing the risk of privacy leakage.

[0003] Existing federated clustering methods mainly include methods based on k-means, fuzzy-c-means, spectral clustering, and matrix decomposition. These methods have significant limitations when dealing with large-scale, high-dimensional, nonlinear and complex data sets, and the clustering performance is not ideal. For example, the federated clustering method based on k-means is easily affected by outliers, is sensitive to the selection of initial centers, and has difficulty in dealing with clustering problems of non-convex shapes; the federated clustering algorithm based on spectral clustering takes a lot of time to construct the similarity matrix and calculate the eigenvector, and is not suitable for large-scale data sets. In order to overcome the dimensionality curse and feature selection problems in high-dimensional and complex data sets, some scholars have applied advanced representation learning methods such as contrastive learning to federated clustering in the latest research, and proposed federated clustering methods based on contrastive learning, but these methods are difficult to cope with data heterogeneity scenarios.

[0004] In addition to clustering performance issues, existing methods also have major flaws in privacy protection. Most federated clustering methods require the client to upload local cluster centers or model parameter updates to the server during operation. However, this information contains sensitive information of the client and is private data, which should be fully protected. Existing studies have shown that attackers can successfully restore the original information in the data set through K-nearest neighbor attacks and reconstruction attacks on cluster centers, as well as gradient attacks and member inference attacks on model parameters, posing a serious threat to user privacy. Although there are methods that apply localized differential privacy technology to protect the centroid matrix that needs to be uploaded, this method introduces a lot of noise, making it difficult to strike a balance between privacy protection and clustering performance, resulting in poor algorithm usability. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide a federated deep clustering method, system, processing device and storage medium based on differential privacy, which can improve clustering performance and privacy protection capabilities.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions: In a first aspect, a federated deep clustering method based on differential privacy is provided, comprising:

[0007] The central server trains the global model based on the public data set and the local clustering model of each client to obtain a trained global model;

[0008] Each client uses the trained global model to predict the local data set and obtain the corresponding clustering assignment results.

[0009] Furthermore, the central server trains the global model based on the public data set and the local clustering model of each client to obtain a trained global model, including:

[0010] The central server pre-trains the global model based on the public data set to obtain model parameters;

[0011] Each client negotiates a random number between two clients according to the federal security aggregation protocol to complete the negotiation of random numbers;

[0012] Each client uses the LoRA method to reparameterize the parameters of the convolutional layer in the local clustering model to obtain the initial global model parameters;

[0013] Each client initializes the local clustering model according to the global model parameters sent by the central server, and trains the local clustering model according to the local data set;

[0014] Each client calculates the parameter update value of the local clustering model according to the global model parameters, clips the parameter update value according to the clipping threshold, and adds Gaussian noise to obtain the parameter update value of the local clustering model after clipping and adding Gaussian noise;

[0015] Based on the federal security aggregation protocol, each client encrypts the parameter update value of the local clustering model after pruning and adding Gaussian noise according to the corresponding negotiated random number, obtains the parameter update value encrypted by the random number, and sends it to the central server;

[0016] The central server aggregates the parameter update values ​​of the local clustering model of each client after random number encryption, and updates the global model;

[0017] Repeat the above steps for several rounds of federated learning, update the global model, and obtain the final trained global model.

[0018] Secondly, a federated deep clustering method based on differential privacy is provided, including:

[0019] Customize the corresponding local clustering model for each client;

[0020] The central server trains the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client;

[0021] Each client uses the corresponding trained local clustering model to predict the local data set and obtain the corresponding clustering allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients.

[0022] The central server uses the Lagrange interpolation method to obtain the global clustering result based on the trained global model and the secret shared fragments generated by each client. Each client reassigns clustering labels to the local data set according to the global clustering result.

[0023] Furthermore, the central server trains the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client, including:

[0024] The central server pre-trains the global model based on the public data set to obtain model parameters;

[0025] Each client negotiates a random number between two clients according to the federal security aggregation protocol to complete the negotiation of random numbers;

[0026] Each client uses the LoRA method to reparameterize the parameters of the convolutional layer in the local clustering model to obtain the initial global model parameters;

[0027] Each client initializes the local clustering model according to the global model parameters sent by the central server, and trains the local clustering model according to the local data set;

[0028] Each client calculates the parameter update value of the local clustering model according to the global model parameters, clips the parameter update value according to the clipping threshold, and adds Gaussian noise to obtain the parameter update value of the local clustering model after clipping and adding Gaussian noise;

[0029] Based on the federal security aggregation protocol, each client encrypts the parameter update value of the local clustering model after pruning and adding Gaussian noise according to the corresponding negotiated random number, obtains the parameter update value encrypted by the random number, and sends it to the central server;

[0030] The central server aggregates the parameter update values ​​of the local clustering model of each client after random number encryption, updates the global model, and each client assigns a value to the corresponding local clustering model based on the updated global model to obtain the updated local clustering model of each client;

[0031] Repeat the above process of training the local clustering model, perform several rounds of federated learning, update the global model and the local clustering model of each client, and obtain the final trained global model and the local clustering model of each client.

[0032] Furthermore, each client uses the corresponding trained local clustering model to predict the local data set to obtain the corresponding clustering allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients, including:

[0033] Each client uses the corresponding trained local clustering model to predict the local data set and obtain the corresponding clustering allocation result;

[0034] Each client calculates the corresponding local cluster center according to the corresponding cluster allocation result;

[0035] The central server and all clients determine the parameters used to generate the secret sharing fragments;

[0036] Each client divides the corresponding local cluster center into l segments and samples s noise vectors;

[0037] Each client performs Lagrangian encoding on the local cluster center based on the determined parameters, calculates the secret sharing fragment of the local cluster center, and sends it to the rest of the clients.

[0038] Furthermore, the central server uses Lagrange interpolation method to obtain a global clustering result based on the trained global model and the secret sharing fragments generated by each client, and each client reassigns clustering labels to the local data set according to the global clustering result, including:

[0039] The central server randomly generates clustering results for the local clustering centers of all clients;

[0040] Each client calculates the global cluster center based on the secret shared local cluster center according to the corresponding clustering result;

[0041] Each client calculates the distance from the secret sharing fragment in the corresponding local cluster center to the global cluster center, and sends the calculated distance to the central server;

[0042] The central server uses Lagrange interpolation method to decode the calculated distance and restore the real distance;

[0043] The central server regenerates the clustering results based on the real distance and sends them to the corresponding client;

[0044] Repeat the above steps until the iteration converges to obtain the global clustering result;

[0045] Each client reassigns clustering labels to the local dataset based on the global clustering results.

[0046] Thirdly, a federated deep clustering system based on differential privacy is provided, including:

[0047] The model training module is used for the central server to train the global model based on the public data set and the local clustering model of each client to obtain a trained global model;

[0048] The prediction module is used for each client to use the trained global model to predict the local data set and obtain the corresponding clustering allocation results.

[0049] Fourthly, a federated deep clustering system based on differential privacy is provided, including:

[0050] Model customization module, used to customize the corresponding local clustering model for each client;

[0051] A model training module is used for the central server to train the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client;

[0052] The cluster allocation determination module is used for each client to use the corresponding trained local clustering model to predict the local data set and obtain the corresponding cluster allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients.

[0053] The global clustering determination module is used for the central server to adopt the Lagrange interpolation method to obtain the global clustering result based on the trained global model and the secret sharing fragments generated by each client. Each client reassigns clustering labels to the local data set according to the global clustering result.

[0054] In a fifth aspect, a processing device is provided, comprising computer program instructions, wherein the computer program instructions, when executed by the processing device, are used to implement the steps corresponding to the above-mentioned federated deep clustering method based on differential privacy.

[0055] In a sixth aspect, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, are used to implement the steps corresponding to the above-mentioned federated deep clustering method based on differential privacy.

[0056] The present invention adopts the above technical solution, which has the following advantages:

[0057] 1. In the present invention, the local clustering model of each client adopts a contrastive clustering model based on contrastive learning to better mine the potential features of the data and improve the clustering effect.

[0058] 2. The present invention adopts a federated security aggregation protocol, which can provide centralized user-level differential privacy protection for the client without relying on a trusted server. Compared with directly perturbing the client-uploaded parameters using a localized differential privacy method, this method of the present invention adds less noise and has less impact on model availability.

[0059] 3. The present invention proposes a set of solutions to improve model availability, which alleviate the adverse effects of differential privacy noise on model clustering performance, including: using pre-training and applying the low-rank adaptation method (LoRA) during fine-tuning to reduce the number of parameters during fine-tuning; optimizing the original comparative clustering loss function, introducing the KL divergence loss term, and improving model performance; proposing a noisy robust model training method to enhance the model's robustness to noise. The present invention can better balance privacy and model availability.

[0060] 4. Data heterogeneity brings certain challenges to the privacy-preserving federated clustering task: data heterogeneity will lead to slow convergence and poor performance of the federated clustering deep model; data heterogeneity will cause the client's local clustering model update to have a large second norm, aggravating the difficulty of balancing privacy and availability. To solve the above problems, in the data heterogeneity scenario, the present invention further designs a federated deep clustering method based on differential privacy to achieve more targeted model training, effectively alleviating the adverse effects of heterogeneity and improving model clustering performance.

[0061] In summary, the present invention can be widely used in the fields of data analysis and network security. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings:

[0063] Figure 1Schematic diagram of the basic framework of the DP-FLDC method provided by one embodiment of the present invention;

[0064] Figure 2 Schematic diagram of the basic framework of the DP-PFLDC method provided by one embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of global clustering inconsistency caused by a personalized clustering level comparison head provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0067] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0068] Although the terms first, second, third, etc. can be used in the text to describe multiple elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms can only be used to distinguish an element, component, region, layer or section from another region, layer or section. Unless the context clearly indicates, terms such as "first", "second" and other numerical terms do not imply order or sequence when used in the text. Therefore, the first element, component, region, layer or section discussed below can be referred to as the second element, component, region, layer or section without departing from the teaching of the example embodiments.

[0069] There are two main shortcomings in the current federated clustering work: 1) Most federated clustering studies focus on simply extending traditional federated clustering methods from centralized scenarios to the distributed framework of federated learning. Due to the natural limitations of traditional federated clustering methods, it is difficult to handle large-scale, high-dimensional and complex data sets, and the clustering accuracy is not high. 2) Most federated clustering methods do not fully consider privacy protection issues and are vulnerable to various privacy attacks, resulting in the leakage of user sensitive data. Even if there are some methods that provide privacy protection, their clustering performance is too poor to be applied in actual production. Take the following scenario as an example: a multinational group has branches in many locations, and each branch has a local data set containing local customer information. The group headquarters hopes to divide customers into different groups through clustering analysis for precision marketing. However, since data sharing is not allowed, clustering analysis performed only locally is ineffective due to the small amount of data and insufficient diversity. Therefore, a method is needed to promote cooperation between branches and improve clustering accuracy by using global knowledge while protecting privacy. The embodiment of the present invention provides a federated deep clustering method based on differential privacy, including: the central server trains the global model based on the public data set and the local clustering model encrypted by the federal security aggregation protocol of each client to obtain a trained global model; each client uses the trained global model to predict the local data set to obtain the corresponding clustering allocation result. The embodiment of the present invention also provides a federated deep clustering method based on differential privacy, including: customizing the corresponding local clustering model for each client; the central server trains the global model based on the public data set and the local clustering model encrypted by the federal security aggregation protocol of each client to obtain the trained local clustering model of each client; each client uses the trained local clustering model to predict the local data set to obtain the corresponding clustering allocation result, and each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients; the central server uses the Lagrangian interpolation method to randomly generate the clustering result to be clustered based on the local cluster center of each client and the secret sharing fragment generated by each client to obtain the global clustering result, and each client re-assigns the clustering label to the local data set according to the global clustering result. The present invention can be applied to the Internet of Things and edge computing environments to perform clustering tasks of distributed high-dimensional complex data while protecting data privacy.

[0070] Example 1

[0071] like Figure 1 As shown, this embodiment provides a federated deep clustering method based on differential privacy, namely, the DP-FLDC method, comprising the following steps:

[0072] 1) Training phase: The central server trains the global model, i.e., the deep clustering model, based on the public data set and each client's local clustering model to obtain a trained global model, specifically:

[0073] 1.1) Let t = 0. The central server pre-trains the global model based on the public data set to obtain the model parameters θ 0 .

[0074] 1.2) Each client U i According to the federal security aggregation protocol, the random number s is negotiated between the two parties. i,j , i and j are both clients, completing the negotiation of random numbers.

[0075] 1.3) Each client U i The LoRA method is used to reparameterize the parameters of the convolutional layer in the local clustering model to obtain the initial global model parameters. (Shared by With private cluster header part composition) to reduce the amount of trainable parameters.

[0076] Specifically, each client U i The local clustering models of all adopt the contrastive clustering model θ based on contrastive learning to better mine the potential features of the data and improve the clustering effect. The contrastive clustering model θ based on contrastive learning includes a data enhancement module, a feature extraction network f(·), an instance-level contrast head g I (·) and cluster-level comparison head g C (·):

[0077] The data augmentation module is used to randomly select N samples from the local dataset As input examples, and the data enhancement methods T a 、T b Acting on the input instance, we get a batch of positive sample pairs (the same instance x i Different enhancements is considered as a positive sample pair).

[0078] The feature extraction network f(·) is used to extract features from a batch of positive sample pairs, and the input instance is obtained through T a 、T b Enhanced feature matrix

[0079] Instance-level comparison head I (·) is used to transform the enhanced feature matrix h a 、h b Mapped to a subspace, we get the feature matrix h a 、h bAfter g I (·) The feature matrix z after projection a 、z b , where the feature matrix z a 、z b The same row eigenvectors of constitutes a positive sample pair, and the rest of the sample pairs are negative sample pairs, so each input instance can form a positive sample pair and 2N-2 negative sample pairs.

[0080] Instance-level comparison head I (·) is also used to perform instance-level contrastive learning using the InfoNCE contrastive loss function, which shortens the distance between positive sample pairs and increases the distance between negative sample pairs. The formula is as follows: k 1 ,k 2 ∈{a,b}, the cosine distance is used to measure the sample similarity:

[0081]

[0082]

[0083]

[0084] in, is the feature vector The similarity between They are the feature matrices z a 、z b The eigenvector of the i-th row in k 1 ,k 2 ∈{a,b};τ I is the temperature coefficient; They are; They are the feature matrices z a 、z b The feature vector of the jth row; N is the number of input instances; L ins is the instance-level contrast loss; is the feature matrix z a The eigenvector of the i-th row The contrast loss of is the feature matrix z b The eigenvector of the i-th row The contrast loss.

[0085] Cluster level comparison head C (·) is used to transform the enhanced feature matrix h a 、h b Projected to another subspace, we get the feature matrix ha 、h b After g C (·) Feature matrix after projection Among them, C is the number of clusters, and the feature matrix y a ,y b Each row of is the feature representation of the instance, which can be regarded as the soft label of the instance. Its kth dimension represents the probability that the instance belongs to the kth cluster. The feature matrix y a ,y b Each column of represents the vector of probabilities of this batch of instances being assigned to a certain cluster. This probability vector can be regarded as the representation of this cluster. a ,y b The same column eigenvectors of constitute a positive sample pair, and the rest of the sample pairs are negative sample pairs, so each cluster can form a positive sample pair and 2C-2 negative sample pairs.

[0086] Cluster level comparison head C (·) is also used to use the InfoNCE contrast loss function to perform cluster-level contrastive learning at the cluster level, shortening the distance between positive sample pairs and increasing the distance between negative sample pairs. The loss function formula is as follows:

[0087]

[0088]

[0089]

[0090] in, is the feature vector The similarity between They are the feature matrices y a ,y b A column of eigenvectors in k 1 ,k 2 ∈{a,b};τ C is the temperature coefficient; L clu is the cluster-level contrast loss; is the eigenvector of column i The contrast loss of is the eigenvector of column i The contrast loss of ; H(Y) is the entropy of instance clustering assignment probability under two data augmentations, and Column vector Each dimension in represents the probability that an instance belongs to a certain cluster. The feature matrices are The sum of the probabilities of each dimension in . Adding this term helps avoid assigning most instances to the same cluster.

[0091] In summary, the overall loss function L consists of two parts: instance-level contrast loss and cluster-level contrast loss:

[0092] L=L ins 10L clu (7)

[0093] The ultimate goal of instance-level contrastive learning and cluster-level contrastive learning is to obtain a deep clustering model that can make the overall loss function in the above formula (7) very small.

[0094] Specifically, the LoRA method is essentially a reparameterization technique: when fine-tuning the pre-trained model, for each dense matrix Add a low-rank correction term L i R i , then freeze the dense matrix Turn to training These two parameters are used to fine-tune the dense matrix The purpose is to:

[0095]

[0096] Among them, W i For dense matrices The parameters obtained after reparameterization operation; L i and R i is the introduced low-rank matrix; a, b, r are the dimensions of the matrix, and r is much smaller than a and b.

[0097] 1.4) Each client U i All are based on the global model parameters θ sent by the central server t-1 Initialize the local clustering model According to the local data set, the local clustering model is constructed using the following formula (9): Conduct training.

[0098] Specifically, the present invention optimizes the overall loss function L and introduces a KL divergence loss L KL(P||Q) , in order to improve the prediction confidence of the local clustering model, the optimized overall loss function L is:

[0099] L=L ins +L clu +L KL (P||Q) (9)

[0100]

[0101] Among them, Q is the cluster level contrast head g C (·) predicted sample cluster soft label; P is the auxiliary target distribution constructed by sharpening according to formula (10); p ij is the sample point z obtained after sharpening operation i The probability of being assigned to cluster j; q ij is the sample point z i The probability of being assigned to cluster j; q ij′ The sample point z predicted by the model i The probability of being assigned to cluster j′; KL(·||·) is the KL divergence of the two distributions. The purpose of adding this item is to make the cluster soft label Q closer to the target distribution P. P is a distribution that tends to be one-hot encoded. Its role here is similar to the pseudo label in self-supervised learning to improve the confidence of model prediction.

[0102] More specifically, in order to enhance the robustness of the local clustering model to noise, the present invention proposes a noisy robust model training method, which performs noisy robust model training on the optimized overall loss function (9), and rewrites the original loss function into the average of multiple noisy model losses, where L is the above-mentioned optimized overall loss function:

[0103]

[0104] in, is the parameter of the local clustering model of the i-th client in the t-th round; E represents the expectation of the loss function value, Δ~N(0,σ r 2 ) indicates that Δ obeys N(0,σ r 2 ) Normal distribution, Δ j is the jth Gaussian noise sampled; σ r is the robust training noise standard deviation; K is the number of sampled noises. Formula (11) is intended to show that the loss value of the optimal model parameters sought by the present invention should be in a relatively smooth space. In other words, in this space, the loss function value of the model is small and remains stable in the vicinity of the noise disturbance.

[0105] 1.5) Each client is based on the global model parameter θ t-1 , calculate the parameter update value of the local clustering model Update the parameter value according to the clipping threshold S Crop and add Gaussian noise N(0,S 2 σ 2 / m), and get the parameter update value of the local clustering model after pruning and adding Gaussian noise Where m is the number of clients and σ is the differential privacy noise scale.

[0106] Specifically, the parameter update values ​​of the local clustering model are: The parameter update values ​​of the local clustering model after pruning and adding Gaussian noise are:

[0107] 1.6) Each client updates the parameters of the local clustering model after pruning and adding Gaussian noise based on the corresponding negotiated random numbers based on the federated security aggregation protocol Encrypt and get That is, the parameter update value encrypted by the random number And sent to the central server.

[0108] Specifically, the client negotiates a random number s between each other. i,j , the disturbance parameter update value get

[0109]

[0110] 1.7) The central server aggregates the parameter update values ​​of the local clustering model of each client after random number encryption Update the global model θ t =θ t-1 +η g Δθ t , where η g is the global learning rate.

[0111] 1.8) Let t = t + 1, go to step 1.1), perform t rounds of federated learning, update the global model, and obtain the final trained global model.

[0112] 2) Inference phase: Each client uses the trained global model to predict the local data set and obtain the corresponding clustering assignment results.

[0113] Example 2

[0114] When the client data participating in federated learning is heterogeneous, in order to overcome the challenges brought by data heterogeneity, this embodiment further designs a DP-PFLDC method based on the DP-FLDC method in Example 1 to achieve more targeted model training, alleviate the problems of client drift and excessive local model updates, improve clustering performance, and achieve global consistency of clustering.

[0115] like Figure 2 As shown, this embodiment provides a federated deep clustering method based on differential privacy, namely, the DP-PFLDC method, comprising the following steps:

[0116] 1) Training phase: A model personalization method based on parameter decoupling is adopted to customize a corresponding local clustering model for each client. The central server trains the global model, i.e., the deep clustering model, and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain a trained global model and a local clustering model of each client. The training phase of this embodiment differs from the training phase of Embodiment 1 in two aspects: first, a model personalization method based on parameter decoupling is adopted to customize a local clustering model for each client; second, a loss function with a global knowledge correction term is adopted, specifically:

[0117] 1.1) A model personalization method based on parameter decoupling is used to customize the corresponding local clustering model for each client.

[0118] Specifically, in this embodiment, the local clustering model of each client is divided into two parts: a shared part and a private part. That is, the local clustering model of each client includes a shared data enhancement module, a feature extraction network f(·) and an instance-level comparison head g I (·) and each client’s private cluster-level comparison header g C (·), where the client-private cluster-level comparison header g C (·) will always remain locally on the client and will not participate in federated aggregation. The reason for this is that due to the different data distributions on the clients, they are prone to disagreement regarding the final cluster label predictions, so the client needs a personalized cluster-level comparison header g C (·), but they are easier to reach a consensus on feature representation, so we can use everyone’s knowledge to jointly learn a globally shared feature extraction network.

[0119] The shared data augmentation module is used to randomly select N samples from the local dataset. As input examples, and the data enhancement methods T a 、T b Act on the input instance to obtain a batch of positive sample pairs.

[0120] The shared feature extraction network f(·) is used to extract features from the enhanced samples, and the input instance is obtained through T a 、T b Enhanced feature matrix

[0121] Instance-level comparison header for sharing I (·) is used to transform the enhanced feature matrix h a 、h b Mapped to a subspace, we get the feature matrix h a 、hb After g I (·) The feature matrix z after projection a 、z b , where the feature matrix z a 、z b The same row eigenvectors of constitutes a positive sample pair, and the rest of the sample pairs are negative sample pairs, so each input instance can form a positive sample pair and 2N-2 negative sample pairs.

[0122] Instance-level comparison header for sharing I (·) is also used to perform instance-level contrastive learning using the InfoNCE contrastive loss function, which brings the distance between positive sample pairs closer and the distance between negative sample pairs farther away.

[0123] Each client's private cluster-level comparison header g C (·) is used to transform the enhanced feature matrix h a 、h b Projected to another subspace, we get the feature matrix h a 、h b After g C (·) Feature matrix after projection Among them, C is the number of clusters, and the feature matrix y a ,y b Each row of is the feature representation of the instance, which can be regarded as the soft label of the instance. Its kth dimension represents the probability that the instance belongs to the kth cluster. The feature matrix y a ,y b Each column of represents the vector of probabilities of this batch of instances being assigned to a certain cluster. This probability vector can be regarded as the representation of this cluster. a ,y b The same column eigenvectors of constitute a positive sample pair, and the rest of the sample pairs are negative sample pairs, so each cluster can form a positive sample pair and 2C-2 negative sample pairs.

[0124] Each client's private cluster-level comparison header g C (·) is also used to perform cluster-level contrastive learning at the cluster level using the InfoNCE contrastive loss function, which shortens the distance between positive sample pairs and increases the distance between negative sample pairs.

[0125] Specifically, before each round of learning begins, the client uses the feature extraction network and instance-level comparison head of the global model as the feature extraction network f(·) and instance-level comparison head g of the local clustering model. I(·) assignment, that is, before each round of learning begins, the local clustering models of all clients have the same feature extraction network f(·) and instance-level comparison head g I (·). The cluster-level comparison head g C (·) It is always updated only through client-side local training and does not participate in federated aggregation, so it is independent and personalized.

[0126] 1.2) Let t = 0. The central server pre-trains the global model based on the public data set and obtains the model parameters θ 0 .

[0127] 1.3) Each client U i According to the federal security aggregation protocol, the random number s is negotiated between the two parties. i,j , i and j are both clients, completing the negotiation of random numbers.

[0128] 1.4) Each client U i The LoRA method is used to reparameterize the parameters of the convolutional layer in the local clustering model to obtain the initial global model parameters. (Shared by With private cluster header part composition) to reduce the amount of trainable parameters.

[0129] 1.5) Each client U i All are based on the global model parameters θ sent by the central server t-1 Initialize the local clustering model According to the local data set, the following loss function (13) with global knowledge correction term is used to train the local clustering model Training is performed. This embodiment limits the impact of updating the local clustering model by adding a global knowledge correction term to the loss function of the above formula (9), so as to prevent the local clustering model trained by the client from deviating too much from the global model and causing client drift. The loss function with the global knowledge correction term is:

[0130]

[0131] in, is the local clustering model parameter of the i-th client in the t-th round; is the sample feature representation extracted by the global model in round t-1; is the sample feature representation extracted by the local clustering model of the i-th client in the t-th round. These sample feature representations are obtained through the sample feature extraction network f(·) and the instance-level contrast head g I (·) Feature matrix obtained by projecting to the subspace; ‖·‖ 2is the L2 norm of the matrix, that is, the Euclidean distance between feature representations; α is the weight hyperparameter of the global knowledge correction term; B is the batch size.

[0132] By introducing a global knowledge correction term, we can measure the difference between the local model (i.e., the local clustering model) and the global model, punish the local models of the client that deviate too much from the global model, and adjust their update direction. This can prevent the local model from losing global knowledge and falling into the local optimum, and alleviate the client drift problem caused by data heterogeneity.

[0133] Since the clustering model lacks label guidance during training, the personalized cluster-level comparison head used in the DP-PFLDC method mentioned above will cause samples of the same category to generate different cluster labels on the cluster-level comparison heads of different clients. This makes the clustering results limited to each client, and it is impossible to obtain clustering results from a global perspective, which hinders global data analysis. Figure 3 As shown, client 1 classifies "cat" into cluster one, while client 2 classifies "cat" into cluster zero.

[0134] 1.6) Each client is based on the global model parameter θ t-1 , calculate the parameter update value of the local clustering model Update the parameter value according to the clipping threshold S Crop and add Gaussian noise N(0,S 2 σ 2 / m), and get the parameter update value of the local clustering model after pruning and adding Gaussian noise Where m is the number of clients and σ is the differential privacy noise scale.

[0135] 1.7) Each client updates the parameters of the local clustering model after pruning and adding Gaussian noise based on the corresponding negotiated random numbers based on the federated security aggregation protocol Encrypt and get That is, the parameter update value encrypted by the random number And sent to the central server.

[0136] 1.8) The central server aggregates the parameter update values ​​of each client's encrypted local clustering model Update the global model θ t =θ t-1 +η g Δ9 t , where η g is the global learning rate, and each client is based on the updated global model θ t is the feature extraction network f(·) and instance-level comparison head g of the corresponding local clustering model I(·) assignment, and obtain the updated local clustering model of each client T is the total number of federated learning training rounds.

[0137] 1.9) Let t = t + 1, go to step 1.5), perform t rounds of federated learning, update the global model and the local clustering model of each client, and obtain the final trained global model and the local clustering model of each client.

[0138] Since the clustering model lacks label guidance during training, the personalized cluster-level comparison head used in the method of Example 1 will cause samples of the same category to generate different cluster labels on the cluster-level comparison heads of different clients, which makes the clustering results limited to each client and cannot obtain clustering results from a global perspective, hindering global data analysis. Figure 3 As shown, client 1 classifies "cat" into cluster 1, while client 2 classifies "cat" into cluster 0. To align with other existing federated clustering studies, this embodiment is updated based on the reasoning phase of the method in embodiment 1, and proposes a privacy-preserving cluster label redistribution method, which will reassign cluster labels to client data from a global perspective after model training is completed, including:

[0139] 2) Reasoning Phase 1: Each client uses the corresponding trained local clustering model to predict the local data set and obtain the corresponding clustering assignment result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients. Specifically:

[0140] 2.1) Each client U i The corresponding trained local clustering model is used to classify the local dataset D i Make predictions and get the corresponding clustering assignment results R i .

[0141] 2.2) Each client U i According to the corresponding clustering distribution results R i , calculate the corresponding local cluster center X i ={x i,1 ,…,x i,C}, where x i,1 ,…,x i,C For client U i The 1st to Cth local cluster centers of , where C is the number of clusters.

[0142] 2.3) The central server and all clients determine two sets of parameters {β 1 ,…,β l+w}, and Where N is the total number of clients; l and w are hyperparameters, l represents the number of segments when segmenting the data in the next step; w is the number of sampled noises, which must satisfy 2l+2w-1≤N; is a finite field of order q; {β 1 ,…,β l+w} and {α 1 ,…,α N} are two sets of parameters belonging to the q-order finite field).

[0143] 2.4) Each client U i The corresponding local cluster center x i,j Divide into l segments x (i,j),1 ,…,x (i,j),l , and sample w noise vectors z j,l+1 ,…,z j,l+w .

[0144] 2.5) Each client U i Based on the determined parameters, the local cluster center {x i,1 ,…,x i,C} Perform Lagrangian coding and calculate the local cluster center x by the following formula (14): i,j Secret sharing fragment And calculate the secret shared fragment Send to other clients U i′ ,(i′∈[N]):

[0145]

[0146] Among them, x (i,j),r is the local cluster center x i,j The rth segment after segmentation; α i′ is the parameter set {α 1 ,…,α N} the i′th parameter; β v is the parameter set {β 1 ,…,β l+w} the vth parameter in β r is the parameter set {β 1 ,…,β l+w}.

[0147] 3) Reasoning Phase II: The central server uses the Lagrange interpolation method to obtain the global clustering result based on the trained global model and the secret shared fragments generated by each client. Each client reassigns clustering labels to the local data set based on the global clustering result. Specifically:

[0148] 3.1) The central server is for all clients U i The local cluster center Xi Randomly generate clustering results {S 1 ,…,S C} and sent to the corresponding client U i .

[0149] 3.2) Each client U i According to the corresponding clustering results S h ,h∈[C], in the local cluster center X of the secret sharing i Calculate the global cluster center

[0150] 3.3) Each client U i The secret sharing fragments in the corresponding local cluster centers are calculated Go to the hth global cluster center calculated by the i-th client Distance And the calculated distance Sent to the central server.

[0151] 3.4) The central server uses the Lagrange interpolation method to calculate the distance Decode and recover the real distance d (j,k),h ,(j∈[N],k∈[C],h∈[C]).

[0152] 3.5) The central server calculates the actual distance d (j,k),h Regenerate clustering results 1 ,…,S C} and sent to the corresponding client U i .

[0153] 3.6) Repeat steps 3.2) to 3.5) until the iterations converge, that is, until the similarity of multiple clustering results is high, and obtain the global clustering result {S 1 ,…,S C}.

[0154] 3.7) Each client U i According to the global clustering results {S 1 ,…,S C}, re-create the local dataset D i Assign cluster labels.

[0155] Specifically, each client U i The corresponding clustering assignment results R i is divided into x i,j The cluster label of the data point that is the local cluster center is changed to h.

[0156] Specifically, due to the lack of label guidance during clustering model training, the personalized cluster-level comparison head used in the DP-PFLDC algorithm will cause samples of the same category to generate different cluster labels on the cluster-level comparison heads of different clients, which makes the clustering results limited to each client and cannot obtain clustering results from a global perspective, hindering global data analysis. Figure 3 As shown, client 1 classifies "cat" into cluster one, while client 2 classifies "cat" into cluster zero. To align with other existing federated clustering studies, the present invention proposes a privacy-preserving cluster label redistribution algorithm, which will reallocate cluster labels to client data from a global perspective after model training. The label redistribution algorithm here is mainly based on SecFC, which adopts a secure multi-party computing method and uses Lagrangian coding for secret sharing. It can safely perform k-means clustering on the client's local cluster center to obtain a clustering result with a global perspective. The client can reallocate globally unified cluster labels to local data based on the result.

[0157] The methods of Embodiment 1 and Embodiment 2 of the present invention are both general methods under the framework of federated learning and are applicable to multiple fields, including business, finance, medical treatment, social media, etc. For example, they can be applied to:

[0158] Business field: Various branches of a chain store can collaboratively use federated clustering to train clustering models to help divide customer groups and achieve precision marketing.

[0159] Medical field: Multiple medical institutions can jointly train a clustering model for diagnosing fundus images, automatically classify different types of fundus lesion images into different clusters, and assist doctors in making decisions.

[0160] Mobile phone application areas: Mobile phone manufacturers can use federated clustering to train smart album models using users’ local album data to achieve the function of automatically classifying pictures.

[0161] The specific data type depends on the field and target task, such as customer transaction data, fundus image data, or user album pictures.

[0162] The technical effects of the federated deep clustering method based on differential privacy of the present invention are described in detail below through specific embodiments:

[0163] 1. Experimental setup:

[0164] Datasets: CIFAR-10, CIFAR-100.

[0165] Data set partitioning: uniform random allocation in data homogeneous scenarios; randomly select M categories of data for each client in data heterogeneous scenarios.

[0166] Clustering performance indicators: normalized mutual information (NMI), adjusted Rand index (ARI) and clustering accuracy (ACC).

[0167] Hyperparameter setting: The privacy budget (ε, δ) of differential privacy is set to (8, 1e-4).

[0168] 2. Clustering performance evaluation:

[0169] Tables 1 and 2 below show the comparison of clustering performance and privacy protection capability of the existing federated clustering methods K-Fed, FFCM, DP-FedC, CCFC, CCFC++ and the DP-FLDC method of the present invention on the CIFAR10 and CIFAR100 datasets in data homogeneity scenarios:

[0170] Table 1: Clustering performance comparison of federated clustering methods on CIFAR10 in data homogeneity scenario

[0171]

[0172] Table 2: Clustering performance comparison of federated clustering methods on CIFAR100 in data homogeneity scenario

[0173]

[0174] From the perspective of privacy, only DP-FedC and the method of the present invention have the privacy protection capability. From the perspective of clustering performance, the method of the present invention is close to the most advanced CCFC++ algorithm. Therefore, it can be shown that the method of the present invention not only protects privacy, but also has good clustering performance.

[0175] Tables 3 and 4 below show the comparison of clustering performance and privacy protection capabilities of the existing federated clustering methods K-Fed, FFCM, DP-FedC, CCFC, CCFC++ and the DP-PFLDC method of the present invention on the CIFAR10 and CIFAR100 datasets in data heterogeneity scenarios. On CIFAR-10 and CIFAR-100, the number of data categories M owned by the client is set to 4 and 8 respectively:

[0176] Table 3: Clustering performance comparison of federated clustering methods on CIFAR10 in data heterogeneity scenarios

[0177]

[0178] Table 4: Clustering performance comparison of federated clustering methods on CIFAR100 in heterogeneous data scenarios

[0179]

[0180] From the experimental results, the method of the present invention performs better in dealing with data heterogeneity. It not only has the ability to protect privacy, but also has better clustering performance.

[0181] 3. Ablation experiment:

[0182] In order to verify the contribution of the three model usability improvement methods proposed above in balancing privacy and usability, ablation experiments were conducted on the CIFAR10 dataset, as shown in Table 5 below. In Table 5, DP-FLDC without the usability improvement method is called NavieDP-FLDC, and different usability improvement methods are gradually added to NavieDP-FLDC, including LoRA-based parameter efficient fine-tuning (LoRA Finetune), loss function optimization strategy (KL Loss) and noisy robust model training (Noisy Model Training). These methods have different angles and are compatible with each other and can be applied to one method at the same time. The experimental results show that the three usability improvement methods proposed in this invention are all helpful in improving the clustering effect of the model and help the model achieve a better balance between privacy and usability:

[0183] Table 5 Ablation analysis of usability improvement methods on CIFAR10<

[0184]

[0185] In order to verify the contribution of the personalized method proposed above in dealing with the challenges brought by data heterogeneity, an ablation experiment was conducted on the CIFAR-10 dataset. The results are shown in Table 6 below. The DP-PFLDC without any personalized measures is called NavieDP-PFLDC. Personalized federated learning methods are gradually applied to NavieDP-PFLDC. These personalized methods include providing a personalized clustering head (PersonalClusterHead) for each client and adding a global knowledge correction term (CorrectionTerm) to the loss function to control the degree of deviation between the local model and the global model. The experimental results show that both methods play a positive role in overcoming data heterogeneity and improving clustering performance:

[0186] Table 6: Ablation analysis of personalized methods on CIFAR10

[0187]

[0188] Example 3

[0189] This embodiment provides a federated deep clustering system based on differential privacy, including:

[0190] The model training module is used by the central server to train the global model based on the public data set and the local clustering model of each client to obtain a trained global model.

[0191] The prediction module is used for each client to use the trained global model to predict the local data set and obtain the corresponding clustering allocation results.

[0192] The system provided in this embodiment is used to execute the method of the above-mentioned embodiment 1. Please refer to the above-mentioned embodiment for the specific process and detailed contents, which will not be repeated here.

[0193] Example 4

[0194] This embodiment provides a federated deep clustering system based on differential privacy, including:

[0195] The model customization module is used to customize the corresponding local clustering model for each client.

[0196] The model training module is used for the central server to train the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client.

[0197] The cluster allocation determination module is used for each client to use the corresponding trained local clustering model to predict the local data set and obtain the corresponding cluster allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients.

[0198] The global clustering determination module is used for the central server to adopt the Lagrange interpolation method to obtain the global clustering result based on the trained global model and the secret sharing fragments generated by each client. Each client reassigns clustering labels to the local data set according to the global clustering result.

[0199] The system provided in this embodiment is used to execute the method of the above-mentioned embodiment 2. Please refer to the above-mentioned embodiment for the specific process and detailed contents, which will not be repeated here.

[0200] Example 5

[0201] This embodiment provides a processing device corresponding to the federated deep clustering method based on differential privacy provided in Embodiment 1 or 2. The processing device can be applicable to a client processing device, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of Embodiment 1 or 2.

[0202] The processing device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The memory stores a computer program that can be run on the processing device, and when the processing device runs the computer program, the federated deep clustering method based on differential privacy provided in this embodiment 1 or 2 is executed.

[0203] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0204] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0205] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0206] Those skilled in the art will understand that the structure of the above-mentioned computing device is only a partial structure related to the solution of the present invention, and does not constitute a limitation on the computing device to which the solution of the present invention is applied. The specific computing device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0207] Example 6

[0208] This embodiment provides a computer program product corresponding to the federated deep clustering method based on differential privacy provided in Embodiment 1 or Embodiment 2. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing the federated deep clustering method based on differential privacy described in Embodiment 1 or Embodiment 2.

[0209] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0210] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0211] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0214] The above embodiments are only used to illustrate the present invention, wherein the structure, connection mode and manufacturing process of each component may be changed. Any equivalent transformations and improvements based on the technical solution of the present invention should not be excluded from the protection scope of the present invention.

Claims

1. A federated deep clustering method based on differential privacy, characterized in that: include: The central server trains the global model based on the public data set and the local clustering model of each client to obtain a trained global model; Each client uses the trained global model to predict the local data set and obtain the corresponding clustering assignment results.

2. A federated deep clustering method based on differential privacy as claimed in claim 1, characterized in that: The central server trains the global model based on the public data set and the local clustering model of each client to obtain a trained global model, including: The central server pre-trains the global model based on the public data set to obtain model parameters; Each client negotiates a random number between two clients according to the federal security aggregation protocol to complete the negotiation of random numbers; Each client uses the LoRA method to reparameterize the parameters of the convolutional layer in the local clustering model to obtain the initial global model parameters; Each client initializes the local clustering model according to the global model parameters sent by the central server, and trains the local clustering model according to the local data set; Each client calculates the parameter update value of the local clustering model according to the global model parameters, clips the parameter update value according to the clipping threshold, and adds Gaussian noise to obtain the parameter update value of the local clustering model after clipping and adding Gaussian noise; Based on the federal security aggregation protocol, each client encrypts the parameter update value of the local clustering model after pruning and adding Gaussian noise according to the corresponding negotiated random number, obtains the parameter update value encrypted by the random number, and sends it to the central server; The central server aggregates the parameter update values ​​of the local clustering model of each client after random number encryption, and updates the global model; Repeat the above steps for several rounds of federated learning, update the global model, and obtain the final trained global model.

3. A federated deep clustering method based on differential privacy, characterized in that: include: Customize the corresponding local clustering model for each client; The central server trains the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client; Each client uses the corresponding trained local clustering model to predict the local data set and obtain the corresponding clustering allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients. The central server uses the Lagrange interpolation method to obtain the global clustering result based on the trained global model and the secret shared fragments generated by each client. Each client reassigns clustering labels to the local data set according to the global clustering result.

4. A federated deep clustering method based on differential privacy as claimed in claim 3, characterized in that: The central server trains the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client, including: The central server pre-trains the global model based on the public data set to obtain model parameters; Each client negotiates a random number between two clients according to the federal security aggregation protocol to complete the negotiation of random numbers; Each client uses the LoRA method to reparameterize the parameters of the convolutional layer in the local clustering model to obtain the initial global model parameters; Each client initializes the local clustering model according to the global model parameters sent by the central server, and trains the local clustering model according to the local data set; Each client calculates the parameter update value of the local clustering model according to the global model parameters, clips the parameter update value according to the clipping threshold, and adds Gaussian noise to obtain the parameter update value of the local clustering model after clipping and adding Gaussian noise; Based on the federal security aggregation protocol, each client encrypts the parameter update value of the local clustering model after pruning and adding Gaussian noise according to the corresponding negotiated random number, obtains the parameter update value encrypted by the random number, and sends it to the central server; The central server aggregates the parameter update values ​​of the local clustering model of each client after random number encryption, updates the global model, and each client assigns a value to the corresponding local clustering model based on the updated global model to obtain the updated local clustering model of each client; Repeat the above process of training the local clustering model, perform several rounds of federated learning, update the global model and the local clustering model of each client, and obtain the final trained global model and the local clustering model of each client.

5. The federated deep clustering method based on differential privacy as claimed in claim 3, characterized in that: Each client uses the corresponding trained local clustering model to predict the local data set and obtain the corresponding clustering allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients, including: Each client uses the corresponding trained local clustering model to predict the local data set and obtain the corresponding clustering allocation result; Each client calculates the corresponding local cluster center according to the corresponding cluster allocation result; The central server and all clients determine the parameters used to generate the secret sharing fragments; Each client divides the corresponding local cluster center into l segments and samples s noise vectors; Each client performs Lagrangian encoding on the local cluster center based on the determined parameters, calculates the secret sharing fragment of the local cluster center, and sends it to the rest of the clients.

6. A federated deep clustering method based on differential privacy as claimed in claim 3, characterized in that: The central server uses the Lagrange interpolation method to obtain a global clustering result based on the trained global model and the secret sharing fragments generated by each client. Each client reassigns clustering labels to the local data set according to the global clustering result, including: The central server randomly generates clustering results for the local clustering centers of all clients; Each client calculates the global cluster center based on the secret shared local cluster center according to the corresponding clustering result; Each client calculates the distance from the secret sharing fragment in the corresponding local cluster center to the global cluster center, and sends the calculated distance to the central server; The central server uses Lagrange interpolation method to decode the calculated distance and restore the real distance; The central server regenerates the clustering results based on the real distance and sends them to the corresponding client; Repeat the above steps until the iteration converges to obtain the global clustering result; Each client reassigns clustering labels to the local dataset based on the global clustering results.

7. A federated deep clustering system based on differential privacy, characterized in that: include: The model training module is used for the central server to train the global model based on the public data set and the local clustering model of each client to obtain a trained global model; The prediction module is used for each client to use the trained global model to predict the local data set and obtain the corresponding clustering allocation results.

8. A federated deep clustering system based on differential privacy, characterized in that: include: Model customization module, used to customize the corresponding local clustering model for each client; A model training module is used for the central server to train the global model and the local clustering model of each client based on the public data set and the local clustering model of each client to obtain the trained global model and the local clustering model of each client; The cluster allocation determination module is used for each client to use the corresponding trained local clustering model to predict the local data set and obtain the corresponding cluster allocation result. Each client also performs Lagrangian encoding on the local cluster center to generate a secret sharing fragment to share with other clients. The global clustering determination module is used for the central server to adopt the Lagrange interpolation method to obtain the global clustering result based on the trained global model and the secret sharing fragments generated by each client. Each client reassigns clustering labels to the local data set according to the global clustering result.

9. A processing device, characterized in that: It includes computer program instructions, wherein the computer program instructions, when executed by a processing device, are used to implement the steps corresponding to the federated deep clustering method based on differential privacy as described in any one of claims 1 or 3.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, wherein the computer program instructions, when executed by a processor, are used to implement the steps corresponding to the federated deep clustering method based on differential privacy as described in any one of claims 1 or 3.

Images