Federated contrastive clustering learning method and system for non-independent and identically distributed data

Abstract

The present invention discloses a federated comparative clustering learning method and system for non-independent and identically distributed data, belonging to the field of federated learning technology. The method comprises: sending a shared model and a prototype matrix to selected clients on a central server side for cluster comparative training. During cluster comparative training, the model does not focus on the distribution of the data, but only on the comparative relationship between data cluster information. Based on the characteristic of self-supervised comparative learning that focuses on local features, the dependence on the global distribution is alleviated, thereby improving the offset of the model caused by unbalanced data distribution, and can better represent the class distribution of the data and improve the Non-IID problem caused by class imbalance. At the same time, the dependence of federated learning on labeled data can be eliminated, and the technical problems of low model accuracy caused by the Non-IID data distribution problem and inability to be applied in actual production due to dependence on labeled data in existing federated learning methods can be solved.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning technology, and more specifically, relates to a federated comparative clustering learning method and system for non-independent and identically distributed data. Background Art

[0002] In the current Internet+ era, people generate and store vast amounts of data through edge devices such as smartphones, Internet of Things (IoT), and wearables. This data provides a vast learning data sample for the further development and application of deep learning. Deep learning provides the algorithmic foundation for intelligent applications that serve billions of people every day. However, with the rapid development of deep learning, the architecture of uploading personal data to cloud servers for centralized storage, processing, and model training has become difficult to address growing concerns about personal data privacy and security. Furthermore, the high costs and latency associated with this architecture are unacceptable. Therefore, in 2016, the Google team proposed Federated Learning (FL), a distributed approach that decouples training data from the devices used to train the models. While the original data remains on the user device, the federated model is trained collaboratively between the user device and a central server. The central server only receives and aggregates local computational updates and analysis results to enhance the global model. The new model is then shared with clients for knowledge sharing, ensuring high security and privacy for user data.

[0003] While federated learning approaches have addressed security and privacy concerns to a certain extent, they also present numerous challenges. The first is the non-IID data distribution problem: excessively complex and disordered data distributions prevent conventional federated learning algorithms from effectively extracting globally relevant data features from data that follows a non-IID data distribution. Furthermore, issues such as insufficient data volume, a limited number of users with limited data types, or excessively discrete data distributions between users further exacerbate the non-IID nature of the data. To mitigate the impact of non-IID data on the model, existing federated learning algorithms typically improve the overall data distribution at the data level by sharing user data and features, and augmenting small amounts of user data with data augmentation, rather than addressing the non-IID problem from an algorithmic perspective. Furthermore, conventional federated learning algorithms typically utilize machine learning methods based on labeled data for training, relying on this data. However, in real-world applications, users possess vast amounts of unlabeled data, making federated learning infeasible for practical production applications. Summary of the Invention

[0004] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a federated comparative clustering learning method and system for non-independent and identically distributed data, which is used to solve the technical problems of low model accuracy caused by the existence of Non-IID data distribution problem and inability to be applied in actual production due to reliance on labeled data in existing federated learning methods.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a federated contrastive clustering learning method for non-independent and identically distributed data, comprising:

[0006] Perform the following steps on the central server:

[0007] A11. Initialize the vector representation of K cluster centers to obtain the prototype matrix; initialize the shared model;

[0008] A12. Randomly select m clients from the client set, and send the shared model and prototype matrix to each of the m selected clients for training.

[0009] A13. After collecting the trained shared models and prototype matrices returned by m clients, weight the trained shared models returned by each client to obtain an aggregate model, and weight the trained prototype matrices returned by each client to obtain an aggregate matrix.

[0010] A14. Update the shared model to the aggregate model. After updating the prototype matrix to the aggregate matrix, normalize the prototype matrix and repeat steps A12-A13 until the preset number of iterations is reached. The shared model at this point is the trained model.

[0011] Among them, after the client receives the shared model and prototype matrix sent by the central server, it inputs the local data set in the client into the shared model for clustering comparison training, specifically including: performing the following operations on each local data sample in the local data set in the client:

[0012] B11. Perform two random data enhancements on the local data samples to obtain a first comparison sample and a second comparison sample;

[0013] B12. Input the first comparison sample and the second comparison sample into the received shared model respectively to obtain the first sample feature and the second sample feature;

[0014] B13, mapping and matching the first sample feature and the second sample feature with the vector representation of each cluster center in the received prototype matrix to obtain a first feature encoding vector and a second feature encoding vector;

[0015] B14. Update the parameters in the received shared model and the received prototype matrix by minimizing the cross entropy loss between the first sample feature and the second feature encoding vector and the cross entropy loss between the second sample feature and the first feature encoding vector.

[0016] Further preferably, the loss function L(z s , z t )for:

[0017] L(z s ,z t )=l(z s ,q t )+l(z t ,q s )

[0018]

[0019]

[0020] Among them, z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector.

[0021] Further preferably, each client performs the above cluster comparison training in parallel.

[0022] In a second aspect, the present invention provides a federated contrastive clustering learning method for non-independent and identically distributed data, comprising:

[0023] Perform the following steps on the central server:

[0024] A21. Initialize the vector representations of K cluster centers to obtain the initial representation C0 of the prototype matrix; construct a prototype array of length m to store the prototype matrices returned by the client according to the client number, and initialize them to C0 respectively; initialize the shared model;

[0025] A22. Randomly select m clients from the client set, and send the shared model and prototype matrix to the selected m clients for training. In the process of sending the shared model and prototype matrix to the t-th client, randomly select a prototype matrix from the prototype array other than the prototype matrix corresponding to the t-th client, and record it as matrix C. P , the matrix C P At the same time, it is sent to the t-th client for training; t = 1, 2, ..., m;

[0026] A23. After collecting the trained shared models and prototype matrices returned by m clients, store the prototype matrices returned by each client in the corresponding position of the prototype array, and perform weighted summation of the trained shared models returned by each client using the proportion of the client's data volume in the total data volume as a weight to obtain an aggregate model, and perform weighted summation of the trained prototype matrices returned by each client to obtain an aggregate matrix;

[0027] A24. Update the shared model to the aggregate model. After updating the prototype matrix to the aggregate matrix, normalize the prototype matrix and repeat steps A22-A23 until the preset number of iterations is reached. The shared model at this point is the trained model.

[0028] Among them, when the client receives the shared model, prototype matrix and matrix C sent by the central server P After that, the local data set in the client is input into the shared model for clustering comparison training, which specifically includes: performing the following operations on each local data sample in the local data set in the client:

[0029] B21. Perform two random data enhancements on the local data samples to obtain a first comparison sample and a second comparison sample;

[0030] B22. Input the first comparison sample and the second comparison sample into the received shared model respectively to obtain the first sample feature and the second sample feature;

[0031] B23, mapping and matching the first sample feature and the second sample feature with the vector representation of each cluster center in the received prototype matrix to obtain a first feature encoding vector and a second feature encoding vector;

[0032] The first sample feature and the second sample feature are respectively compared with the received matrix C P The vector representations of the cluster centers in are mapped and matched to obtain the third feature coding vector and the fourth feature coding vector;

[0033] B24. Update the parameters and prototype matrix in the shared model by minimizing the cross entropy loss between the first sample feature and the second feature encoding vector, the cross entropy loss between the second sample feature and the first feature encoding vector, the cross entropy loss between the first sample feature and the fourth feature encoding vector, and the cross entropy loss between the second sample feature and the third feature encoding vector.

[0034] Further preferably, the loss function L(z s , z t )for:

[0035] L(z s ,z t )=l(z s ,q t )+l(z t ,q s )+l′(z s ,q t ')+l′(z t ,q s ')

[0036]

[0037]

[0038]

[0039]

[0040] Among them, z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q s ' is the third feature encoding vector; q t ' is the fourth feature code vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector; q t '(k) is the kth feature code in the fourth feature code vector; d k is the matrix C P The vector representation of the kth cluster center in ; q s '(k) is the kth feature code in the third feature code vector.

[0041] Further preferably, the temperature coefficient is initialized by the central server and then sent to each client.

[0042] Further preferably, the loss function L(z s , z t )for:

[0043] L(z s ,z t )=l(z s ,q t )+l(z t ,q s )+l″(z s ,q t ')+l″(z t ,q s ')

[0044]

[0045]

[0046]

[0047]

[0048] Among them, z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q s ' is the third feature encoding vector; q t ' is the fourth feature code vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector; q t '(k) is the kth feature code in the fourth feature code vector; τ' is the temperature difference coefficient; d k is the matrix C P The vector representation of the kth cluster center in ; q s '(k) is the kth feature code in the third feature code vector.

[0049] Further preferably, the temperature coefficient and temperature difference coefficient are initialized by the central server and then sent to each client.

[0050] Further preferably, the temperature difference coefficient is smaller than the temperature coefficient.

[0051] Further preferably, each client performs the above cluster comparison training in parallel.

[0052] In a third aspect, the present invention provides a federated contrastive clustering learning system for non-independent and identically distributed data, comprising: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it executes the federated contrastive clustering learning method provided in the first aspect of the present invention and / or the federated contrastive clustering learning method provided in the second aspect of the present invention.

[0053] In a fourth aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the federated contrastive clustering learning method provided in the first aspect of the present invention and / or the federated contrastive clustering learning method provided in the second aspect of the present invention.

[0054] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0055] 1. The first aspect of the present invention provides a federated contrastive clustering learning method. A central server sends a shared model and a prototype matrix to each selected client for cluster contrastive training. During cluster contrastive training, the model does not focus on the distribution of the data, but only on the contrast relationship between the data cluster information. Based on the characteristic of self-supervised contrastive learning that focuses on local features, it greatly alleviates the dependence on the global distribution, can effectively improve the model deviation caused by uneven data distribution in the global range, can better represent the class distribution of the data and improve the Non-IID problem caused by class imbalance, and can also eliminate the dependence of federated learning on labeled data. After the central server collects the trained shared model and prototype matrix returned by all clients, in order to ensure that the prototype matrix can effectively reflect the global cluster center, the prototype matrix is ​​averaged and then normalized. Before normalization, the prototype matrix has already integrated the local cluster center of each client. Normalization on this basis can ensure that the prototype still reflects the global cluster center. Based on the above method, the present invention can solve the technical problems of the existing federated learning method, such as low model accuracy due to the Non-IID data distribution problem and inability to be applied in actual production due to reliance on labeled data.

[0056] 2. The second aspect of the present invention provides a federated contrast clustering learning method for sharing a central vector. By sharing a central vector, the federated contrast clustering learning method provided in the first aspect is improved in a higher dimension, that is, starting directly from the perspective of data distribution, the prototype matrix of different users is used to improve the data offset of the user itself. In the method, each user shares the prototype matrix to the global, and uses the local prototype matrix and the random global prototype matrix to map the local data during the training process. The loss function calculates the loss based on the deviation of the local data from the two. In this way, the local prototype matrix will cover the local data distribution and the data distribution of other users represented by the global prototype matrix as much as possible, rather than simply reflecting the local data distribution, thereby achieving the effect of improving the model. At the same time, the prototype matrix generated by the present invention reflects the mixture between the distribution of local data in high-dimensional space and the distribution of other user data, which is usually not able to be semantically interpreted and the attacker cannot reversely obtain the user's original data from the matrix, so it can effectively ensure the security of user data.

[0057] 3. The third aspect of the present invention provides a federated contrast clustering learning method based on temperature difference and shared center vector. On the basis of the federated contrast clustering learning method of shared center vector provided in the second aspect, the temperature difference coefficient is introduced to enable some cluster centers to migrate to non-local data distribution areas more quickly during the training process. By setting different temperature parameters for the local prototype and the global prototype and designing different loss functions for different prototypes, the local prototype matrix can effectively learn the characteristics of different data distributions at the same time, ensuring that the shared model can correctly identify features belonging to different distributions, thereby making the obtained global feature space representation more uniform and increasing the accuracy of the data features learned by the model. At the same time, since the loss function based on the temperature difference coefficient utilizes the characteristics of user data distributions being far away from each other, it reduces the possibility of overfitting. The gradient direction of the model during training will be more stable, enabling the model to converge more accurately toward the local optimal point, thereby accelerating the model convergence speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of the framework of a federated contrastive clustering learning method for non-independent and identically distributed data provided by the present invention;

[0059] Figure 2 This is a flowchart of performing cluster comparison training in the client provided in Example 1 of the present invention;

[0060] Figure 3 Flowchart of the federated contrastive clustering learning method provided in Example 1 of the present invention;

[0061] Figure 4Schematic diagram of the relationship between user data distribution and cluster center distribution provided in Example 1 of the present invention; (a) shows the distribution of cluster centers under ideal conditions; (b) shows the distribution and change trend of cluster centers under Non-IID data distribution before training; (c) shows the distribution of cluster centers after training;

[0062] Figure 5 A schematic diagram of the client update process provided in Example 3 of the present invention;

[0063] Figure 6 This is a flowchart of the federated contrastive clustering learning method based on temperature difference and shared center vector provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0065] The theory of federated learning was proposed to solve the problem of data silos formed in the context of privacy protection. It is a framework for training models with the help of distributed machine learning ideas. It is divided into central servers and clients according to the different roles involved in training.

[0066] Clients can be low-performance mobile devices. Each client has its own local private data for model training. During training, data is typically not directly transmitted or exchanged to ensure data privacy and security. Each client remains independent of each other and can train in parallel, greatly improving training efficiency. The central server, typically composed of high-performance servers, is primarily responsible for scheduling the various clients during federated learning training, collecting trained models from the clients for aggregation, and distributing the aggregated models to each participating client.

[0067] In federated learning training, the client participants can be heterogeneous devices, allowing federated learning to leverage a wide range of idle devices. Furthermore, the client hardware requirements are low, allowing training to be performed on mobile terminals and IoT devices, lowering the barrier to entry for federated learning and effectively integrating low-power, fragmented devices. Furthermore, during the training process, participants can join or leave the federated learning task at any time, significantly enhancing its flexibility.

[0068] With the continuous development of the information age, the ever-increasing amount of data and the increasing number of users are constantly raising the bar for federated learning. This presents both opportunities and challenges for federated learning. The massive amount of data generated by a vast number of users from diverse regions, professions, and hobbies is an invaluable asset for federated learning. However, this also presents many problems, the first of which is the problem of non-IID data distribution. This overly complex and disordered data distribution prevents conventional federated learning algorithms from effectively extracting globally relevant data features from data that follows a non-IID data distribution. Problems such as insufficient data volume, a limited number of users with too few data types, or excessively discrete data distributions between users further exacerbate the data's non-IID nature. Furthermore, conventional federated learning algorithms typically use machine learning methods based on labeled data for training, but the massive amount of unlabeled data held by users in real-world applications makes these algorithms challenging to implement.

[0069] In order to solve the above problems, Figure 1 As shown, the present invention proposes a federated contrastive clustering learning method for non-independent and identically distributed data, which is applied to distributed systems, as follows:

[0070] Example 1

[0071] A federated contrastive clustering learning method for non-IID data, including:

[0072] Perform the following steps on the central server:

[0073] A11. Initialize the vector representation of K cluster centers to obtain the prototype matrix; initialize the shared model;

[0074] A12. Randomly select m clients from the client set, and send the shared model and prototype matrix to each of the m selected clients for training.

[0075] A13. After collecting the trained shared models and prototype matrices returned by m clients, weight the trained shared models returned by each client to obtain an aggregate model, and weight the trained prototype matrices returned by each client to obtain an aggregate matrix.

[0076] A14. Update the shared model to the aggregate model. After updating the prototype matrix to the aggregate matrix, normalize the prototype matrix and repeat steps A12-A13 until a preset number of iterations (500 in this embodiment) is reached. The shared model at this point is the trained model.

[0077] Among them, when the client receives the shared model and prototype matrix sent by the central server, it inputs the local data set in the client into the shared model for clustering comparison training, such as Figure 2 Specifically, the following operations are performed on each local data sample in the local data set in the client:

[0078] B11. Perform two random data enhancements on the local data samples to obtain a first comparison sample and a second comparison sample;

[0079] B12. Input the first comparison sample and the second comparison sample into the received shared model respectively to obtain the first sample feature and the second sample feature;

[0080] B13, mapping and matching the first sample feature and the second sample feature with the vector representation of each cluster center in the received prototype matrix to obtain a first feature encoding vector and a second feature encoding vector;

[0081] Specifically, the feature encoding vector represents the similarity between the sample feature and the vector of each cluster center in the prototype matrix; it can be measured by cosine similarity, or by directly calculating the inner product of the sample feature and the cluster center vector.

[0082] B14. Update the parameters in the received shared model and the received prototype matrix by minimizing the cross entropy loss between the first sample feature and the second feature encoding vector and the cross entropy loss between the second sample feature and the first feature encoding vector.

[0083] Preferably, each client performs the above cluster comparison training in parallel.

[0084] In this embodiment, the above step B14 solves an exchange prediction problem, and the corresponding loss function represents the degree of fit between the sample features and the feature encoding; specifically, the loss function L(z) when performing cluster comparison training on the shared model in the above step B14 is s , z t )for:

[0085] L(z s ,z t )=l(z s ,qt)+l(z t ,q s )

[0086]

[0087]

[0088] Among them, z s is the first sample feature; zt is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector.

[0089] Specifically, the loss function L consists of two items, namely, t To predict the feature code q s And through the sample feature z s To predict the feature code q t , these two terms form a swap prediction problem. To find a solution that minimizes the loss function L, we need to solve this swap prediction problem. Each term in function L represents the cross-entropy loss between the feature encoding and the probability obtained by softmaxing the dot product of the sample feature and all vectors in the prototype matrix.

[0090] During the training process, the loss function is minimized by updating the parameters in the received shared model and the vector representation in the received prototype matrix. In each training process, only the sample features and prototype matrix of the current batch are used to calculate the feature coding. That is to say, the prototype matrix will be updated according to the current data distribution during each training process without the need to recalculate the cluster distribution from scratch when new data is introduced. By using the prototype matrix to calculate the coding, it is required that the features of all samples in the current batch (image samples are used in this embodiment) are evenly distributed to each cluster center. This equal distribution constraint ensures that the feature coding of different images in each batch is different, otherwise the model will collapse to a situation where the same coding is generated for all samples.

[0091] It should be noted that in order to solve the technical problem that conventional federated learning methods rely on labeled data, the present invention takes into account that unsupervised learning allows the model to be trained only with samples without providing their corresponding labels during training, and proposes a federated learning method based on unsupervised deep learning methods. In unsupervised deep learning methods, contrastive learning does not rely on data labels and focuses on the comparison between local samples and is insensitive to the global distribution of data. At the same time, clustering-based deep learning algorithms can better represent the class distribution of data and improve the Non-IID problem caused by class imbalance. Therefore, in order to better improve the damage caused by Non-IID data distribution to the model from the algorithm level and solve the problem of the algorithm's dependence on labeled data, the present invention proposes a federated learning method based on clustering contrastive learning, which uses the characteristics of clustering-based methods and contrastive learning methods that are insensitive to global data distribution to respectively improve the Non-IID problem caused by class imbalance between user data and the impact of Non-IID data distribution on the model.

[0092] Specifically, when the above-mentioned client performs cluster comparison training, it needs to normalize the prototype matrix after updating it through back propagation to ensure the normal distribution of cluster centers. However, if the cluster centers are normalized at the client, the distribution of cluster centers will be greatly changed, because the prototype matrix at the client is obtained by training the user's local data and can only reflect its local cluster center but not the global cluster center. In order to ensure that the prototype matrix can effectively reflect the global cluster center, this embodiment puts the normalization step of the prototype matrix on the central server for processing. After the central server receives the prototype matrices from all clients and averages them, it normalizes the obtained prototype matrix. In this way, before normalization, the prototype matrix has already integrated the local cluster centers of each client. Normalization on this basis can ensure that the prototype still reflects the global cluster center without deviating too far.

[0093] Furthermore, the above-mentioned local dataset is a local private dataset, which is determined by the specific task of the training model, such as image classification tasks, as well as other visual tasks such as object detection and image segmentation. Taking the image classification task as an example, in contrastive learning, in order to obtain more positive or negative samples, the image cropping method is usually used to expand the number of samples. However, conventional image cropping methods significantly increase the memory and computing requirements by increasing the number of crops or views in a single batch training. Since user devices are usually in a resource-constrained state in a federated environment, in order to expand the number of samples while minimizing the memory and computing resource requirements of the training process, this embodiment introduces a multi-crop strategy to reduce the memory and computing consumption of conventional cropping methods. This method uses two standard resolution crops and samples V additional low-resolution crops that only cover a small part of the image. By using low-resolution crops, the memory and computing consumption brought by conventional cropping methods can be effectively reduced.

[0094] Next, the workflow of the federated contrastive clustering learning method provided in this embodiment will be described in detail. Figure 3 As shown in the figure, the central server first determines the number of prototype centers and initializes the prototype matrix C0 accordingly. Then, the model w0 is initialized. Both the model and the prototype are initialized with normalized random values. At the same time, the client participating in the training is selected on the user side and data is allocated to each client according to the data partitioning method. After the model and prototype are prepared and the data is allocated, the model and prototype matrix are distributed from the central server to each client participating in the training. Client k receives the model w i and the prototype matrix C i After that, read the local data set to start the above clustering comparison training. After completing the local training on the client, the model and the prototype matrix Client k transmits the trained model and prototype back to the central server for aggregation. After receiving the updated models and prototypes from all clients, the central server aggregates and updates the model parameters and prototype matrix using a weighted average method, and normalizes the prototype matrix to obtain the model w i+1 and prototype C i+1 , then increment round i by one and determine whether the model has converged. If not, the model and prototype are distributed to the client for training again. If converged, the final model is returned. The pseudo code of the federated contrastive clustering learning method provided in this embodiment is shown in Table 1.

[0095] Table 1

[0096]

[0097]

[0098] Example 2

[0099] Although the federated contrastive clustering learning method provided in Example 1 of the present invention can improve the Non-IID problem and the technical problem of dependence on labeled data faced by federated learning, the above method only focuses on a single sample (such as an image) itself and is not sensitive to data distribution. Therefore, the prototype matrix can be corrected while correcting the shared model to ensure that the model does not deviate significantly. However, when the degree of Non-IID is large, users with a large amount of data will still cause the prototype matrix to deviate significantly toward the user's local distribution, resulting in the prototype matrix being unable to effectively reflect the global data feature distribution. Moreover, the contrastive learning introduced therein has a characteristic, namely, focusing on the feature comparison between samples and the data distribution of local samples. In a federated environment, this will cause the model generated by a single client to be too biased towards the local data distribution, resulting in overfitting.

[0100] To address the above issues, many existing frameworks enhance the models by sharing data or pre-training models to utilize more global information for training and improve the effectiveness of federated learning. For example, the FedCA algorithm uses methods of sharing data features and preparing additional data sets. This method can expand the data for users with too little data or too few types of data and at the same time narrow the data distribution between users. However, this also brings about the problem of privacy leakage: when using this method, the user's data is transmitted to other user devices for training. Not only may the data be intercepted by hackers during transmission, but it may also be used by malicious users when training on the user device. This goes against the original intention of federated learning to protect privacy.

[0101] In order to utilize the method of sharing user information to solve the deviation problem of the federated contrastive clustering learning method provided in Example 1 of the present invention and avoid its potential privacy leakage risk, this embodiment proposes a federated contrastive clustering learning method with shared center vectors. This method shares the user's local prototype with the global one, and uses the prototype matrix of other users to map when the user maps through the local prototype to correct the deviation of the prototype matrix. Compared with the existing methods that use methods such as sharing data, features, or introducing pre-trained models to alleviate the damage to the model effect caused by the excessive influence of users with a large amount of data on the model under the Non-IID data distribution, the federated contrastive clustering learning method with shared center vectors provided in this embodiment can avoid data leakage and does not require the introduction of additional pre-trained models to enhance the model effect.

[0102] Specifically, the federated contrastive clustering learning method for sharing a central vector provided in this embodiment includes:

[0103] Perform the following steps on the central server:

[0104] A21. Initialize the vector representations of K cluster centers to obtain the initial representation C0 of the prototype matrix; construct a prototype array of length m to store the prototype matrices returned by the client according to the client number, and initialize them to C0 respectively; initialize the shared model;

[0105] A22. Randomly select m clients from the client set, and send the shared model and prototype matrix to the selected m clients for training. In the process of sending the shared model and prototype matrix to the t-th client, randomly select a prototype matrix from the prototype array other than the prototype matrix corresponding to the t-th client, and record it as matrix C. P , the matrix C P At the same time, it is sent to the t-th client for training; t = 1, 2, ..., m;

[0106] A23. After collecting the trained shared models and prototype matrices returned by m clients, store the prototype matrices returned by each client in the corresponding position of the prototype array, and perform weighted summation of the trained shared models returned by each client using the proportion of the client's data volume in the total data volume as a weight to obtain an aggregate model, and perform weighted summation of the trained prototype matrices returned by each client to obtain an aggregate matrix;

[0107] A24. Update the shared model to the aggregate model. After updating the prototype matrix to the aggregate matrix, normalize the prototype matrix and repeat steps A22-A23 until a preset number of iterations (500 in this embodiment) is reached. The shared model at this point is the trained model.

[0108] Among them, when the client receives the shared model, prototype matrix and matrix C sent by the central server P After that, the local data set in the client is input into the shared model for clustering comparison training, which specifically includes: performing the following operations on each local data sample in the local data set in the client:

[0109] B21. Perform two random data enhancements on the local data samples to obtain a first comparison sample and a second comparison sample;

[0110] B22. Input the first comparison sample and the second comparison sample into the received shared model respectively to obtain the first sample feature and the second sample feature;

[0111] B23, mapping and matching the first sample feature and the second sample feature with the vector representation of each cluster center in the received prototype matrix to obtain a first feature encoding vector and a second feature encoding vector;

[0112] The first sample feature and the second sample feature are respectively combined with the received matrix C P The vector representations of the cluster centers in are mapped and matched to obtain the third feature coding vector and the fourth feature coding vector;

[0113] Specifically, the feature encoding vector represents the similarity between the sample feature and the vector of each cluster center in the prototype matrix; it can be measured by cosine similarity, or by directly calculating the inner product of the sample feature and the cluster center vector.

[0114] B24. Update the parameters and prototype matrix in the shared model by minimizing the cross entropy loss between the first sample feature and the second feature encoding vector, the cross entropy loss between the second sample feature and the first feature encoding vector, the cross entropy loss between the first sample feature and the fourth feature encoding vector, and the cross entropy loss between the second sample feature and the third feature encoding vector.

[0115] Preferably, the temperature coefficient is initialized by the central server and then sent to each client.

[0116] It should be noted that the conventional way of sharing data or feature vectors is essentially to solve the offset caused by its own data by adding user data, thereby alleviating the model's inability to correctly reflect the global data distribution due to overly extreme user data. The federated contrast clustering learning method provided in Example 1 of the present invention can improve the algorithm in a higher dimension, that is, directly starting from the perspective of data distribution, and using the prototype matrices of different users to improve the user's own data offset. In the federated contrast clustering learning method provided in Example 1 of the present invention, each user shares the prototype matrix to the global one, and during the training process, the local prototype matrix and the random global prototype matrix are used to map the local data. The loss function calculates the loss based on the deviation of the local data from the two. In this way, the local prototype matrix will cover the local data distribution and the data distribution of other users represented by the global prototype matrix as much as possible, rather than simply reflecting the local data distribution, thereby achieving the effect of improving the model. On this basis, the prototype matrix generated for the client by the federated contrastive clustering learning method provided in Example 1 of the present invention reflects the mixture between the distribution of local data in high-dimensional space and the distribution of other user data. It is usually not able to be semantically interpreted and the attacker cannot obtain the original user data from the matrix in reverse. Therefore, the federated contrastive clustering learning method of sharing the central vector provided in this embodiment can also effectively share global information for each user while ensuring security.

[0117] Specifically, sharing the cluster center vector means sharing the prototype matrix that has not been normalized after the local training on the client is completed. The reason for sharing the prototype matrix that has not been normalized here is that normalization will affect the original distribution of the data. Although normalization can effectively alleviate the large deviation of the cluster distribution during the global aggregation process, the prototype matrix shared globally should maintain its original distribution as much as possible. In order to achieve the sharing of cluster center vectors, a prototype array is maintained in this embodiment. The length of the array is the number of users participating in the training, and the latest prototype matrix of the user is stored according to the user number. The array is initialized with the original prototype matrix. During the training process, the central server will randomly select a prototype matrix of another client and send it to the specified client. Locally, the client will not only perform the loss calculation for the exchange prediction problem on its own prototype, but also perform the loss calculation for the exchange prediction problem on the prototype matrix sent by the server.

[0118] Specifically, the loss function L(z s , z t )for:

[0119] L(z s ,z t )=l(z s ,q t )+l(z t ,q s )+l′(z s ,q t ')+l′(z t ,q s ')

[0120]

[0121]

[0122]

[0123]

[0124] Among them, z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q s ' is the third feature encoding vector; q t ' is the fourth feature code vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; qs (k) is the kth feature code in the first feature code vector; q t '(k) is the kth feature code in the fourth feature code vector; d k is the matrix C P The vector representation of the kth cluster center in ; q s '(k) is the kth feature code in the third feature code vector. The pseudo code of the federated contrastive clustering learning method for sharing the center vector provided in this embodiment is shown in Table 2.

[0125] Table 2

[0126]

[0127]

[0128] Example 3

[0129] The federated contrast clustering learning method for sharing center vectors provided in Example 2 uses a clustering-based contrast learning algorithm as a basis for training, and improves the model effect by sharing clusters, but it still has defects, because if the data characteristics under the Non-IID data distribution cannot be better utilized, simply sharing global information cannot effectively solve the model offset caused by the user's local data training. Therefore, considering the data distribution differences between different users under the Non-IID data distribution, the federated contrast clustering learning method for sharing center vectors provided in Example 2 still has room for improvement. During the training process, in order to enable the prototype matrix to better reflect the global data distribution, the local prototype matrix should cover the local data distribution and the data distribution of the remaining users represented by the global prototype matrix as much as possible. During training, some cluster centers in the local prototype matrix will move closer to the local data distribution, and the other cluster centers will be biased towards the data distribution represented by the global prototype matrix, and ultimately achieve the effect of covering both distributions at the same time. The federated contrast clustering learning method of the shared center vector provided in Example 2 uses the same loss function for the user's local prototype and the global prototype when calculating the loss, which will make the cluster center move towards the two data distributions at the same speed. However, because the user's local data distribution is closer to the local prototype matrix and farther away from the global prototype matrix under the Non-IID data distribution, the cluster center that moves closer to the data distribution represented by the global prototype matrix needs to move at a faster speed to ensure that it reaches the target area at the same time as the cluster center that moves towards the local data distribution. Therefore, in order to solve this problem, this embodiment proposes a federated contrast clustering learning method based on temperature difference and shared center vector on the basis of the federated contrast clustering learning method of the shared center vector provided in Example 2. This method introduces the temperature difference coefficient τ' on the basis of the federated contrast clustering learning method of the shared center vector provided in Example 2 and uses the global prototype array C to calculate the temperature difference coefficient τ'. g Used when calculating losses, for the following reasons:

[0130] When learning cluster centers, the algorithm often encounters some tricky problems, such as empty clusters and collapsed parameter problems. When the discriminant model learns the boundaries between different classes, its optimal decision boundary assigns all inputs to a certain class, which leaves no samples distributed around other cluster centers. This is called the empty cluster problem. In this embodiment, since the cluster centers are constantly changing with user local training and global aggregation, this problem is implicitly avoided. The collapsed parameter problem means that if most of the images in the dataset are assigned to a few cluster centers, then the model parameters θ will specifically distinguish only these few classes, resulting in overfitting. In a real federated learning environment, different user devices have different data sets. These data sets come from various user behaviors and habits. Specifically, sample data sets (such as image data sets) come from users' personal preferences, work, etc. Therefore, these data sets usually have obvious preferences. As time goes by, the internal data sets of user devices will contain more and more pictures of certain categories. At the same time, since everyone's personal preferences and other influencing factors are likely to be different, the types of pictures gathered locally by each user are often inconsistent, which is reflected in the overall data distribution as a highly non-IID data distribution. The collapsed parameter problem is very easy to occur when the number of images in each category is highly unbalanced under the non-IID data distribution. In this invention, images are sampled using a uniform distribution based on classes or pseudo-labels, and consistency constraints are used to solve this problem. However, in addition to the problem of collapsed parameters, the data distribution in a federated environment also gives researchers a very important revelation - in a federated environment, due to user interests and other factors, user local data is highly cohesive in categories (tending to have data of a few specific categories rather than evenly owning each category), and the cluster centers of data between users are far away from each other (from a statistical point of view, when the number of users is large enough, two users are randomly selected, and the data categories they have are likely to be far apart due to inconsistencies in personal preferences, habits and environments). When the algorithm has already effectively avoided empty clusters and collapsed parameters, reasonable use of this Non-IID data distribution characteristic will help the model better learn the overall data distribution and achieve better model effects.

[0131] Based on the above conclusions, we further analyzed the relationship between user data distribution and cluster center distribution. The specific situation is as follows: Figure 4 As shown, Figure 4 Figure (a) shows the distribution of cluster centers under ideal conditions; Figure 4 Figure (b) shows the distribution and changing trend of cluster centers under Non-IID data distribution before training; Figure 4 Figure (c) shows the distribution of cluster centers after training. Figure 4In the figure, the circle represents the global class space, the triangle represents the cluster center vector of the user's local prototype, and its distribution represents the class space that the model can recognize; the darker gray area represents the user's local data distribution, and the lighter gray area represents the data distribution of other users represented by the global prototype matrix. It can be seen that there is no overlap between the lighter gray area and the darker gray area in the figure, indicating that the data distribution between different users is far away from each other, and the area is narrow, indicating that each user only has data of a few categories. Figure 4 As can be seen from Figure (a) in the figure, in an ideal state, the cluster centers of the user's local prototype matrix are evenly distributed in the global range, which means that it can correctly identify the types of all data. Figure 4 In Figure (b), we can see that under the Non-IID data distribution, the cluster centers of the user's local prototype matrix are closer to the user's local data distribution. During the training process, the arrows indicate the direction of movement of the cluster centers. In order to enable the user's local prototype matrix to simultaneously identify the categories represented by the two distributions, some cluster centers will move toward the direction of the local data distribution, while the other cluster centers will move toward the data distribution of the remaining users. Figure 4 In Figure (c), we see that if all cluster centers migrate at the same rate, some will cover the local data distribution earlier, while the remaining cluster centers will not be able to cover the data distribution of other users. Based on the above analysis, we can reasonably speculate that if some cluster centers can be migrated to non-local data distribution areas more quickly during training, the model performance will be improved.

[0132] Specifically, considering that the federated contrastive clustering learning method of the shared center vector provided in Example 2 uses the same loss function for the user local prototype and the global prototype when calculating the loss, this will make the cluster centers move towards the two different data distributions at the same speed. However, since the category distribution that can be identified by the user local prototype matrix under the Non-IID data distribution is closer to the local data distribution and away from the other user data distributions represented by the global prototype matrix, the consistency of the cluster center movement speed will cause some cluster centers in the local prototype to converge to the local data distribution, while the remaining cluster centers that move towards other user data distributions have not yet reached the target area, or the cluster centers that move towards other user data distributions have reached the target area but the cluster centers that move towards the local data distribution have been overfitted. Therefore, the cluster centers that move towards the data distribution represented by the global prototype matrix need to have a faster movement speed to ensure that they reach the target area at the same time as the cluster centers that move towards the local data distribution.

[0133] In conventional contrastive learning algorithms, the contrastive loss function typically uses a temperature coefficient, τ, to adjust the focus on edge samples. Within a reasonable range, the smaller the temperature coefficient, the more the model focuses on separating the sample from its most similar samples, that is, paying more attention to samples that are not yet far away. The temperature coefficient τ is typically between 0.1 and 1. This temperature coefficient τ enables the contrastive loss function to self-discover edge negative samples, a property crucial for contrastive learning algorithms to learn high-quality feature representations. The purpose of self-discovery of edge negative samples is to, instead of pushing samples already on the edge further away, focus on pushing samples that are not yet far away as far away as possible, thereby making the resulting feature representation space more uniform.

[0134] Based on the above analysis, in order to enable cluster centers to move toward other user data distributions at a faster rate than toward the local data distribution during training, thereby achieving the effect of cluster centers covering both local and non-local user data distributions, this embodiment utilizes temperature differences based on shared center vectors and proposes a federated clustering contrastive algorithm based on center vectors and temperature differences. This embodiment further expands the effectiveness of the temperature coefficient in cluster-based contrastive learning. To enable cluster centers to move toward other user data distributions at a faster rate, this embodiment redesigns the contrastive loss function used in the algorithm. Unlike Example 2, where the client uses the same temperature parameter for the loss functions for both local and other user prototypes, the contrastive loss function results in a larger calculated loss value for local data relative to other user data distributions. This results in a faster rate of cluster center movement toward other user data distributions, facilitating the cluster center's simultaneous coverage of both local and non-local user data distributions after training. Specifically, this embodiment uses a new temperature parameter when the client uses local data to calculate the loss value for the exchange prediction problem against the prototype matrix of other users. By setting different temperature parameters for the local prototype and the global prototype, the newly introduced temperature coefficient enables the local prototype matrix to effectively learn the characteristics of different data distributions at the same time, ensuring that the model encoder can correctly identify features belonging to different distributions, thereby making the global feature space representation more uniform. After the introduction of the new temperature coefficient, the new loss function L(z s , z t )for:

[0135] L(z s ,z t )=l(z s ,q t )+l(z t ,q s )+l″(z s ,q t ')+l″(zt ,q s ′)

[0136]

[0137]

[0138]

[0139]

[0140] Among them, the loss function l" used by the non-local prototype; z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q s ′ is the third feature encoding vector; q t ′ is the fourth feature encoding vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector; q t ′(k) is the kth feature code in the fourth feature code vector; τ′ is the temperature difference coefficient for the non-local prototype loss function, which is used to adjust the speed of migration of the cluster center in the local prototype matrix to other user data distributions; d k is the matrix C P The vector representation of the kth cluster center in ; q s '(k) is the kth feature code in the third feature code vector. In this embodiment, the temperature coefficient and the temperature difference coefficient are initialized by the central server and then sent to each client.

[0141] Preferably, the temperature difference coefficient τ′ is smaller than the temperature coefficient τ to accelerate the migration of cluster centers to other user data distributions. It should be noted that the temperature parameter used to calculate the exchange prediction loss of data for the non-local prototype matrix should be smaller than the temperature parameter for the local prototype, which can increase the exchange prediction loss for the non-local prototype matrix. This semantically tells the model that the current data cluster center distribution is farther away from the data distribution represented by the global prototype matrix than the local data distribution. To simultaneously cover both distributions, the model should accelerate the migration of cluster centers to the non-local data distribution, which is consistent with the analysis of non-IID cluster center distributions mentioned above. Furthermore, based on the aforementioned analysis, it can be seen that the effectiveness of this method should improve as the degree of non-IID data distribution increases. However, it should be noted that this does not mean that this method will not be effective in the case of IID data distribution, as IID data distribution does not necessarily mean that the data distribution is completely identical. As long as there are deviations in the data distribution between different users, this method will be effective. Since the loss function based on the temperature difference coefficient takes advantage of the fact that user data distributions are far apart from each other, it reduces the possibility of overfitting. The gradient direction of the model during training will be more stable, allowing the model to converge more accurately towards the local optimum. Therefore, the federated contrastive clustering learning method based on temperature difference and shared center vector provided in this embodiment can theoretically improve the convergence speed of the model to a certain extent, which will be helpful for the training of the model on edge devices. Specifically, Figure 5 As shown, when training is performed locally on the user, when the data is enhanced to obtain X s , X t After that, the feature Z is obtained through the encoder s , Z t , and map it through the local matrix C to obtain the feature code Q s , Q t , then Z s , Q t and Z t , Q s Use τ,C and τ′,C respectively g The exchange prediction is performed to obtain the loss L, and then the model and local prototype C are updated according to the loss L.

[0142] like Figure 6 The flowchart of the federated contrastive clustering learning method based on temperature difference and shared center vector provided by this embodiment is shown. First, the center number K of the prototype is determined in the central server and the prototype matrix C0 is initialized accordingly. Then, the model w0 is initialized and the global prototype array C is filled according to C0 initialization. g, then initialize the temperature difference coefficient τ', where both the model and the prototype are initialized with normalized random values. At the same time, the client participating in the training is selected on the user side and data is allocated to each client according to the data partitioning method. After the model and prototype are prepared and the data is allocated, a prototype C' other than the user's own prototype is randomly selected from the global prototype array g , distribute it together with the model and prototype matrix from the central server to each client participating in the training. Client k receives the global prototype C' g , model w i and the prototype matrix C i After that, the local data is Figure 6 The process in Calculate the loss value of the global prototype matrix and the local prototype matrix respectively, and add the two as the final loss value for back propagation, update the model and the local prototype matrix to obtain a new model and the prototype matrix After receiving the updated models and prototypes from all clients, the central server first updates the global prototype array using the unnormalized prototype matrix, and then aggregates and updates the model parameters and prototype matrix using a weighted average method and normalizes the prototype matrix to obtain the model w i+1 and prototype C i+1 , then add one to the round i and determine whether the model has converged. If not, the model and prototype are distributed to the client for training again. If converged, the final model is returned.

[0143] The pseudo code of the federated contrastive clustering learning method based on temperature difference and shared center vector provided in this embodiment is shown in Table 3.

[0144] Table 3

[0145]

[0146]

[0147] It should be noted that, except for the loss function, other technical means are the same as those in Example 2 and will not be described in detail here.

[0148] In order to illustrate the performance of the federated contrast clustering learning method provided in Examples 1-3 of the present invention, the present invention uses MNIST, CIFAR-10 and CIFAR-100 data sets as training and test data sets in the federated learning process. Wherein, MNIST is a handwritten digital black and white image data set, and CIFAR-10 and CIFAR-100 are color image data sets that are closer to ordinary objects. Wherein, the MNIST data set contains ten kinds of handwritten digital pictures from 0 to 9, consisting of 60,000 pictures, and the picture size is 28x28. The CIFAR-10 data set contains a total of 10 categories of color pictures of airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships and trucks, consisting of 50,000 training pictures and 10,000 test pictures, with 6,000 pictures for each category, and the size of each picture is 32×32. The CIFAR-100 data set is similar to the CIFAR-10 composition, except that it contains 100 categories of pictures, with 600 pictures for each category. This experiment uses three data sets for image classification training and to verify the model effect. Specifically, the experimental data set is divided into two forms that conform to IID and Non-IID data distribution. The IID data distribution adopts random sampling and average distribution, while the Non-IID data distribution adopts the Dirichlet distribution function with a distribution coefficient α of 0.5 to randomly allocate the number of pictures of each category to each user. Under different data distributions of different data sets, the methods provided in Examples 1-3 of the present invention are compared with the existing Swav algorithm and FedCA algorithm, and the accuracy of the shared model (ResNet-18 network is used in this experiment) is shown in Table 4.

[0149] Table 4

[0150]

[0151]

[0152] As can be seen from Table 4, the accuracy of the method provided by Example 3 of the present invention significantly exceeds that of the method provided by Example 1 of the present invention on all datasets due to the simultaneous introduction of center vector sharing and temperature differences. The accuracy comparison relationship of each method on the three datasets is basically consistent. Specifically, taking the CIFAR-10 dataset as an example, the accuracy of the method provided by Example 3 of the present invention under the IID data distribution is 72.34%, which is 3.1% higher than FedCA. This performance is almost as good as Swav (72.5%) under the non-federated learning environment. At the same time, the accuracy of the method provided by Example 3 of the present invention under the non-IID data distribution is 3.9% higher than FedCA. This shows that the federated contrastive learning algorithm can benefit from the proposed method, and the improvement is more obvious under the non-IID data distribution. Compared with the method provided in the basic embodiment 1 of the present invention, the accuracy of the method provided in embodiment 2 of the present invention decreased by 0.2% under the IID data distribution, while the accuracy increased by 0.9% under the Non-IID data distribution; the accuracy of the method provided in embodiment 3 of the present invention increased by 2.7% under the IID data distribution, while the accuracy increased by 5.7% under the Non-IID data distribution.

[0153] In summary, the present invention combines the characteristics of non-independent and identically distributed data with federated learning and contrastive learning algorithms, starting from the information contained in the user's local data set and the global data distribution, and simultaneously mines the user's local data characteristics and the cluster center distribution between the global data through the shared center vector and temperature coefficient difference method to obtain a more effective feature extraction model. Specifically, the present invention discloses three new federated contrastive clustering learning methods for non-independent and identically distributed data, starting from the characteristics of non-independent and identically distributed data and privacy protection, to obtain a more effective network model. It includes: defining the model structure and contrasting clustering methods to retain the category information of the data; utilizing user center vector sharing to alleviate the model's deviation towards users with large data volumes under non-independent and identically distributed data distribution; and further introducing the temperature difference coefficient to make the cluster center in training have different migration rates for each user's data distribution, so that the trained cluster center can cover the data distribution of each user globally as much as possible, and obtain a federated learning method that is well adapted to non-independent and identically distributed data.

[0154] Example 4

[0155] A federated contrastive clustering learning system for non-independent and identically distributed data includes: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it performs the federated contrastive clustering learning method provided by Example 1, Example 2 and / or Example 3 of the present invention.

[0156] The relevant technical solutions are the same as those in Examples 1-3 and will not be described in detail here.

[0157] Example 5

[0158] A computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the federated contrastive clustering learning method provided by Example 1, Example 2 and / or Example 3 of the present invention.

[0159] The relevant technical solutions are the same as those in Examples 1-3 and will not be described in detail here.

[0160] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A federated contrastive clustering learning method for non-independent and identically distributed data, characterized by: include: Perform the following steps on the central server: A11. Initialize the vector representation of K cluster centers to obtain the prototype matrix; Initialize the shared model; A12. Randomly select m clients from the client set, and send the shared model and prototype matrix to each of the m selected clients for training. A13. After collecting the trained shared models and prototype matrices returned by m clients, weight the trained shared models returned by each client to obtain an aggregate model, and weight the trained prototype matrices returned by each client to obtain an aggregate matrix. A14. Update the shared model to the aggregate model, update the prototype matrix to the aggregate matrix, normalize the prototype matrix, and repeat steps A12-A13 until a preset number of iterations is reached. The shared model at this time is the trained model; Among them, after the client receives the shared model and prototype matrix sent by the central server, it inputs the local data set in the client into the shared model for clustering comparison training, specifically including: performing the following operations on each local data sample in the local data set in the client: B11. Perform two random data enhancements on the local data samples to obtain a first comparison sample and a second comparison sample; B12. Input the first comparison sample and the second comparison sample into the received shared model respectively to obtain the first sample feature and the second sample feature; B13, mapping and matching the first sample feature and the second sample feature with the vector representation of each cluster center in the received prototype matrix to obtain a first feature encoding vector and a second feature encoding vector; B14. Update the parameters in the received shared model and the received prototype matrix by minimizing the cross entropy loss between the first sample feature and the second feature encoding vector and the cross entropy loss between the second sample feature and the first feature encoding vector.

2. The federated contrastive clustering learning method according to claim 1, characterized in that: Each client executes the cluster contrast training in parallel.

3. A federated contrastive clustering learning method for non-independent and identically distributed data, characterized by: include: Perform the following steps on the central server: A21. Initialize the vector representations of K cluster centers to obtain the initial representation C0 of the prototype matrix; construct a prototype array of length m to store the prototype matrices returned by the client according to the client number, and initialize them to C0 respectively; initialize the shared model; A22. Randomly select m clients from the client set, and send the shared model and prototype matrix to the selected m clients for training. In the process of sending the shared model and prototype matrix to the t-th client, a prototype matrix other than the prototype matrix corresponding to the t-th client is randomly selected from the prototype array and recorded as matrix C P , the matrix C P At the same time, it is sent to the t-th client for training; t = 1, 2, ..., m; A23. After collecting the trained shared models and prototype matrices returned by m clients, store the prototype matrices returned by each client in the corresponding position of the prototype array, and perform weighted summation of the trained shared models returned by each client using the proportion of the client's data volume in the total data volume as a weight to obtain an aggregate model, and perform weighted summation of the trained prototype matrices returned by each client to obtain an aggregate matrix; A24, updating the shared model to the aggregate model, updating the prototype matrix to the aggregate matrix, normalizing the prototype matrix, and iterating through steps A22-A23 until a preset number of iterations is reached; The shared model at this time is the trained model; Among them, when the client receives the shared model, prototype matrix and matrix C sent by the central server P After that, the local data set in the client is input into the shared model for clustering comparison training, which specifically includes: performing the following operations on each local data sample in the local data set in the client: B21. Perform two random data enhancements on the local data samples to obtain a first comparison sample and a second comparison sample; B22. Input the first comparison sample and the second comparison sample into the received shared model respectively to obtain the first sample feature and the second sample feature; B23, mapping and matching the first sample feature and the second sample feature with the vector representation of each cluster center in the received prototype matrix to obtain a first feature encoding vector and a second feature encoding vector; The first sample feature and the second sample feature are respectively compared with the received matrix C P The vector representations of the cluster centers in are mapped and matched to obtain the third feature coding vector and the fourth feature coding vector; B24. Update the parameters and prototype matrix in the shared model by minimizing the cross entropy loss between the first sample feature and the second feature encoding vector, the cross entropy loss between the second sample feature and the first feature encoding vector, the cross entropy loss between the first sample feature and the fourth feature encoding vector, and the cross entropy loss between the second sample feature and the third feature encoding vector.

4. The federated contrastive clustering learning method according to claim 3, characterized in that: The loss function L(z s , z t )for: L(z s ,z t )=l(z s ,q t )+l(z t ,q s )+l′(z s ,q t ')+l′(z t ,q s ') Among them, z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q s ′ is the third feature encoding vector; q t ′ is the fourth feature encoding vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector; q t ′(k) is the kth feature code in the fourth feature code vector; d k is the vector representation of the kth cluster center in the matrix CP; q s ′(k) is the kth feature code in the third feature code vector.

5. The federated contrastive clustering learning method according to claim 4, characterized in that: The temperature coefficient is initialized by the central server and then sent to each client.

6. The federated contrastive clustering learning method according to claim 3, characterized in that: The loss function L(z s , z t )for: L(z s ,z t )=l(z s ,q t )+l(z t ,q s )+l″(z s ,q t ′)+l″(z t ,q s ′) Among them, z s is the first sample feature; z t is the second sample feature; q s is the first feature encoding vector; q t is the second feature encoding vector; q s ' is the third feature encoding vector; q t ' is the fourth feature code vector; q t (k) is the kth feature code in the second feature code vector; c k is the vector representation of the kth cluster center in the prototype matrix; τ is the temperature coefficient; q s (k) is the kth feature code in the first feature code vector; q t '(k) is the kth feature code in the fourth feature code vector; τ' is the temperature difference coefficient; d k is the matrix C P The vector representation of the kth cluster center in ; q s '(k) is the kth feature code in the third feature code vector.

7. The federated contrastive clustering learning method according to claim 6, characterized in that: The temperature coefficient and the temperature difference coefficient are both initialized by the central server and then sent to each client.

8. The federated contrastive clustering learning method according to claim 6, characterized in that: The temperature variation coefficient is smaller than the temperature coefficient.

9. The federated contrastive clustering learning method according to any one of claims 3 to 8, characterized in that: Each client executes the cluster contrast training in parallel.

10. A federated contrastive clustering learning system for non-independent and identically distributed data, characterized by: include: A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the federated contrastive clustering learning method according to any one of claims 1 to 2 and / or the federated contrastive clustering learning method according to any one of claims 3 to 9 is executed.

Images