Federal unsupervised image classification model training method, classification method and device based on semantic clustering

Abstract

The application provides a kind of federated unsupervised image classification model training method based on semantic clustering, classification method and equipment: obtaining client local data set;Obtain initial model, including semantic clustering model and pre-training obtained encoder network;Local data set is randomly enhanced twice to generate two views, input initial encoder network, extract feature vector and construct contrast loss, and train to obtain encoder network;Local data set is input into encoder network to extract feature vector, and extract the Top-K nearest neighbor sample of sample, calculate the vector value of sample belonging to different clusters using a preset function, to obtain the category of sample;Local data set is used to train initial model, and clustering loss is constructed to obtain initial image classification model;Based on the model parameters of each client, a shared model with a self-labeling module is constructed, and the initial image classification model is updated according to the shared model parameters.The image classification model provided by the application has high accuracy and is suitable for non-independent and identically distributed scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a federated unsupervised image classification model training method and device based on semantic clustering. BACKGROUND

[0002] With the popularity of intelligent devices, federated learning has become the most commonly used method for sharing privacy-protected models and has been widely applied in many scenarios such as user habit prediction, personalized recommendation, and wireless network optimization. Existing federated learning methods usually only consider supervised training settings, in which client data is fully labeled. However, local data containing complex annotations is unrealistic for Internet of Things applications, because users always have different habits and usage frequencies. For example, assume there is a photo classifier application that can automatically classify pictures in a photo album. In this case, if users of the application are unwilling to annotate these private and sensitive pictures themselves, the service provider will only be able to use limited public pictures on the central server. Therefore, in many real Internet of Things scenarios, client data may not be fully labeled, and only a small amount of labeled data is available on the server.

[0003] Existing federated learning methods mainly use federated semi-supervised learning in scenarios lacking label data. The goal of federated semi-supervised learning is to learn consistency between multiple clients. Some works use consistency loss between clients for distributed training of labeled and unlabeled data, or consider semi-supervised training diversity scaling aggregation algorithms that exchange local model outputs between mobile devices instead of model parameter exchange used in typical frameworks. However, when there are no available labels for downstream tasks, the performance of these methods is poor. At the same time, unlike idealized distribution conditions, data between Internet of Things devices is usually non-identically distributed due to different user habits and usage frequencies, which also leads to a decline in the performance of shared models. SUMMARY

[0004] In view of this, the embodiments of the present application provide a federated unsupervised image classification model training method and device based on semantic clustering to eliminate or improve one or more defects in the prior art, solve the problem of poor performance of existing federated unsupervised learning and unsuitability for non-identically distributed scenarios.

[0005] In one aspect, the present application provides a federated unsupervised image classification model training method based on semantic clustering, characterized in that the method is executed on each client and includes the following steps:

[0006] Obtain a local data set, which contains multiple samples, each sample containing an image;

[0007] obtaining an initial model, the initial model comprising a semantic clustering model and a pre-trained encoder network; wherein samples of the local dataset are subjected to twice random augmentation to generate a first view and a second view; the first view and the second view are input into the initial encoder network to extract a first feature vector and a second feature vector; the local dataset is used to train the initial encoder network, and a contrast loss between the first feature vector and the second feature vector is constructed to obtain a trained encoder network; the local dataset is input into the semantic clustering model in batches, and the encoder network is used to extract a feature vector of a corresponding sample; Top-K neighbor samples of the corresponding sample are extracted from the feature vector based on a preset neural network, and vector values of different clusters to which the corresponding sample belongs are calculated by a preset Softmax function to obtain a class of the corresponding sample; Figure One obtaining an initial model, the initial model comprising a semantic clustering model and a pre-trained encoder network; wherein samples of the local dataset are subjected to twice random augmentation to generate a first view and a second view; the first view and the second view are input into the initial encoder network to extract a first feature vector and a second feature vector; the local dataset is used to train the initial encoder network, and a contrast loss between the first feature vector and the second feature vector is constructed to obtain a trained encoder network; the local dataset is input into the semantic clustering model in batches, and the encoder network is used to extract a feature vector of a corresponding sample; Top-K neighbor samples of the corresponding sample are extracted from the feature vector based on a preset neural network, and vector values of different clusters to which the corresponding sample belongs are calculated by a preset Softmax function to obtain a class of the corresponding sample;

[0008] using the local dataset to train the initial model, and constructing a clustering loss, and iteratively updating parameters of the initial model by using the clustering loss to obtain an initial image classification model;

[0009] sending model parameters of the initial image classification model to a global server to generate a shared model; the shared model is obtained by the global server according to weighted aggregation of initial image classification model parameters of each client; receiving parameters of the shared model, and updating the initial image classification model by using an exponential moving average to obtain a final image classification model; wherein the shared model further comprises a self-labeling module, the self-labeling module sets a pseudo label based on a class obtained by the shared model, and constructs a cross-entropy loss between the class obtained by the shared model and the corresponding pseudo label, and updates the shared model parameters by using the cross-entropy loss.

[0010] In some embodiments of the present application, the samples of the local dataset are subjected to twice random augmentation, and the random augmentation at least comprises one or more combined operations of spatial transformation cropping, rotation, saturation adjustment, contrast adjustment, hue adjustment, color adjustment, brightness adjustment and grayscale adjustment.

[0011] In some embodiments of the present application, the contrast loss between the first feature vector and the second feature vector is constructed, and the contrast loss adopts a normalized temperature cross-entropy loss.

[0012] In some embodiments of the present application, the calculation formula of the contrast loss is:

[0013]

[0014] wherein, represents the contrastive loss; i, j represent the first view and the second view, respectively; z i j represents the first feature vector and the second feature vector, respectively; sim(z i j ) represents the similarity measure of the first view and the second view; τ is a temperature factor; M represents the number of samples in the local data set; m represents the serial number of samples in the local data set.

[0015] In some embodiments of the application, the initial model is trained using the local data set, and a clustering loss is constructed; the calculation formula of the clustering loss is:

[0016]

[0017] wherein, represents the clustering loss; x represents a single sample in the local data set x c ; represents a single nearest neighbor image sample in the set of adjacent samples N x of x; q(·) represents a preset function; <·> represents a dot product operator symbol; λ represents a weight; k represents a cluster; p k represents the probability of being assigned to cluster k.

[0018] In some embodiments of the application, the parameters of the shared model are received, and the initial image classification model is updated using an exponential moving average, and the calculation formula is:

[0019]

[0020] wherein, q c represents the initial image classification model parameter of client c; q g represents the shared model parameter; t represents the tth round of aggregation of the shared model parameter; μ represents a preset threshold; ξ represents the weight of the initial image classification model parameter and the shared model parameter in the update, respectively.

[0021] In some embodiments of the application, further comprising:

[0022] calculating the model divergence of the initial image classification model during training, when the model divergence is greater than the preset threshold, the client updates using the weight of the shared model; when the model divergence is less than or equal to the preset threshold, the client updates using the combination of its initial image classification model and the weight of the shared model.

[0023] ​​In some embodiments of the present application, samples with a confidence greater than a preset confidence threshold are selected based on the preset confidence threshold, and pseudo labels are set for the corresponding samples based on the categories obtained by the shared model. A cross-entropy loss is constructed between the categories obtained by the shared model and the corresponding pseudo labels, and the cross-entropy loss calculation formula is:

[0024]

[0025] wherein L self represents the cross-entropy loss; x represents a single sample in a dataset x g of the global server; sigma represents the preset confidence threshold; p(x) represents the output of the shared model; represents the pseudo label of sample x; H(·) represents the standard cross-entropy loss on the pseudo label.

[0026] In another aspect, the present application provides a federated unsupervised image classification method based on semantic clustering, characterized in that the method is performed on a client side and comprises the following steps:

[0027] obtaining an image to be classified;

[0028] inputting the image into an image classification model obtained by the federated unsupervised image classification model training method based on semantic clustering according to any one of the above embodiments to obtain the category of the image.

[0029] In another aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0030] The present application has at least the following beneficial effects:

[0031] The application provides a federated unsupervised image classification model training method and device based on semantic clustering, comprising: obtaining a local data set of a client to construct a training set of a model. An initial model is obtained, including a semantic clustering model and a pre-trained encoder network. In the training of the encoder network, random data enhancement is used as a pre-task of contrast learning, and the local data set is randomly enhanced twice to generate two views, which are input into the initial encoder network to extract feature vectors. A normalized temperature cross-entropy loss function is used for contrast learning to train the encoder network. The feature vectors of the local data set are extracted by using the trained encoder network, a nearest neighbor semantic clustering method is used, prior knowledge obtained by contrast learning is integrated into a clustering loss function, samples are classified according to feature similarity, and unsupervised learning is realized. A shared model is constructed based on the weighted aggregation of model parameters of each client. Considering the non-independent and identically distributed characteristics of the data of each client, a dynamic updating mechanism is designed, and the initial image classification model is updated by using the shared model parameter according to the weight divergence degree of the client model. In view of the clustering error problem existing in the clustering process, a self-labeling module is set in the shared model, highly confident prediction samples are used to correct the clustering error, and the classification performance of the model is improved. The image classification model trained based on the unsupervised learning method provided by the application has high accuracy, scalability and is suitable for non-independent and identically distributed scenes.

[0032] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0033] To those skilled in the art, it will be understood that the objects and advantages of the application realized by the application are not limited to the above specific description, and the above and other objects realized by the application will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0034] The drawings described herein are used to provide further understanding of the application, constitute a part of the application, and do not constitute a limitation of the application. In the drawings:

[0035] Figure 1 The figure is a schematic diagram of the steps of the federated unsupervised image classification model training method based on semantic clustering in an embodiment of the application.

[0036] Figure 2 The figure is a schematic diagram of the structure flow of the federated unsupervised image classification model training method based on semantic clustering in an embodiment of the application. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0038] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0039] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0040] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0041] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0042] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0043] To address the issues of poor performance and inapplicability to non-independent and identically distributed scenarios in existing federated unsupervised learning methods, this invention provides a training method for federated unsupervised image classification models based on semantic clustering, such as... Figure 1 As shown, this method is executed on each client and includes the following steps S101 to S104:

[0044] Step S101: Obtain the local dataset. The local dataset contains multiple samples, and each sample contains one image.

[0045] Step S102: Obtain the initial model, which includes a semantic clustering model and a pre-trained encoder network. Specifically, the samples from the local dataset are randomly augmented twice to generate a first view and a second view; the first view and the second view... Figure OneThe initial encoder network is input to the same dataset, and a first feature vector and a second feature vector are extracted. The initial encoder network is trained using a local dataset, and a contrast loss is constructed between the first and second feature vectors to obtain a trained encoder network. The local dataset is input into the semantic clustering model in batches, and the encoder network is used to extract feature vectors for corresponding samples. Based on a preset neural network, the Top-K nearest neighbors of the corresponding samples are extracted from the feature vectors. The vector values ​​of the corresponding samples belonging to different clusters are calculated using a preset Softmax function to obtain the category of the corresponding samples.

[0046] Step S103: Train the initial model using the local dataset and construct a clustering loss. Iterate the parameters of the initial model using the clustering loss to obtain the initial image classification model.

[0047] Step S104: Send the model parameters of the initial image classification model to the global server to generate a shared model. The shared model is obtained by the global server through weighted aggregation of the initial image classification model parameters from each client. Receive the parameters of the shared model and update the initial image classification model using an exponential moving average to obtain the final image classification model. The shared model also includes a self-labeling module, which sets pseudo-labels for the categories obtained based on the shared model and constructs a cross-entropy loss between the categories obtained based on the shared model and the corresponding pseudo-labels. The cross-entropy loss is then used to update the shared model parameters.

[0048] This invention proposes a federated unsupervised learning framework based on semantic clustering, relying on traditional distributed learning architectures and deep neural network frameworks for model training. The deep neural network framework can be PyTorch, TensorFlow, etc.

[0049] like Figure 2 The diagram shown is an overall flowchart of the training method for a federated unsupervised image classification model based on semantic clustering.

[0050] In step S101, the local dataset of each client is obtained, and local training is performed on each client.

[0051] In step S102, each client obtains an initial model from the global server, which includes a semantic clustering model and a pre-trained encoder network.

[0052] First, the initial encoder network is trained. In the unsupervised learning scenario, since there is no available label, a precondition must be defined to determine which samples are the same class. Therefore, data augmentation is used to constrain the model prediction to be unaffected by noise, while based on contrastive learning, a general model representation is learned by maximizing the consistency between augmented image samples on each client. The local dataset or randomly sampled small batch samples are used to perform two random augmentations on the samples to generate the first view and the second view.

[0053] In some embodiments, the random augmentation employed by the present application at least includes one or more combined operations of spatial transformation cropping, rotation, adjusting saturation, adjusting contrast, adjusting hue, adjusting color, adjusting brightness, and adjusting grayscale. At the same time, the same augmentation strategy is employed for each client in the federated learning framework, and the best strategy is not searched on different datasets.

[0054] The first view and the second view Figure One are input into the initial encoder network to generate semantic feature representations of the two views, and then the semantic feature representations of the two views are transformed through a preset fully connected layer nonlinear function to generate a first feature vector and a second feature vector, respectively. The first feature vector and the second feature vector are used to calculate the contrastive loss.

[0055] According to observations, image samples with similar high-order features will be closer, so in some embodiments, the normalized temperature cross-entropy is used as the contrastive loss of the present application, and the contrastive loss calculation formula of each client can be represented as formula (1):

[0056]

[0057] Wherein, sim(z i ,z j is calculated as formula (2):

[0058]

[0059] In formula (1) and (2), represents the contrastive loss; i, j represent the first view and the second view, respectively; z i ,z j represent the first feature vector and the second feature vector, respectively; sim(z i ,z j represents the similarity measure of the first view and the second view; τ is the temperature factor; M represents the number of samples in the local dataset or randomly sampled small batch samples; m represents the sequence number of the sample in the local dataset or randomly sampled small batch samples; [·] T represents the vector transpose.

[0060] The initial encoder network is trained using a local dataset or randomly sampled mini-batch samples, and the parameters of the initial encoder network are iterated using contrastive loss to obtain a trained encoder network.

[0061] The trained encoder network is applied to the semantic clustering model to map sample images into feature representations. Specifically, the local dataset is input into the semantic clustering model in batches. The encoder network extracts the feature vectors of the corresponding samples. Based on a preset neural network, the Top-K nearest neighbors of the corresponding samples are extracted from the feature vectors. A preset function is used to calculate the vector values ​​of the input image samples to which they belong in different clusters using a preset Softmax function. The category of the corresponding sample is finally determined based on the vector values.

[0062] In step S103, the initial model is trained using the local dataset, and a clustering loss is constructed. The parameters of the initial model are iterated using the clustering loss to obtain the initial image classification model.

[0063] In some embodiments, N is defined x As the neighboring sample set of the input image sample, the probability of an image sample being assigned to the corresponding cluster is denoted as p. Therefore, the clustering loss of each client is calculated as shown in formula (3):

[0064]

[0065] Where, p k The calculation formula is shown in formula (4):

[0066]

[0067] In formulas (3) and (4), The clustering loss is represented by x; x represents the local dataset x. c A single sample in; N represents the neighboring sample set of x. x In the image, a single nearest neighbor image sample is used; q(·) represents the preset function; <·> represents the dot product operator; λ represents the weight; k represents the cluster; p k This represents the probability that x is assigned to cluster k.

[0068] In step S104, inspired by the divergence of weights caused by non-independent and identically distributed data, this invention designs a dynamic update mechanism to dynamically update the parameters of the initial image classification model based on the weight divergence.

[0069] In one round of training, after training the initial image classification model for each client based on the local dataset, each client sends the parameters of its own model to the global server, where the global server... Figure 2The cloud server is used to represent the global server. After obtaining the parameters of the initial image classification model of each client, the global server performs weighted aggregation to build a shared model, and sends the parameters of the shared model to each client. Each client receives the parameters of the shared model, and updates the initial image classification model using the exponential moving average to obtain the final image classification model.

[0070] In some embodiments, the initial image classification model is updated using the exponential moving average, and the calculation formula is shown in formula (5):

[0071]

[0072] wherein q c represents the initial image classification model parameter of the client c; q g represents the shared model parameter; t represents the tth round of shared model parameter aggregation; μ represents a preset threshold; and ξ represents the weight of the initial image classification model parameter and the shared model parameter in the update.

[0073] The model divergence of the initial image classification model during training is measured by Since the data of each client is unbalanced and not independent and identically distributed, the initial image classification model parameter q c of each client may have a large variance. In the present application, a preset threshold is defined, and the model divergence of the initial image classification model during training is calculated. When the model divergence is greater than the preset threshold, the client updates using the weight of the shared model; when the model divergence is less than or equal to the preset threshold, the client updates using the combination of the initial image classification model and the weight of the shared model.

[0074] In the semantic clustering process, each image sample has K neighbors, so it is inevitable to assign some samples to the wrong clusters. In the present application, a self-labeling module is designed in the shared model to tend to assign samples with highly confident predictions to the correct clusters. Thus, by using these highly confident prediction samples, the errors in the clustering process can be reliably corrected, further improving the performance of the image classification model.

[0075] Specifically, after the global server performs weighted aggregation on the initial image classification model parameters of each client, a preset confidence threshold is defined, and samples with a confidence greater than the preset confidence threshold are selected. For each trusted sample, a pseudo label is assigned to the predicted cluster, and the cross-entropy loss between the class obtained based on the shared model and the corresponding pseudo label is constructed to update the shared model parameters.

[0076] In some embodiments, the calculation formula of the cross-entropy loss is shown in formula (6):

[0077]

[0078] wherein, L self represents cross-entropy loss; x represents a single sample in the dataset x of the global server; sigma represents a preset threshold; p(x) represents the output of the shared model; g represents a pseudo label of the sample x; represents a pseudo label of the sample x; the argmax function is used to change the probability distribution into a single-hot distribution; H(·) represents a standard cross-entropy loss on the pseudo label.

[0079] The self-labeling module allows the shared model to correct itself, thereby improving the performance of the shared model.

[0080] The application further provides a federated unsupervised image classification method based on semantic clustering, which comprises the following steps S201-S202:

[0081] Step S201: obtaining an image to be classified.

[0082] Step S202: inputting the image into an image classification model obtained by the federated unsupervised image classification model training method based on semantic clustering as described above, so as to obtain the class of the image.

[0083] The application further provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the federated unsupervised image classification model training method based on semantic clustering and the federated unsupervised image classification method based on semantic clustering.

[0084] Corresponding to the above method, the application further provides a device comprising a computer device, the computer device comprising a processor and a memory, the memory having computer instructions stored therein, and the processor being configured to execute the computer instructions stored in the memory, so that the device implements the steps of the method as described above when the computer instructions are executed by the processor.

[0085] The application further provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the federated unsupervised image classification model training method based on semantic clustering and the federated unsupervised image classification method based on semantic clustering.

[0086] In summary, the application provides a federated unsupervised image classification model training method and classification method and device based on semantic clustering, comprising: obtaining a local data set of a client to construct a training set of the model. An initial model is obtained, including a semantic clustering model and a pre-trained encoder network. In the training of the encoder network, random data augmentation is used as a pre-task of contrast learning, and the local data set is randomly augmented twice to generate two views, which are input into the initial encoder network to extract feature vectors; and a normalized temperature cross-entropy loss function is used for contrast learning to train the encoder network. The feature vectors of the local data set are extracted using the trained encoder network, a nearest neighbor semantic clustering method is used, prior knowledge obtained by contrast learning is integrated into a clustering loss function, samples are classified according to feature similarity, and unsupervised learning is realized. A shared model is constructed based on the weighted aggregation of the model parameters of each client, considering the non-independent and identically distributed characteristics of the data of each client, a dynamic updating mechanism is designed, and the initial image classification model is updated using the shared model parameters according to the weight divergence degree of the client model; in view of the clustering error problem existing in the clustering process, a self-labeling module is set in the shared model, highly confident predicted samples are used to correct the clustering error, and the classification performance of the model is improved. The image classification model trained based on the unsupervised learning method provided by the application has high accuracy, scalability and is suitable for non-independent and identically distributed scenarios.

[0087] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The choice of hardware or software implementation is a matter of design choice and will depend on the particular application and constraints of the technology solution. Those of skill in the art will be able to make such design choices without departing from the scope of the application. When implemented in hardware, the hardware can be implemented in, for example, electronic circuitry, an application specific integrated circuit (ASIC), suitable firmware, a plug-in card, a function card, etc. When implemented in software, the elements of the application are the program or code segments that implement the necessary tasks. The program or code segments can be stored in a machine readable medium, or transmitted by carrier waves in a data signal over a transmission medium or communication link.

[0088] It is to be understood that the application is not limited to the particular configurations and processes described hereinabove and shown in the drawings. For simplicity, detailed descriptions of well-known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the application.

[0089] In this disclosure, features described and / or illustrated with respect to one implementation can be used in the same manner or in an analogous manner in one or more other implementations, and / or in combination with or in place of features of other implementations.

[0090] The above descriptions are only the preferred embodiments of the present application, not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of the present application.

Claims

1. A method for training a federated unsupervised image classification model based on semantic clustering, characterized in that, The method is executed at each client and includes the following steps: Obtaining a local dataset, the local dataset containing a plurality of samples, each sample containing an image; Obtaining an initial model, the initial model including a semantic clustering model and a pre-trained encoder network; wherein the samples of the local dataset are subjected to twice random augmentation to generate a first view and a second view; the first view and the second view are input into the initial encoder network to extract a first feature vector and a second feature vector; the initial encoder network is trained using the local dataset, and a contrast loss between the first feature vector and the second feature vector is constructed to obtain a trained encoder network; the local dataset is input into the semantic clustering model in batches, and the encoder network is used to extract a feature vector of a corresponding sample; Top-K nearest neighbor samples of the corresponding sample are extracted from the feature vector based on a preset neural network, and vector values of different clusters to which the corresponding sample belongs are calculated by a preset Softmax function to obtain a class of the corresponding sample; The initial model is trained using the local dataset, and a clustering loss is constructed; the parameters of the initial model are iterated using the clustering loss to obtain an initial image classification model; The model parameters of the initial image classification model are sent to a global server to generate a shared model; the shared model is obtained by the global server according to the weighted aggregation of the initial image classification model parameters of each client; the parameters of the shared model are received, and the initial image classification model is updated using an exponential moving average to obtain a final image classification model; wherein the shared model further includes a self-labeling module, the self-labeling module sets a pseudo label based on the class obtained by the shared model, and constructs a cross-entropy loss between the class obtained by the shared model and the corresponding pseudo label, and the shared model parameters are updated using the cross-entropy loss. 2.The method of claim 1, wherein, The samples of the local dataset are subjected to twice random augmentation, and the random augmentation at least includes one or more combined operations of spatial transformation cropping, rotation, saturation adjustment, contrast adjustment, hue adjustment, color adjustment, brightness adjustment, and grayscale adjustment. 3.The method of claim 1, wherein, The contrast loss between the first feature vector and the second feature vector is constructed, and the contrast loss uses a normalized temperature cross-entropy loss.

4. The method of claim 3, wherein the method further comprises: The calculation formula of the contrast loss is: wherein, represents the contrast loss; i, j represent the first view and the second view, respectively; z i ,z j represent the first feature vector and the second feature vector, respectively; sim(z i ,z j ) represents a similarity measure of the first view and the second view; τ is a temperature factor; M represents the number of samples in the local dataset; m represents the serial number of samples in the local dataset. 5.The method of claim 1, wherein, The initial model is trained using the local dataset, and a clustering loss is constructed; the calculation formula of the clustering loss is: wherein, represents the clustering loss; x represents a single sample in the local dataset x c ; represents a single nearest neighbor image sample in the set of neighboring samples N x of x; q(·) represents a pre-set function; <·> represents a dot product operator symbol; λ represents a weight; k represents a cluster; p k represents a probability of being assigned to cluster k. 6.The method of claim 1, wherein, The parameters of the shared model are received, and the initial image classification model is updated using an exponential moving average, and the calculation formula is: wherein q c denotes the initial image classification model parameters of the client c; q g denotes the shared model parameters; t denotes the t-th round of the shared model parameter aggregation; μ denotes a preset threshold; and ξ denotes the weight of the initial image classification model parameters and the shared model parameters in the update, respectively.

7. The method of claim 6, wherein the method further comprises: Further comprising: Calculating the model divergence of the initial image classification model during training, when the model divergence is greater than the preset threshold, the client updates using the weight of the shared model; When the model divergence is less than or equal to the preset threshold, the client updates using the combination of the initial image classification model and the weight of the shared model. 8.The method of claim 1, wherein, Based on a preset confidence threshold, samples with a confidence greater than the preset confidence threshold are selected, and a pseudo label is set for the corresponding sample based on a category obtained by the shared model, a cross-entropy loss between the category obtained by the shared model and the corresponding pseudo label is constructed, and the cross-entropy loss calculation formula is: where L self represents the cross-entropy loss; x represents a single sample in the dataset x g of the global server; σ represents the pre-set confidence threshold; p(x) represents the output of the shared model; represents the pseudo label of sample x; H(·) represents the standard cross-entropy loss on the pseudo label.

9. A federated unsupervised image classification method based on semantic clustering, characterized in that, The method is executed on a client and includes the following steps: Obtaining an image to be classified; Inputting the image into the image classification model obtained by the federated unsupervised image classification model training method based on semantic clustering in any one of claims 1 to 8 to obtain a category of the image.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of the method in any one of claims 1 to 9.

Images