Task migration based federated unsupervised image classification model training method, classification method and device

Abstract

The application provides a task migration-based federated unsupervised image classification model training method, a classification method and equipment: a local data set containing complete labels and partial labels is acquired; an initial neural network model is acquired, which comprises an adaptive incremental layer and a deep migration module; the adaptive incremental layer is a fully connected layer added after each convolutional layer of the initial neural network model; a competition mechanism of domain classification and domain confusion is constructed in the deep migration module, and a knowledge distillation method is used to save information between related categories; the local data set is used to train the model, a joint loss of domain classification loss, domain confusion loss and soft label loss is constructed, and the task priority is determined according to the average precision of the classification task, so that an initial image classification model is obtained; a shared model is constructed based on the parameters of each client model, and the initial image classification model is updated according to the shared model parameters. The image classification model provided by the application has high precision and can retain personalized local models.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a federated unsupervised image classification model training method and classification method based on task migration and equipment. BACKGROUND

[0002] With the rapid increase of Internet of Things devices, mobile devices, smartphones, etc. are connected through an open communication network platform, which opens up new possibilities for improving the quality of service of emerging applications through data sharing. The federated learning technology is effective for solving privacy protection training in the Internet of Things scenario, such as user habit prediction, personalized recommendation, and wireless network optimization, and its success is partly due to the training of a large amount of labeled data on multiple clients. The existing joint learning method cannot achieve generalization performance through a large amount of training on labeled data. However, in the real Internet of Things scenario, due to user habits or lack of sufficient professional knowledge to correctly label data, the client's data is always accompanied by very few labels. For example, a mobile fitness application can correct the user's body posture, but in this case, the user may not be able to evaluate whether his own posture is qualified, so he cannot provide the corresponding data label for the application. Therefore, the performance advantage brought by large labeled data sets is at the cost of cost and limited applications.

[0003] Traditional federated learning usually relies on a large amount of labeled data to improve model performance, but in a semi-supervised environment, due to the lack of training data labels, traditional distributed model optimization algorithms are no longer applicable. Therefore, federated semi-supervised learning is introduced, which combines semi-supervised learning methods with the federated learning framework, iteratively optimizes the shared model through local supervised training and knowledge transfer between devices. Existing methods have achieved some success in corresponding application fields, but still have the following two problems: first, traditional federated semi-supervised learning usually implements knowledge transfer between labeled data and unlabeled data in the device local data set based on semi-supervised algorithms, ignoring the transfer needs between devices. Second, since federated learning aims to train a general shared model, it is difficult to balance model consistency and personalization while preserving the unique task requirements of each client. SUMMARY

[0004] In view of this, the embodiments of the present application provide a federated unsupervised image classification model training method and classification method based on task migration and equipment to eliminate or improve one or more defects in the prior art, solve the problem of low model precision, inability to realize knowledge transfer between labeled data and unlabeled data, and inability to preserve personalized local models in the existing federated semi-supervised learning method in a sparse label environment.

[0005] In one aspect, the present application provides a task migration-based federated unsupervised image classification model training method, characterized in that the method is executed on each client and comprises the following steps:

[0006] obtaining a local data set, wherein the local data set comprises source data with complete category labels and target data with partial category labels, and each data contains an image sample;

[0007] obtaining an initial neural network model, wherein the initial neural network model comprises an adaptive incremental layer and a deep migration module; the adaptive incremental layer is a fully connected layer added after each convolutional layer of the initial neural network model; the image samples of the local data set are input into the initial neural network model in batches for feature extraction, a preset domain classifier is used to determine whether the corresponding image sample belongs to the source data or the target data, and a preset pre-mixing layer is used to align the domains through domain confusion to construct a domain confusion competition mechanism; a knowledge distillation method is used to calculate the relationship values between each category using the source data, and the average of the relationship values is taken as the soft label of the target data having relevance with the corresponding source data to output the category of the corresponding image sample;

[0008] the initial neural network is trained using the local data set, domain classification loss, domain confusion loss and soft label loss are constructed, a joint loss is constructed according to the domain classification loss, the domain confusion loss and the soft label loss, the average precision of each classification task is calculated, the weight of each task in the loss function is determined according to the average precision, the parameters of the initial neural network model are iterated using the joint loss, and an initial image classification model is obtained;

[0009] the model parameters of the initial image classification model are sent to a global server to generate a shared model; the shared model is obtained by the global server according to the weighted aggregation of the initial image classification model parameters of each client; the parameters of the shared model are received, the initial image classification model is updated based on the adaptive incremental layer, and a final image classification model is obtained.

[0010] In some embodiments of the present application, the loss function of the domain classification loss is defined as a Softmax cross-entropy loss function, and the calculation formula is:

[0011]

[0012] wherein, represents the domain classification loss; d represents one data in the local data set D k ; y represents the image category determined by the initial neural network model; y d represents the category label of data d; p d represents the feature vector corresponding to the network output.

[0013] In some embodiments of the present application, the calculation formula of the domain confusion loss is:

[0014]

[0015] wherein L conf represents the domain confusion loss; d represents one data in the local data set D k ; p d represents the feature vector of the corresponding network output.

[0016] In some embodiments of the present application, the calculation formula of the soft label loss is:

[0017]

[0018] wherein L soft represents the soft label loss; d represents one data in the local data set D k ; y represents the image category determined by the initial neural network model; y soft represents the soft label of data d; q represents the network output after knowledge distillation.

[0019] In some embodiments of the present application, the domain classification loss, the domain confusion loss and the soft label loss are combined by weighting to construct a joint loss, and the calculation formula of the joint loss is:

[0020]

[0021] wherein L tra represents the joint loss; represents the domain classification loss; L conf represents the domain confusion loss; L soft represents the soft label loss.

[0022] In some embodiments of the present application, the average precision of each classification task is calculated, and the weight of each task in the loss function is determined according to the average precision, and further comprising:

[0023] The average precision of each classification task is calculated as a key performance indicator of the initial neural network model, and the key performance indicator is used as an index for measuring the weight of each task in the loss function, and the weight of each task is defined as:

[0024]

[0025] wherein λ is used to control the relative priority of each task; κ t represents the average precision of the task.

[0026] In some embodiments of the present application, a dynamic scaling cross-entropy loss is used instead of a cross-entropy loss to reduce the weight of simple classification tasks, and the calculation formula of the total image classification loss is:

[0027]

[0028] wherein L cla represents the total image classification loss; t represents a task in the total classification task T; d represents a data in the local data set D k . c represents a classification task containing class c.

[0029] In some embodiments of the present application, each client and the global server construct an image classification system, and the system defines an objective function by minimizing a total loss function, and the calculation formula of the objective function is:

[0030]

[0031] L k (ω)=η1L cla (X s , X t ; ω) + η2L tra (X s , X t ; ω);

[0032] wherein L total (ω) represents the objective function; D represents a set of all client local data sets; k represents a client in all clients K; D k represents the local data set of client k; L k (ω) represents the loss function of client k; η1 and η2 are used to balance multiple optimization objectives; L cla represents the total image classification loss; L tra represents the joint loss; X s represents the source data in the local data set of client k; X t represents the target data in the local data set of client k; ω represents the initial neural network model parameter.

[0033] In another aspect, the present application provides a task transfer-based federated unsupervised image classification method, which is characterized in that the method is executed at a client and comprises the following steps:

[0034] obtaining an image to be classified;

[0035] inputting the image into an image classification model obtained by the task transfer-based federated unsupervised image classification model training method according to any one of the above embodiments to obtain the class of the image.

[0036] In another aspect, the application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method according to any one of the above mentioned aspects.

[0037] The application has at least the following advantages:

[0038] The application provides a task migration-based federated unsupervised image classification model training method and device, a task migration-based federated unsupervised image classification model classification method and device. A local data set containing complete labels and partial labels is obtained to construct a source domain and a target domain. An initial neural network model is obtained, including an adaptive incremental layer and a deep migration module. The adaptive incremental layer adds a fully connected layer after each convolutional layer of the initial neural network model, retains the personalized local model of the client when the model is updated, and improves the generalization ability of the model. A competition mechanism of domain classification and domain confusion is constructed in the deep migration module, and a knowledge distillation method is used to realize domain confusion between complete label data and partial label data, reduce the distance between the source domain and the target domain, realize the training of partial label data, and design a soft label to adjust the information between classes, realize task migration, and effectively perform semi-supervised learning on the client. The local data set is used to train the model, a joint loss of domain classification loss, domain confusion loss and soft label loss is constructed, and a dynamic task loss is introduced to automatically adjust the weight between tasks and determine the task priority. Finally, an initial image classification model is trained. A shared model is constructed based on the model parameters of each client, and the initial image classification model is updated according to the shared model parameters. The image classification model trained by the method provided by the application has high precision and can retain the local personalized model.

[0039] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the specification as well as in the appended claims.

[0040] Those skilled in the art will appreciate that the objects and advantages of the application can be realized and attained by means summarized fully in the following description, and particularly pointed out in the appended claims. Accordingly, the application is not defined by the summary description but by the claims. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0042] Figure 1 The steps of the task migration-based federated unsupervised image classification model training method in an embodiment of the application are shown in the schematic diagram.

[0043] Figure 2 This is a schematic diagram of the structural flow of a federated unsupervised image classification model training method based on task transfer in one embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0045] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0046] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0047] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0048] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0049] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0050] To address the problems of low model accuracy, inability to achieve knowledge transfer between labeled and unlabeled data, and inability to preserve personalized local models in existing federated semi-supervised learning methods under sparse label environments, this invention provides a task-transfer-based federated unsupervised image classification model training method, such as... Figure 1 As shown, this method is executed on each client and includes the following steps S101 to S104:

[0051] Step S101: Obtain the local dataset, which includes source data with complete category labels and target data with partial category labels, each containing one image sample.

[0052] Step S102: obtaining an initial neural network model, wherein the initial neural network model comprises an adaptive incremental layer and a deep migration module; the adaptive incremental layer is an added fully connected layer after each convolutional layer of the initial neural network model; image samples of a local data set are input into the initial neural network model in batches for feature extraction, a preset domain classifier is used to determine that the corresponding image samples belong to source data or target data, and a preset pre-mixing layer is used to align the domains through domain confusion to construct a domain confusion competition mechanism; a knowledge distillation method is used to calculate the relationship values between each category using the source data, and the average of the relationship values is taken as the soft label of the target data having relevance with the corresponding source data to output the category of the corresponding image sample.

[0053] Step S103: training the initial neural network using the local data set, constructing a domain classification loss, a domain confusion loss and a soft label loss, constructing a joint loss according to the domain classification loss, the domain confusion loss and the soft label loss, calculating the average precision of each classification task, determining the weight of each task in the loss function according to the average precision, and using the joint loss to iteratively update the parameters of the initial neural network model to obtain an initial image classification model.

[0054] Step S104: sending the model parameters of the initial image classification model to a global server to generate a shared model; wherein the shared model is obtained by the global server according to the weighted aggregation of the initial image classification model parameters of each client; receiving the parameters of the shared model, updating the initial image classification model based on the adaptive incremental layer to obtain a final image classification model.

[0055] The application proposes a federated unsupervised learning framework based on domain and task migration, which relies on a traditional distributed learning architecture and a deep neural network framework for model training. The deep neural network framework can be selected from PyTorch, TensorFlow, etc.

[0056] In step S101, for example, it is assumed that a group of clients C={C1, C2,..., C k} and a global server G, each client has a local data set D k ={X s , X t}, wherein X s represents source data containing complete category labels, and X t represents target data containing only partial category labels.

[0057] Obtain the local data set D k corresponding to the client C k , which is used to train the local model of the client C k .

[0058] In step S102, an initial neural network model is obtained, and an exemplary VGG-Net deep neural network is selected. The initial neural network model includes an adaptive incremental layer and a deep migration module, wherein the adaptive incremental layer is used to retain the client personalized local model when updating the client local model according to the shared model parameters; and the deep migration module is used for domain confusion and task migration.

[0059] Specifically, the client C k inputs the local data set D k into the initial neural network model to perform feature extraction on each image sample. The present application adds a domain confusion layer before the last fully connected layer of the initial neural network, which is used to align the domain through domain confusion, wherein the source domain refers to source data containing complete class labels, and the target domain refers to target data containing only partial class labels. Each image sample is classified by using a preset domain classifier, that is, it is judged whether the image sample data is source data or target data, so as to learn the representation of the target domain with sparse labels from the source domain with complete labels.

[0060] In some embodiments, for any data (image sample) in the local data set, the loss function of the domain classification loss can be defined as a Softmax cross-entropy loss function, and the calculation formula is shown in formula (1):

[0061]

[0062] In formula (1), Ld represents the domain classification loss; d represents one data in the local data set D k ; y represents the image category determined by the initial neural network model; y d represents the class label of the data d; p d represents the feature vector corresponding to the network output.

[0063] Then, domain confusion is performed, and in some embodiments, the calculation formula of the domain confusion loss is shown in formula (2):

[0064]

[0065] In formula (2), L conf represents the domain confusion loss; d represents one data in the local data set D k ; and p d represents the feature vector corresponding to the network output.

[0066] Based on the domain classification loss and the domain confusion loss, the initial neural network model can be trained to classify source data and target data, and to confuse source data and target data. The domain classification loss and the domain confusion loss together constitute a domain confusion competition mechanism, the domain classification loss is used for better classification, and the domain confusion loss is used to maximize the confusion of source data and target data.

[0067] In order to better align the source data and the target data, a knowledge distillation method is used to save the information between related categories. The relationship values between each category are calculated using the source data, and the average of the relationship values is taken as the soft label of the target data having relevance to the corresponding source data. For example, for the image of a horse, the soft label is a probability vector, in which the values of the horse and the deer are closer, and the value of the bird image is greatly different.

[0068] In some embodiments, a soft label loss is used instead of a standard Softmax loss as the task transfer loss, and the calculation formula of the soft label loss is shown in formula (3):

[0069]

[0070] In formula (3), L soft represents the soft label loss; d represents one data in the local data set D k ; y represents the image category determined by the initial neural network model; y soft represents the soft label of the data d; and q represents the network output after knowledge distillation.

[0071] In step S103, the initial neural network is trained using the local data set, the domain classification loss, the domain confusion loss, and the soft label loss are constructed, the joint loss is constructed according to the domain classification loss, the domain confusion loss, and the soft label loss, the parameters of the initial neural network model are iterated using the joint loss, and the initial image classification model is obtained.

[0072] In some embodiments, the domain classification loss, the domain confusion loss, and the soft label loss are combined by weighting to construct the joint loss, and the calculation formula of the joint loss is shown in formula (4):

[0073]

[0074] In formula (4), L tra represents the joint loss; represents the domain classification loss; L conf represents the domain confusion loss; L soft represents the soft label loss.

[0075] By minimizing the joint loss, the domain and the classification task can be aligned at the same time, and the knowledge transfer between the source data containing complete category labels and the target data containing partial category labels can be realized.

[0076] Meanwhile, due to the lack of a considerable part of data labels in the local data set, the initial neural network will produce chaotic results. Inspired by the dynamic task priority, the application makes further improvements, using the key performance indicator as an indicator to measure the weight of each task in the loss function, to optimize the network performance.

[0077] In the application, the average precision of each classification task is calculated as the key performance indicator of the initial neural network model, and the key performance indicator is used as an indicator to measure the weight of each task in the loss function, and the weight of each task can be defined as shown in formula (5):

[0078]

[0079] In formula (5), λ is used to control the relative priority of each task; κ t represents the average precision of the task.

[0080] The dynamic scaling cross-entropy loss is used instead of the cross-entropy loss to reduce the weight of the simple classification task. For example, the average precision of the horse classification task is lower than that of the cat classification task, so the weight of the cat classification task is reduced, and the calculation formula of the total image classification loss is shown in formula (6):

[0081]

[0082] Where, L cla represents the total image classification loss; t represents a task in the total classification task T; d represents a data in the local data set D k ; L c represents the classification task containing class c.

[0083] In step S104, the initial image classification model parameters of each client are sent to the global server to generate a shared model. The shared model is obtained by the global server according to the weighted aggregation of the initial image classification model parameters of each client. Each client receives the parameters of the shared model, updates the initial image classification model based on the adaptive incremental layer to obtain the final image classification model.

[0084] Specifically, the adaptive incremental layer refers to adding a fully connected layer after each convolutional layer of the initial neural network model, and the elements of the layer are composed of 0 and 1. During the training process of the initial neural network model of each client, the parameters of the adaptive incremental layer remain fixed, and the weights of the incremental layer are learned by back propagation in the case of a given cross-entropy loss. When the global server updates the shared model, the fully connected layer is added after the convolutional layer to retain the local personalized model of each client.

[0085] In some embodiments, the image classification system is built based on each client and a global server, and the image classification system defines an objective function by minimizing a total loss function, and the objective function is calculated as shown in formula (7):

[0086]

[0087] In formula (7), L total (ω) represents the objective function; D represents a set of all client local data sets; k represents one of the K clients; D k represents the local data set of the client k; L k (ω) represents the loss function of the client k.

[0088] For each client local image classification model, the loss function is shown in formula (8):

[0089] L k (ω)=η1L cla (X s , X t ; ω) + η2L tra (X s , X t ; ω); (8)

[0090] In formula (8), L k (ω) represents the loss function of the client k; η1 and η2 are used to balance multiple optimization objectives; L cla represents the total image classification loss; L tra represents the joint loss; X s represents the source data in the local data set of the client k; X t represents the target data in the local data set of the client k; ω represents the initial neural network model parameters.

[0091] L cla is used to improve the performance of the model classifier, L tra is used to confuse the domain to better learn the representation of the target data, and η1 and η2 are used to balance multiple optimization objectives.

[0092] The application also provides a task transfer-based federated unsupervised image classification method, which is executed at a client and includes the following steps S201-S202:

[0093] Step S201: obtaining an image to be classified.

[0094] Step S202: inputting the image into an image classification model obtained by the task transfer-based federated unsupervised image classification model training method described above to obtain the class of the image.

[0095] The application further provides a computer readable storage medium, which stores a computer program, and the program, when executed by a processor, implements the steps of the task migration based federated unsupervised image classification model training method and the task migration based federated unsupervised image classification method.

[0096] Corresponding to the above method, the application further provides a device, which comprises a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the device implements the steps of the above method.

[0097] The application further provides a computer readable storage medium, which stores a computer program, and the program, when executed by a processor, implements the steps of the above edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), an internal memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0098] To sum up, the application provides a task migration based federated unsupervised image classification model training method, a classification method and a device, which comprise the following steps: obtaining a local data set containing complete labels and partial labels, and constructing a source domain and a target domain. An initial neural network model is obtained, which comprises an adaptive incremental layer and a deep migration module. The adaptive incremental layer is a fully connected layer added after each convolutional layer of the initial neural network model, which can retain the personalized local model of the client when the model is updated, and improve the generalization ability of the model. In the deep migration module, a competition mechanism of domain classification and domain confusion is constructed, and a knowledge distillation method is used to realize domain confusion between complete label data and partial label data, reduce the distance between the source domain and the target domain, realize the training of partial label data, and design a soft label to adjust the information between classes, realize task migration, and effectively perform semi-supervised learning on the client. The local data set is used to train the model, a joint loss of domain classification loss, domain confusion loss and soft label loss is constructed, a dynamic task loss is introduced to automatically adjust the weight between tasks, determine the priority of the task, and finally train an initial image classification model. A shared model is constructed based on the model parameters of each client, and the initial image classification model is updated according to the shared model parameters. The image classification model trained by the method provided by the application has high precision and can retain the personalized local model.

[0099] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, for example, the hardware can comprise an electronic circuit, an Application Specific Integrated Circuit (ASIC), a suitable firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are the program or code segments to perform a specific task. The program or code segments can be stored in a machine-readable medium, or transmitted by a carrier wave as data signals over a transmission medium or communication link.

[0100] It is to be understood that the application is not limited to the particular configurations and processes described herein and shown in the drawings, which can be varied in accordance with the particular needs of the application. For the sake of brevity, conventional techniques and methods related to making and using the application can not be described in detail herein. In the above embodiments, several specific steps are described and illustrated in order to provide a thorough understanding of the present application. However, the process of the present application can be practiced with less than all of the described specific steps, or with additional steps, or with different order of the steps, without departing from the spirit of the application.

[0101] In the present application, features described and / or illustrated in connection with one embodiment can be used in the same or a similar way or in conjunction with or in place of features of another embodiment.

[0102] The above description is intended to be illustrative and not restrictive. Many embodiments of the application will be apparent to those of skill in the art upon reviewing the above description. The scope of the application should, therefore, be determined not with reference to the above description, but instead with reference to the appended claims, along with their full scope of equivalents.

Claims

1. A method for training a federated unsupervised image classification model based on task transfer, characterized in that, The method is executed on each client and includes the following steps: Obtain a local dataset, which includes source data with complete category labels and target data with partial category labels, each containing one image sample; An initial neural network model is obtained, comprising an adaptive incremental layer and a deep transfer module. The adaptive incremental layer is a fully connected layer added after each convolutional layer of the initial neural network model. Image samples from the local dataset are input into the initial neural network model in batches for feature extraction. A preset domain classifier is used to determine whether the corresponding image sample belongs to the source data or the target data. A preset pre-confusion layer is used to align the domains through domain confusion and construct a domain confusion competition mechanism. A knowledge distillation method is used to calculate the relationship values ​​between each category using the source data, and the average value of each relationship value is taken as the soft label of the target data that is related to the corresponding source data, so as to output the category of the corresponding image sample. The initial neural network is trained using the local dataset to construct domain classification loss, domain confusion loss, and soft label loss. A joint loss is constructed based on the domain classification loss, the domain confusion loss, and the soft label loss. The average accuracy of each classification task is calculated, and the weight of each task in the loss function is determined based on the average accuracy. The parameters of the initial neural network model are iterated using the joint loss to obtain the initial image classification model. The model parameters of the initial image classification model are sent to the global server to generate a shared model; the shared model is obtained by the global server through weighted aggregation of the initial image classification model parameters of each client; the parameters of the shared model are received, and the initial image classification model is updated based on the adaptive incremental layer to obtain the final image classification model.

2. The method for training a federated unsupervised image classification model based on task transfer as described in claim 1, characterized in that, The loss function for the domain classification loss is defined as the Softmax cross-entropy loss function, and its calculation formula is: in, d represents the domain classification loss; d represents the local dataset D. k One of the data points; y represents the image category determined by the initial neural network model; y d p represents the category label of data d; d This represents the feature vector output by the corresponding network.

3. The method for training a federated unsupervised image classification model based on task transfer as described in claim 2, characterized in that, The formula for calculating the domain confusion loss is: Among them, L conf d represents the domain confusion loss; d represents the local dataset D. k One of the data points; p d This represents the feature vector output by the corresponding network.

4. The method for training a federated unsupervised image classification model based on task transfer as described in claim 3, characterized in that, The formula for calculating the soft tag loss is: Among them, L soft d represents the soft-label loss; d represents the local dataset D. k One of the data points; y represents the image category determined by the initial neural network model; y soft d represents the soft label for data d; q represents the network output after knowledge distillation.

5. The method for training a federated unsupervised image classification model based on task transfer as described in claim 4, characterized in that, The domain classification loss, the domain confusion loss, and the soft label loss are combined using a weighted average to construct a joint loss, which is calculated as follows: Among them, L tra Indicates the joint loss; L represents the domain classification loss; conf L represents the domain confusion loss; soft This represents the soft tag loss.

6. The method for training a federated unsupervised image classification model based on task transfer as described in claim 1, characterized in that, Calculating the average accuracy of each classification task, and determining the weight of each task in the loss function based on the average accuracy, further includes: The average accuracy of each classification task is calculated and used as a key performance indicator of the initial neural network model. This key performance indicator is then used to measure the weight of each task class in the loss function. The weight of each task class is defined as follows: Where λ is used to control the relative priority of each type of task; k t This indicates the average accuracy of the task.

7. The method for training a federated unsupervised image classification model based on task transfer as described in claim 6, characterized in that, A dynamically scaled cross-entropy loss is used instead of the traditional cross-entropy loss to reduce the weight of simple classification tasks. The total image classification loss is calculated as follows: Among them, L cla d represents the total image classification loss; t represents one task in the total classification task T; d represents the local dataset D. k One of the data points; L c This represents a classification task involving category c.

8. The method for training a federated unsupervised image classification model based on task transfer as described in claim 7, characterized in that, Each client and the global server construct an image classification system. The system defines an objective function by minimizing the total loss function. The objective function is calculated as follows: L k (ω)=η1L cla (X s ,X t ;ω)+η2L tra (X s ,X t ;oh); Among them, L total (ω) represents the objective function; D represents the set of all client local datasets; k represents one client in K; D k L represents the local dataset of client k; k (ω) represents the loss function of client k; η1 and η2 are used to balance multiple optimization objectives; L cla L represents the total image classification loss; tra X represents the joint loss; s X represents the source data in the local dataset of client k; t ω represents the target data in the local dataset of client k; ω represents the parameters of the initial neural network model.

9. A federated unsupervised image classification method based on task transfer, characterized in that, This method is executed on the client side and includes the following steps: Obtain the image to be classified; The image is input into an image classification model obtained by the task transfer-based federated unsupervised image classification model training method as described in any one of claims 1 to 8, to obtain the category of the image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 9.

Images