Training method of image classification model, image classification method and related device

Abstract

The application discloses a training method of an image classification model, an image classification method and related devices. The method comprises the following steps: inputting source domain images and target domain images of multiple source domains into an encoder respectively to map to the same feature space, so as to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images; all the source domain images are labeled, the number of the target domain images is multiple, and at least part of the target domain images are unlabeled; performing cross-domain reconstruction on the source domain features and the target domain features by using a generator to obtain cross-domain reconstructed images, performing intra-domain reconstruction on the source domain features and the target domain features by using the generator to obtain intra-domain reconstructed images, and determining a reconstruction loss; setting a training label for the unlabeled target domain images and determining a classification loss based on the source domain features and the target domain features by using a classifier; and adjusting parameters of the encoder, the generator and the classifier based on the reconstruction loss and the classification loss. The above scheme can improve the accuracy of image classification of the image classification model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a training method of an image classification model, an image classification method and related devices. BACKGROUND

[0002] With the rise of the field of computer vision, image classification as an important branch of the field of computer vision is increasingly valued. When there are a large number of images to be classified, a trained image classification model is usually obtained by training a model, and the trained image classification model is used for image classification to set a corresponding label for an input image to determine the class of the image to improve the classification efficiency. In the prior art, the training of the image classification model is extremely dependent on accurate labeled images. Once the labeled images are insufficient, the training effect of the image classification model will be poor, and the training effect of the image classification model directly determines the accuracy of image classification. Therefore, how to improve the accuracy of image classification by the image classification model has become a problem to be solved. SUMMARY

[0003] The technical problem solved by the present application is to provide a training method of an image classification model, an image classification method and related devices, which can improve the accuracy of image classification by the image classification model.

[0004] To solve the above technical problem, the first aspect of the present application provides a training method of an image classification model, which comprises: inputting source domain images and target domain images of a plurality of source domains into an encoder to map the source domain images and the target domain images to the same feature space, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images; wherein all the source domain images are labeled, and the number of the target domain images is multiple and at least part of them are unlabeled; using a generator to perform cross-domain reconstruction on the source domain features and the target domain features to obtain cross-domain reconstructed images, using the generator to perform intra-domain reconstruction on the source domain features and the target domain features to obtain intra-domain reconstructed images, and determining a reconstruction loss based on the cross-domain reconstructed images, the intra-domain reconstructed images, the source domain images and the target domain images; using a classifier to set a training label for the unlabeled target domain images based on the source domain features and the target domain features and determine the confidence of the training label, and determining a classification loss based on the confidence of the training label; and adjusting parameters of the encoder, the generator and the classifier based on the reconstruction loss and the classification loss to obtain a trained image classification model.

[0005] To solve the above technical problems, the second aspect of the present application provides an image classification method, which comprises: obtaining a to-be-processed image; inputting the to-be-processed image into an encoder to map the to-be-processed image to a feature space, and obtaining a to-be-processed feature corresponding to the to-be-processed image; inputting the to-be-processed feature into a classifier to set a target label for the to-be-processed image based on the to-be-processed feature, and obtaining a classification result of the to-be-processed image; wherein the encoder and the classifier belong to an image classification model, and the image classification model is obtained after being trained based on the method of the first aspect.

[0006] To solve the above technical problems, the third aspect of the present application provides an electronic device, which comprises a memory and a processor coupled with each other, wherein the memory stores program data, and the processor invokes the program data to execute the method of the first aspect or the second aspect.

[0007] To solve the above technical problems, the fourth aspect of the present application provides a computer readable storage medium, which stores program data, and the program data is executed by a processor to implement the method of the first aspect or the second aspect.

[0008] The above scheme inputs source domain images and target domain images from multiple source domains into an encoder respectively to map the source domain images and the target domain images to the same feature space, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images, wherein all the source domain images are labeled, the number of the target domain images is multiple and at least part of the target domain images are unlabeled, so that in the training process, the information of the source domain can be used to assist the learning of the target domain, to improve the training efficiency and accuracy, and the images of the target domain do not need to be all labeled, to reduce the training difficulty, the source domain features and the target domain features are reconstructed from the source domain to the target domain and from the target domain to the source domain by using a generator, to obtain cross-domain reconstructed images, the source domain features and the target domain features are reconstructed within the domain from the source domain to the source domain and from the target domain to the target domain by using the generator, to obtain intra-domain reconstructed images, the reconstruction loss is determined by using the cross-domain reconstructed images, the intra-domain reconstructed images, the source domain images and the target domain images, the training label is set for the unlabeled target domain images and the confidence of the training label is determined based on the source domain features and the target domain features by using a classifier, the classification loss corresponding to the classifier is determined based on the confidence of the training label, and then the encoder, the generator and the classifier are adjusted based on the reconstruction loss and the classification loss, so that the cross-domain reconstructed images and the intra-domain reconstructed images generated by the generator correct the encoder, so that the encoder can map the source domain images and the target domain images of multiple source domains to the same feature space, so as to migrate the source domain features containing rich labels from different source domains to the target domain, to reduce the difference between the source domain features and the target domain features, and then the classifier can obtain the training label with higher confidence when setting the training label based on the source domain features and the target domain features, to improve the accuracy and robustness of the image classification model after training. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0010] Figure 1 is a flowchart of an embodiment of the training method of the image classification model of the present application;

[0011] Figure 2 is a flowchart of another embodiment of the training method of the image classification model of the present application;

[0012] Figure 3 is an application scenario diagram of an embodiment corresponding to the training method of the image classification model of the present application;

[0013] Figure 4This is a flowchart illustrating one embodiment of the image classification method of this application;

[0014] Figure 5 This is a schematic diagram of the structure of one embodiment of the electronic device of this application;

[0015] Figure 6 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0017] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0018] The image classification model training method provided in this application is used to train the image classification model, which is used to classify images and assign labels to unlabeled images to determine the image category. Therefore, the image classification model training method and the execution subject of the image classification method provided in this application are processors capable of calling images.

[0019] Please see Figure 1 , Figure 1 This is a flowchart illustrating one implementation method of the image classification model training method of this application. The method includes:

[0020] S101: Input source domain images and target domain images from multiple source domains into the encoder respectively, so that the source domain images and target domain images are mapped to the same feature space, and obtain the source domain features corresponding to the source domain images and the target domain features corresponding to the target domain images. Here, all source domain images are labeled, and the number of target domain images is multiple and at least some are unlabeled.

[0021] Specifically, the source domain images come from different source domains, all the source domain images are labeled, the number of target domains is one, at least part of the target domain images in the target domain are unlabeled, the source domain images and the target domain images from multiple source domains are respectively input into an encoder to map the source domain images and the target domain images to the same feature space, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images, wherein all the source domain images are labeled, the number of target domain images is multiple and at least part of them are unlabeled, so that in the training process, the information of the source domain can be used to assist the learning of the target domain, improve the training efficiency and accuracy, and the images of the target domain do not need to be all labeled, reducing the training difficulty.

[0022] In an application mode, the source domain images and the target domain images of multiple source domains are respectively input into the same encoder to map the source domain images and the target domain images to the same feature space for feature extraction, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images.

[0023] In another application mode, the source domain images of multiple source domains are collectively input into the encoder corresponding to the source domain, and the target domain images are input into the encoder corresponding to the target domain, to map the source domain images and the target domain images to the same feature space for feature extraction, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images, wherein the encoder corresponding to the source domain and the encoder corresponding to the target domain share the weights.

[0024] In an application scenario, based on the preset categories of image classification, the source domain images from multiple source domains are obtained from the open source labeled data sets such as ImageNet and Caltech 256, and the target domain images are all unlabeled or part of them are labeled, the source domain images are input into the source domain encoder, and the target domain images are input into the target domain encoder, to map the source domain images and the target domain images to the same feature space by the source domain encoder and the target domain encoder, to obtain source domain features corresponding to the source domain images and target domain features, wherein the weights of the source domain encoder and the target domain encoder are shared, and the same feature space is collectively referred to as a latent transformation layer.

[0025] S102: using the generator to perform cross-domain reconstruction on the source domain features and the target domain features to obtain cross-domain reconstructed images, using the generator to perform intra-domain reconstruction on the source domain features and the target domain features to obtain intra-domain reconstructed images, and determining a reconstruction loss based on the cross-domain reconstructed images, the intra-domain reconstructed images, the source domain images and the target domain images.

[0026] Specifically, the generator is used to perform source domain to target domain and target domain to source domain cross-domain reconstruction on the source domain features and the target domain features to obtain cross-domain reconstructed images, the generator is used to perform source domain to source domain and target domain to target domain intra-domain reconstruction on the source domain features and the target domain features to obtain intra-domain reconstructed images, and the cross-domain reconstructed images, the intra-domain reconstructed images, the source domain images and the target domain images are used to determine the reconstruction loss, wherein the intra-domain includes source domain to source domain and target domain to target domain, and the cross-domain includes source domain to target domain and target domain to source domain.

[0027] In an application mode, the generator is used to perform cross-domain reconstruction on the source domain features and the target domain features respectively to obtain cross-domain reconstructed images corresponding to the source domain and the target domain respectively, and the generator is used to perform intra-domain reconstruction on the source domain features and the target domain features respectively to obtain intra-domain reconstructed images corresponding to the source domain and the target domain respectively.

[0028] In another application mode, the generator includes a source domain generator corresponding to the source domain and a target domain generator corresponding to the target domain, the source domain generator is used to perform cross-domain reconstruction on the target domain features to obtain a cross-domain reconstructed image corresponding to the source domain, the target domain generator is used to perform cross-domain reconstruction on the source domain features to obtain a cross-domain reconstructed image corresponding to the target domain, the source domain generator is used to perform intra-domain reconstruction on the source domain features to obtain an intra-domain reconstructed image corresponding to the source domain, and the target domain generator is used to perform intra-domain reconstruction on the target domain features to obtain an intra-domain reconstructed image corresponding to the target domain.

[0029] Further, a cross-domain source domain loss is determined based on the difference between the cross-domain reconstructed image corresponding to the source domain and the target domain image, a cross-domain target domain loss is determined based on the difference between the cross-domain reconstructed image corresponding to the target domain and the source domain image, an intra-domain source domain loss is determined based on the difference between the intra-domain reconstructed image corresponding to the source domain and the source domain image, an intra-domain target domain loss is determined based on the difference between the intra-domain reconstructed image corresponding to the target domain and the target domain image, and the reconstruction loss is determined by using the cross-domain source domain loss, the cross-domain target domain loss, the intra-domain source domain loss and the intra-domain target domain loss.

[0030] S103: The classifier is used to set a training label for the unlabeled target domain image based on the source domain features and the target domain features and determine the confidence of the training label, and a classification loss is determined based on the confidence of the training label.

[0031] Specifically, the classifier is used to set a training label for the unlabeled target domain image based on the source domain features and the target domain features and determine the confidence of the training label, and the classification loss corresponding to the classifier is determined based on the confidence of the training label.

[0032] In an application mode, the source domain features and the target domain features are input into a classifier, the source domain features correspond to labeled tags of source domain images, the classifier sets training tags for unlabeled target domain images based on similarities between the source domain features and the target domain features, the confidence of the training tags is positively correlated with the similarities, and a classification loss corresponding to the classifier is determined based on the confidence of the training tags corresponding to all the unlabeled target domain images.

[0033] In another application mode, the image categories correspond to a plurality of preset tags, the source domain features and the target domain features are input into a classifier, the source domain features correspond to labeled tags of source domain images, the classifier determines probabilities that the unlabeled target domain images belong to each preset tag based on the source domain features and the target domain features, and a preset tag with the highest probability is taken as a training tag, the confidence of the training tag is positively correlated with the probability, and a classification loss corresponding to the classifier is determined based on the confidence of the training tags corresponding to all the unlabeled target domain images.

[0034] S104: Adjusting parameters of the encoder, the generator and the classifier based on the reconstruction loss and the classification loss to obtain a trained image classification model.

[0035] Specifically, the encoder, the generator and the classifier are adjusted based on the reconstruction loss and the classification loss, so that the generator generates cross-domain reconstructed images and intra-domain reconstructed images to correct the encoder, and the encoder can map source domain images and target domain images of a plurality of source domains to the same feature space, thereby migrating source domain features containing rich labels from different source domains to the target domain, reducing the difference between the source domain features and the target domain features, and further enabling the classifier to obtain training tags with higher confidence when setting the training tags based on the source domain features and the target domain features, thereby improving the accuracy and robustness of the image classification model after training in image classification.

[0036] In an application mode, the encoder, the generator and the classifier are adjusted based on the reconstruction loss and the classification loss until a preset convergence condition is met, to obtain a trained image classification model, the preset convergence condition is related to the reconstruction loss and the classification loss, and when the sum of the reconstruction loss and the classification loss is less than a loss threshold, the training process is ended to obtain the trained image classification model.

[0037] In another application mode, the encoder, the generator and the classifier are adjusted based on the reconstruction loss and the classification loss until a preset convergence condition is met, to obtain a trained image classification model, the preset convergence condition is related to the reconstruction loss, the classification loss and the number of adjustments, and when the sum of the reconstruction loss and the classification loss is less than a loss threshold and the number of adjustments exceeds a number threshold, the training process is ended to obtain an image classification model with higher stability.

[0038] The above scheme inputs source domain images and target domain images from multiple source domains into an encoder respectively to map the source domain images and the target domain images to the same feature space, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images, wherein all the source domain images are labeled, the number of the target domain images is multiple and at least part of the target domain images are unlabeled, so that in the training process, the information of the source domain can be used to assist the learning of the target domain, to improve the training efficiency and accuracy, and the images of the target domain do not need to be all labeled, to reduce the training difficulty, the generator is used to perform cross-domain reconstruction of the source domain to the target domain and the target domain to the source domain based on the source domain features and the target domain features, to obtain cross-domain reconstructed images, the generator is used to perform intra-domain reconstruction of the source domain to the source domain and the target domain to the target domain based on the source domain features and the target domain features, to obtain intra-domain reconstructed images, the reconstruction loss is determined based on the cross-domain reconstructed images, the intra-domain reconstructed images, the source domain images and the target domain images, the classifier is used to set a training label for the unlabeled target domain images based on the source domain features and the target domain features and to determine the confidence of the training label, the classification loss corresponding to the classifier is determined based on the confidence of the training label, and then the encoder, the generator and the classifier are adjusted based on the reconstruction loss and the classification loss, so that the cross-domain reconstructed images and the intra-domain reconstructed images generated by the generator correct the encoder, so that the encoder can map the source domain images and the target domain images of multiple source domains to the same feature space, so that the source domain features containing rich labels from different source domains are migrated to the target domain, to reduce the difference between the source domain features and the target domain features, and then the classifier can obtain a training label with higher confidence when setting the training label based on the source domain features and the target domain features, to improve the accuracy and robustness of the image classification model after the training is completed.

[0039] Please refer to Figure 2 , Figure 2 is a flowchart of another embodiment of the training method of the image classification model of the present application, and the method comprises:

[0040] S201: input source domain images and target domain images of multiple source domains into an encoder respectively to map the source domain images and the target domain images to the same feature space, to obtain source domain features corresponding to the source domain images and target domain features corresponding to the target domain images, wherein all the source domain images are labeled, and the number of the target domain images is multiple and at least part of the target domain images are unlabeled.

[0041] Specifically, please refer to Figure 3 , Figure 3 is an application scenario diagram of an embodiment of the training method of the image classification model of the present application, the source domain images come from different source domains, all the source domain images are labeled, the number of the target domain is one, and the target domain includes labeled samples and unlabeled samples.

[0042] Further, the source domain images are denoted as xi , the labeled samples in the target domain image are denoted as x T-L , the unlabeled samples are denoted as x T-UL , the target domain image x T = T-UL ∪x T-L , the source domain image x i is input to an encoder , and the source domain feature is obtained as The target domain image x T is input to the encoder E T , and the target domain feature is obtained as E T ( T ).

[0043] S202: Cross-domain reconstruction of the source domain feature and the target domain feature is performed by using a generator to obtain a cross-domain reconstructed image, and intra-domain reconstruction of the source domain feature and the target domain feature is performed by using the generator to obtain an intra-domain reconstructed image, and a reconstruction loss is determined based on the cross-domain reconstructed image, the intra-domain reconstructed image, the source domain image, and the target domain image.

[0044] Specifically, the generator includes a source domain generator and a target domain generator, wherein the source domain generator corresponds to the source domain, and the target domain generator corresponds to the target domain.

[0045] Further, the cross-domain reconstruction of the source domain feature and the target domain feature by using the generator to obtain the cross-domain reconstructed image includes: cross-domain reconstruction of the target domain feature by using the source domain generator to obtain a first cross-domain source domain reconstructed image, and cross-domain reconstruction of the source domain feature by using the source domain generator to obtain a first cross-domain target domain reconstructed image; cross-domain reconstruction of the target domain feature by using the target domain generator to obtain a second cross-domain source domain reconstructed image, and cross-domain reconstruction of the source domain feature by using the target domain generator to obtain a second cross-domain target domain reconstructed image; wherein the cross-domain reconstructed image includes the first cross-domain source domain reconstructed image, the second cross-domain source domain reconstructed image, the first cross-domain target domain reconstructed image, and the second cross-domain target domain reconstructed image.

[0046] Specifically, please refer again to Figure 3 , the cross-domain reconstruction of the target domain feature E ST (x T ) by using the source domain generator G T obtains a first cross-domain source domain reconstructed image Here, the first cross-domain source domain reconstructed image is marked as a first type of source domain image after cross-domain reconstruction, the cross-domain reconstruction of the source domain feature by using the source domain generator G ST obtains a first cross-domain target domain reconstructed image Here, the first cross-domain target domain reconstructed image x′ tThe first type of target domain image after cross-domain reconstruction is marked as x.

[0047] Further, the target domain generator G TS is used to perform cross-domain reconstruction on the target domain feature E T (x T ), to obtain a second cross-domain source domain reconstructed image x′ i =G TS (E T (x T )),wherein the second cross-domain source domain reconstructed image x′ i is marked as the second type of source domain image after cross-domain reconstruction, and the target domain generator G TS is used to perform cross-domain reconstruction on the source domain feature , to obtain a second cross-domain target domain reconstructed image , wherein the second cross-domain target domain reconstructed image is marked as the second type of target domain image after cross-domain reconstruction. Therefore, when performing image cross-domain migration reconstruction, the features extracted by the feature space are used as input, the mapping relationship between the source domain and the target domain and the mapping relationship between the target domain and the source domain are learned, and the generality and robustness of the features from multiple source domains or target domains in the latent transformation layer corresponding to the feature space are ensured through stable mapping relationship.

[0048] It can be understood that the generator is used to perform intra-domain reconstruction on the source domain feature and the target domain feature to obtain an intra-domain reconstructed image, including: the source domain generator is used to perform intra-domain reconstruction on the source domain feature to obtain an intra-domain source domain reconstructed image, and the target domain generator is used to perform intra-domain reconstruction on the target domain feature to obtain an intra-domain target domain reconstructed image; wherein the intra-domain reconstructed image includes the intra-domain source domain reconstructed image and the intra-domain target domain reconstructed image.

[0049] Specifically, referring again to Figure 3 , the source domain generator G ST is used to perform intra-domain reconstruction on the source domain feature , to obtain an intra-domain source domain reconstructed image , and the target domain generator G TS is used to perform intra-domain reconstruction on the target domain feature E T (x T ), to obtain an intra-domain target domain reconstructed image x″ t =G TS (E T (x T )). Therefore, when performing image intra-domain reconstruction, the features extracted by the feature space are used as input, the correlation relationship between the source domain and the source domain and the correlation relationship between the target domain and the target domain are learned, and the robustness during intra-domain reconstruction in the latent transformation layer corresponding to the feature space is ensured through stable correlation relationship.

[0050] Further, after the source domain generator and the target domain generator perform the intra-domain reconstruction and the cross-domain reconstruction on the source domain features and the target domain features, a reconstruction loss can be finally determined, and the encoder can be optimized based on the reconstruction loss to make the encoder effectively reduce the difference in data distribution in different data domains when mapping the source domain images and the target domain images to the feature space for feature extraction, so that the features of the target domain and the source domain share similar distribution, and the training efficiency and the accuracy of the training label setting are improved.

[0051] It can be understood that, based on the cross-domain reconstructed images, the intra-domain reconstructed images, the source domain images and the target domain images, the reconstruction loss is determined, including: determining an intra-domain reconstruction loss based on the difference between the intra-domain source domain reconstructed image and the source domain image, and the difference between the intra-domain target domain reconstructed image and the target domain image; determining a cross-domain reconstruction loss based on the difference between the first cross-domain source domain reconstructed image and the second cross-domain source domain reconstructed image and the source domain image, and the difference between the first cross-domain target domain reconstructed image and the second cross-domain target domain reconstructed image and the target domain image; and determining the reconstruction loss based on the cross-domain reconstruction loss and the intra-domain reconstruction loss.

[0052] Specifically, the difference between the intra-domain source domain reconstructed image x" i and the source domain image x i , and the difference between the intra-domain target domain reconstructed image x" t and the target domain image x T are compared to determine an intra-domain reconstruction loss L r1 .

[0053] Further, the first cross-domain source domain reconstructed image x' is the first type of source domain image after cross-domain reconstruction, the second cross-domain source domain reconstructed image x' i is the second type of source domain image after cross-domain reconstruction, the first cross-domain target domain reconstructed image x' t is the first type of target domain image after cross-domain reconstruction, and the second cross-domain target domain reconstructed image x" is the second type of target domain image after cross-domain reconstruction. The difference between the first cross-domain source domain reconstructed image x' and the second cross-domain source domain reconstructed image x' i and the source domain image x i , and the difference between the first cross-domain target domain reconstructed image x' t and the second cross-domain target domain reconstructed image x" and the target domain image x T are compared to determine a cross-domain reconstruction loss L r2 . Therefore, the intra-domain reconstruction loss L r1 and the cross-domain reconstruction loss L r2 can be obtained to improve the accuracy of the reconstruction loss L R . R .

[0054] Optionally, after determining the reconstruction loss based on the cross-domain reconstructed image, the intra-domain reconstructed image, the source domain image and the target domain image, the method comprises: dividing the first cross-domain source domain reconstructed image by using the source domain discriminator to obtain source domain true samples with a similarity to the source domain image greater than a similarity threshold and source domain false samples with a similarity to the source domain image less than or equal to the similarity threshold; wherein the source domain discriminator corresponds to the source domain generator; determining a first discrimination loss corresponding to the source domain discriminator based on the source domain true samples and the source domain false samples; dividing the second cross-domain target domain reconstructed image by using the target domain discriminator to obtain first target domain true samples with a similarity to the target domain image greater than the similarity threshold and having been labeled, second target domain true samples with a similarity to the target domain image greater than the similarity threshold and having not been labeled, and target domain false samples with a similarity to the target domain image less than or equal to the similarity threshold; wherein the target domain discriminator corresponds to the target domain generator; determining a second discrimination loss corresponding to the target domain discriminator based on the first target domain true samples, the second target domain true samples and the target domain false samples.

[0055] Specifically, the source domain corresponds to a source domain discriminator D S , the first cross-domain source domain reconstructed image X S is divided by using the source domain discriminator D according to a similarity to the source domain image x i to obtain source domain true samples with a similarity to the source domain image x i greater than a similarity threshold and source domain false samples with a similarity to the source domain image x i less than or equal to the similarity threshold, and a first discrimination loss corresponding to the source domain discriminator is determined by using the source domain true samples and the source domain false samples. The above process is represented by the following formula:

[0056]

[0057] wherein L GAN_S represents the first discrimination loss.

[0058] Further, the target domain corresponds to a target domain discriminator D T , the second cross-domain target domain reconstructed image X T is divided by using the target domain discriminator D according to a similarity to the target domain image x T to obtain target domain true samples with a similarity to the target domain image greater than the similarity threshold and target domain false samples with a similarity to the target domain image less than or equal to the similarity threshold, wherein the target domain true samples having been labeled are first target domain true samples and the target domain true samples having not been labeled are second target domain true samples, and a second discrimination loss corresponding to the target domain discriminator is determined by using the first target domain true samples, the second target domain true samples and the target domain false samples. The above process is represented by the following formula:

[0059]

[0060] wherein, L GAN_T represents the second discriminant loss. Therefore, when the image cross-domain migration reconstruction is performed, the source domain discriminator and the target domain discriminator are used for classification processing, and the first discriminant loss and the second discriminant loss are obtained, so that the encoder can be adjusted based on the first discriminant loss and the second discriminant loss, and the accuracy of feature extraction by the encoder is improved.

[0061] Further, after determining the second discriminant loss corresponding to the target domain discriminator based on the first target domain true sample, the second target domain true sample and the target domain false sample, the method comprises: obtaining a source domain consistency loss based on the difference of the first cross-domain source domain reconstructed image and the second cross-domain source domain reconstructed image with respect to the source domain image.

[0062] Specifically, the source domain consistency loss is obtained based on the difference of the first cross-domain source domain reconstructed image x and the second cross-domain source domain reconstructed image x i ′ with respect to the source domain image x i . The above process is represented by the following formula:

[0063]

[0064] wherein, L s The source domain consistency loss indicates that the source domain image is a more accurate and stable image labeled. When calculating the total loss of the image classification model, the source domain consistency loss is added to the total loss, the weight of the source domain in the total loss is improved, and the stability of the entire image classification model is improved.

[0065] S203: Using the classifier to obtain the probability of each unlabeled target domain image belonging to multiple types of preset labels based on the source domain feature and the target domain feature, and determining the training label corresponding to each unlabeled target domain image from the preset labels.

[0066] Specifically, the target domain image corresponds to multiple image categories, each image category corresponds to a preset label, and the classifier is used to obtain the probability of each unlabeled target domain image belonging to multiple types of preset labels based on the source domain feature and the target domain feature.

[0067] Further, the training label corresponding to each unlabeled target domain image is determined from the preset labels, for example, the preset label with the highest probability is taken as the training label.

[0068] S204: Determining the information entropy corresponding to the training label based on the probability, and determining the confidence corresponding to the training label using the information entropy.

[0069] Specifically, the information entropy corresponding to the training label is determined based on the probability, for an event, the smaller the probability of occurrence, the greater the amount of information, the greater the probability of occurrence, the smaller the amount of information, so as to determine the information entropy by the probability, use the information entropy as the uncertainty of the training label, the greater the entropy, the greater the uncertainty, therefore, the corresponding weight of each unlabeled sample is given, the weight is used as the evaluation standard to evaluate the confidence of the sample, thereby improving the accuracy of the confidence of the training label.

[0070] S205: based on the confidence, all unlabeled target domain images are divided into high-confidence samples and low-confidence samples, and the high-confidence samples and the corresponding training labels are marked as labeled target domain images.

[0071] Specifically, referring again to Figure 3 , the confidence of the training label corresponding to the unlabeled target domain image is compared with the confidence threshold, the target domain image greater than the confidence threshold is taken as the high-confidence sample, and the target domain image less than or equal to the confidence threshold is taken as the low-confidence sample.

[0072] S206: based on the high-confidence samples and the low-confidence samples, the classification loss corresponding to the classifier is determined.

[0073] Specifically, the updated labeled sample using the high-confidence sample is denoted as x' T_L , the updated unlabeled sample using the low-confidence sample is denoted as x' T_UL , the updated target domain image is x' T =x' T_L ∪x' T_UL , the classification loss corresponding to the classifier is determined using the updated target domain image x' T and the source domain image x i , and the above process is represented by the following formula:

[0074]

[0075] Wherein, L C is the classification loss, the high-confidence sample and the corresponding training label are taken as the labeled target domain image, thereby updating the labeled sample for the next training, and in the training process, the target domain image is converted from the unlabeled sample to the labeled sample, thereby improving the training efficiency of the image classification model.

[0076] S207: based on the reconstruction loss and the classification loss, the parameters of the encoder, the generator and the classifier are adjusted to obtain the trained image classification model.

[0077] Specifically, based on the reconstruction loss, the classification loss, the first discriminator loss, the second discriminator loss and the source domain consistency loss, the parameters of the encoder, the generator and the classifier are adjusted to obtain the trained image classification model.

[0078] It can be understood that, based on the reconstruction loss L R , the classification loss L c , the first discriminator loss L GAN_S , the second discriminator loss L GAN_T and the source domain consistency loss L s , the total loss L of the image classification model can be determined, and by constraining the total loss L until the preset convergence condition is reached, the trained image classification model can be obtained. The above process is represented by the following formula:

[0079] L = min (L R + L c + L GAN_S + L GAN_T + L s ) (5)

[0080] Optionally, in determining the total loss L, the reconstruction loss L R , the classification loss L c , the first discriminator loss L GAN_S , the second discriminator loss L GAN_T and the source domain consistency loss L s may also be weighted and summed, which is not specifically limited in the present application, so that the parameters of the image classification model are adjusted by multiple losses, the accuracy of the image classification model is improved, and the accuracy of the image classification model in image classification is improved.

[0081] In the present embodiment, when reconstructing in the image domain, the features extracted by the feature space are used as input to learn the correlation between the source domain and the source domain and the target domain and the target domain, and when reconstructing in the image cross-domain migration, the features extracted by the feature space are used as input to learn the mapping relationship between the source domain and the target domain and the target domain and the source domain, and the universality and robustness of the features from multiple source domains or target domains in the latent transformation layer corresponding to the feature space are ensured by stabilizing the correlation and mapping relationship, and when reconstructing in the image cross-domain migration, the source domain discriminator and the target domain discriminator are used for classification processing, and the first discriminator loss and the second discriminator loss are obtained, so that the encoder can be adjusted based on the first discriminator loss and the second discriminator loss, the accuracy of the encoder in feature extraction is improved, the source domain consistency loss is added to the total loss when calculating the total loss of the image classification model, the weight of the source domain in the total loss is improved, and the stability of the entire image classification model is improved. The parameters of the image classification model are adjusted by multiple losses, the accuracy of the image classification model is improved, and the accuracy of the image classification model in image classification is improved.

[0082] Please refer to Figure 4 , Figure 4 is a flowchart of an embodiment of an image classification method of the present application, which comprises the following steps:

[0083] S401: obtaining an image to be processed.

[0084] Specifically, an image to be processed is obtained, and image preprocessing is performed on the image to be processed to match the image to be processed with an encoder. The preprocessing includes at least one of denoising, scaling and padding.

[0085] S402: inputting the image to be processed into the encoder to map the image to be processed to a feature space, to obtain a to-be-processed feature corresponding to the image to be processed.

[0086] Specifically, the image to be processed is inputted into the encoder to map the image to be processed to the feature space for feature extraction by the encoder, to obtain the to-be-processed feature corresponding to the image to be processed.

[0087] S403: inputting the to-be-processed feature into a classifier to set a target label for the image to be processed based on the to-be-processed feature, to obtain a classification result of the image to be processed.

[0088] Specifically, the to-be-processed feature is inputted into the classifier to determine a target label corresponding to the image to be processed from preset labels according to the to-be-processed feature, to obtain a classification result corresponding to the image to be processed. The encoder and the classifier belong to an image classification model, which is obtained after being trained based on the method in any of the above embodiments. For details, please refer to the detailed description of the method embodiments above, which will not be repeated here.

[0089] Please refer to Figure 5 , Figure 5 is a structural diagram of an embodiment of an electronic device of the present application, which comprises a memory 501 and a processor 502 coupled with each other, wherein the memory 501 stores program data (not shown in the figure), and the processor 502 invokes the program data to implement the method in any of the above embodiments. For details, please refer to the detailed description of the method embodiments above, which will not be repeated here.

[0090] Please refer to Figure 6 , Figure 6 is a structural diagram of an embodiment of a computer readable storage medium of the present application, which stores program data 600. When the program data 600 is executed by a processor, the method in any of the above embodiments is implemented. For details, please refer to the detailed description of the method embodiments above, which will not be repeated here.

[0091] It should be noted that the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0092] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0093] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0094] The above is only the embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A training method for an image classification model, characterized in that, The method includes: Multiple source domain images and target domain images are input into an encoder to map the source domain images and the target domain images to the same feature space, thereby obtaining source domain features corresponding to the source domain images and target domain features corresponding to the target domain images; wherein, all the source domain images are labeled, and the number of target domain images is multiple, and at least some are unlabeled; A cross-domain reconstruction image is obtained by using a generator to reconstruct the source and target domain features across domains. Then, an intra-domain reconstruction image is obtained by using the generator to reconstruct the source and target domain features within the same domain. A reconstruction loss is determined based on the cross-domain reconstruction image, the intra-domain reconstruction image, the source domain image, and the target domain image. The generator includes a source domain generator and a target domain generator. The cross-domain reconstruction image is obtained through the following steps: using the source domain generator to reconstruct the target domain features across domains to obtain a first cross-domain source domain reconstruction image; using the source domain generator to reconstruct the source domain features across domains to obtain a first cross-domain target domain reconstruction image; using the target domain generator to reconstruct the target domain features across domains to obtain a second cross-domain source domain reconstruction image; and using the target domain generator to reconstruct the source domain features across domains to obtain a second cross-domain target domain reconstruction image. The cross-domain reconstruction image includes the first cross-domain source domain reconstruction image, the second cross-domain source domain reconstruction image, the first cross-domain target domain reconstruction image, and the second cross-domain target domain reconstruction image. Using a classifier based on the source domain features and the target domain features, training labels are set for the unlabeled target domain image and the confidence of the training labels is determined. Based on the confidence of the training labels, the classification loss is determined. The parameters of the encoder, generator, and classifier are adjusted based on the reconstruction loss and the classification loss to obtain the trained image classification model.

2. The training method for the image classification model according to claim 1, characterized in that, The step of using a generator to reconstruct the source domain features and target domain features within the domain to obtain an intra-domain reconstructed image includes: The source domain features are reconstructed within the domain using the source domain generator to obtain an in-domain source domain reconstructed image; the target domain features are reconstructed using the target domain generator to obtain an in-domain target domain reconstructed image. The reconstructed image within the domain includes the reconstructed source domain image and the reconstructed target domain image.

3. The training method for the image classification model according to claim 2, characterized in that, Based on the cross-domain reconstructed image, the intra-domain reconstructed image, the source domain image, and the target domain image, the reconstruction loss is determined, including: Based on the difference between the reconstructed source domain image and the source domain image, and the difference between the reconstructed target domain image and the target domain image, the intra-domain reconstruction loss is determined. Based on the differences between the first cross-domain source domain reconstructed image and the second cross-domain source domain reconstructed image relative to the source domain image, and the differences between the first cross-domain target domain reconstructed image and the second cross-domain target domain reconstructed image relative to the target domain image, the cross-domain reconstruction loss is determined; The reconstruction loss is determined based on the cross-domain reconstruction loss and the intra-domain reconstruction loss.

4. The training method for the image classification model according to claim 1, characterized in that, After determining the reconstruction loss based on the cross-domain reconstructed image, the intra-domain reconstructed image, the source domain image, and the target domain image, the process includes: The first cross-domain source domain reconstructed image is divided using a source domain discriminator to obtain true source domain samples with a similarity greater than a similarity threshold to the source domain image, and false source domain samples with a similarity less than or equal to the similarity threshold to the source domain image; wherein, the source domain discriminator corresponds to the source domain generator; Based on the true samples and false samples in the source domain, determine the first discrimination loss corresponding to the source domain discriminator; The second cross-domain target domain reconstructed image is divided using a target domain discriminator to obtain a first target domain true sample that is labeled and has a similarity greater than a similarity threshold with the target domain image, a second target domain true sample that is unlabeled and has a similarity greater than a similarity threshold with the target domain image, and target domain false samples that have a similarity less than or equal to a similarity threshold with the target domain image; wherein, the target domain discriminator corresponds to the target domain generator; Based on the first target domain true samples, the second target domain true samples, and the target domain false samples, the second discrimination loss corresponding to the target domain discriminator is determined.

5. The training method for the image classification model according to claim 4, characterized in that, After determining the second discriminant loss corresponding to the target domain discriminator based on the first target domain true samples, the second target domain true samples, and the target domain false samples, the process includes: Based on the differences between the first cross-domain source domain reconstructed image and the second cross-domain source domain reconstructed image relative to the source domain image, the source domain consistency loss is obtained; The step of adjusting the parameters of the encoder, the generator, and the classifier based on the reconstruction loss and the classification loss to obtain the trained image classification model includes: Based on the reconstruction loss, the classification loss, the first discriminant loss, the second discriminant loss, and the source domain consistency loss, the parameters of the encoder, the generator, and the classifier are adjusted to obtain the trained image classification model.

6. The training method for the image classification model according to claim 1, characterized in that, The step of using a classifier to set training labels for unlabeled target domain images based on the source domain features and the target domain features, and determining the confidence level of the training labels, and determining the classification loss based on the confidence level of the training labels, includes: The classifier is used to obtain the probability that each unlabeled target domain image belongs to multiple types of preset labels based on the source domain features and the target domain features, and the training label corresponding to each unlabeled target domain image is determined from the preset labels; The information entropy corresponding to the training label is determined based on the probability, and the confidence level corresponding to the training label is determined using the information entropy; Based on the confidence level, all unlabeled target domain images are divided into high-confidence samples and low-confidence samples, and the high-confidence samples and their corresponding training labels are labeled as labeled target domain images; Based on the high-confidence samples and the low-confidence samples, the classification loss corresponding to the classifier is determined.

7. An image classification method, characterized in that, The method includes: Obtain the image to be processed; The image to be processed is input into the encoder so that the image to be processed is mapped to the feature space to obtain the features to be processed corresponding to the image to be processed. The features to be processed are input into a classifier, so that the classifier sets a target label for the image to be processed based on the features to be processed, thereby obtaining a classification result for the image to be processed; wherein, the encoder and the classifier belong to an image classification model, and the image classification model is obtained after training based on the method of any one of claims 1-6.

8. An electronic device, characterized in that, include: A memory and a processor are coupled to each other, wherein the memory stores program data, and the processor invokes the program data to perform the method as described in any one of claims 1-6 or 7.

9. A computer-readable storage medium storing program data thereon, characterized in that, When the program data is executed by the processor, it implements the method as described in any one of claims 1-6 or 7.

Images