An image classification method and system based on general domain adaptation
By employing a general domain adaptation-based image classification method, the classifier and feature extractor are trained using open set cross-entropy loss, regularization loss, and neighborhood clustering loss. Combined with confidence-weighted conditional adversarial loss, the problem of insufficient generalization performance in existing methods is solved, and accurate identification of new class samples and optimization of classification boundaries are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-07
- Publication Date
- 2026-04-07
AI Technical Summary
Existing generalized domain adaptation methods have insufficient generalization performance on the target domain. In particular, the classification boundary of open set classifiers is not accurate enough, and they fail to effectively eliminate the domain differences between the source and target domains, resulting in unsatisfactory classification results.
We employ a general domain adaptation-based image classification method, using open set cross-entropy loss, consistency regularization loss, open set classification loss, and neighborhood clustering loss to train an open set classifier and feature extractor. We combine this with confidence-weighted conditional adversarial loss and optimize feature extraction and classification boundaries through pseudo-label clustering and data augmentation techniques.
It can accurately identify new types of samples in the target domain, reduce classification errors, improve the model's generalization ability in different scenarios, and requires no manual assistance.
Smart Images
Figure CN116403018B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of universal domain adaptation, and particularly relates to an image classification method and system based on universal domain adaptation. BACKGROUND
[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.
[0003] The impressive effect of image classification based on deep learning is highly dependent on rich labeled training data. However, collecting a large amount of annotated data in a new field will result in a high annotation cost. Unsupervised domain adaptation is a kind of method for transferring knowledge from a labeled training field (source domain) to a new field without label information (target domain). Traditional domain adaptation methods learn a closed set classification model in a static closed environment, and then classify and detect samples in the target domain, so the traditional adaptation method can only be applied to relatively simple closed data sets.
[0004] The existing work to solve the open class threshold setting is OVANet (derived from the paper OVANet: One-vs-All Network for Universal Domain Adaptation published by Saito K and Saenko K in 2021), which is composed of a feature extractor, a closed set classifier and an open set classifier. OVANet uses an open set classifier to automatically detect new class samples in the target domain, but the classification boundary of the open set classifier trained only by the source domain data is not accurate enough, resulting in reduced generalization performance on the target domain and unsatisfactory classification results. At the same time, the classification results of the closed set classifier for the common classes of the target domain also do not reach the expected standard.
[0005] Firstly, the classification boundary of open set classifier is not accurate enough, because each sub-classifier in open set classifier does not learn the essential features of each known class positive and negative class sufficiently. In the process of open set classifier training, only the source domain sample with the closest probability value is selected as the negative sample of the class (non-class sample) to train the sub-classifier, which is not accurate enough. Because the common class features extracted by the feature extractor of OVANet are relatively dispersed in the feature space, the common class samples are mostly uniformly dispersed around the boundary of the open space. Therefore, even if the source domain sample closest to the known class is selected as the negative class to train the boundary, when a new class sample appears in the open space, this boundary still cannot accurately screen out the new class sample. Secondly, the classification boundary trained by only using the source domain sample with the closest probability value is not comprehensive enough. For a class, the sample with the closest probability value mostly also comes from a certain class, and the classification boundary trained by using the sample of the same class as the negative class is also biased to distinguish the difference between the two classes, and has bias. And such selection of negative samples does not make full use of the remaining known class samples in the source domain.
[0006] Secondly, the target domain features extracted by the feature extractor of OVANet are relatively dispersed in the feature space, especially the new class (unknown class) features of the target domain are not concentrated in the open space. The labels of multiple new classes that do not appear in the source domain are all marked as unknown classes (new classes), and then the new class features extracted are relatively scattered in the open space. In some difficult data sets, there is also a situation that the new class features of the target domain are mixed with the features of the common classes.
[0007] Finally, the model trained only on the source domain does not consider the problem of distribution difference between the source domain and the target domain. Generally, there is an inherent universality problem in the training process of a deep model, that is, the model trained on one training data set (source domain) performs poorly on another test data set (target domain). One important reason for this lack of generalization performance is the domain difference in the data set. The closed set classifier and the open set classifier trained in the source domain in OVANet are directly applied to the target domain, and the domain difference causes their generalization performance to decrease. Therefore, eliminating domain difference and learning domain-invariant representation are the basis for improving the generalization ability of domain adaptation model. However, the conditional adversarial mechanism uses the feature extractor and the domain discriminator to form an adversarial mechanism, aligns the target domain data with the source domain by class, and then eliminates the domain difference of the data. However, this is difficult for general domain adaptation, because here we need to align only the common classes between the two domains and separate the private classes. Incorrectly aligning private classes will cause serious negative transfer, and then affect the new class detection and classification results. SUMMARY
[0008] In order to solve the technical problems in the background art, the application provides a general domain adaptation-based image classification method and system, which can well identify new class samples of different scenes without manual assistance.
[0009] In order to achieve the above-mentioned purpose, the application adopts the following technical solutions:
[0010] The first aspect of the application provides a general domain adaptation-based image classification method, which comprises:
[0011] Obtaining an image to be classified;
[0012] Inputting the image to be classified into a trained classification network to obtain the class of the image to be classified;
[0013] The classification network is used to train an open set classifier and a feature extractor by using source domain data through an open set cross-entropy loss, a consistency regularization loss, an open set classification loss and a neighborhood clustering loss.
[0014] Further, the feature extractor is used to extract the features of the input image.
[0015] The open set classifier is used to obtain the prediction probability of a certain image belonging to each class and the probability of not belonging to each class according to the features of the certain image.
[0016] Further, the open set classifier is composed of a plurality of binary classifiers, the number of binary classifiers is the number of classes of source domain data, and each binary classifier corresponds to a class in the source domain.
[0017] Further, if each binary classifier does not classify a certain image into its corresponding class, the image is a new class sample of the target domain.
[0018] If a certain image is classified into its corresponding class by a binary classifier, the image is classified into the class corresponding to the maximum prediction probability.
[0019] Further, the domain discriminator of the classification network is used for credibility-weighted conditional confrontation to solve the alignment problem between the source domain and the target domain.
[0020] Further, in the training process of the open set classifier, the samples in the target domain are clustered according to pseudo labels, and the clustering centers of each class are calculated; when the samples in the neighborhood of a certain clustering center are all of the same class, the clustering center is added to the next round of training as a representative sample; when the samples in the neighborhood of a certain clustering center are mixed with new class samples, the clustering center and the new class samples in the neighborhood are subjected to Mixup operation and then added to the next round of training.
[0021] Furthermore, the classification network uses data augmentation techniques to process the source domain data, and then uses the augmented source domain data for training the classification network.
[0022] A second aspect of the present invention provides an image classification system based on universal domain adaptation, comprising:
[0023] The image acquisition module is configured to acquire images to be classified.
[0024] The image classification module is configured to input the image to be classified into a trained classification network and obtain the category of the image to be classified.
[0025] The classification network uses source domain data to train an open-set classifier and a feature extractor through open-set cross-entropy loss, consistency regularization loss, open-set classification loss, and neighborhood clustering loss.
[0026] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the image classification method based on universal domain adaptation as described above.
[0027] A fourth aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the image classification method based on universal domain adaptation as described above.
[0028] Compared with the prior art, the beneficial effects of the present invention are:
[0029] This invention provides an image classification method based on universal domain adaptation, which can also identify new class samples in different scenes well without the need for manual assistance.
[0030] This invention provides an image classification method based on universal domain adaptation. It uses an open set classifier to obtain pseudo-labels for samples in the target domain and calculates the cluster center for each category. Clean cluster centers and cluster centers suspected of being new categories are selected and added to the training to strengthen the training of common categories and the learning of classification boundaries, and to correct misclassified new category samples.
[0031] This invention provides an image classification method based on general domain adaptation, which proposes a confidence-weighted conditional adversarial loss, where the weight represents the confidence that each sample is classified into a certain class, in order to prevent new classes from being incorrectly aligned with known classes and causing classification errors. Attached Figure Description
[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0033] Figure 1 This is a diagram of the classification network structure during the training phase of Embodiment 1 of the present invention;
[0034] Figure 2 This is a classification network structure diagram of the testing phase of Embodiment 1 of the present invention. Detailed Implementation
[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0036] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0038] Example 1
[0039] This embodiment provides an image classification method based on universal domain adaptation, which specifically includes the following steps:
[0040] Step 1: Obtain the image to be classified.
[0041] The image to be classified can be a data sample from the target domain dataset used for testing.
[0042] The images to be classified can be images containing animals. The categories of the images to be classified can include cats, dogs, and horses, etc.
[0043] It's important to note that source domain data is used to train the classification network; target domain data is used for testing. The differences between the source and target domains mainly fall into two categories: category differences. For example, the source domain might contain categories like cat, dog, and cow, while the target domain might contain categories like cat, dog, and horse. Here, "horse" is a newly introduced category in the target domain—this is a category difference. Distribution differences are also important. For instance, the sample images of cats in the source domain might be artistic depictions of cats, while the images of cats in the target domain might be real-world photographs of cats—this difference is a distribution difference. Cats and dogs belong to the common class; cows belong to the source domain's private class; and horses belong to the target domain's private class (also known as a new / unknown class). Data representing categories common to both the source and target domains form the common class dataset (open set). Data representing private classes in either the source or target domain form the private dataset; data present in the source domain but not in the target domain forms the source domain's private class dataset; and data present in the target domain but not in the source domain forms the target domain's private class dataset.
[0044] Step 2: Input the image to be classified into the trained classification network to obtain the category of the image.
[0045] like Figure 1 As shown, the classification network includes a feature extractor F and an open-set classifier O (composed of L). s The system consists of a binary classifier F and a domain discriminator D. During testing, only the feature extractor F and the open set classifier O are used; the domain discriminator D is used during the training process.
[0046] like Figure 2 As shown, during the testing process, the feature extractor F is used to extract features from the input image to be classified. Then, these features are input into the open-set classifier O, which obtains the predicted probability of each class and the probability of not belonging to each class. If each binary classifier classifies a sample as not belonging to that class, then the sample belongs to a new class in the target domain. If any binary classifier predicts a sample as belonging to that class, then the sample is classified into the class with the highest predicted probability. That is, the feature extractor is used to extract features from the input image; the open-set classifier is used to obtain the predicted probability of an image belonging to each class and the probability of not belonging to each class based on the features of that image. If no binary classifier classifies an image into its corresponding class, then the image is a new class sample in the target domain; if any binary classifier classifies an image into its corresponding class, then the image is classified into the class with the highest predicted probability. The classification network uses the source domain dataset (source domain data...) and source domain tags ) and target domain data Training is then performed. During training, the feature extractor F, the open-set classifier O, and the adversarial domain discriminator D are all used. The domain discriminator D of the classification network is used for confidence-weighted adversarial conditions, mainly addressing the alignment problem between the source and target domains; the open-set classifier is composed of L... s It consists of an integrated binary classifier, where L s It represents the number of categories in the source domain samples; open set classifiers primarily address the problems of new class detection and sample classification.
[0047] In this context, "conditional adversarial" means aligning the source and target domain samples by category, based on the predicted probabilities output by the open-set classifier O. For example, previously the target domain data was treated as a whole and aligned with all data in the source domain. Now, the goal is to align the target domain data with the corresponding categories in the source domain according to the categories output by the open-set classifier. That is, aligning category 1 in the source domain with category 1 in the target domain.
[0048] The classification network first uses data augmentation techniques (standard random pruning) to augment the source domain samples. The data is processed to obtain the source domain samples after data augmentation. Source domain samples Data augmentation of source domain samples and target domain samples The dataset is composed of two parts: the target domain data and the source domain data. The target domain data is unlabeled because there is a distribution difference between the two data. The target domain data is used for domain alignment. Then, a feature extractor based on a convolutional neural network (CNN) is used to extract features from each image in the dataset, which are then input into an open-set classifier and a domain discriminator. The open-set classifier and the feature extractor are trained using the source domain data and the data-augmented source domain data through cross-entropy loss, consistency regularization loss, open-set classification loss, and neighborhood clustering loss on the open set. The output of the open-set classifier is the predicted probability of each sample in the dataset belonging to each class and the predicted probability of not belonging to that class. This is used as the standard for judging new class samples in the target domain and the classification of samples.
[0049] The cross-entropy loss is:
[0050]
[0051] Where, x s For the source domain sample, y s This is the source domain tag. (B) s This represents a batch of data randomly selected from the source domain dataset. K is the number of classes in the source domain. P(y s |x s ) is the sample x s In category y s The probability of it. This represents the label of the i-th sample in the source domain on the j-th class.
[0052] The consistency regularization loss is:
[0053]
[0054] Where, x s For the source domain sample, y s This is the source domain tag. (B) s This represents a batch of data randomly selected from the source domain dataset. K is the number of classes in the source domain. τ(x) s P(τ(x)) represents the source domain sample after data augmentation. s P(x) represents the probability vector of the enhanced source domain samples after softmax processing. s ) is the probability vector of the source domain samples after softmax.
[0055] The neighborhood clustering loss is:
[0056]
[0057] Where, the memory library Z represents the target feature, the cluster center of the unknown class, and the prototype weight vector; where, f i W represents the target feature. i C represents the prototype weight vector. un They represent the cluster centers of unknown classes; they are L2 normalized; and in each iteration, they continuously update the features in the memory pool Z.
[0058]
[0059]
[0060] Among them, the temperature parameter τ controls the distribution concentration; n t B is the total number of samples in the target domain, k is the number of classes in the source domain, and the temperature parameter τ controls the distribution concentration and is a hyperparameter; t This represents a batch of data randomly sampled from the target domain. f i S represents the features of the target domain samples. ij This represents the similarity between target domain sample i and feature j in the memory database. sThe source domain labels are used. In the domain discriminator, the source and target domain features output by the feature extractor are combined with the class prediction probabilities output by the open-set classifier to align sample data of the same class in the source and target domains. Furthermore, the goal is to align only samples of common classes without affecting samples of new classes in the target domain. Based on this, the cluster centers of the source domain are calculated according to the source domain labels, and pseudo-labels for the target domain data are obtained through the open-set classifier. The cluster centers of the source domain are mapped one-to-one to the feature space of the target domain. By calculating the proportion of samples of the same class in the target domain surrounding (within the neighborhood) of the source domain cluster centers, the confidence level of a sample being classified into that class can be obtained (e.g., ...). Figure 1 Within the dashed box, in the target domain, the cluster center of the source domain for the triangle category has 6 samples from the target domain surrounding it, and all 6 samples are triangles, so the confidence of this category is 6 / 6 = 1; while the cluster center of the rhombus category has 6 samples surrounding it, but only two are rhombuses, so the confidence of this category is 2 / 6 = 0.33. This is because samples of the same category, whether in the target or source domain, should be closer in the feature space. If it's a cluster center of a common class, then the samples around it are mostly samples of the same category. However, the cluster centers of private classes in the source domain are surrounded by many samples of new categories from the target domain. Therefore, by adding the confidence of classifying a sample into the conditional adversarial loss, the influence on private class samples can be reduced, thus reducing the harm of negative transfer. A confidence-weighted conditional adversarial loss is used to train the domain discriminator and feature extractor. The confidence-weighted conditional adversarial loss is as follows:
[0061]
[0062]
[0063] Where, N K R represents the number of samples surrounding the cluster center; K This represents the credibility score for each category. s For the source domain sample, x t For the target domain sample. y s For source domain tags. represents the pseudo-label of the target domain sample. E denotes expectation. Ⅱ denotes indicator function. It is the i-th feature of the source domain. D represents the i-th predicted probability in the source domain. s D represents the source domain dataset. t Represents the target domain dataset; It is the j-th feature of the target domain; n represents the j-th predicted probability in the target domain; sLet |k| be the total number of samples in the source domain and |k| be the total number of classes in the source domain. The adversarial domain discriminator D is structured similarly to a binary classifier. For example, if the source domain sample label is 0, then the target domain sample label is 1. The optimization goal of the adversarial domain discriminator is to distinguish as much as possible whether a sample comes from the source domain or the target domain. There is a gradient inversion layer between the domain discriminator D and the feature extractor F, which multiplies the error (gradient) passed to the gradient inversion layer by a negative number (-λ). Therefore, the adversarial domain discriminator D and the feature extractor F optimize in opposite directions, thus forming an adversarial relationship. The adversarial domain discriminator D aims to distinguish between source domain samples and target domain samples as much as possible, while the feature extractor F aims to extract features that make it impossible for the domain discriminator to distinguish neighboring regions. When the features extracted by the feature extractor F cannot be distinguished as source domain features or target domain features, the effect of eliminating domain differences is achieved, that is, aligning the target domain samples with the source domain samples.
[0064] In the open-set classifier O, based on OVAnet, the feature extractor and open-set classifier are further trained using cross-entropy loss and consistency regularization loss on the open set, on both the source domain data and the data augmentation data, to better learn the essential features of each class; and neighborhood clustering loss is used to shorten the distance between samples of the same class. Then, the samples in the target domain are clustered according to pseudo-labels, and the cluster center of each class in the target domain is calculated; when the samples surrounding a cluster center in the target domain are all samples of the same class, the cluster center is added to the next round of training as a representative sample of that class; when the samples surrounding a cluster center in the target domain are mixed with samples of a new class (i.e., not equal to any class in the source domain), the features of the cluster center are mixed with the features of the surrounding new class samples, and these are added to the next round of training as possible new class samples (e.g., ...). Figure 1 In the lower right corner, within the solid-lined box, we can see that the cluster centers of triangles, circles, and squares are surrounded by samples of the same type, so their cluster centers are used as representative samples in subsequent training. The cluster centers of diamonds, however, are surrounded by samples of the target domain's private class, so they are also highly likely to be private class samples. In subsequent training, the representative samples are used as positive class samples, while the synthesized new class samples are used as negative class samples of the class with the highest predicted probability. Open-set classification loss is used to strengthen the training of the open-set classifier. This process is continuously iterated and updated. The continuous training with representative samples strengthens the learning of the common class, while the synthesized new class samples, during iteration, can strengthen the learning of the classification boundary and correct misclassified new class samples.
[0065] The Mixup operation is as follows:
[0066]
[0067] Where X0 is the feature of the cluster center, and X1 is the feature of the unknown class surrounding it.
[0068] The open set classification loss is calculated as follows:
[0069]
[0070] Where P(y|x) represents the predicted probability of sample x in class y. The first term represents minimizing the entropy of positive class samples, and the second term represents minimizing the entropy of negative class samples, where sample x is... s Treat it as the negative class sample of the class with the second highest predicted probability. s This is a sample from the source domain. s For source domain tags. This represents the probability that the sample label is the true labeled category of the source domain. This represents the probability that a sample label belongs to another category. Open-set classification loss is applied to samples in the source domain, with each source domain category corresponding to a binary classifier. Each binary classifier classifies a sample as belonging to that category. Positive samples are those belonging to that category in the source domain, so the key to applying open-set classification loss is finding suitable negative class samples. Open-set classification loss is similar to binary cross-entropy loss, except that the second term here represents the probability that the sample x is a negative class. s It is treated as a negative class sample of the class with the second highest confidence (predicted probability).
[0071] The method in this embodiment can effectively identify novel class samples in different scenarios without human assistance. It employs a confidence-weighted conditional adversarial loss, where the weights represent the confidence that each sample is classified into a particular class, preventing misclassification of novel classes by misaligning them with known classes. In the target domain, an open-set classifier is used to obtain pseudo-labels for samples and to calculate the cluster centers for each class. Clean cluster centers and cluster centers suspected of being novel classes are selected and added to the training to strengthen the training of common classes and the learning of classification boundaries, thus correcting misclassified novel class samples. Experiments were conducted on open datasets, and the results on multiple datasets demonstrate that the method of this embodiment achieves excellent performance.
[0072] Example 2
[0073] This embodiment provides an image classification system based on universal domain adaptation, which specifically includes the following modules:
[0074] The image acquisition module is configured to acquire images to be classified.
[0075] The image classification module is configured to input the image to be classified into a trained classification network and obtain the category of the image to be classified.
[0076] The classification network uses source domain data to train an open-set classifier and a feature extractor through open-set cross-entropy loss, consistency regularization loss, open-set classification loss, and neighborhood clustering loss.
[0077] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.
[0078] Example 3
[0079] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of an image classification method based on universal domain adaptation as described in Embodiment 1 above.
[0080] Example 4
[0081] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the image classification method based on universal domain adaptation as described in Embodiment 1 above.
[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a processFigure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0086] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An image classification method based on universal domain adaptation, characterized in that, include: Obtain the image to be classified; The image to be classified is input into the trained classification network to obtain the category of the image; The classification network uses source domain data to train an open set classifier and a feature extractor through open set cross-entropy loss, consistency regularization loss, open set classification loss and neighborhood clustering loss. The cross-entropy loss is: ( ) = ; in, For source domain samples, For source domain tags; K represents a batch of data randomly selected from the source domain dataset; K is the number of categories in the source domain; P( ) as a sample In category The probability of it; Indicates the first in the source domain i The sample at the th j Tags on the class; The consistency regularization loss is: ( ) = ; in, ( p( represents the source domain sample after data augmentation); ( p() represents the probability vector of the enhanced source domain samples after softmax processing; () represents the probability vector of the source domain samples after softmax. The neighborhood clustering loss is: Z = [ ]; Wherein, the memory library Z represents the target features, cluster centers of unknown classes, and prototype weight vector; Indicate target features, W i Represents the prototype weight vector. C un Represents the cluster center of the unknown class; Open set classification loss: ( ) = -log (P( )) - (1-P( )); Where P(y|x) represents the predicted probability of sample x in class y; the first term represents minimizing the entropy of positive class samples, and the second term represents minimizing the entropy of negative class samples, where the sample... Treat it as the negative class sample of the class with the second highest predicted probability; P( P( ) represents the probability that the sample label is the true labeled class of the source domain. ) represents the probability that the sample label is another category.
2. The image classification method based on universal domain adaptation as described in claim 1, characterized in that, The feature extractor is used to extract features from the input image; The open set classifier is used to obtain the predicted probability that an image belongs to each category and the probability that it does not belong to each category based on the features of an image.
3. The image classification method based on universal domain adaptation as described in claim 1, characterized in that, The open set classifier consists of several binary classifiers, the number of which is equal to the number of categories in the source domain data. Each binary classifier corresponds to a category in the source domain.
4. The image classification method based on universal domain adaptation as described in claim 3, characterized in that, If none of the binary classifiers classifies an image into its corresponding category, then the image is a new class sample in the target domain. If a binary classifier classifies an image into its corresponding category, then the image is classified into the category with the highest predicted probability.
5. The image classification method based on universal domain adaptation as described in claim 1, characterized in that, The domain discriminator of the classification network is used for confidence-weighted conditional adversarial analysis to solve the problem of source and target domain alignment.
6. The image classification method based on universal domain adaptation as described in claim 1, characterized in that, During training, the open set classifier clusters samples in the target domain according to pseudo-labels and calculates the cluster center for each class. When all samples in the neighborhood of a cluster center belong to the same class, the cluster center is added to the next round of training as a representative sample. When samples in the neighborhood of a cluster center are mixed with samples of a new class, the cluster center is mixed with the samples of the new class in its neighborhood and then added to the next round of training.
7. The image classification method based on universal domain adaptation as described in claim 1, characterized in that, The classification network uses data augmentation techniques to process the source domain data, and then uses the augmented source domain data for training the classification network.
8. An image classification system based on universal domain adaptation, characterized in that, include: The image acquisition module is configured to acquire images to be classified. The image classification module is configured to input the image to be classified into a trained classification network and obtain the category of the image to be classified. The classification network uses source domain data to train an open set classifier and a feature extractor through open set cross-entropy loss, consistency regularization loss, open set classification loss and neighborhood clustering loss. The cross-entropy loss is: ( ) = ; in, For source domain samples, For source domain tags; K represents a batch of data randomly selected from the source domain dataset; K is the number of categories in the source domain; P( ) as a sample In category The probability of it; Indicates the first in the source domain i The sample at the th j Tags on the class; The consistency regularization loss is: ( ) = ; in, ( p( represents the source domain sample after data augmentation); ( p() represents the probability vector of the enhanced source domain samples after softmax processing; () represents the probability vector of the source domain samples after softmax. The neighborhood clustering loss is: Z = [ ]; Wherein, the memory library Z represents the target features, cluster centers of unknown classes, and prototype weight vector; Indicate target features, W i Represents the prototype weight vector. C un Represents the cluster center of the unknown class; Open set classification loss: ( ) = -log (P( )) - (1-P( )); Where P(y|x) represents the predicted probability of sample x in class y; the first term represents minimizing the entropy of positive class samples, and the second term represents minimizing the entropy of negative class samples, where the sample... Treat it as the negative class sample of the class with the second highest predicted probability; P( P( ) represents the probability that the sample label is the true labeled class of the source domain. ) represents the probability that the sample label is another category.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the image classification method based on universal domain adaptation as described in any one of claims 1-7.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the image classification method based on universal domain adaptation as described in any one of claims 1-7.