A method and system for image classification based on joint clustering domain adaptation

Abstract

The present invention proposes a picture classification method and system based on joint clustering domain adaptation, comprising: extracting features of source domain picture data and target domain picture data respectively through a feature extraction network to obtain source domain features and target domain features; explicitly expressing the intra-domain category conditional distribution probability based on the distances from the source domain features and the target domain features to the cluster centers; minimizing the relative entropy distance between the source domain data label and the intra-domain category conditional distribution probability, updating the feature extraction network and the category cluster centers; minimizing the relative entropy distance between the label assignment of the target domain data and the intra-domain category conditional distribution probability, and again updating the feature extraction network and the cluster centers; inputting the picture to be classified into the re-updated feature extraction network to obtain the picture features of the picture to be classified, calculating the probabilities of the picture features and all category centers in the re-updated category cluster centers, and taking the category with the largest probability as the picture classification result of the picture to be classified.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to an image classification method and system based on joint clustering domain adaptation. Background Art

[0002] Existing deep convolutional neural network methods assume that the training dataset and the application scenario have the same distribution. However, the dataset used for model training often differs from the actual application scenario, which can significantly reduce the model's performance. Unsupervised domain adaptation (UDA) provides a method to address the performance degradation caused by domain distribution mismatch when target domain labels are unavailable.

[0003] Traditional unsupervised domain adaptation methods align the feature distributions of different domains by designing a loss function metric and minimizing the difference distance between the domains. Metrics include maximum mean squared deviation (MMD), Kullback-Leibler divergence (KL), and GAN-based domain classifiers. These methods can be summarized as finding a domain-invariant feature space while maintaining the classification accuracy of the source domain.

[0004] The theoretical basis of existing methods is that the distributions of the source and target domains are sufficiently similar in a domain-consistent feature space. However, in practical applications, even if the feature distributions between the two domains are similar, the performance of the target domain on downstream classification tasks will still decline, and negative transfer may occur. Summary of the Invention

[0005] This application proposal hopes to solve the problem of domain adaptation in which the target domain samples and the conditional probability distribution of categories in the feature space are consistent, so as to solve the problem of negative transfer in domain adaptation.

[0006] Specifically, the present invention proposes an image classification method based on joint clustering domain adaptation, which includes:

[0007] Step 1: The features of the source domain image data and the target domain image data can be extracted through the feature extraction network to obtain the source domain features and the target domain features;

[0008] Step 2: Obtain the category cluster center and explicitly express the conditional distribution probability of the category within the domain based on the distance from the source domain feature and the target domain feature to the cluster center;

[0009] Step 3: Update the feature extraction network and category cluster center by minimizing the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category;

[0010] Step 4: Update the feature extraction network and cluster centers again by minimizing the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the categories within the domain;

[0011] Step 5: Input the image to be classified into the updated feature extraction network to obtain the image features of the image to be classified. Calculate the distance between the image features and all category centers in the updated category cluster centers to represent the probability that the image to be classified belongs to each category. Take the category with the largest probability as the image classification result of the image to be classified.

[0012] The image classification method based on joint clustering domain adaptation, wherein step 1 includes extracting features of source domain and target domain image data using the feature extraction network:

[0013]

[0014] in To input image data, Denote the parameter θ R neural network, Represents the output features.

[0015] The image classification method based on joint clustering domain adaptation, wherein step 2 includes:

[0016]

[0017] in represents the feature cluster center of category j pictures, p(y j |x i ,q) represents the image sample x i Belongs to category y j The conditional distribution probability of the category within the domain.

[0018] In the aforementioned image classification method based on joint clustering domain adaptation, the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category is minimized in step 3 as follows:

[0019]

[0020] Where N represents the number of samples in the source domain, Represents the sample x in the source domain i The category label, Represents the source domain sample x i Through neural network The extracted features.

[0021] In the image classification method based on joint clustering domain adaptation, the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the category within the domain is minimized in step 4.

[0022]

[0023] Where N represents the number of samples in the target domain, Represents the source domain sample x i The class label assignment, Represents the source domain sample x i Through neural network The extracted features, γ i Source domain sample x i Label assignments obtained through clustering algorithms.

[0024] The present invention also proposes an image classification system based on joint clustering domain adaptation, which includes:

[0025] Module 1 is used to extract features of source domain image data and target domain image data through a feature extraction network to obtain source domain features and target domain features;

[0026] Module 2 is used to obtain the category cluster center and explicitly express the conditional distribution probability of the category within the domain based on the distance between the source domain feature and the target domain feature and the cluster center;

[0027] Module 3 is used to update the feature extraction network and the category cluster center by minimizing the relative entropy distance between the source domain data label and the conditional distribution probability of the category in the domain;

[0028] Module 4 is used to update the feature extraction network and cluster centers again by minimizing the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the categories within the domain;

[0029] Module 5 is used to input the image to be classified into the updated feature extraction network to obtain the image features of the image to be classified, calculate the distance between the image features and all category centers in the updated category cluster centers, and express the probability that the image to be classified belongs to each category. The category with the largest probability is taken as the image classification result of the image to be classified.

[0030] The image classification system based on joint clustering domain adaptation, wherein the module 1 includes extracting features of source domain and target domain image data using the feature extraction network:

[0031]

[0032] in To input image data, Denote the parameter θ R neural network, Represents the output features.

[0033] The image classification system based on joint clustering domain adaptation, wherein the module 2 includes:

[0034]

[0035] in represents the feature cluster center of category j pictures, p(y j |x i ,q) represents the image sample x i Belongs to category y j The conditional distribution probability of the category within the domain.

[0036] In the aforementioned image classification system based on joint clustering domain adaptation, the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category is minimized in module 3 as follows:

[0037]

[0038] Where N represents the number of samples in the source domain, Represents the sample x in the source domain i The category label, Represents the source domain sample x i Through neural network The extracted features.

[0039] In the aforementioned image classification system based on joint clustering domain adaptation, the relative entropy distance between the label assignment of target domain data and the conditional distribution probability of the category within the domain is minimized in module 4.

[0040]

[0041] Where N represents the number of samples in the target domain, Represents the target domain sample x i The class label assignment, Represents the target domain sample x i Through neural network The extracted features, γ i Source domain sample x i Label assignments obtained through clustering algorithms.

[0042] From the above scheme, it can be seen that the advantages of the present invention are:

[0043] 1. It can solve the negative transfer effect in the domain adaptation algorithm and improve the model accuracy;

[0044] 2. It can better support large-scale data training. The online clustering algorithm does not require offline inference clustering process, ensuring the speed of network training. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION

[0046] While conducting research on domain adaptation, the inventors discovered that this limitation in existing techniques stems from a lack of constraints on the conditional probabilities of the target domain's classes. This prevents the distribution of the target domain in feature space from being similar to the distribution of the samples themselves, impairing the model's performance in the target domain. This limitation can be addressed through a domain adaptation method based on joint clustering.

[0047] The key technical points of the present invention include:

[0048] Key point 1: Introducing a set of learnable parameters to estimate the cluster centers of features, and using the distance from the sample to the cluster center to explicitly express the conditional distribution probability of the category in the domain;

[0049] Key point 2: Update the feature extraction network and cluster center by minimizing the KL distance between data label assignment and category conditional probability.

[0050] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0051] like Figure 1 As shown, the present invention proposes a domain adaptation algorithm based on joint clustering.

[0052] First, use the backbone network to extract the features of the source and target domain image data:

[0053]

[0054] in To input image data, Denote the parameter θ R neural network, Represents output features

[0055] Second, introduce a set of parameters that can be learned using neural network gradient updates to represent the category clustering centers of the features The distances between source and target domain samples and the cluster centers are used to explicitly express the conditional distribution probabilities of categories within the two domains.

[0056]

[0057] in represents the feature cluster center of a certain category j picture, p(y j |x i ,q) represents the sample x i Belongs to category y j The conditional probability distribution of .

[0058] Third, update the feature extraction network and clustering center by minimizing the relative entropy (KL) distance between the source domain data label and the source domain category conditional probability:

[0059]

[0060] Where N represents the number of samples in the source domain, Represents the source domain sample x i The category label, Represents the source domain sample x i Through neural network The extracted features.

[0061] Fourth, update the feature extraction network and cluster center by minimizing the KL distance between the target domain label assignment and the target domain category conditional probability:

[0062]

[0063] Where N represents the number of samples in the target domain, Represents the target domain sample x i The class label assignment, Represents the target domain sample x i Through neural network The extracted features.

[0064] The label assignment γ can be obtained by any clustering algorithm, such as K-Means.

[0065] After the update is completed, the sample x passes through the neural network during the inference process Extracted feature z, calculate z and cluster center The distance between all category centers q in the , indicating the probability that x belongs to a certain category, takes the category q with the largest probability max The corresponding category is the classification result:

[0066]

[0067] Taking image classification as an example, there is an active domain image dataset X S , target domain image dataset X T .

[0068] The parameter θ is introduced R A neural network and a set of cluster centers

[0069] After the source domain image is input into the network, the above formula is used

[0070]

[0071] Update θ R and

[0072] After the target domain image is input into the network, the above formula is used

[0073]

[0074] Update θ R and

[0075] After training is completed, input the test set image X, network θ R Output feature Z, and compare Z with Calculate the category conditional distribution probability:

[0076]

[0077] Take the category q with the largest category conditional distribution probability max As the classification result.

[0078] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiment.

[0079] The present invention also proposes an image classification system based on joint clustering domain adaptation, which includes:

[0080] Module 1 is used to extract features of source domain image data and target domain image data through a feature extraction network to obtain source domain features and target domain features;

[0081] Module 2 is used to obtain the category cluster center and explicitly express the conditional distribution probability of the category within the domain based on the distance between the source domain feature and the target domain feature and the cluster center;

[0082] Module 3 is used to update the feature extraction network and the category cluster center by minimizing the relative entropy distance between the source domain data label and the conditional distribution probability of the category in the domain;

[0083] Module 4 is used to update the feature extraction network and cluster centers again by minimizing the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the categories within the domain;

[0084] Module 5 is used to input the image to be classified into the updated feature extraction network to obtain the image features of the image to be classified, calculate the distance between the image features and all category centers in the updated category cluster centers, and express the probability that the image to be classified belongs to each category. The category with the largest probability is taken as the image classification result of the image to be classified.

[0085] The image classification system based on joint clustering domain adaptation, wherein the module 1 includes extracting features of source domain and target domain image data using the feature extraction network:

[0086]

[0087] in To input image data, Denote the parameter θ R neural network, Represents the output features.

[0088] The image classification system based on joint clustering domain adaptation, wherein the module 2 includes:

[0089]

[0090] in represents the feature cluster center of category j pictures, p(y j |x i ,w) represents the image sample x i Belongs to category y j The conditional distribution probability of the category within the domain.

[0091] In the aforementioned image classification system based on joint clustering domain adaptation, the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category is minimized in module 3 as follows:

[0092]

[0093] Where N represents the number of samples in the source domain, Represents the sample x in the source domain i The category label, Represents the source domain sample x i Through neural network The extracted features.

[0094] In the aforementioned image classification system based on joint clustering domain adaptation, the relative entropy distance between the label assignment of target domain data and the conditional distribution probability of the category within the domain is minimized in module 4.

[0095]

[0096] Where N represents the number of samples in the target domain, Represents the target domain sample x i The class label assignment, Represents the target domain sample x i Through neural network The extracted features, γ iSource domain sample x i Label assignments obtained through clustering algorithms.

Claims

1. A method for image classification based on joint clustering domain adaptation, characterized in that: include: Step 1: Extract the features of the source domain image data and the target domain image data through the feature extraction network to obtain the source domain features and the target domain features; Step 2: Obtain the category cluster center and explicitly express the conditional distribution probability of the category within the domain based on the distance from the source domain feature and the target domain feature to the cluster center; Step 3: Update the feature extraction network and category cluster center by minimizing the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category; Step 4: Update the feature extraction network and cluster centers again by minimizing the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the categories within the domain; Step 5: Input the image to be classified into the updated feature extraction network to obtain the image features of the image to be classified. Calculate the distance between the image features and all category centers in the updated category cluster centers to represent the probability that the image to be classified belongs to each category. The category with the highest probability is selected as the image classification result of the image to be classified. Step 1 includes extracting features of the source domain and target domain image data using the feature extraction network: in To input image data, Denote the parameter θ R neural network, represents the output features; Characterized in that step 2 includes: in q j Indicates the cluster center of the highlighted category j, The parameter of the neural network gradient represents the category cluster center of the feature, p(y j |x i ,q) represents the image sample x i The domain-conditional distribution probability of image belonging to category j; In step 3, the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category is minimized as follows: Where N represents the number of samples in the source domain, Represents the sample x in the source domain i The category label, Represents the source domain sample x i Through neural network The extracted features; In step 4, the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the category within the domain is minimized as follows: Where N represents the number of samples in the target domain, Represents the source domain sample x i The class label assignment, Represents the source domain sample x i Through neural network The extracted features, γ i Represents the source domain sample x i Label assignments obtained through clustering algorithms.

2. An image classification system based on joint clustering domain adaptation, characterized in that: include: Module 1 is used to extract features of source domain image data and target domain image data through a feature extraction network to obtain source domain features and target domain features; Module 2 is used to obtain the category cluster center and explicitly express the conditional distribution probability of the category within the domain based on the distance between the source domain feature and the target domain feature and the cluster center; Module 3 is used to update the feature extraction network and the category cluster center by minimizing the relative entropy distance between the source domain data label and the conditional distribution probability of the category in the domain; Module 4 is used to update the feature extraction network and cluster centers again by minimizing the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the categories within the domain; Module 5 is used to input the image to be classified into the updated feature extraction network, obtain the image features of the image to be classified, calculate the distance between the image features and all category centers in the updated category cluster centers, and express the probability that the image to be classified belongs to each category. The category with the highest probability is selected as the image classification result of the image to be classified; The module 1 includes extracting features of source domain and target domain image data using the feature extraction network: in To input image data, Denote the parameter θ R neural network, represents the output features; This module 2 includes: in q j Indicates the cluster center of the highlighted category j, The parameter of the neural network gradient represents the category cluster center of the feature, p(y j |x i ,q) represents the image sample x i The domain-conditional distribution probability of image belonging to category j; In module 3, the relative entropy distance between the source domain data label and the conditional distribution probability of the domain category is minimized as follows: Where N represents the number of samples in the source domain, Represents the sample x in the source domain i The category label, Represents the source domain sample x i Through neural network The extracted features; In module 4, the relative entropy distance between the label assignment of the target domain data and the conditional distribution probability of the category within the domain is minimized as follows: Where N represents the number of samples in the target domain, Represents the target domain sample x i The class label assignment, Represents the target domain sample x i Through neural network The extracted features, γ i Represents the source domain sample x i Label assignments obtained through clustering algorithms.

Images