Universal domain adaptive image classification method based on distance entropy weighting

Abstract

The invention discloses a universal domain adaptive image classification method based on distance entropy weighting, and the method specifically comprises the steps: 1, obtaining source domain data and target domain data, 2, building a model, and carrying out the pre-training; step 3, mapping the source domain data and the target domain data to a hypersphere; 4, setting all prototypes of the source domain as 0; step 5, calculating a source domain category weight # imgabs0 # and a target domain sample weight # imgabs1 #; step 6, performing adversarial training by using a weight weighting training model and a domain discriminator D, and determining whether the data is from a source domain or a target domain; obtaining weighted confrontation loss; and 7, screening a target domain sample to obtain a target domain auxiliary domain, and outputting an image for classification. According to the universal domain adaptive image classification method based on distance entropy weighting disclosed by the invention, the problem that image classification is not clear due to the fact that a model in the prior art cannot effectively distinguish and recognize private categories of a target domain is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of transfer learning, and particularly relates to a general domain adaptation image classification method based on distance entropy weighting. Background Art

[0002] In the setting of general domain adaptation, general domain adaptation is not restricted by any prior knowledge. The source domain and the target domain may share a common label set, and each domain may have a private label set or outlier classes. General domain adaptation first tends to find a shared label space across domains, and then aligns the data distributions in the common label set. Finally, a classifier is trained on the matched source-labeled data for the unlabeled target data. In the test phase, the classifier trained in general domain adaptation assigns accurate labels to the target samples belonging to the shared label space and labels the samples in the outlier classes as unknown. Solving the general domain adaptation problem faces three difficulties: ① alignment of the common classes in the source domain and the target domain ② screening of the private classes in the source domain ③ separation of the unknown classes in the target domain. Simple methods based on uncertainty thresholds prevent the model from considering the potential complexity existing between known and unknown samples in the high-dimensional feature space. Rejection of unknown classes based on clustering and consistency criteria, but these methods themselves lack certainty for the number of clusters and the consistency criteria, thus resulting in the model being unable to effectively distinguish and identify the private classes in the target domain, further affecting the overall adaptation effect.

[0003] Most of the existing general domain adaptation methods first train a classifier for known classes through the source domain, and then rely on a single threshold to distinguish unknown target samples. Simple threshold-based methods prevent the model from considering the potential complexity existing between known and unknown samples in the high-dimensional feature space. Secondly, there are also some methods that reject unknown classes based on clustering and consistency criteria, but these methods themselves lack certainty for the number of clusters and the consistency criteria, thus resulting in the model being unable to effectively distinguish and identify the private classes in the target domain, further affecting the overall adaptation effect. Summary of the Invention

[0004] The purpose of the present invention is to provide a general domain adaptation image classification method based on distance entropy weighting, which solves the problem in the prior art that the private classes in the target domain cannot be effectively distinguished and identified, resulting in unclear image classification.

[0005] The technical solution adopted by the present invention is that a general domain adaptation image classification method based on distance entropy weighting specifically includes the following steps: Step 1, obtain source domain data and target domain data to form a data set for the image classification task; Step 2, establish a deep learning model and pre-train it, including a feature extractor F, a classifier C, and a domain discriminator D; Step 3, map the source domain data and the target domain data to a hypersphere to obtain the source domain class prototypes ; Step 4, initialize the source domain class prototypes , and set all source domain class prototypes to 0; Step 5, calculate the source domain class weights , separate the source domain private classes and shared classes; and calculate the target domain sample weights ; Step 6, use the weights obtained in Step 5 to weighted train the model, and perform adversarial training using the domain discriminator D to determine whether the data comes from the source domain or the target domain; obtain the weighted adversarial loss; Step 7, screen the target domain samples, calculate the unknown class prototypes, obtain the target domain auxiliary domain, and output the images for classification.

[0006] The features of the present invention also lie in that In Step 1, the source domain data and the target domain data are respectively: the source domain data obeys the distribution , the source domain label is , the number of samples is , that is , the target domain data obeys the distribution , the number of samples is , that is .

[0007] Step 2 is specifically: initially train the deep learning model using the source domain data, and use the backbone part of the pre-trained ResNet50 network as the feature extractor F; then, transfer the extracted features to a predefined residual block, and the residual block includes a convolutional layer, a normalization layer, and a residual connection; finally, transfer the extracted features to a fully connected layer, and the fully connected layer adjusts the output dimension according to the number of input categories for classification tasks; the model is trained by minimizing the cross-entropy loss function, and the cross-entropy loss calculation formula is: (1); In the formula, represents the total number of source domain samples, respectively represent the source domain samples and their class labels from the source domain ; represents the total number of source domain classes, represents the label of the sample in the th class, taking values of 0 or 1, is the output probability value of the sample in the th class, and multi-round training is performed until a model with classification ability for the source domain data is obtained.

[0008] Step 3 is specifically as follows: Feed the source domain samples and the target domain samples into the feature extractor F obtained in Step 2 to obtain -dimensional features and . Calculate the Euclidean norm of the feature vectors, and use the Euclidean norm to standardize the original feature vectors into unit feature vectors and . At this time, the source domain data and the target domain data have been mapped onto the unit hypersphere.

[0009] Step 4 is specifically as follows: Initially, set all class prototypes in the source domain to 0. According to the labels of the source domain samples, stack their feature vectors into the corresponding class prototypes, and divide the accumulated features after stacking the samples of each category by the number of samples in that category for averaging; then normalize the prototypes of all categories so that the L2 norm of each prototype vector is 1, obtaining the representative features of each category and maintaining the unit length of the class prototypes.

[0010] Calculating the source domain class weights in Step 5 is specifically as follows: Calculate the predicted values and output scores obtained by all target domain samples through the model, and statistically average and normalize the output scores of the target domain samples predicted for each category to obtain the class weights at the source domain class level .

[0011] Calculating the target domain sample-level weights, the weight of the th target domain sample is specifically as follows: For the target domain sample , obtain the feature through the feature extractor F, calculate its distance to the th class prototype , and normalize it to . Perform softmax processing on the normalized distance to obtain the similarity of the sample to each prototype vector, that is . Calculate the distance entropy of the sample, and the specific formula is as follows: (2); In the formula, represents the total number of source domain classes, represents the similarity between the sample and the th class prototype vector.

[0012] Step 6 is as follows: use the weights obtained in step 5 for weighted training, increase the distance between prototypes, and encourage samples to be close to the class prototype; and use the domain discriminator D for adversarial training to classify the input feature representation and determine whether the data comes from the source domain or the target domain. Source samples and target samples with different confidence levels provide different levels of loss, namely, weighted adversarial loss. Specifically: (3); In the formula, represents the total number of source domain samples, represents the total number of samples in the target domain, Respectively represent the source domain Source domain samples and their category labels; and Represent the source domain category weight and target domain sample weight respectively; Indicates that the source domain samples pass through the feature extractor The obtained features are fed into the domain discriminator The output value obtained; Indicates that the target domain sample passes through the feature extractor The obtained features are fed into the domain discriminator The output value obtained; Use each batch of source domain samples to update the corresponding class prototype. and class prototype , the update rule is , calculate the similarity between all prototypes and other prototypes, and get the prototype dispersion loss, which is: (4); In the formula, represents the total number of source domain categories, and The source domain The class prototype and Class prototype; Next, in order to bring each category sample closer to the corresponding class prototype, the intra-class consistency loss is introduced: (5); In the formula, represents the total number of source domain samples, Respectively represent the source domain Source domain samples and their category labels; represents the total number of source domain categories, represents the sample characteristics, The source domain Class prototype; Finally, the total loss of the model is: (6); In the formula, is the loss weight, represents the weighted adversarial loss, represents the prototype dispersion loss, represents the intra-class consistency loss.

[0013] Step 7 is specifically as follows: First, calculate the distance entropy of all target domain samples, select 5% of the target domain samples with larger entropy values, calculate the mean of the feature vectors of these samples, and use this mean as the class prototype of the unknown class, and add it to the source domain as the unknown class samples of the target domain to form a target auxiliary domain; input the target domain data into the classification network, use the total loss function to train the network and continuously update the target auxiliary domain, the source domain class weights and the target domain sample weights, and finally obtain the classification result.

[0014] Compared with the prior art, the beneficial effects of the present invention are: (1) The general domain adaptive image classification method based on distance entropy weighting provided by the present invention maps the source domain and target domain samples to a hypersphere to obtain a more compact and discriminative representation, avoiding the loss of effective capture of the potential structure of the data by using simple clustering to obtain clusters. Through class prototype learning on the representation on the hypersphere, the class prototypes of the source domain can accurately represent the characteristics of each class in the source domain. For the source domain samples, a clustering loss function is used to keep the representations of the samples in the same class on the hypersphere as compact as possible, thereby improving the intra-class similarity and ensuring the close aggregation of the same-class samples in the high-dimensional space; at the same time, in order to increase the inter-class separability, a prototype dispersion loss is introduced, so that the representations of different classes are as far away from each other as possible on the hypersphere, thereby enhancing the discrimination between different classes and further improving the classification accuracy.

[0015] (2) The general domain adaptive image classification method based on distance entropy weighting provided by the present invention dynamically adjusts the weights of adversarial training based on the entropy values of these distances, avoiding the use of simple thresholds for division. Specifically, the known class samples have a higher weight in adversarial training, while the weight of the unknown class samples is lower, which can effectively avoid the interference of the unknown class samples on the training process and ensure that the model can better distinguish the public classes and private classes in the target domain.

[0016] (3) The general domain adaptive image classification method based on distance entropy weighting provided by the present invention constructs unknown class prototypes, and enhances the model's ability to identify unknown classes through these unknown class prototypes, improving the model's adaptability to new classes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic flowchart of the general domain adaptive image classification method based on distance entropy weighting of the present invention; Figure 2It is the network structure diagram of the general domain adaptive image classification method based on distance entropy weighting of the present invention. Detailed implementation manners

[0018] The present invention will be further explained below in conjunction with the accompanying drawings and specific implementation manners.

[0019] Embodiment 1 The present invention provides a general domain adaptive image classification method based on distance entropy weighting. As Figure 1-2 shown, the specific steps are as follows: Step 1: Obtain source domain data and target domain data to form a data set for the image classification task. The source domain data obeys the distribution , the source domain label is , the number of samples is , that is . The target domain data obeys the distribution , the number of samples is , that is .

[0020] Next, use the source domain data to preliminarily train the network. The network uses the backbone part of the pre-trained ResNet50 network as the feature extractor F. Then, the extracted features are passed to a predefined residual block, which includes a series of convolutional layers, normalization layers, and residual connections for extracting higher-level features. Finally, these features are passed to a fully connected layer, which adjusts the output dimension according to the number of input classes for the classification task. The network is trained by minimizing the cross-entropy loss function. The cross-entropy loss calculation formula is: (1); In the formula, represents the total number of source domain samples, respectively represent the source domain sample and its class label from the source domain . represents the total number of source domain classes, represents the label of the sample in the th class, taking values of 0 or 1, is the output probability value of the sample in the th class.

[0021] After training for a certain number of rounds, a model with good classification ability for the source domain data can be obtained.

[0022] Step 2: Map the source and target domain data to the hypersphere to obtain the source domain class prototype . As Figure 2 , map the source domain sample and the target domain sample Input into the feature extraction network obtained in Step 1 to obtain -dimensional features and , calculate the Euclidean norm of the feature vector, and use the Euclidean norm to standardize the original feature vector into a unit vector and . At this time, the source and target domain data have been mapped onto the unit hypersphere. This mapping method can reduce the amplitude difference of the features while preserving the information in their directions. The key to this mapping process is to use the points on the hypersphere to represent the features in order to better capture and distinguish the differences between the source domain and the target domain.

[0023] Step 3: Initialize the source domain class prototypes . Initially, temporarily set all source domain class prototypes to 0. According to the labels of the source domain samples, stack their feature vectors into the corresponding class prototypes, and divide the accumulated features after stacking the samples of each category by the number of samples in that category for averaging. Then normalize all category prototypes to ensure that the L2 norm of each prototype vector is 1, obtain the representative features of each category, and maintain the unit length of these class prototypes to avoid the influence caused by the feature scale differences between different categories.

[0024] Step 4: Calculate the source domain class weights using the model's prediction ability , and separate the source domain private classes and shared classes. For the source domain private classes, which are the classes not present in the target domain, the target domain samples will not be over-predicted onto the private classes in the feature space. Specifically, calculate the prediction values and output scores obtained by all target domain samples through the model. To avoid excessive differences in scores between classes, calculate the mean of the output scores of the target domain samples predicted for each class and normalize it to obtain the weights at the source domain class level.

[0025] Step 5: Calculate the target domain sample weights . For the target domain sample , obtain the feature through the feature extraction network, calculate its distance to the th class prototype , and normalize it to . Perform softmax processing on the normalized distance to obtain the similarity of the sample to each prototype vector, that is . Calculate the sample distance entropy, and the specific formula is as follows: (2); In the formula, represents the total number of source domain classes, represents the sample and the The similarity of the prototype vectors of the classes.

[0026] Next, use Get the sample weight .

[0027] Step 6: Use the weights obtained in steps 4 and 5 for weighted training, while increasing the distance between prototypes and ensuring compactness within the class to encourage samples to be close to their class prototypes. The model uses the domain discriminator D for adversarial training to classify the input feature representations to determine whether they are from the source domain or the target domain. Source samples and target samples with different confidence levels should provide different levels of loss. The weighted adversarial loss is obtained: (3); In the formula, represents the total number of source domain samples, represents the total number of samples in the target domain, Respectively represent the source domain The source domain samples and their category labels. and They represent the source domain category weight and target domain sample weight respectively. Indicates that the source domain samples pass through the feature extractor The obtained features are fed into the domain discriminator The output value obtained. Indicates that the target domain sample passes through the feature extractor The obtained features are fed into the domain discriminator The output value obtained.

[0028] Use each batch of source domain samples to update the corresponding class prototype. and class prototype , the update rule is , calculate the similarity between all prototypes and other prototypes, and get the prototype dispersion loss, which is: (4); In the formula, represents the total number of source domain categories, and The source domain The class prototype and A class prototype.

[0029] Next, in order to bring each category sample closer to the corresponding class prototype, the intra-class consistency loss is introduced: (5); In the formula, represents the total number of source domain samples, Respectively represent the source domain Source domain samples and their class labels. Denotes the total number of source domain classes, Denotes the sample features, Denotes the th class prototype of the source domain.

[0030] Finally, the total loss of the model is obtained as: (6); In the formula, is the loss weight, Denotes the weighted adversarial loss, Denotes the prototype dispersion loss, Denotes the intra-class consistency loss.

[0031] Step 7: Screen the target domain samples to calculate the unknown class prototypes and obtain the target domain auxiliary domain. First, calculate the distance entropy of all target domain samples. The distance entropy is used to measure the distribution uncertainty of each target domain sample and its neighboring samples. Specifically, the calculation of the distance entropy involves the distance distribution between the target domain sample and its nearest neighbor, so as to evaluate the dispersion degree of the sample in the feature space. Samples with higher distance entropy are usually located at the boundary of the feature space or in relatively fuzzy regions. These samples usually represent regions where the class division is not clear and have high uncertainty. Select 5% of the target domain samples with larger entropy values, calculate the mean of the feature vectors of these samples, and use this mean as the class prototype of the unknown class, and add it to the source domain as the unknown class samples of the target domain to form the target auxiliary domain. This class prototype can represent the feature distribution of those samples in the target domain that are difficult to classify. By updating the unknown class prototype, the representation of the unknown class is improved, and at the same time, the model's recognition ability for unknown classes in the target domain is enhanced. Keep the same update mode and frequency as the original class prototype in the source domain.

[0032] Example 2 To verify the effectiveness of the method of the present invention, the method of the present invention was performance-tested with two existing methods UAN and CMU on the Office-Home dataset. Office-Home contains 4 domains: Art, (A), Clipart (C), Product (P) and Real-World (R). The samples were collected as follows: 2,427, 4,365, 4,439 and 4,357. This dataset includes 65 classes and a total of 15,588 images. Select the first 10 classes arranged in alphabetical order as the common classes of the source domain and the target domain, the next 5 classes are defined as the private classes of the source domain, and the remaining classes are used as the unknown classes of the target domain.

[0033] Table 1 Experimental results of general domain adaptation methods on the Office-Home dataset

[0034] In Table 1, OS is the weighted combination of the accuracy OS* of the model on |C_S| + 1 classes, i.e., the known classes, and the accuracy UNK on an unknown class, that is: (7); HOS, on the other hand, is and the harmonic mean of, and will only get a high score when the algorithm performs well on both known and unknown samples, that is: (8); The ours method shown in Table 1 is the method of the present invention. As can be seen from the table, compared with the method applicable to the adaptation in the field of image classification under the general setting, the method of the present invention shows effectiveness on the Office-Home dataset. In summary, the present invention proposes a general domain adaptation image classification method based on distance entropy weighting. This method combines two techniques, distance entropy weighting and prototype consistency. By effectively utilizing the relationship between the internal structure information of the target domain and the source domain, it aims to solve the cross-domain classification problem. In this method, distance entropy weighting is used to measure the class uncertainty of the target domain samples, so as to be able to identify and process the samples of unknown classes in the target domain. And prototype consistency is used to ensure that the distributions of the known classes in the source domain and the target domain are fully aligned in the feature space, so as to reduce the distribution difference between the source domain and the target domain, and further improve the performance of the model on the target domain.

[0035] Specifically, the method of the present invention first calculates the distance entropy of the target domain samples and screens out those samples with high uncertainty, which usually represent the unknown classes that are difficult to classify in the target domain. By weighting these high-entropy samples and performing feature learning, the classification ability of the model on the unknown classes in the target domain can be effectively enhanced. At the same time, in order to ensure the consistency of the known classes between the source domain and the target domain, the method adopts a prototype consistency constraint, which prompts each known class in the source domain and the target domain to have a similar representation in the feature space. The introduction of prototype consistency not only helps to maintain the class structure between the source domain and the target domain, but also avoids class confusion caused by distribution differences.

[0036] The advantage of this method is that it can improve the recognition ability of the unknown classes in the target domain while ensuring the distribution alignment of the known classes between the source domain and the target domain. Through this dual constraint mechanism, the model can significantly improve its adaptability and generalization performance when facing cross-domain classification tasks, reduce the influence of the distribution deviation between the source domain and the target domain on the classification results, and thus achieve more accurate and transferable image classification.

[0037] Example 3 This embodiment provides a general domain adaptive image classification method based on distance entropy weighting, as Figure 1-2 shown, and specifically includes the following steps: Step 1: Obtain source domain data and target domain data to form a dataset for the image classification task; Step 2: Establish a deep learning model and pre-train it, including a feature extractor F, a classifier C, and a domain discriminator D; Step 3: Map the source domain data and target domain data to a hypersphere to obtain source domain class prototypes ; Step 4: Initialize the source domain class prototypes , and set all source domain class prototypes to 0; Step 5: Calculate the source domain class weights , separate the source domain private classes and shared classes; and calculate the target domain sample weights ; Step 6: Use the weights obtained in Step 5 to weighted train the model, and use the domain discriminator D for adversarial training to determine whether the data comes from the source domain or the target domain; obtain the weighted adversarial loss; Step 7: Screen the target domain samples, calculate the unknown class prototypes, obtain the target domain auxiliary domain, and output the image for classification.

[0038] Embodiment 4 On the basis of Embodiment 3, in Step 1, the source domain data and target domain data are respectively: the source domain data obeys the distribution , the source domain label is , the number of samples is , that is , the target domain data obeys the distribution , the number of samples is , that is .

[0039] Step 2 is specifically: initially train the deep learning model using the source domain data, and use the backbone part of the pre-trained ResNet50 network as the feature extractor F; then, transfer the extracted features to a predefined residual block, and the residual block includes a convolutional layer, a normalization layer, and a residual connection; finally, transfer the extracted features to a fully connected layer, and the fully connected layer adjusts the output dimension according to the number of input categories for the classification task; the model is trained by minimizing the cross-entropy loss function, and the cross-entropy loss calculation formula is: (1); In the formula, represents the total number of source domain samples, respectively represent the source domain from the source domain ; the source domain samples and their class labels. represents the total number of source domain categories, represents the label of the sample in the th category, taking values of 0 or 1, is the output probability value of the sample in the th category.

[0040] Train in multiple rounds until a model with classification ability for source domain data is obtained.

[0041] Example 5 Based on Example 4, step 3 is specifically as follows: Feed the source domain sample and the target domain sample into the feature extractor F obtained in step 2 to obtain -dimensional features and , calculate the Euclidean norm of the feature vectors, and use the Euclidean norm to standardize the original feature vectors into unit feature vectors and . At this time, the source domain data and the target domain data have been mapped onto the unit hypersphere.

[0042] Step 4 is specifically as follows: Initially, set all source domain class prototypes to 0. According to the labels of the source domain samples, superimpose their feature vectors onto the corresponding class prototypes, and average the accumulated features after superimposing all samples of each category by dividing by the number of samples in that category; then perform normalization processing on the prototypes of all categories so that the L2 norm of each prototype vector is 1, obtain the representative features of each category, and maintain the unit length of the class prototypes.

[0043] Calculating the source domain class weights in step 5 is specifically as follows: Calculate the predicted values and output scores obtained by all target domain samples through the model, and statistically average and normalize the output scores of the target domain samples predicted to each category to obtain the class weights at the source domain category level.

[0044] Calculating the target domain sample-level weights, the th target domain sample weight is specifically as follows: For the target domain sample , obtain the feature through the feature extractor F, calculate its distance to the th class prototype , and normalize it to . Perform softmax processing on the normalized distance to obtain the similarity of the sample to each prototype vector, that is, , calculate the sample distance entropy, and the specific formula is as follows: (2); In the formula, represents the total number of source domain categories, represents the similarity between the sample and the th class prototype vector.

[0045] Step 6 is specifically as follows: Use the weights obtained in Step 5 for weighted training, while increasing the distance between prototypes to encourage samples to approach the class prototypes; and use the domain discriminator D for adversarial training to classify the input feature representation to determine whether the data comes from the source domain or the target domain. Source samples and target samples with different confidences provide different levels of loss, that is, the weighted adversarial loss. Specifically: (3); In the formula, represents the total number of source domain samples, represents the total number of target domain samples, respectively represent the source domain samples from the source domain and their class labels. and respectively represent the source domain class weight and the target domain sample weight. represents the output value obtained by feeding the features of the source domain samples passing through the feature extractor into the domain discriminator represents the output value obtained by feeding the features of the target domain samples passing through the feature extractor into the domain discriminator

[0046] Update the corresponding class prototypes using the source domain samples in each batch. For the sample and the class prototype , the update rule is , calculate the similarity between all prototypes and other prototypes to obtain the prototype dispersion loss. Specifically: (4); In the formula, represents the total number of source domain categories, and represent the th class prototype and the th class prototype in the source domain.

[0047] Next, in order to narrow the compactness between samples of each category and the corresponding class prototypes, introduce the intra-class consistency loss: (5); In the formula, represents the total number of source domain samples, respectively represent the source domain samples and their class labels from the source domain in the source domain. represents the total number of source domain classes, represents the sample features, represents the th class prototype in the source domain.

[0048] Finally, the total loss of the model is obtained as: (6); In the formula, is the loss weight, represents the weighted adversarial loss, represents the prototype dispersion loss, represents the intra-class consistency loss.

[0049] Step 7 is specifically as follows: First, calculate the distance entropy of all target domain samples, select 5% of the target domain samples with larger entropy values, calculate the mean of the feature vectors of these samples, and use this mean as the class prototype of the unknown class, and add it to the source domain as the unknown class samples of the target domain to form the target auxiliary domain. Input the target domain data into the classification network, use the total loss function to train the network and continuously update the target auxiliary domain, the source domain class weights, and the target domain sample weights, and finally obtain the classification result.

[0050] Example 6 Implement image classification on the Office31 dataset using the method provided by the present invention. The specific steps are as follows: First, obtain the source domain data and the target domain data to form the dataset for the image classification task. For the office31 dataset, take the ten shared classes of the office31 dataset and the caltech-256 dataset as the source domain and the target domain

[0051] common classes. In alphabetical order, the next 10 classes are used as the source domain private classes, and the last 11 classes are used as the unknown classes of the target domain.

[0052] Then, map the source and target domain data to the hypersphere. As Figure 2 , map the source domain sample and the target domain sample Input into the trained feature extraction network to obtain -dimensional features and , calculate the Euclidean norm of the feature vector, and normalize the original feature vector to a unit vector using the Euclidean norm and .

[0053] Next, initialize the source domain class prototypes . Initially, set all source domain class prototypes to 0 temporarily. According to the labels of the source domain samples, stack their feature vectors into the corresponding class prototypes, and divide the accumulated features after stacking the samples of each category by the number of samples in that category for averaging. Then, normalize all class prototypes to ensure that the L2 norm of each prototype vector is 1, obtain the representative features of each category, and maintain the unit length of these class prototypes to avoid the influence caused by the feature scale differences between different categories

[0054] Calculate the source domain class weights using the model's prediction ability , and separate the source domain private classes and shared classes. For the source domain private classes, which are the classes not in the target domain, the target domain samples will not be over-predicted to the private classes in the feature space. Specifically, calculate the prediction values and output scores obtained by all target domain samples through the model. To avoid large differences in scores between classes, calculate the mean of the output scores of the target domain samples predicted to each class and normalize it to obtain the weights at the source domain class level

[0055] Calculate the target domain sample weights . For the target domain samples , obtain the features through the feature extraction network , calculate the distance to the th class prototype , and normalize it to . Perform softmax processing on the normalized distance to obtain the similarity of the sample to each prototype vector, that is . Next, use to obtain the sample weight .

[0056] Use the domain discriminator D for adversarial training to classify the input feature representations to determine whether they are from the source domain or the target domain. Source samples and target samples with different confidences provide different levels of losses

[0057] Update the corresponding class prototypes using each batch of source domain samples. For example, for the sample and the class prototype , the update rule is . Calculate the similarity between all prototypes and other prototypes. Next, use the intra-class consistency loss to bring each category sample closer to the corresponding class prototype.

[0058] Count the distance entropy of all target domain samples. Distance entropy is used to measure the distribution uncertainty of each target domain sample and its neighboring samples. Samples with higher distance entropy are usually located at the boundary of the feature space or in a more ambiguous area. These samples usually represent areas where the classification is unclear and have higher uncertainty. Select 5% of the target domain samples with larger entropy values, calculate the mean of the feature vectors of these samples, and use the mean as the class prototype of the unknown class. Add the source domain as the unknown class sample of the target domain to form the target auxiliary domain. This class prototype can represent the feature distribution of those samples in the target domain that are difficult to classify. By updating the unknown class prototype, the representation of the unknown class is improved, and the model's recognition ability of the unknown class in the target domain is enhanced. Maintain the same update mode and frequency as the original class prototype of the source domain.

[0059] The test results on the office31 dataset are compared with other general domain adaptive image classification methods, as shown in Table 2. The ours in Table 2 is the test result brought by the method provided by the present invention. It can be concluded that the method provided by the present invention has highly balanced OS and HOS performance, is more suitable for complex migration scenarios, and is significantly better than existing methods under HOS conditions. There is no performance shortcoming in all tasks. At the same time, it also verifies the effectiveness of the hypersphere mapping combined with the domain adaptive strategy, achieving better migration performance and clearer classified output images.

[0060] Table 2 Experimental results of general domain adaptation methods on Office31 dataset

Claims

1. A single target tracking method based on prompt learning, characterized in that: The specific steps include: Step 1: Obtain source domain data and target domain data to form a data set for the image classification task; Step 2: Establish a deep learning model and pre-train it. The deep learning model includes a feature extractor F, a classifier C, and a domain discriminator D. Step 3: Map the source domain data and target domain data to the hypersphere to obtain the source domain class prototype ; Step 4: Initialize the source domain class prototype , set all class prototypes in the source domain to 0; Step 5: Calculate the source domain category weight , separate the source domain private class and shared class; and calculate the target domain sample weight ; Step 6: Use the weighted training model obtained in step 5 and the domain discriminator D to perform adversarial training to determine whether the data comes from the source domain or the target domain. Get the weighted adversarial loss; Step 7: Filter the target domain samples, calculate the unknown class prototype, obtain the target domain auxiliary domain, and output the image for classification.

2. The general domain adaptive image classification method based on distance entropy weighting according to claim 1 is characterized in that: The source domain data and target domain data in step 1 are: Follow the distribution , the source domain label is , the number of samples is ,Right now , target domain data Follow the distribution , the number of samples is ,Right now .

3. The general domain adaptive image classification method based on distance entropy weighting according to claim 2 is characterized in that: Step 2 is as follows: use the source domain data to preliminarily train the deep learning model, and use the backbone of the pre-trained ResNet50 network as the feature extractor F; Then, the extracted features are passed to a predefined residual block, which includes a convolutional layer, a normalization layer, and a residual connection; finally, the extracted features are passed to a fully connected layer, which adjusts the output dimension according to the number of input categories for classification tasks; the model is trained by reducing the cross entropy loss function, and the cross entropy loss is calculated as: （1） In the formula, represents the total number of source domain samples, Respectively represent the source domain Source domain samples and their category labels; represents the total number of source domain categories, Indicates that the sample is in The label of each category has a value of 0 or 1. For this sample in Output probability values ​​for each category; multiple rounds of training are performed until a model with classification capabilities for source domain data is obtained.

4. The general domain adaptive image classification method based on distance entropy weighting according to claim 3 is characterized in that: The step 3 specifically includes: and target domain samples Send it to the feature extractor F obtained in step 2, and get Dimensional Features and , calculate the Euclidean norm of the eigenvector, and use the Euclidean norm to normalize the original eigenvector to the unit eigenvector and , at this time, the source domain data and the target domain data have been mapped to the unit hypersphere.

5. The general domain adaptive image classification method based on distance entropy weighting according to claim 4 is characterized in that: The specific step 4 is as follows: initially, all class prototypes in the source domain are set to 0, and according to the labels of the source domain samples, their feature vectors are The samples of each category are superimposed on the corresponding class prototype, and the accumulated features after superposition are divided by the number of samples of that category for averaging; then the prototypes of all categories are normalized so that each prototype The L2 norm of the vector is 1, which obtains the representative features of each category and maintains the unit length of the class prototype.

6. The general domain adaptive image classification method based on distance entropy weighting according to claim 1, characterized in that: Calculate the source domain category weights as described in step 5 Specifically, the prediction values ​​and output scores of all target domain samples obtained by the model are calculated, and the mean of the output scores of target domain samples predicted to each category are statistically normalized to obtain the category weights at the source domain category level. .

7. The general domain adaptive image classification method based on distance entropy weighting according to claim 1, characterized in that: Calculate the target domain sample-level weights as described in step 5, The target domain sample weight Specifically: For the target domain sample , the feature is obtained through the feature extractor F , calculate its Class prototype Distance , and normalized to , perform softmax processing on the normalized distance to obtain the similarity of the sample to each prototype vector, that is, , calculate the sample distance entropy, the specific formula is as follows: （2） In the formula, represents the total number of source domain categories, Indicates the sample and The similarity of the prototype vectors of the classes.

8. The general domain adaptive image classification method based on distance entropy weighting according to claim 2, characterized in that: The step 6 specifically includes: using the weights obtained in step 5 for weighted training, while increasing the distance between prototypes to encourage samples to approach the class prototype; And use the domain discriminator D for adversarial training to classify the input feature representation and determine whether the data comes from the source domain or the target domain. Source samples and target samples with different confidence levels provide different levels of loss, namely weighted adversarial loss, which is specifically: （3） In the formula, represents the total number of source domain samples, represents the total number of samples in the target domain, Respectively represent the source domain Source domain samples and their category labels; and Represent the source domain category weight and target domain sample weight respectively; Indicates that the source domain samples pass through the feature extractor The obtained features are fed into the domain discriminator The output value obtained; Indicates that the target domain sample passes through the feature extractor The obtained features are fed into the domain discriminator The output value obtained; Use each batch of source domain samples to update the corresponding class prototype. and class prototype , the update rule is , calculate the similarity between all prototypes and other prototypes, and get the prototype dispersion loss, which is: （4） In the formula, represents the total number of source domain categories, and The source domain The class prototype and Class prototype; Next, in order to bring each category sample closer to the corresponding class prototype, the intra-class consistency loss is introduced: （5） In the formula, represents the total number of source domain samples, Respectively represent the source domain Source domain samples and their category labels; represents the total number of source domain categories, represents the sample characteristics, The source domain Class prototype; Finally, the total loss of the model is: （6） In the formula, is the loss weight, represents the weighted adversarial loss, represents the prototype dispersion loss, represents the intra-class consistency loss.

9. The general domain adaptive image classification method based on distance entropy weighting according to claim 1, characterized in that: The step 7 is specifically as follows: first, the distance entropy of all target domain samples is counted, 5% of the target domain samples with larger entropy values ​​are selected, the mean of the feature vectors of these samples is calculated, and the mean is used as the class prototype of the unknown class, and the source domain is added as the unknown class sample of the target domain to form a target auxiliary domain; the target domain data is input into the classification network, the network is trained using the total loss function, and the target auxiliary domain, source domain category weights and target domain sample weights are continuously updated to finally obtain the classification result.

Images