A domain-adaptive image classification method based on regularization and joint autonomous training

Abstract

This invention discloses a domain-adaptive image classification method based on regularization-coupled autonomous training, comprising the following steps: 1. Obtaining a source domain dataset and a target domain dataset; 2. Building a domain-adaptive network model comprising a feature extractor, a source domain classifier, and a target domain classifier; 3. Training the domain-adaptive network model using a regularization-coupled autonomous training strategy; and 4. Using the trained domain-adaptive network model for image classification. This method achieves automatic cross-domain generalization of the model and robustness to noisy data, overcoming the limitations of traditional domain-adaptive methods, such as high training overhead and reliance on high-quality datasets, and is more suitable for practical image classification applications.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a domain-adaptive image learning method, and in particular to a domain-adaptive image classification method based on regularization combined with autonomous training. Background Art

[0002] Image classification is a key research area in computer vision, widely used in areas such as autonomous driving, facial recognition, security monitoring, and medical imaging diagnosis. Currently, deep neural networks have become a key tool for image classification. They tend to be trained on large datasets, assuming that the training and test sets have the same data distribution. However, in real-world problems, the distribution of training datasets often differs significantly from the actual scene. For example, in facial recognition, if a model trained on Asian faces is used to identify Westerners, the model's performance will significantly decline. Taking autonomous driving as another example, the training samples in synthetic simulations often differ from those on actual roads, resulting in reduced performance of models trained in synthetic simulations in real-world applications.

[0003] In recent years, domain adaptation models and algorithms have been proposed to address these issues. Domain adaptation, a sub-discipline of transfer learning, addresses inter-domain transfer tasks where the class space and feature space are aligned, but the class distributions between domains are shifted. Domain adaptation uses a labeled dataset to train a model and applies it to an unlabeled dataset with a similar distribution. The labeled dataset is called the source domain, and the unlabeled dataset is called the target domain. Feature alignment is a popular domain adaptation algorithm, which transfers knowledge by reducing the gap between the feature distributions of the source and target domains. However, due to inherent limitations, feature alignment algorithms cannot achieve robust performance. Autonomous training, derived from unsupervised learning, has made some progress as an alternative to feature alignment. In domain adaptation tasks, autonomous training generates pseudo-labels for unlabeled samples in the target domain using a source domain classifier. The model is then trained jointly using the source domain labeled data and the target domain pseudo-labeled data. However, due to the shift in class distributions between domains, some pseudo-labels are noisy. Therefore, directly training the model using the noisy source domain labels and pseudo-labels will result in error accumulation and increase transfer error. Classic methods filter pseudo-labels by setting confidence thresholds or confidence weighting. However, these methods require complex threshold adjustments and re-assignment of weights when dealing with different tasks, and the improvement in model performance is still inconsistent.

[0004] In addition to the problem of inter-domain class distribution shift, the presence of noisy labels in the dataset is also a significant factor affecting model performance. The vast majority of labels in large-scale image datasets such as ImageNet and CIFAR-10 are manually annotated, and domain adaptation models trained on these datasets perform well. However, due to the cost of manual annotation and the difficulty of data cleaning, datasets often contain a certain amount of noisy labels. In image classification problems, the classic cross-entropy loss function can cause models to overfit to noisy labels when exposed to noisy data, resulting in poor generalization. Existing methods for addressing noisy labels generally rely on label correction, reweighting, and the use of noise-robust loss functions. Label correction and reweighting are prohibitively expensive, making them impractical for large-scale practical application. Furthermore, the use of noise-robust loss functions in feature alignment and autonomous training can lead to underfitting of the model to the data, severely limiting their application prospects in real-world image classification problems. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention proposes a domain-adaptive image classification method based on regularization and joint autonomous training, in order to achieve automatic cross-domain generalization of the image classification model and the robustness of the image classification model to noisy data, thereby improving the classification performance of the domain-adaptive method and the image classification model when facing the problems of inter-domain class distribution shift and noisy labels.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0007] The present invention provides a domain-adaptive image classification method based on regularization combined with autonomous training, which comprises the following steps:

[0008] Step 1. Obtain the source domain dataset and target domain dataset ;in, Indicates the i Zhang source domain image, express The corresponding source domain label, Indicates the j A target domain image, Represents the source domain dataset The total number of source domain samples, Represents the target domain dataset The number of target domain samples in ;

[0009] Step 2. Build a domain-adaptive network model, including: Feature Extractor , source domain classifier , target domain classifier ;

[0010] The feature extractor Depend on T The feature extraction unit consists of a convolution layer, a ReLU activation function layer, a pooling layer and a fully connected layer.

[0011] The source domain classifier and target domain classifier They have the same network structure, which includes an input convolution layer, downsampling convolutional layers, There are three residual convolution layers and one fully connected layer, and each convolution layer is connected to a BN layer and a ReLU activation function;

[0012] Step 3. Regularized joint autonomous training strategy for domain adaptive network model;

[0013] Step 3.1. i Zhang Yuan domain image and the aforementioned j target domain image Input feature extractor separately In, and through T After processing by the feature extraction unit, we get i Zhang Yuan domain image The eigenvector of Hedi j target domain image The eigenvector of ;

[0014] Step 3.2. j target domain image The eigenvector of Input the source domain classifier Classify in and get the source domain classifier For the first j target domain image Classification results , and then the classification results After sharpening, the j target domain image Pseudo labels ;

[0015] The said j target domain image The eigenvector of Input the target domain classifier Classify in and get the target domain classifier For the first j target domain image Classification results ;

[0016] Use formula (1) to build the target domain classifier For the j target domain image Pseudo labels And the classification results The mean square error loss function :

[0017] (1)

[0018] Step 3.3. i Zhang Yuan domain image The eigenvector of Input the source domain classifier respectively and target domain classifier Perform classification and obtain the source domain classifier For the first i Zhang Yuan domain image Classification results and target domain classifier For the first i Zhang Yuan domain image Classification results ;

[0019] Use formula (2) to build the source domain classifier About the Source Domain Classifier For the first i Zhang Yuan domain image Classification results And the true source domain label Early learning regularization loss function :

[0020] (2)

[0021] In formula (2), Represents the source domain classifier For the first i Zhang Yuan domain image The classification results of The probability corresponding to each category, Indicates the i Zhang Yuan domain image The true source domain label Middle The probability corresponding to each category, Represents the source domain dataset The total number of categories of images in , represents the regularization parameter;

[0022] Use formula (3) to construct feature extractor About the Source Domain Classifier For the first i Zhang Yuan domain image Classification results , target domain classifier For the first i Zhang Yuan domain image Classification results and i Zhang Yuan domain image The true source domain label The joint loss function :

[0023] (3)

[0024] In formula (3), Represents the target domain classifier For the first i Zhang Yuan domain image The classification results of The probability corresponding to each category;

[0025] Step 4. Use the stochastic gradient descent method to perform regularization-based joint autonomous training on the domain adaptation network model to obtain the optimal image classification model.

[0026] The domain-adaptive image classification method based on regularization combined with autonomous training of the present invention is characterized in that step 4 comprises:

[0027] Step 4.1. Use formula (4) to construct the overall objective function of the domain adaptive network model:

[0028] (4)

[0029] Step 4.2. Set the maximum number of iterations to , the current number of iterations is , and initialize ;

[0030] Step 4.3. From the source domain dataset and target domain dataset The batch size is randomly selected from The image sample is taken as the The source domain training samples and target domain training samples of the iteration;

[0031] Step 4.4. Using formula (5) in the Optimize the target domain classifier in iterations , and get the The target domain classifier of the iteration :

[0032] (5)

[0033] In formula (5), Represents the target domain classifier For the In the iteration The first target domain training sample The classification results of image samples, Indicates the In the iteration The first target domain training sample Pseudo labels for image samples;

[0034] Step 4.5. Use equations (6) and (7) to calculate the Optimize the source domain classifier in iterations and feature extractor , and get the The source domain classifier of the iteration and feature extractor :

[0035] (6)

[0036] (7)

[0037] In formula (6) and formula (7), and Represents the source domain classifier and target domain classifier For the In the iteration The source domain training samples The classification results of image samples, Indicates the In the iteration The source domain training samples The true source domain labels of image samples, and Respectively represent In the iteration The source domain training samples Classification results of image samples and Middle The probability corresponding to each category, Indicates the In the iteration The source domain training samples The true source domain labels of image samples Middle The probability corresponding to each category, Representative The learning rate of the model in iterations, Representative The correction coefficient of the regularization loss function in the early iteration is obtained by formula (8);

[0038] (8)

[0039] Step 4.6. Judgment Or whether formula (4) converges, if so, it means that the training of the domain adaptive network model is completed and the optimal image classification model is obtained; otherwise, Assign to After that, update the learning rate , return to step 4.3 and execute sequentially.

[0040] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute the domain adaptive image classification method, and the processor is configured to execute the program stored in the memory.

[0041] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the domain-adaptive image classification method when executed by a processor.

[0042] Compared with the prior art, the beneficial effects of the present invention are embodied in:

[0043] 1. The present invention proposes a domain-adaptive image classification method based on regularization and autonomous training, which realizes the automatic cross-domain generalization of the model and the robustness of the model to noisy data. It overcomes the problems that traditional domain-adaptive methods are limited by the high training overhead and dependence on high-quality datasets and cannot be widely used, and is more suitable for the actual application scenarios of image classification.

[0044] 2. The present invention proposes a joint autonomous training strategy, which introduces a target domain classifier on the basis of the standard autonomous training method which only contains a source domain classifier. The source domain classifier and the feature extractor are explicitly jointly optimized through three steps: generating pseudo labels by the source domain classifier, training the target domain classifier using the pseudo labels, and optimizing the source domain classifier and feature extractor in the source domain. This realizes automatic cross-domain generalization of pseudo labels and effectively migrates the source domain information to the target domain. It makes up for the defects of excessive overhead and poor migration effect caused by the classic method of solving the problem of inter-domain class distribution offset by setting confidence thresholds or confidence weightings, and effectively improves the usability of this classification method in actual image classification scenarios.

[0045] 3. Based on the early learning phenomenon of deep neural networks, the present invention proposes an early learning regularization loss function, which implicitly prevents the model from memorizing noisy labels. It overcomes the problem that noisy labels and erroneous pseudo-labels in the dataset interfere with the classification performance of the image model without adding additional overhead, and realizes an image classification method that is more robust to noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the method of the present invention;

[0047] Figure 2 It is a structural diagram of the model of the present invention. DETAILED DESCRIPTION

[0048] In this embodiment, Figure 1 As shown in FIG, a domain-adaptive image classification method based on regularization combined with autonomous training includes the following steps:

[0049] Step 1. Obtain the source domain dataset and target domain dataset ;in, Indicates the i Zhang source domain image, express The corresponding source domain label, Indicates the j A target domain image, Represents the source domain dataset The total number of source domain samples, Represents the target domain dataset The number of target domain samples in ;

[0050] In this example, we used VisDA-2017, Office-Home, and Office-31. VisDA2017 is a UDA dataset with two domains: Synthetic and Real, encompassing 12 object categories. Office-Home offers four distinct domains: Real-World, Product, Artistic, and Clip Art, each containing 64 classes. Office-31 encompasses three domains with a total of 31 object categories: Amazon, DSLR, and Webcam.

[0051] Step 2. Figure 2 As shown, a domain adaptive network model is built, which includes: feature extractor , source domain classifier , target domain classifier ;

[0052] Feature Extractor Depend on The feature extraction unit consists of a convolution layer, a ReLU activation function layer, a pooling layer and a fully connected layer.

[0053] Source domain classifier and target domain classifier They have the same network structure, which includes an input convolution layer, downsampling convolutional layers, A residual convolution layer, a fully connected layer, and each convolution layer is connected to a BN layer and a ReLU activation function; in this embodiment, Set to 5, Set to 5, Set to 4;

[0054] Step 3. Regularized joint autonomous training strategy for domain adaptive network model;

[0055] Step 3.1. i Zhang Yuan domain image Hedi j target domain image Input feature extractor separately In, and through T After processing by the feature extraction unit, we get i Zhang Yuan domain image The eigenvector of Hedi j target domain image The eigenvector of ; The feature vector is obtained by expanding the feature map output by the convolution layer, and its dimension is 512;

[0056] Step 3.2. j target domain image The eigenvector of Input source domain classifier Classify in and get the source domain classifier For the first j target domain image Classification results , and then the classification results After sharpening, the j target domain image Pseudo labels ;

[0057] No. j target domain image The eigenvector of Input target domain classifier Classify in and get the target domain classifier For the first j target domain image Classification results ;

[0058] Building a target domain classifier For the j target domain image Pseudo labels And the classification results The mean square error loss function , and use formula (1) to train the target domain classifier :

[0059] (1)

[0060] Step 3.3. i Zhang Yuan domain image The eigenvector of Input source domain classifier separately and target domain classifier Perform classification and obtain the source domain classifier For the first i Zhang Yuan domain image Classification results and target domain classifier For the first i Zhang Yuan domain image Classification results ;

[0061] Use formula (2) to build the source domain classifier About the Source Domain Classifier For the first i Zhang Yuan domain image Classification results And the true source domain label Early learning regularization loss function :

[0062] (2)

[0063] In formula (2), Represents the source domain classifier For the first i Zhang Yuan domain image The classification results of The probability corresponding to the category, Indicates the i Zhang Yuan domain image The true source domain label Middle The probability corresponding to each category, Represents the source domain dataset The total number of categories of images in , represents the regularization parameter;

[0064] Use formula (3) to construct feature extractor About the Source Domain Classifier For the first i Zhang Yuan domain image Classification results , target domain classifier For the first i Zhang Yuan domain image Classification results and i Zhang Yuan domain image The true source domain label The joint loss function :

[0065] (3)

[0066] In formula (3), Represents the target domain classifier For the first i Zhang Yuan domain image The classification results of The probability corresponding to each category;

[0067] Step 4. Use the stochastic gradient descent method to perform regularization-based joint autonomous training on the domain adaptation network model to obtain the optimal image classification model;

[0068] Step 4.1. Use formula (4) to construct the overall objective function of the domain adaptive network model:

[0069] (4)

[0070] Step 4.2. Set the maximum number of iterations to , the current number of iterations is , and initialize ; In this embodiment, the maximum number of iterations Set to 150 times, set the The learning rate of the model in the iteration is , the initial learning rate Set to 5e-3, the decay factor Set to 0.1, and the learning rate is decayed every 50 iterations.

[0071] Step 4.3. From the source domain dataset and target domain dataset The batch size is randomly selected from The image sample is taken as the The source domain training samples and target domain training samples of the iteration;

[0072] Step 4.4. Using formula (5) in the Optimize the target domain classifier in iterations , and get the The target domain classifier of the iteration :

[0073] (5)

[0074] In formula (5), Represents the target domain classifier For the In the iteration The first target domain training sample The classification results of image samples, Indicates the In the iteration The first target domain training sample Pseudo labels for image samples;

[0075] Step 4.5. Use equations (6) and (7) to calculate the Optimize the source domain classifier in iterations and feature extractor , and get the The source domain classifier of the iteration and feature extractor :

[0076] (6)

[0077] (7)

[0078] In formula (6) and formula (7), and Represents the source domain classifier and target domain classifier For the In the iteration The source domain training samples The classification results of image samples, Indicates the In the iteration The source domain training samples The true source domain labels of image samples, and Respectively represent In the iteration The source domain training samples Classification results of image samples and Middle The probability corresponding to each category, Indicates the In the iteration The source domain training samples The true source domain labels of image samples Middle The probability corresponding to each category, Representative The learning rate of the model in iterations, Representative The correction coefficient of the regularization loss function in the early iteration is obtained by formula (8);

[0079] (8)

[0080] Step 4.6. Judgment Or whether formula (4) converges, if so, it means that the training of the domain adaptive network model is completed and the optimal image classification model is obtained; otherwise, Assign to After that, update the learning rate , the update of the learning rate is expressed as its linear decay, that is, ,in, is the attenuation factor. In this embodiment, Set it to 0.1 and return to step 4.3 to execute the sequence.

[0081] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0082] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed. Specific embodiment:

[0084] To illustrate the classification performance of the proposed method, we compared it with the classification accuracy of a classic domain adaptation algorithm on VisDA-2017, Office-Home, and Office-31. The classification results are shown in Tables 1, 2, and 3, and the ablation experiment results are shown in Table 4.

[0085] Table 1 Average classification accuracy of model experiments on the VisDA-2017 dataset (unit: %)

[0086]

[0087] Table 1 shows the experimental classification accuracy of our method on the VisDA-2017 dataset. Our method is the complete method proposed in this paper. Our method achieves the highest average classification accuracy on the VisDA-2017 dataset.

[0088] Table 2: Classification accuracy of the model experiment on the Office-Home dataset (unit: %)

[0089]

[0090] Table 2 shows the classification accuracy results of our method for 12 pairs of transfer tasks on the Office-Home dataset. Our method achieved the highest accuracy in all 12 transfer tasks. In the case of domain transfer, standard self-training methods suffer from reduced pseudo-label quality. For example, in the Pr-Cl task, where domain distributions vary widely, other methods achieved an average accuracy of 53.3%, while our method achieved an accuracy of 74.7%, significantly outperforming other methods.

[0091] Table 3: Classification accuracy of the model experiment on the Office-31 dataset (unit: %)

[0092]

[0093] Table 3 shows the classification accuracy of our model experiments on the Office-31 dataset. Our method achieved the highest accuracy in four of the six transfer tasks. Notably, in previous work, CDAN and MDD, as feature alignment methods, generally performed better for large-to-small tasks (AW and AD), while GTA, as a pixel-level adaptation method, achieved higher accuracy for small-to-small tasks (DA and WA). However, our method achieved the highest accuracy on the DA and WA tasks, second and third highest on the AW and AD tasks, respectively, achieving the highest average accuracy across all six tasks, demonstrating the effectiveness and high accuracy of our method.

[0094] Table 4 Ablation experiments of the present invention on the VisDA-2017 dataset (unit: %)

[0095]

[0096] Table 4 studies the role of each part of the present invention in training. ELR-loss represents the early regularization loss function. Ours w / o ELR-loss means that Ours deletes the ELR loss function. Ours+Entropy uses standard entropy instead of ELR-loss. Observe the results in Table 4. Although the average accuracy of Ours+Entropy on VisDA-2017 is about 5.5% higher than that of Ours w / o ELR-loss, it is still about 12.5% ​​behind Ours, which shows that ELR-loss is a better regularization method than standard entropy. FixMatch is a standard autonomous training method. Although its performance is better than Ours w / o ELR-loss, it lags behind Ours by about 14.5%. This shows that in the case of inter-domain class distribution shift, Ours has achieved higher robustness than standard autonomous training by introducing ELR-loss to solve the pseudo-label noise problem.

Claims

1. A domain-adaptive image classification method based on regularization combined with autonomous training, characterized in that: The steps include: Step 1. Obtain the source domain dataset and target domain dataset ;in, Indicates the i Zhang source domain image, express The corresponding source domain label, Indicates the j A target domain image, Represents the source domain dataset The total number of source domain samples, Represents the target domain dataset The number of target domain samples in ; Step 2. Build a domain-adaptive network model, including: Feature Extractor , source domain classifier , target domain classifier ; The feature extractor Depend on T The feature extraction unit consists of a convolution layer, a ReLU activation function layer, a pooling layer and a fully connected layer. The source domain classifier and target domain classifier They have the same network structure, which includes an input convolution layer, downsampling convolutional layers, There are three residual convolution layers and one fully connected layer, and each convolution layer is connected to a BN layer and a ReLU activation function; Step 3. Regularized joint autonomous training strategy for domain adaptive network model; Step 3.

1. i Zhang Yuan domain image and the aforementioned j target domain image Input feature extractor separately In, and through T After processing by the feature extraction unit, we get i Zhang Yuan domain image The eigenvector of Hedi j target domain image The eigenvector of ; Step 3.

2. j target domain image The eigenvector of Input the source domain classifier Classify in and get the source domain classifier For the first j target domain image Classification results , and then the classification results After sharpening, the j target domain image Pseudo labels ; The said j target domain image The eigenvector of Input the target domain classifier Classify in and get the target domain classifier For the first j target domain image Classification results ; Use formula (1) to build the target domain classifier For the j target domain image Pseudo labels And the classification results The mean square error loss function : (1) Step 3.

3. i Zhang Yuan domain image The eigenvector of Input the source domain classifier respectively and target domain classifier Perform classification and obtain the source domain classifier For the first i Zhang Yuan domain image Classification results and target domain classifier For the first i Zhang Yuan domain image Classification results ; Use formula (2) to build the source domain classifier About the Source Domain Classifier For the first i Zhang Yuan domain image Classification results And the true source domain label Early learning regularization loss function : (2) In formula (2), Represents the source domain classifier For the first i Zhang Yuan domain image The classification results of The probability corresponding to each category, Indicates the i Zhang Yuan domain image The true source domain label Middle The probability corresponding to each category, Represents the source domain dataset The total number of categories of images in , represents the regularization parameter; Use formula (3) to construct feature extractor About the Source Domain Classifier For the first i Zhang Yuan domain image Classification results , target domain classifier For the first i Zhang Yuan domain image Classification results and i Zhang Yuan domain image The true source domain label The joint loss function : (3) In formula (3), Represents the target domain classifier For the first i Zhang Yuan domain image The classification results of The probability corresponding to each category; Step 4. Use the stochastic gradient descent method to perform regularization-based joint autonomous training on the domain adaptation network model to obtain the optimal image classification model.

2. The domain-adaptive image classification method based on regularization combined with autonomous training according to claim 1, characterized in that: The step 4 comprises: Step 4.

1. Use formula (4) to construct the overall objective function of the domain adaptive network model: (4) Step 4.

2. Set the maximum number of iterations to , the current number of iterations is , and initialize ; Step 4.

3. From the source domain dataset and target domain dataset The batch size is randomly selected from The image sample is taken as the The source domain training samples and target domain training samples of the iteration; Step 4.

4. Using formula (5) in the Optimize the target domain classifier in iterations , and get the The target domain classifier of the iteration : (5) In formula (5), Represents the target domain classifier For the In the iteration The first target domain training sample The classification results of image samples, Indicates the In the iteration The first target domain training sample Pseudo labels for image samples; Step 4.

5. Use equations (6) and (7) to calculate the Optimize the source domain classifier in iterations and feature extractor , and get the The source domain classifier of the iteration and feature extractor : (6) (7) In formula (6) and formula (7), and Represents the source domain classifier and target domain classifier For the In the iteration The source domain training samples The classification results of image samples, Indicates the In the iteration The source domain training samples The true source domain labels of image samples, and Respectively represent In the iteration The source domain training samples Classification results of image samples and Middle The probability corresponding to each category, Indicates the In the iteration The source domain training samples The true source domain labels of image samples Middle The probability corresponding to each category, Representative The learning rate of the model in iterations, Representative The correction coefficient of the regularization loss function in the early iteration is obtained by formula (8); (8) Step 4.

6. Judgment Or whether formula (4) converges, if so, it means that the training of the domain adaptive network model is completed and the optimal image classification model is obtained; otherwise, Assign to After that, update the learning rate , return to step 4.3 and execute sequentially.

3. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the domain adaptive image classification method according to claim 1 or 2, and the processor is configured to execute the program stored in the memory.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the domain adaptive image classification method according to claim 1 or 2 are executed.

Images