Image classification method and system based on label noise domain adaptation
Through adaptive noise detection and metanet weighted sample loss, the problem of low accuracy caused by label noise in image classification is solved, and higher image classification accuracy and better generalization performance are achieved.
Patent Information
- Application Number
- CN202210567062.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-23
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2042-05-23
AI Technical Summary
The existing domain adaptive method in image classification tasks has low classification accuracy due to the existence of label noise, making it difficult to apply in actual tasks.
Adopting the image classification method based on label noise domain adaptation, the contribution of noise samples to the classifier is reduced through adaptive noise detection and meta-network weighted sample loss, and the noise samples are corrected through adaptive noise correction, and the image data samples are calculated to the source domain. The contribution weights aligned with the target domain task until the total model loss converges, and a robust noise domain adaptation model is obtained.
It effectively reduces the impact of label noise and improves the accuracy of image classification, so that the model has better generalization performance in real scenes and obtains more accurate image classification results.
Smart Images
Figure CN114842267B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an image classification method and system based on label noise domain adaptation, belonging to the technical field of image processing. Background Art
[0002] Domain Adaptation learning is an important branch of transfer learning. It can more concretely point out the goal of transfer: finding the correlation between homogeneous data sets / tasks. Domain Adaptation learning attempts to use the source domain with rich labels to help the target domain with scarce labels, and has achieved significant performance improvements on many tasks.
[0003] In recent years, SOTA unsupervised domain adaptation methods can be roughly divided into three categories: methods based on difference metrics, methods based on game confrontation between classifiers and domain discriminators, and methods based on reconstruction to generate cross-domain features. The method based on difference metrics aims to measure the distribution differences between domains and achieve alignment of the source domain and the target domain in the feature space by narrowing the distribution differences. The method based on the game confrontation between the classifier and the domain discriminator extracts domain-invariant features through adversarial learning, and trains the classifier based on the domain-invariant features, so that the classifier trained with source domain data also has good robustness to the target domain. The method based on reconstruction to generate cross-domain features is an adversarial method at the sample level. It generates samples with high similarity to the target domain based on GAN (Generative Adversarial Network), and can effectively generate target domain features.
[0004] However, most of the current domain adaptation methods still assume that the rich labels in the source domain are correct, but accurate manual annotations are time-consuming and labor-intensive and are usually difficult to collect, which limits the application of domain adaptation methods in real-world tasks such as image classification.
[0005] In real-world tasks, rich image data is more easily obtained from social media and the Internet, but it is difficult for such image datasets to meet the condition that all data are correctly labeled. Image classification model training tends to favor high-loss samples, that is, samples with higher confidence label noise. Therefore, traditional domain adaptation methods cannot achieve good results in image classification processing.
[0006] In image classification processing, due to the existence of label noise in image data samples, on the one hand, it greatly affects the learning of image classification models, and on the other hand, it greatly interferes with the learning of category domain features, resulting in relatively low image classification accuracy. The classification accuracy needs to be improved.
[0007] The above problems are those that should be considered and solved in the process of image classification. Summary of the invention
[0008] The purpose of the present invention is to provide an image classification method and system based on label noise domain adaptation to solve the problem that the domain adaptation method in the prior art has low accuracy and insufficient precision in image classification tasks, which needs to be improved.
[0009] The technical solution of the present invention is:
[0010] An image classification method based on label noise domain adaptation comprises the following steps:
[0011] S1. Preprocess the image data in the source domain and the target domain to obtain a source domain dataset consisting of image data samples in the source domain and a target domain dataset consisting of image data samples in the target domain, respectively, and the image data samples in the source domain contain label noise;
[0012] S2. Build an image classification model, perform adaptive noise detection on the image data samples in the source domain and obtain sample weights. Use the threshold method to divide the source domain data set into a noise sample set and a clean sample set based on the obtained sample weights. After adaptive noise correction is performed on the clean sample set, calculate the contribution weight of the image data sample to the category domain in the process of aligning the source domain and the target domain tasks. The total model loss is obtained from the obtained contribution weight and the sample weight until the total model loss converges and the training is performed to obtain a robust noise domain adaptation model.
[0013] S3. The robust noise domain adaptation model obtained in step S2 performs image classification on the image data samples in the target domain, and outputs the predicted labels for the image data samples in the target domain, that is, obtains the image classification results of the target domain.
[0014] Furthermore, in step S2, a robust noise domain adaptation model is obtained by training, specifically,
[0015] S21, initialize the multi-layer perceptron classifier and the meta-network, and construct an image classification model for solving label noise scenarios by the feature extractor, the multi-layer perceptron classifier and the meta-network;
[0016] S22, adaptive noise detection, inputting the image data samples of the source domain into the image classification model constructed in step S21 to obtain a probability output, calculating the sample loss according to the result of the probability output, and based on the characteristic that clean samples have smaller losses than noise samples, using a neural network with only one hidden layer as a meta-network, and adaptively obtaining sample weights according to the gradient direction through meta-learning behavior, and using the obtained sample weights to divide the source domain data set into a noise sample set and a clean sample set using a threshold method;
[0017] S23, adaptive noise correction, calculating the prototype center for the clean sample set obtained in step S22, and using it to correct the noise sample set to obtain a corrected probability vector;
[0018] S24, calculating the contribution weight of the image data sample to the task alignment of the source domain and the target domain, specifically, inputting the image data sample of the target domain into the image classification model constructed in step S21 to obtain a pseudo label vector; then using the one-hot vector based on the true label for the clean sample set obtained in step S22, using the corrected probability vector obtained in step S23 for the noise sample set obtained in step S22, and using the pseudo label vector for the image data sample of the target domain, to calculate the contribution weight of the image data sample to the category field in the process of task alignment of the source domain and the target domain;
[0019] S25, using the sample weights obtained in step S22 to weight the cross entropy loss of the source domain image samples;
[0020] S26, obtaining the task alignment loss based on the local maximum mean difference according to the contribution weight obtained in step S24;
[0021] S27. Set the dynamic balance parameter λ to balance the cross entropy loss of the source domain image samples obtained in S25 and the task alignment loss based on the local maximum mean difference obtained in S26, calculate the total model loss and perform back propagation, and update the network parameters; determine whether the total model loss converges. When the total model loss converges, the update process ends and the final classification model, i.e., the robust noise domain adaptation model, is obtained; when the total model loss does not converge, return to step S22.
[0022] Furthermore, in step S22, based on the characteristic that clean samples have smaller losses than noise samples, a neural network with only one hidden layer is used as a meta-network, and sample weights are adaptively obtained according to the gradient direction through meta-learning behavior. The obtained sample weights are used to divide the data set of the source domain into a noise sample set and a clean sample set using a threshold method, specifically,
[0023] S221. For the image data samples in the source domain, the weight function of the meta-network is used Mapping sample weights, input is sample loss, output is sample weight in is the source domain sample, and the meta-network function parameter α depends on its On learning performance, namely:
[0024]
[0025] Where V is the number of samples in the validation set, and V is less than the total number of image data samples in the source domain. vali (·,·) is the cross entropy loss calculated on the validation set. Consists of correctly labeled source domain samples, and are the image samples and image labels in the validation set, respectively. is the classifier parameter affected by the meta-network function parameter α in the network update, represents the solution space of the meta-network function parameter α;
[0026] S222, set the threshold thres, if the sample weight obtained in step S221 Will is marked as a noise instance; otherwise, it is marked as a clean instance, i.e. in, and They represent the clean sample set and the noisy sample set after the dataset of the source domain is split.
[0027] Furthermore, in step S23, the prototype center is calculated for the clean sample set obtained in step S22, and is used to correct the noise sample set to obtain a corrected probability vector, specifically,
[0028] S231: Divide the sample set of the source domain in step S22 into a clean sample set For each category, calculate the prototype center
[0029] S232: Divide the sample set in the source domain of step S22 into a noise sample set For each noise sample, its category label is corrected based on the prototype classifier, and the probability prediction of the corrected probability vector corresponding to category c is calculated as follows:
[0030]
[0031] Where K is the total number of categories, Clean sample set The prototype center of the jth class, Φ1(·, ·) is used to measure the similarity between the source domain instance and the prototype of a specific class.
[0032] Furthermore, in step S24, the contribution weight of the image data sample to the category domain in the process of aligning the source domain and the target domain task is calculated, that is:
[0033]
[0034] Among them, P is the prototype classifier for noise correction, G is the classifier for source domain / target domain classification, and are the clean sample set and the noisy sample set after the source domain dataset is split, F is the feature extractor, and X t is the dataset of the target domain.
[0035] Furthermore, in step S25, the cross entropy loss of the source domain image sample is weighted using the sample weight obtained in step S22:
[0036]
[0037] in, Is The parameterized weight function, It is a classification model parameterized by the network parameter θ, including the feature extractor F and the classifier G. and are the image samples and image labels in the source domain dataset, n s is the number of samples from the dataset.
[0038] Further, in step S26, the contribution weight obtained in step S24 is used to obtain the task alignment loss based on the local maximum mean difference:
[0039]
[0040] Among them, K is the total number of categories, X s represents all image data samples in the source domain, X t represents all image data samples in the target domain, Φ2 is the mapping from the feature space to the reproducing kernel Hilbert space, and Respectively represent the contribution weights of source domain and target domain instances to class k in the source domain and target domain task alignment, and are source domain samples and target domain samples respectively.
[0041] Furthermore, the contribution weight of the source domain instance to the k-class in the task alignment between the source domain and the target domain is The contribution weight of the target domain instance to the k classes in the task alignment between the source and target domains All are calculated by the following formula:
[0042]
[0043] in, represents the sample distribution of the data set, represents the contribution weight of the i-th sample to category k, which is calculated in step S24.
[0044] Further, in step S27, the dynamic balance parameter λ is set to balance the cross entropy loss of the source domain image samples obtained in S25 and the task alignment loss based on the local maximum mean difference obtained in S26, the total model loss is calculated and back propagated, and the network parameters are updated, specifically,
[0045] S271, balance the classification loss by the balance parameter λ and task alignment loss The total model loss is calculated as And perform back propagation;
[0046] S272, the image classification model parameters are updated by the following formula:
[0047]
[0048] Among them, λ is the balance parameter, which balances the weight of classification learning loss and task alignment learning loss during training, η θ represents the gradient descent rate of parameter θ, (·) represents the gradient of θ;
[0049] S273. The weight function of sample loss is updated by the following formula:
[0050]
[0051] Among them, η α is the gradient descent rate of parameter α, represents the gradient of α, J vali (·,·) is the cross entropy loss calculated on the validation set, and are the image samples and image labels in the validation set, respectively. is the classifier parameter affected by the meta-network function parameter α in the network update, For the classification model to sample prediction.
[0052] A system using any of the above-mentioned image classification methods based on label noise domain adaptation comprises a data preprocessing module, a model training module and an image classification processing module,
[0053] Data preprocessing module: preprocess the image data in the source domain and the target domain to obtain a source domain dataset consisting of image data samples in the source domain and a target domain dataset consisting of image data samples in the target domain, respectively. The image data samples in the source domain contain label noise.
[0054] Model training module: build an image classification model, perform adaptive noise detection on the image data samples in the source domain and obtain sample weights. Use the threshold method to divide the source domain data set into a noise sample set and a clean sample set based on the obtained sample weights. After adaptive noise correction is performed on the clean sample set, calculate the contribution weight of the image data sample to the category field during the task alignment process between the source domain and the target domain. The total model loss is obtained from the obtained contribution weight and the sample weight until the total model loss converges and the robust noise domain adaptation model is obtained through training.
[0055] Image classification processing module: The obtained robust noise domain adaptation model is used to perform image classification on the image data samples in the target domain, and the predicted labels of the image data samples in the target domain are output, that is, the image classification results of the target domain are obtained.
[0056] The beneficial effects of the present invention are as follows: the image classification method and system based on label noise domain adaptation can reduce the influence of label noise and effectively improve the accuracy of image classification. The method, for the image data samples in the source domain with label noise, reduces the contribution of noise samples to the classifier through adaptive noise detection and based on the weighted sample loss of the meta-network; corrects the noise samples through adaptive noise correction, so that the noise samples obtain labels with higher confidence and re-invest in learning, reduces the influence of noise samples on the alignment of source domain and target domain tasks, and makes the final robust noise domain adaptation model obtained by optimization have better generalization performance in this case that is closer to the real scene, so as to obtain more accurate image classification results. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 It is a flowchart of an image classification method based on label noise domain adaptation according to an embodiment of the present invention.
[0058] Figure 2 Schematic diagram for explaining the robust noise domain adaptation model in the embodiment.
[0059] Figure 3 This is a schematic diagram of the performance comparison between the embodiment method and the classic convolutional neural network AlexetNet, the metric domain adaptation method DAN, the adversarial domain adaptation method DANN and ADDA DETAILED DESCRIPTION
[0060] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0061] Example
[0062] An image classification method based on label noise domain adaptation, such as Figure 1 , including the following steps,
[0063] S1. Preprocess the image data in the source domain and the target domain to obtain a source domain dataset consisting of image data samples in the source domain and a target domain dataset consisting of image data samples in the target domain, respectively, and the image data samples in the source domain contain label noise;
[0064] Before model training, the image data is unified into the format required by the network model input through preprocessing methods such as resizing and random cropping. At the same time, the image data samples in the source domain have noise labels, while the image data samples in the target domain have no noise labels.
[0065] S2. Build an image classification model, perform adaptive noise detection on the image data samples in the source domain and obtain sample weights. Use the threshold method to divide the data set into a noise sample set and a clean sample set based on the obtained sample weights. After adaptive noise correction is performed on the clean sample set, calculate the contribution weight of the image data sample to the category domain in the process of aligning the source domain and the target domain tasks. Use the obtained contribution weights and sample weights to obtain the total model loss until the total model loss converges and the training is performed to obtain a robust noise domain adaptation model. Figure 2 ;
[0066] S21. Initialize the classifier and meta-network. The feature extractor, classifier, and meta-network jointly construct an image classification model that solves the label noise scenario. First, use the ResNet50 pre-trained on ImageNet as the feature extractor. Pre-train the feature extractor so that the feature extractor has excellent feature extraction capabilities. Randomly initialize the classifier and meta-network to form the classification model used. Then cold-start the classification model and perform domain adaptive training so that the model has preliminary feature extraction and classification capabilities.
[0067] S22, adaptive noise detection, inputting the image data samples of the source domain into the image classification model constructed in step S21 to obtain a probability output, calculating the sample loss according to the result of the probability output, and based on the characteristic that clean samples have smaller losses than noise samples, using a neural network with only one hidden layer as a meta-network, and adaptively obtaining sample weights according to the gradient direction through meta-learning behavior, and using the obtained sample weights to divide the source domain data set into a noise sample set and a clean sample set using a threshold method;
[0068] In step S22, based on the characteristic that clean samples have smaller losses than noise samples, a neural network with only one hidden layer is used as a meta-network, and sample weights are adaptively obtained according to the gradient direction through meta-learning behavior. The obtained sample weights are used to divide the source domain data set into a noise sample set and a clean sample set using a threshold method, specifically,
[0069] S221. For the image data samples in the source domain, the weight function of the meta-network is used Mapping sample weights, input is sample loss, output is sample weight in is the source domain sample, and the meta-network function parameter α depends on its On learning performance, namely:
[0070]
[0071] Where V is the number of samples in the validation set, and V is less than the total number of image data samples in the source domain. vali (·,·) is the cross entropy loss calculated on the validation set. It consists of correctly labeled source domain samples, and are the image samples and image labels in the validation set, respectively. is the classifier parameter affected by the meta-network function parameter α in the network update, represents the solution space of the meta-network function parameter α;
[0072] S222, set the threshold thres, if the sample weight obtained in step S221 Will is marked as a noise instance; otherwise, it is marked as a clean instance, i.e. in, and They represent the clean sample set and the noisy sample set after the dataset of the source domain is split.
[0073] S23, adaptive noise correction, calculating the prototype center for the clean sample set obtained in step S22, and using it to correct the noise sample set to obtain a corrected probability vector;
[0074] S231: Divide the sample set of the source domain in step S22 into a clean sample set For each category, calculate the prototype center
[0075] S232: Divide the sample set in the source domain of step S22 into a noise sample set For each noise sample, its category label is corrected based on the prototype classifier, and the probability prediction of the corrected probability vector corresponding to category c is calculated as follows:
[0076]
[0077] Where K is the total number of categories, For clean sample set The prototype center of the jth class, Φ1(·, ·) is used to measure the similarity between the source domain instance and the prototype of a specific class.
[0078] S24, calculate the contribution weight of the image data sample to the task alignment of the source domain and the target domain, specifically, input the image data sample of the target domain into the image classification model constructed in step S21 to obtain a pseudo label vector; then use the one-hot vector based on the true label for the clean sample set obtained in step S22, use the corrected probability vector obtained in step S23 for the noise sample set obtained in step S22, and use the pseudo label vector for the image data sample of the target domain, and calculate the contribution weight of the image data sample to the category field in the process of task alignment of the source domain and the target domain; that is:
[0079]
[0080] Among them, P is the prototype classifier for noise correction, G is the classifier for source domain / target domain classification, and are the clean sample set and the noise sample set after the source domain dataset is split, respectively, F is the feature extractor, X t is the dataset of the target domain.
[0081] S25, using the sample weights obtained in step S22 to weight the cross entropy loss of the source domain image samples; to reduce the model's inclination toward high-loss samples, that is, to reduce the contribution weight of high-confidence noise samples to model training; in step S25, using the sample weights obtained in step S22 to weight the cross entropy loss of the source domain image samples:
[0082]
[0083] in, Is The parameterized weight function, It is a classification model parameterized by the network parameter θ, including the feature extractor F and the classifier G. and are the image samples and image labels in the source domain dataset, n s is the number of samples from the dataset.
[0084] S26, obtaining a task alignment loss based on the local maximum mean difference based on the contribution weight obtained in step S24, so as to strengthen the alignment of features of the same category between the source domain and the target domain and the feature difference between different categories;
[0085] In step S26, the contribution weights obtained in step S24 are used to obtain the task alignment loss based on the local maximum mean difference
[0086]
[0087] Among them, K is the total number of categories, X s represents all image data samples in the source domain, X t represents all image data samples in the target domain, Φ2 is the mapping from the feature space to the reproducing kernel Hilbert space, and Respectively represent the contribution weights of source domain and target domain instances to class k in the source domain and target domain task alignment, and They are source domain samples and target domain samples respectively.
[0088] The contribution weight of the source domain instance to the k classes in the task alignment between the source domain and the target domain The contribution weight of the target domain instance to the k classes in the task alignment between the source and target domains All are calculated by the following formula:
[0089]
[0090] in, represents the sample distribution of the data set, represents the contribution weight of the i-th sample to category k, which is calculated in step S24.
[0091] S27. Set the dynamic balance parameter λ to balance the cross entropy loss of the source domain image samples obtained in S25 and the task alignment loss based on the local maximum mean difference obtained in S26, calculate the total model loss and perform back propagation, and update the network parameters; determine whether the total model loss converges. When the total model loss converges, the update process ends and the final classification model, i.e., the robust noise domain adaptation model, is obtained; when the total model loss does not converge, return to step S22.
[0092] In step S27, the dynamic balance parameter λ is set to balance the cross entropy loss of the source domain image samples obtained in S25 and the task alignment loss based on the local maximum mean difference obtained in S26, the total model loss is calculated and back propagated, and the network parameters are updated, specifically,
[0093] S271, balance the classification loss by the balance parameter λ and task alignment loss The total model loss is calculated as And perform back propagation;
[0094] S272, the image classification model parameters are updated by the following formula:
[0095]
[0096] Among them, λ is the balance parameter, which balances the weight of classification learning loss and task alignment learning loss during training, η θ represents the gradient descent rate of parameter θ, (·) represents the gradient of θ;
[0097] S273. The weight function of sample loss is updated by the following formula:
[0098]
[0099] Among them, η α is the gradient descent rate of parameter α, (·) represents the gradient of α, J vali (·,·) is the cross entropy loss calculated on the validation set, and are the image samples and image labels in the validation set, respectively. is the classifier parameter affected by the meta-network function parameter α in the network update, For the classification model to sample prediction.
[0100] S3. The robust noise domain adaptation model obtained in step S2 performs image classification on the image data samples in the target domain, and outputs the predicted labels for the image data samples in the target domain, that is, obtains the image classification results of the target domain.
[0101] This image classification method based on label noise domain adaptation can reduce the impact of label noise and effectively improve the accuracy of image classification. In view of the label noise in source domain samples, this method reduces the contribution of noise samples to the classifier through adaptive noise detection and meta-network weighted sample loss; corrects the noise samples through adaptive noise correction so that the noise samples obtain higher confidence labels and re-invest in learning, reducing the impact of noise samples on the alignment of source and target domain tasks, so that the final optimized robust noise domain adaptation model can have better generalization performance in this situation that is closer to the real scene, thereby obtaining more accurate image classification results.
[0102] The embodiment further provides a system using any of the above-mentioned image classification methods based on label noise domain adaptation, including a data preprocessing module, a model training module and an image classification processing module.
[0103] Data preprocessing module: preprocess the image data in the source domain and the target domain to obtain a source domain dataset consisting of image data samples in the source domain and a target domain dataset consisting of image data samples in the target domain, respectively. The image data samples in the source domain contain label noise.
[0104] Model training module: build an image classification model, perform adaptive noise detection on the image data samples in the source domain and obtain sample weights. Use the threshold method to divide the source domain data set into a noise sample set and a clean sample set based on the obtained sample weights. After adaptive noise correction is performed on the clean sample set, calculate the contribution weight of the image data sample to the category field during the task alignment process between the source domain and the target domain. The total model loss is obtained from the obtained contribution weight and the sample weight until the total model loss converges and the robust noise domain adaptation model is obtained through training.
[0105] Image classification processing module: The obtained robust noise domain adaptation model is used to perform image classification on the image data samples in the target domain, and the predicted labels of the image data samples in the target domain are output, that is, the image classification results of the target domain are obtained.
[0106] In this image classification system based on label noise domain adaptation, Figure 2The model training module is used to train and obtain a robust noise domain adaptation model. In the model training process, after the feature extractor F extracts features from the input image data samples, the extracted features are input to the multi-layer perceptual classifier G; the multi-layer perceptual classifier outputs a probability vector to the meta-network M for the features extracted by the feature extractor; the meta-network M is only used in the training process to perform adaptive noise detection on the image data samples in the source domain and obtain sample weights according to the sample loss calculated based on the probability vector output by the multi-layer perceptual classifier G. The source domain data set is divided into a noise sample set and a clean sample set using the threshold method based on the obtained sample weights. After the prototype classifier P performs adaptive noise correction on the clean sample set, the contribution weight of the image data sample to the category domain in the process of task alignment between the source domain and the target domain is calculated, and the cross entropy loss of the source domain image sample is obtained from the sample weight, and the task alignment loss based on the local maximum mean difference is obtained from the obtained contribution weight, and then the total model loss is obtained from the task alignment loss and the cross entropy loss. Back propagation is performed to update the network parameters until the total model loss converges, and the robust noise domain adaptation model is obtained through training.
[0107] In this image classification system based on label noise domain adaptation, the image classification processing module is used to output the image classification results of the image data samples in the target domain. The robust noise domain adaptation model obtained by the model training module includes a feature extractor and a multi-layer perceptual classifier. After the feature extractor extracts features from the image data samples in the target domain, the extracted features are input into the multi-layer perceptual classifier; the multi-layer perceptual classifier outputs the predicted labels for the image data samples in the target domain, that is, the image classification results of the target domain are obtained.
[0108] This image classification method and system based on label noise domain adaptation transfers the source domain knowledge containing label noise to the target domain without labels for the classification of image data samples in the target domain. By constructing an end-to-end network model, the image data samples of the source domain and the image data samples of the target domain are input into a common feature extractor and a multi-layer perceptron classifier. Then, when training the generative model, it mainly includes two learning stages: adaptive label noise detection and adaptive label noise correction. First, adaptive noise detection is used to identify noise samples. Based on the characteristic that clean samples have smaller losses than noise samples, the meta network weighted instance loss is used to reduce the contribution of noise samples to classification learning. At the same time, the data set is divided based on the threshold method to further reduce the impact of noise samples on task pulling. Then, the noise samples are corrected by adaptive label noise correction, so that the noise samples obtain more accurate labels and are re-entered into learning. After iterative optimization of the classification model, the final model obtained after convergence is finally used to classify the target image to obtain the classification result.
[0109] The image classification method based on label noise domain adaptation in the embodiment is experimentally simulated by taking the Digits dataset as an example as follows:
[0110] To simulate the learning scenario of domain adaptive image classification task with noise in the source domain, the Digits dataset is manually contaminated. Taking the Digits dataset with 40% noise labels as an example, the method of the embodiment is processed as follows:
[0111] S1. Data preprocessing: convert the image data samples in the source domain and the image data samples in the target domain into three-channel images with a size of 32*32;
[0112] S2. Build an image classification model, perform adaptive noise detection and adaptive noise correction on image data samples, reduce the contribution of noise samples to the classifier based on meta-network weighted sample loss, divide the data set based on the threshold method to further reduce the impact of noise samples on task pulling, and train to obtain a robust noise domain adaptation model;
[0113] S21. Use the LeNet model as the feature extractor F. Pre-train the feature extractor based on samples and corresponding true labels on a large dataset unrelated to this task, and then initialize the weight function w of the multi-layer perceptron classifier G.
[0114] S22, input the source domain samples and noise labels into the feature extractor and classifier training, and complete the cold start of the model after 8 iterations; calculate the cross entropy loss of the source domain samples, and input the sample loss into the weight function of the meta-network M to obtain the sample weight, i.e., the noise confidence; set a threshold of 2 / batch_size based on the weight to separate the noise samples and the source domain samples, where batch_size is the size of the sample block in training;
[0115] S23, calculate the prototype center of each category according to the category classification of the clean sample set obtained in step S22 For the noise sample set obtained in step S22, the distance to the center of each category prototype is calculated and modified based on the prototype classifier.
[0116] S24. Calculate the contribution weight of the image data sample to the task alignment of the source domain and the target domain. Specifically, for the clean samples in the source domain, use the one-hot vector of its true label, for the noisy samples in the source domain, use the probability vector corrected by the prototype classifier, and for the unlabeled samples in the target domain, use the pseudo-label probability vector output by the classification model to calculate the contribution weight of the image data sample to the category domain in the process of task alignment of the source domain and the target domain.
[0117] S25. Input the source domain / target domain samples into the classification model to obtain the sample features. Combined with the contribution weight of the sample to the category domain in step S24, the local maximum mean difference is calculated. The conditional distribution difference between the source domain and the target domain is calculated through the local maximum mean difference LMMD, and the distribution distance between the domains is narrowed to achieve the migration of source domain knowledge to the target domain.
[0118] S26. Use the contribution weights calculated in step S24 to weight the cross entropy classification loss of the source domain samples, minimize the local maximum mean difference in step S25, calculate the task alignment loss, balance the classification loss and the task alignment loss to calculate the total model loss:
[0119] S27, calculate the gradient of the overall loss, back-propagate and update the network parameters, update the pseudo labels of all target domain data every 20 rounds, and calculate the average accuracy of each category of target domain data every 50 rounds until the loss converges. Save the optimal target domain model as the robust noise domain adaptation model.
[0120] S3. The obtained robust noise domain adaptation model outputs the predicted label of the target domain image data sample, that is, the image classification result in the target domain.
[0121] Figure 3 This is a schematic diagram of the performance comparison of the embodiment method with the classic convolutional neural network AlexetNet, the deep domain adaptation network DAN, the adversarial training neural network DANN and the adversarial discriminant domain adaptation ADDA on the Digits dataset under three migration tasks. Among them, mnist, svhn, and usps represent three different domains on Digits, and the symbol → represents a group of migration tasks, such as mnist→svhn represents migration from the noisy dataset mnist to the unlabeled dataset svhn, and AVG represents the average performance of the three migration tasks. Figure 3 In the embodiment, the image classification method based on label noise domain adaptation is RoDAC. In the three migration tasks under the Digits dataset, compared with the traditional domain adaptation method, the embodiment method achieved a higher average classification accuracy and reached the highest value in the average performance of the three tasks.
[0122] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. An image classification method based on label noise domain adaptation, characterized by: The following steps are included: S1. Preprocess the image data in the source domain and the target domain to obtain a source domain dataset consisting of image data samples in the source domain and a target domain dataset consisting of image data samples in the target domain, respectively, and the image data samples in the source domain contain label noise; S2. Build an image classification model, perform adaptive noise detection on the image data samples in the source domain and obtain sample weights. Use the threshold method to divide the source domain data set into a noise sample set and a clean sample set based on the obtained sample weights. After adaptive noise correction is performed on the clean sample set, calculate the contribution weight of the image data sample to the category field in the process of aligning the source domain and the target domain tasks. The total model loss is obtained from the obtained contribution weight and the sample weight until the total model loss converges and the training is obtained to obtain a robust noise domain adaptation model. Specifically, S21, initialize the multi-layer perceptron classifier and the meta-network, and construct an image classification model for solving label noise scenarios by the feature extractor, the multi-layer perceptron classifier and the meta-network; S22, adaptive noise detection, inputting the image data samples of the source domain into the image classification model constructed in step S21 to obtain a probability output, calculating the sample loss according to the result of the probability output, and based on the characteristic that clean samples have smaller losses than noise samples, using a neural network with only one hidden layer as a meta-network, and adaptively obtaining sample weights according to the gradient direction through meta-learning behavior, and using the obtained sample weights to divide the source domain data set into a noise sample set and a clean sample set using a threshold method; S23, adaptive noise correction, calculating the prototype center for the clean sample set obtained in step S22, and using it to correct the noise sample set to obtain a corrected probability vector; S24, calculating the contribution weight of the image data sample to the task alignment of the source domain and the target domain, specifically, inputting the image data sample of the target domain into the image classification model constructed in step S21 to obtain a pseudo label vector; then using the one-hot vector based on the true label for the clean sample set obtained in step S22, using the corrected probability vector obtained in step S23 for the noise sample set obtained in step S22, and using the pseudo label vector for the image data sample of the target domain, to calculate the contribution weight of the image data sample to the category field in the process of task alignment of the source domain and the target domain; S25, using the sample weights obtained in step S22 to weight the cross entropy loss of the source domain image samples; S26, obtaining the task alignment loss based on the local maximum mean difference according to the contribution weight obtained in step S24; S27, setting the dynamic balance parameter λ to balance the cross entropy loss of the source domain image samples obtained in S25 and the task alignment loss based on the local maximum mean difference obtained in S26, calculating the total model loss and performing back propagation, and updating the network parameters; judging whether the total model loss converges, when the total model loss converges, the update process ends and the final classification model, i.e., the robust noise domain adaptation model, is obtained; when the total model loss does not converge, returning to step S22; S3. The robust noise domain adaptation model obtained in step S2 performs image classification on the image data samples in the target domain, and outputs the predicted labels for the image data samples in the target domain, that is, obtains the image classification results of the target domain.
2. The image classification method based on label noise domain adaptation as claimed in claim 1, characterized in that: In step S22, based on the characteristic that clean samples have smaller losses than noise samples, a neural network with only one hidden layer is used as a meta-network, and sample weights are adaptively obtained according to the gradient direction through meta-learning behavior. The obtained sample weights are used to divide the source domain data set into a noise sample set and a clean sample set using a threshold method, specifically, S221. For the image data samples in the source domain, the weight function of the meta-network is used Mapping sample weights, input is sample loss, output is sample weight in is the source domain sample, and the meta-network function parameter α depends on its On learning performance, namely: Where V is the number of samples in the validation set, and V is less than the total number of image data samples in the source domain. vali (·,·) is the cross entropy loss calculated on the validation set. Consists of correctly labeled source domain samples, and are the image samples and image labels in the validation set, respectively. is the classifier parameter affected by the meta-network function parameter α in the network update, represents the solution space of the meta-network function parameter α; S222, set the threshold thres, if the sample weight obtained in step S221 Will is marked as a noise instance; otherwise, it is marked as a clean instance, i.e. in, and They represent the clean sample set and the noisy sample set after the dataset of the source domain is split.
3. The image classification method based on label noise domain adaptation as claimed in claim 1, characterized in that: In step S23, the prototype center is calculated for the clean sample set obtained in step S22, and is used to correct the noise sample set to obtain a corrected probability vector, specifically, S231: Divide the sample set of the source domain in step S22 into a clean sample set For each category, calculate the prototype center S232: Divide the sample set in the source domain of step S22 into a noise sample set For each noise sample, its category label is corrected based on the prototype classifier, and the probability prediction of the corrected probability vector corresponding to category c is calculated as follows: Where K is the total number of categories, For clean sample set The prototype center of the jth class, Φ1(.,.) is used to measure the similarity between the source domain instance and the prototype of a specific class.
4. The image classification method based on label noise domain adaptation as claimed in claim 1, characterized in that: In step S24, the contribution weight of the image data sample to the category domain in the process of aligning the source domain and the target domain task is calculated, that is: Among them, P is the prototype classifier for noise correction, G is the classifier for source domain / target domain classification, and are the clean sample set and the noisy sample set after the source domain dataset is split, F is the feature extractor, and X t is the dataset of the target domain.
5. The image classification method based on label noise domain adaptation according to any one of claims 1 to 4, characterized in that: In step S25, the cross entropy loss of the source domain image sample is weighted using the sample weight obtained in step S22: in, Is The parameterized weight function, It is a classification model parameterized by the network parameter θ, including the feature extractor F and the classifier G. and are the image samples and image labels in the source domain dataset, n s is the number of samples from the dataset.
6. The image classification method based on label noise domain adaptation according to any one of claims 1 to 4, characterized in that: In step S26, the contribution weights obtained in step S24 are used to obtain the task alignment loss based on the local maximum mean difference Among them, K is the total number of categories, X s represents all image data samples in the source domain, X t represents all image data samples in the target domain, Φ2 is the mapping from the feature space to the reproducing kernel Hilbert space, and Respectively represent the contribution weights of source domain and target domain instances to class k in the source domain and target domain task alignment, and They are source domain samples and target domain samples respectively.
7. The image classification method based on label noise domain adaptation as claimed in claim 6, characterized in that: The contribution weight of the source domain instance to the k classes in the task alignment between the source domain and the target domain The contribution weight of the target domain instance to the k classes in the task alignment between the source and target domains All are calculated by the following formula: in, represents the sample distribution of the data set, represents the contribution weight of the i-th sample to category k, which is calculated in step S24.
8. The image classification method based on label noise domain adaptation according to any one of claims 1 to 4, characterized in that: In step S27, the dynamic balance parameter λ is set to balance the cross entropy loss of the source domain image samples obtained in S25 and the task alignment loss based on the local maximum mean difference obtained in S26, the total model loss is calculated and back propagated, and the network parameters are updated, specifically, S271, balance the classification loss by the balance parameter λ and task alignment loss The total model loss is calculated as And perform back propagation; S272, the image classification model parameters are updated by the following formula: Among them, λ is the balance parameter, which balances the weight of classification learning loss and task alignment learning loss during training, η θ represents the gradient descent rate of parameter θ, represents the gradient of θ; S273. The weight function of sample loss is updated by the following formula: Among them, η α is the gradient descent rate of parameter α, represents the gradient of α, J vali (·,·) is the cross entropy loss calculated on the validation set, and are the image samples and image labels in the validation set, respectively. is the classifier parameter affected by the meta-network function parameter α in the network update, For the classification model to sample prediction.
9. A system using the image classification method based on label noise domain adaptation according to any one of claims 1 to 8, characterized in that: Including data preprocessing module, model training module and image classification processing module, Data preprocessing module: preprocess the image data in the source domain and the target domain to obtain a source domain dataset consisting of image data samples in the source domain and a target domain dataset consisting of image data samples in the target domain, respectively. The image data samples in the source domain contain label noise. Model training module: build an image classification model, perform adaptive noise detection on the image data samples in the source domain and obtain sample weights. Use the threshold method to divide the source domain data set into a noise sample set and a clean sample set based on the obtained sample weights. After adaptive noise correction is performed on the clean sample set, calculate the contribution weight of the image data sample to the category field during the task alignment process between the source domain and the target domain. The total model loss is obtained from the obtained contribution weight and the sample weight until the total model loss converges and the robust noise domain adaptation model is obtained through training. Image classification processing module: The obtained robust noise domain adaptation model is used to perform image classification on the image data samples in the target domain, and the predicted labels of the image data samples in the target domain are output, that is, the image classification results of the target domain are obtained.