Image classification method, system, device and medium based on active domain adaptation

Abstract

The application provides an image classification method, system, device and medium based on active domain adaptation, the method comprises the following steps: initializing the labeled data set and the unlabeled data set according to the obtained source domain image data set and target domain image data set respectively, and after the target domain image classification model is obtained by initializing and training the preset deep neural network model according to the labeled data set, establishing the information scoring model and the information sampling model according to the labeled data set, the unlabeled data set and the target domain image classification model, and performing active domain adaptation learning iterative training on the target domain image classification model, updating the labeled data set and the unlabeled data set until the preset iteration stopping condition is reached, and the image classification result is obtained by inputting the image to be classified into the target domain image classification model for classification prediction. The application designs a sampling and training strategy based on the heterogeneity of sample differences, effectively improves the classification accuracy and generalization ability of the model, and has strong universality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image classification method, system, computer device, and storage medium based on active domain adaptation. Background Technology

[0002] Image classification is a fundamental task in computer vision and image processing, providing effective technical support for tasks such as autonomous driving and remote sensing image processing. Machine learning-based image classification is widely used due to its intelligence and efficiency; however, its classification effectiveness depends heavily on the annotation of training samples, and the large amount of annotation work inevitably consumes significant human and time resources. Therefore, active domain adaptation methods are highly popular because they can find the most informative target domain samples for annotation based on effective annotations of source domain data, making them most beneficial for target domain classification, and effectively reducing the workload of manual sample annotation.

[0003] Existing image classification methods based on active domain adaptation can improve the performance of the model in the target domain to a certain extent by designing an active learning algorithm to select the most informative batch of samples in the unlabeled target domain data in the domain adaptation scenario. However, they still have certain application defects: (1) They require additional finely designed network modules (such as domain discriminators) to evaluate the uncertainty and representativeness of the samples, which means that they can only be applied in a certain specific image classification domain adaptation scenario, which limits the scalability and universality of the method; (2) They do not pay attention to the differences between different samples in the target domain and the source domain samples, and do not establish customized design sampling and training strategies for samples with different differences. Instead, they apply a unified training objective to all target domain samples, which makes the model unable to generalize well to scenarios with different degrees of domain differences, resulting in poor image classification performance.

[0004] Therefore, there is an urgent need to provide a general image classification method based on active domain adaptation that can effectively improve classification accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide an active domain adaptive image classification method. By employing a target data partitioning strategy based on domain representativeness and uncertainty, an information scoring system based on Gaussian mixture models, and an automatic data partitioning method with corresponding sampling functions, combined with a progressive learning strategy, this method achieves stable training and strong adaptability in active domain adaptive classification learning. It addresses the shortcomings of existing active domain adaptive image classification applications, effectively improves the model's classification accuracy and generalization ability, and can adapt to different active domain adaptive scenarios, thus possessing strong versatility.

[0006] To achieve the above objectives, it is necessary to provide an image classification method, system, computer device, and storage medium based on active domain adaptation to address the aforementioned technical problems.

[0007] In a first aspect, embodiments of the present invention provide an image classification method based on active domain adaptation, the method comprising the following steps:

[0008] The labeled and unlabeled datasets are initialized based on the obtained source domain image dataset and target domain image dataset, respectively; the source domain image dataset is a labeled image dataset; the target domain image dataset is an unlabeled image dataset.

[0009] The preset deep neural network model is initialized and trained based on the labeled dataset to obtain a target domain image classification model; the preset deep neural network model includes a feature extractor and a classifier.

[0010] Based on the labeled dataset, the unlabeled dataset, and the target domain image classification model, an information scoring model and an information sampling model are established.

[0011] Based on the information scoring model and the information sampling model, the target domain image classification model is subjected to active domain adaptive learning iterative training to update the labeled dataset and the unlabeled dataset until the preset iteration stopping condition is reached.

[0012] The image to be classified is obtained and input into the target domain image classification model for classification prediction to obtain the image classification result.

[0013] Further, the steps of establishing the information scoring model and the information sampling model based on the labeled dataset, the unlabeled dataset, and the target domain image classification model include:

[0014] Based on the labeled dataset and the target domain image classification model, the similarity labels of each sample in the unlabeled dataset are obtained, and the information scoring model is established based on the similarity labels and the corresponding model prediction probability vectors.

[0015] Based on the information scoring model, information scoring sets for the unlabeled dataset and the labeled dataset are obtained respectively, and the information sampling model is constructed based on the information scoring sets for the unlabeled dataset and the labeled dataset.

[0016] Further, the step of obtaining the similarity labels of each sample in the unlabeled dataset based on the labeled dataset and the target domain image classification model includes:

[0017] The labeled dataset is used to extract features using the target domain image classification model to obtain the corresponding labeled sample features;

[0018] Based on the labeled sample features, the category centers are obtained; the category centers are represented as follows:

[0019]

[0020] Among them, A c Let represent the category center corresponding to category c, c∈{1,…,C}, where C represents the total number of categories; 1{·} represents the indicator function; G(·) represents the feature extractor of the target domain image classification model; L represents the labeled dataset, and L=S∪T, where S represents the source domain image dataset; T represents the labeled samples in the target domain image dataset during the training of the target domain image classification model; x and y represent the image data of the labeled dataset and the predicted category of the corresponding target domain image classification model, respectively.

[0021] Based on the category centers, the similarity of each sample in each category in the unlabeled dataset is calculated using the topk metric.

[0022] The maximum class similarity among the class similarities of each sample is selected, and the class corresponding to the maximum class similarity is used as the corresponding similarity label; the similarity label is represented as:

[0023]

[0024] in, Indicates sample x in the unlabeled dataset u The similarity label; topk(·) represents the index list of the first k elements after sorting the vector in descending order of element value; IoU(·) represents the intersection-union function.

[0025] Furthermore, the information scoring model is expressed as:

[0026]

[0027] in, Indicates sample x in the unlabeled dataset u Information rating; P c (x u ) represents sample x in the unlabeled dataset. u The predicted probability vector P(x) u The c-th element of ).

[0028] Further, the step of constructing the information sampling model based on the information scoring set of the unlabeled dataset and the labeled dataset includes:

[0029] Based on the information score set of the unlabeled dataset, the unlabeled dataset is classified according to domain representativeness and uncertainty to obtain subset classification categories of the unlabeled dataset; the subset classification categories include confident and consistent class, uncertain and consistent class, uncertain and inconsistent class, and confident and inconsistent class;

[0030] Based on the category division of the subset and the information scoring set of the labeled dataset, a sample observation label segmentation function is constructed; the sample observation label segmentation function is expressed as follows:

[0031]

[0032] Where q(x) l ,y l ) represents the sample (x) in the labeled dataset. l ,y l The Gaussian component indices of ) are given by P(x), where 1, 2, 3, and 4 represent the Gaussian component indices corresponding to the confident and consistent class, the uncertain and consistent class, the uncertain and inconsistent class, and the confident and inconsistent class, respectively; τ represents the confidence threshold; P(x) l ) represents sample x l The predicted probability vector;

[0033] Based on the sample observation label segmentation function and the information scoring sets of the unlabeled dataset and the labeled dataset, a sampling model training set is constructed; the sampling model training set is represented as follows:

[0034]

[0035] In the formula,

[0036]

[0037] in, This indicates that samples (x) in the labeled dataset L are... l ,y l Information rating; This represents sample x in the unlabeled dataset U. u Information rating; D L and D U These represent the information score sets of the labeled dataset L and the unlabeled dataset U, respectively.

[0038] The information sampling model is obtained by training a preset Gaussian mixture model based on the sampling training dataset; the information sampling model is represented as follows:

[0039]

[0040] In the formula,

[0041] W={(π k ,μ k ,σ k |1≤k≤K}

[0042]

[0043] Where Pr(z=k│x,W) represents the probability that a sample x in the unlabeled dataset is assigned to the k-th Gaussian component, z represents the discrete variable, and W represents the parameters of the Gaussian mixture model; π k μ represents the weight of the k-th Gaussian component, where K represents the number of Gaussian components; k and σ k Let N(:; μ) represent the mean and variance of the k-th Gaussian component, respectively; k ,σ k ) represents the Gaussian distribution of the k-th component; This represents the information score for sample x in the unlabeled dataset; This represents the probability density function output of the information sampling model's information score for sample x.

[0044] Further, the step of performing active domain adaptive learning iterative training on the target domain image classification model based on the information scoring model and the information sampling model, and updating the labeled dataset and the unlabeled dataset, includes:

[0045] Based on the information scoring model and the information sampling model, the unlabeled dataset is divided according to the subset division category to obtain corresponding unlabeled data subsets; the unlabeled data subsets include a confident and consistent subset, an uncertain and consistent subset, an uncertain and inconsistent subset, and a confident and inconsistent subset;

[0046] The samples in the uncertain and inconsistent subset are sorted in descending order according to their corresponding classification probabilities. A preset number of unlabeled samples are obtained from top to bottom and labeled. Based on the corresponding labeling results, the labeled dataset and the unlabeled dataset are updated.

[0047] Based on the updated labeled dataset and the unlabeled dataset, and a preset loss function, the parameters of the target domain image classification model are optimized, and active domain adaptive learning is performed iteratively based on the optimized target domain image classification model to update the labeled dataset and the unlabeled dataset.

[0048] Furthermore, the preset loss function includes the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function;

[0049] The step of optimizing the parameters of the target domain image classification model based on the updated labeled dataset and the unlabeled dataset, and a preset loss function includes:

[0050] The labeled dataset, the confident and consistent subset, and the uncertain and consistent subset are respectively input into the target domain image classification model for prediction analysis, and the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function are respectively established based on the corresponding prediction results.

[0051] Based on the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function, the total training loss value of the model is obtained, and the parameters of the target domain image classification model are optimized based on the total training loss value; the total training loss value of the model is expressed as:

[0052] Loss = L sup +λ c L con +λ e L ent

[0053] In the formula,

[0054]

[0055]

[0056]

[0057] Where Loss represents the total training loss of the model; L sup L con and L ent Let λ represent the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function, respectively; c and λ e represent the weights corresponding to the regularization loss function and the conditional entropy loss function, respectively; Aug(·) represents the function that further perturbs the sample input; L, and represents the labeled dataset, the confident and consistent subset, and the uncertain and consistent subset, respectively.

[0058] Secondly, embodiments of the present invention provide an image classification system based on active domain adaptation, the system comprising:

[0059] The data acquisition module is used to initialize a labeled dataset and an unlabeled dataset based on the acquired source domain image dataset and target domain image dataset, respectively; the source domain image dataset is a labeled image dataset; the target domain image dataset is an unlabeled image dataset;

[0060] An initial training module is used to initialize and train a preset deep neural network model based on the labeled dataset to obtain a target domain image classification model; the preset deep neural network model includes a feature extractor and a classifier.

[0061] The scoring and sampling module is used to establish an information scoring model and an information sampling model based on the labeled dataset, the unlabeled dataset, and the target domain image classification model.

[0062] The learning and training module is used to perform active domain adaptive learning iterative training on the target domain image classification model based on the information scoring model and the information sampling model, and update the labeled dataset and the unlabeled dataset until a preset iteration stopping condition is reached.

[0063] The image classification module is used to acquire the image to be classified, input the image to be classified into the target domain image classification model for classification prediction, and obtain the image classification result.

[0064] Thirdly, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0065] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0066] This application provides an active domain adaptive image classification method and system. The method initializes labeled and unlabeled datasets based on the acquired source and target domain image datasets, respectively. After initializing and training a pre-defined deep neural network model using the labeled dataset to obtain a target domain image classification model, an information scoring model and an information sampling model are established based on the labeled, unlabeled, and target domain image classification models. The target domain image classification model is then iteratively trained using active domain adaptive learning, updating the labeled and unlabeled datasets until a pre-defined iteration stopping condition is met. Finally, the image to be classified is input into the target domain image classification model for classification prediction to obtain the image classification result. Compared with existing technologies, this active domain adaptive image classification method, by designing sampling and training strategies based on the anisotropy of sample differences, effectively improves the model's classification accuracy and generalization ability, and has strong versatility. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the architecture of image classification based on active domain adaptation in an embodiment of the present invention;

[0068] Figure 2 This is a flowchart illustrating the image classification method based on active domain adaptation in an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram illustrating the consistency between the target domain image classification model prediction and the similarity label in an embodiment of the present invention;

[0070] Figure 4 This is a schematic diagram of the algorithm flow for training a preset Gaussian mixture model based on a sampled training dataset in an embodiment of the present invention;

[0071] Figure 5 This is a schematic diagram of the structure of the image classification system based on active domain adaptation in an embodiment of the present invention;

[0072] Figure 6 This is an internal structural diagram of the computer device in an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and beneficial effects of this application clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the embodiments described below are only part of the embodiments of the present invention and are used to illustrate the present invention, but are not intended to limit the scope of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0074] The image classification method based on active domain adaptation provided in this invention addresses the problems of existing active domain adaptive learning methods, which require additional complex model design and fail to consider the differences between samples in the target domain and samples in the source domain. These methods suffer from weak generalization ability and negatively impact classification accuracy. The overall architecture of this invention is as follows: Figure 1 As shown, it mainly includes Figure 1 (a) shows the sampling phase (data partitioning steps) and Figure 1 The training phase is shown in (b). Figure 1 As shown in (a), this invention innovatively proposes an information scoring model that integrates uncertainty and target domain representativeness, and uses this model to divide target domain samples into four sample subsets. Simultaneously, to avoid heuristic thresholding methods from further subdividing the data, a mechanism is proposed to automate data sampling by training a Gaussian mixture model using a semi-supervised expectation-maximization algorithm. This divides unlabeled target data into four sample categories, and one of these categories is selected as the subset with the highest information content for active learning and labeling. Figure 1As shown in (b), this invention proposes to design customized learning strategies for different categories of target samples during model training to maximize the performance of the model.

[0075] This invention's active domain adaptive image classification method can be applied to image classification of any domain and any type, according to... Figure 1 The method architecture shown completes the active domain adaptive iterative training of the target domain image classification model, and performs prediction processing on the image to be classified by the target domain image classification model to obtain accurate and effective image classification results; the following embodiments will describe the active domain adaptive image classification method of the present invention in detail.

[0076] In one embodiment, such as Figure 2 As shown, an image classification method based on active domain adaptation is provided, including the following steps:

[0077] S11. Initialize the labeled dataset and the unlabeled dataset according to the obtained source domain image dataset and target domain image dataset respectively; the source domain image dataset is a labeled image dataset; the target domain image dataset is an unlabeled image dataset, and subsequently, active domain adaptive learning will be used to filter and label samples with high labeling value in the target domain image dataset;

[0078] In active domain adaptive learning, samples in the target domain image dataset are continuously labeled, and the labeled and unlabeled samples participating in model training are continuously updated until the number of labeled samples selected in the target domain image dataset reaches the corresponding maximum labeling limit. Therefore, to facilitate the description of the active domain adaptive learning process, this embodiment introduces labeled and unlabeled datasets to record labeled and unlabeled samples during the learning process, respectively: In the initial stage, the labeled sample set only includes the source domain image dataset, and the unlabeled dataset includes all samples in the target domain image dataset; Through continuous adaptive learning, the labeled sample set will gradually increase the number of labeled samples in the target domain image dataset (excluding the source domain image dataset), while the unlabeled dataset will gradually decrease as some samples in the target domain image dataset are labeled; It should be noted that the images in the source and target domain image datasets are not limited to a specific domain or type. In principle, different types of images can also be directly applied to the method of this invention for effective classification and prediction after adaptive processing;

[0079] S12. Initialize and train the preset deep neural network model based on the labeled dataset to obtain the target domain image classification model; the preset deep neural network model includes a feature extractor and a classifier; wherein, the initial training of the preset deep neural network model can be carried out using existing training methods in conjunction with the standard cross-entropy loss function, and no specific restrictions are imposed here;

[0080] S13. Based on the labeled dataset, the unlabeled dataset, and the target domain image classification model, establish an information scoring model and an information sampling model. The information scoring model can be understood as a model that comprehensively scores the domain representativeness and uncertainty of each unlabeled sample based on the feature similarity between unlabeled samples in the target domain image dataset and samples in the source domain image dataset, and determines the subset classification category of the unlabeled dataset based on the information scoring. Correspondingly, the information sampling model can be understood as a four-component Gaussian mixture model trained using a semi-supervised expectation-maximization algorithm, based on the scoring from the information scoring model and combined with supervised information constructed from the labeled sample set, to classify and sample the unlabeled dataset according to subset classification.

[0081] Specifically, the steps of establishing the information scoring model and the information sampling model based on the labeled dataset, the unlabeled dataset, and the target domain image classification model include:

[0082] Based on the labeled dataset and the target domain image classification model, similarity labels are obtained for each sample in the unlabeled dataset. Then, based on the similarity labels and the corresponding model prediction probability vectors, the information scoring model is established. The similarity label can be understood as estimating reliable category centers based on the labeled dataset, and then calculating the feature similarity between each unlabeled target domain sample and each category center to define a similarity-based label. This embodiment considers that the initial training of the target domain image classification model is based on the labeled dataset, and the model's feature extraction from the labeled data is relatively accurate. Therefore, reliable category centers can be obtained based on the labeled data to establish the data feature distribution for each category, thus providing effective support for reasonable difference assessment of unlabeled samples. Specifically, the step of obtaining similarity labels for each sample in the unlabeled dataset based on the labeled dataset and the target domain image classification model includes:

[0083] The labeled dataset is used to extract features using the target domain image classification model to obtain the corresponding labeled sample features;

[0084] Based on the labeled sample features, each category center is obtained; wherein, each category center is obtained by averaging the features of the currently labeled samples, and the category center is represented as:

[0085]

[0086] Among them, A cLet represent the category center corresponding to category c, c∈{1,…,C}, where C represents the total number of categories; 1{·} represents the indicator function; G(·) represents the feature extractor of the target domain image classification model; L represents the labeled dataset, and L=S∪T, where S represents the source domain image dataset; T represents the labeled samples in the target domain image dataset during the training of the target domain image classification model; x and y represent the image data of the labeled dataset and the predicted category of the corresponding target domain image classification model, respectively. It should be noted that in the first active learning round, the category center is calculated using only the source domain image dataset. After that, as the active learning rounds progress, the set of labeled samples selected from the target domain image dataset will be merged with the source domain image dataset to form the labeled dataset, which will then participate in the calculation of subsequent category centers to accurately update the distribution of category features.

[0087] Based on the category centers, the similarity of each sample in each category in the unlabeled dataset is calculated using the topk metric.

[0088] The maximum class similarity among the class similarities of each sample is selected, and the class corresponding to the maximum class similarity is used as the corresponding similarity label; the similarity label is represented as:

[0089]

[0090] in, Indicates sample x in the unlabeled dataset u The similarity label; topk(·) represents the index list of the first k elements after sorting the vectors in descending order of element value; IoU(·) represents the intersection-union function, used to return the degree of overlap between two feature index lists, where the value of k can be set according to the actual application requirements. In this embodiment, it is preferred to set it to a value much smaller than the feature dimension. For example, if the feature extractor of the target domain image classification model is ResNet-34, then the value of k can be set to 32; if the feature extractor of the target domain image classification model is ResNet-50, then the value of k can be set to 64; G(x u ) represents sample x u Features; A c This represents the category center of the c-th category;

[0091] Considering that samples possessing both domain representativeness and uncertainty have the greatest information content, i.e., the greatest annotation value, this embodiment integrates these two indicators into a single information scoring model to comprehensively consider both domain representativeness and uncertainty. After determining the similarity between the categories of each sample in the unlabeled dataset and the labeled samples according to the above steps, and combining this with the prediction classification results of the corresponding target domain image classification model, an information scoring model for comprehensively scoring unlabeled samples can be obtained. This information scoring model is expressed as:

[0092]

[0093] in, Indicates sample x in the unlabeled dataset u Information rating; P c (x u ) represents sample x in the unlabeled dataset. u The predicted probability vector P(x) u The c-th element of )

[0094] It should be noted that, according to the theory behind the top-k metric, the more common elements shared in a pair of samples in their feature lists, the higher the probability that they belong to the same category. When the value of k is significantly less than the feature completeness dimension, the top-k metric can be analogous to determining whether two image samples are similar under low-resolution conditions. As mentioned earlier, the target domain image classification model is initialized with a large number of labeled source domain image datasets. Therefore, samples in the target domain that are similar to those in the source domain (lower domain representativeness) should have more accurate and stable features than those specific to the target domain (higher domain representativeness). For this reason, for target domain samples with low domain representativeness, even when comparing their features with the category center at low resolution, the labels obtained tend to be consistent with the model's predicted category; while for target domain samples with high domain representativeness, the model is not familiar with them, and the features extracted by the model are not accurate and stable enough. Therefore, when comparing them with the category center at low resolution, the labels obtained will have a greater probability of being inconsistent with the model's predicted category. In summary, as... Figure 3 As shown, the model-predicted categories and similarity labels for target domain samples similar to the source domain tend to be more consistent. Figure 3 (a) indicates that the model-predicted categories and similarity labels for target domain samples that are dissimilar to the source domain tend to be inconsistent. Figure 3(b) When the value of k is significantly smaller than the full dimension of the feature vector, it can be considered that the smaller the consistency between the model's predicted category and the similarity label, the greater the domain representativeness of the target domain sample. That is, the information scoring model given above is reliable and reasonable, and can effectively evaluate the domain representativeness and uncertainty of unlabeled samples. At the same time, based on the above information scoring model, it can be seen that according to whether the model's predicted category and the similarity-based label are the same, the unlabeled target samples can be divided into two subsets: consistent and inconsistent target subsets. According to whether the confidence of the model's prediction is greater than a given threshold, these two subsets can be further divided into a confidence sample subset and an uncertain sample subset. That is, the unlabeled samples can be reasonably divided into confident and consistent, uncertain and consistent, uncertain and inconsistent, and confident and inconsistent. Based on this division pattern, the corresponding information sampling model can be constructed in the following way, giving an automated sampling mechanism to divide the unlabeled dataset according to the above four categories, so as to avoid the adverse effects of manual heuristic data division.

[0095] Based on the information scoring model, information score sets for the unlabeled dataset and the labeled dataset are obtained respectively. Then, based on the information score sets of the unlabeled dataset and the labeled dataset, the information sampling model is constructed. The construction process of the information sampling model can be understood as considering that the information score values ​​of different categories of unlabeled samples correspond to different numerical ranges. A supervised information learning four-component Gaussian mixture model (GMM) is constructed using labeled samples. A sampling function is established to fit the sample type distribution in the unlabeled dataset. Then, based on the trained Gaussian mixture model, different types of samples are sampled and labeled. Specifically, the step of constructing the information sampling model based on the information score sets of the unlabeled dataset and the labeled dataset includes:

[0096] Based on the information score set of the unlabeled dataset, the unlabeled dataset is classified according to domain representativeness and uncertainty to obtain subset classification categories of the unlabeled dataset; the subset classification categories include confident and consistent class, uncertain and consistent class, uncertain and inconsistent class, and confident and inconsistent class;

[0097] Based on the subset classification and the information score set of the labeled dataset, a sample observation label segmentation function is constructed; wherein, the sample observation label segmentation function can be understood as a function used to obtain the observation label corresponding to the information score of the labeled sample, so as to assign all labeled samples to the corresponding Gaussian components; specifically, the sample observation label segmentation function is expressed as follows:

[0098]

[0099] Where q(x) l ,yl ) represents the sample (x) in the labeled dataset. l ,y l The Gaussian component indices of ) are given by P(x), where 1, 2, 3, and 4 represent the Gaussian component indices corresponding to the confident and consistent class, the uncertain and consistent class, the uncertain and inconsistent class, and the confident and inconsistent class, respectively; τ represents the confidence threshold, used to divide the labeled samples into the confidence set and the uncertain set; P(x l ) represents sample x l The predicted probability vector;

[0100] Based on the sample observation label segmentation function and the information scoring sets of the unlabeled dataset and the labeled dataset, a sampling model training set is constructed; the sampling model training set is represented as follows:

[0101]

[0102] In the formula,

[0103]

[0104] in, This indicates that samples (x) in the labeled dataset L are... l ,y l Information rating; This represents sample x in the unlabeled dataset U. u Information rating; D L and D U These represent the information score sets of the labeled dataset L and the unlabeled dataset U, respectively.

[0105] The information sampling model is obtained by training a preset Gaussian mixture model based on the sampling training dataset; the information sampling model is represented as follows:

[0106]

[0107] In the formula,

[0108] W={(π k ,μ k ,σ k |1≤k≤K}

[0109]

[0110] Where Pr(z=k│x,W) represents the probability that a sample x in the unlabeled dataset is assigned to the k-th Gaussian component, z represents the discrete variable, and W represents the parameters of the Gaussian mixture model; π k μ represents the weight of the k-th Gaussian component, where K represents the number of Gaussian components; k and σ kLet N(:; μ) represent the mean and variance of the k-th Gaussian component, respectively; k ,σ k ) represents the Gaussian distribution of the k-th component; This represents the information score for sample x in the unlabeled dataset; This represents the probability density function output of the information sampling model's information score for sample x.

[0111] It should be noted that the process of training a pre-defined Gaussian mixture model based on the sampled training dataset is as follows: Figure 4 As shown, the coefficient α = |D L | / |D U |,D L and D U Let L and U represent the information score sets of the labeled dataset and the unlabeled dataset, respectively. That is, α is used as the weighting coefficient of the labeled and unlabeled sample sets in the sampling training dataset. The four-component GMM model is trained using the semi-supervised expectation maximum algorithm until the parameters converge to obtain a model that is used to sample the unlabeled data to be labeled in the current round of the unlabeled dataset.

[0112] S14. Based on the information scoring model and the information sampling model, perform active domain adaptive learning iterative training on the target domain image classification model, updating the labeled dataset and the unlabeled dataset until a preset iteration stopping condition is reached; wherein, the process of active domain adaptive learning iterative training can also be understood as dividing the unlabeled dataset according to the above information sampling model, and selecting samples with both uncertainty and target domain representativeness (i.e., selecting samples to be labeled from unlabeled samples with Gaussian component index 3) from the unlabeled samples in the target domain image dataset for effective labeling; specifically, the step of performing active domain adaptive learning iterative training on the target domain image classification model based on the information scoring model and the information sampling model, and updating the labeled dataset and the unlabeled dataset includes:

[0113] Based on the information scoring model and the information sampling model, the unlabeled dataset is divided according to the subset division category to obtain corresponding unlabeled data subsets; the unlabeled data subsets include a confident and consistent subset, an uncertain and consistent subset, an uncertain and inconsistent subset, and a confident and inconsistent subset;

[0114] The samples in the uncertain and inconsistent subset are sorted in descending order according to their corresponding classification probabilities. A preset number of unlabeled samples are obtained from top to bottom and labeled. The labeled dataset and the unlabeled dataset are updated according to the labeling results. The classification probability of each sample in the uncertain and inconsistent subset can be represented as Pr(z=3│x,W). Since there is an upper limit to the number of labeled samples in active domain adaptive learning, in each round of active learning sampling steps, a batch of samples (b samples, where b=B / R, B and R are the total labeling limit and the number of labeling rounds, respectively) with a higher probability of Pr(z=3│x,W) are selected for labeling and added to the labeled dataset. At the same time, they are removed from the unlabeled dataset, and the labeled dataset and the unlabeled dataset are updated.

[0115] Based on the updated labeled and unlabeled datasets and a preset loss function, the parameters of the target domain image classification model are optimized. Then, active domain adaptive learning is iteratively trained using the optimized target domain image classification model, updating the labeled and unlabeled datasets. The parameter optimization process can be understood as optimizing the labeled and unlabeled datasets based on three subsets other than the uncertain and inconsistent subset corresponding to Gaussian component index 3. in, These refer to the confident and consistent subset, the uncertain and consistent subset, and the confident and inconsistent subset, respectively. Pr(x) is a vector formed by concatenating the probabilities corresponding to different components for sample x. The process of iteratively training the target domain image classification model and adjusting the model parameters according to the loss value of each training round calculated by the preset loss function is as follows:

[0116] In principle, the loss function used for training the target domain image classification model can be the same as the loss function used in existing active domain adaptive learning. However, to improve the accuracy of model prediction, this embodiment preferably sets different loss functions according to the different classifications of the training data. This allows for a more accurate and effective measurement of the iterative loss during model training, providing a reliable basis for the reasonable optimization of model parameters. This not only improves the accuracy of model classification prediction but also enhances the robustness of the sampling strategy and ensures the model's generalization ability. It should be noted that this applies to the confidence and inconsistency subset. The samples in this dataset exhibit extreme inconsistency between model predictions and similarity-based labels. This can be understood as the most unique part of the target domain image data distribution relative to the source domain image data distribution. Considering that these samples have excessively high target domain representativeness and are unsuitable for model training, this embodiment preferably discards these samples temporarily in each round of training for the target domain image classification model. Only samples from the unlabeled dataset that are classified into a confident and consistent subset and an uncertain and consistent subset in the current round are selected for training in the current round. If they are classified into other types of subsets in subsequent rounds of learning and training, the comprehensiveness and rationality of the dataset application can still be guaranteed.

[0117] Specifically, the preset loss function includes a cross-entropy loss function, a regularization loss function, and a conditional entropy loss function. Correspondingly, the step of optimizing the parameters of the target domain image classification model based on the updated labeled dataset and the unlabeled dataset, and the preset loss function, includes:

[0118] The labeled dataset, the confident and consistent subset, and the uncertain and consistent subset are respectively input into the target domain image classification model for prediction analysis. Based on the corresponding prediction results, the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function are established respectively. The cross-entropy loss function can be understood as the standard cross-entropy loss function used in model training, which calculates the iterative loss when training with samples from the labeled dataset. The regularization loss function can be understood as taking into account the confident and consistent subset... The samples in the model demonstrate confident and reliable predictions, tending to be close to the class centers in the target domain feature space. When training with these samples, consistency regularization is applied to enhance the consistency of predictions under different perturbations. The iterative loss function is calculated based on this consistent subset. It has the characteristic of predicting with a high degree of reliability but low confidence, and the optimal iterative loss calculation function is selected for training with this part of the samples;

[0119] Based on the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function, the total training loss value of the model is obtained, and the parameters of the target domain image classification model are optimized based on the total training loss value; the total training loss value of the model is expressed as:

[0120] Loss = L sup +λ c L con +λ e L ent

[0121] In the formula,

[0122]

[0123]

[0124]

[0125] Where Loss represents the total training loss of the model; L sup L con and L ent Let λ represent the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function, respectively; c and λ e represents the weights corresponding to the regularization loss function and the conditional entropy loss function, respectively. In practical applications, these are preferably set to 0.5 and 0.1 to ensure training stability; Aug(·) represents the function that further perturbs the sample input, performing data augmentation operations on the samples; L, and These represent labeled datasets, confident and consistent subsets, and uncertain and consistent subsets, respectively.

[0126] S15. Obtain the image to be classified and input the image to be classified into the target domain image classification model for classification prediction to obtain the image classification result; wherein, the image to be classified can be understood as an image of the same type as the source domain image dataset and the target domain image dataset, and can be reliably and effectively classified and predicted by the target domain image classification model with optimal parameters obtained by the above-mentioned active domain adaptive training, to obtain the corresponding image classification result for use in subsequent research.

[0127] This application's embodiments propose a technical solution that initializes labeled and unlabeled datasets based on the acquired source domain image dataset and target domain image dataset, respectively. After initializing and training a preset deep neural network model using the labeled dataset to obtain a target domain image classification model, an information scoring model and an information sampling model are established based on the labeled, unlabeled, and target domain image classification models. The target domain image classification model undergoes active domain adaptive learning iterative training, updating the labeled and unlabeled datasets until a preset iteration stopping condition is reached. The acquired image to be classified is then input into the target domain image classification model for classification prediction to obtain the image classification result. This not only proposes a general active domain adaptive learning model capable of handling scenarios with varying degrees of domain differences, but also... This domain adaptation framework can be widely applied to various domain adaptation problems, including unsupervised, semi-supervised, and passive domain adaptation. It proposes an innovative target data partitioning strategy based on domain representativeness and uncertainty, achieving more stable training, better domain adaptation performance, and stronger versatility through a progressive learning strategy. Furthermore, it proposes a novel information scoring system based on the GMM model and its corresponding sampling function for automatic data partitioning, achieving superior performance compared to state-of-the-art techniques in the same direction for active domain adaptation tasks. In other words, it effectively improves the classification accuracy and generalization ability of image classification models while adapting to different active domain adaptation scenarios, demonstrating strong versatility.

[0128] In one embodiment, such as Figure 5 As shown, an active domain-adaptive image classification system is provided, the system comprising:

[0129] Data acquisition module 1 is used to initialize labeled and unlabeled datasets based on the acquired source domain image dataset and target domain image dataset, respectively; the source domain image dataset is a labeled image dataset; the target domain image dataset is an unlabeled image dataset;

[0130] Initial training module 2 is used to initialize and train a preset deep neural network model based on the labeled dataset to obtain a target domain image classification model; the preset deep neural network model includes a feature extractor and a classifier.

[0131] The scoring and sampling module 3 is used to establish an information scoring model and an information sampling model based on the labeled dataset, the unlabeled dataset, and the target domain image classification model.

[0132] Learning and training module 4 is used to perform active domain adaptive learning iterative training on the target domain image classification model based on the information scoring model and the information sampling model, and update the labeled dataset and the unlabeled dataset until a preset iteration stopping condition is reached.

[0133] Image classification module 5 is used to acquire the image to be classified and input the image to be classified into the target domain image classification model for classification prediction to obtain the image classification result.

[0134] For specific limitations regarding the active domain-adaptive image classification system, please refer to the limitations of the active domain-adaptive image classification method above, which will not be repeated here. Each module in the above-mentioned active domain-adaptive image classification system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0135] Figure 6 An internal structural diagram of a computer device is shown in one embodiment. This computer device may specifically be a terminal or a server. Figure 6 As shown, the computer device includes a processor, memory, network interface, display, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an active domain-adaptive image classification method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0136] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computing devices may include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0137] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0138] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0139] In summary, the present invention provides an image classification method, system, computer device, and storage medium based on active domain adaptation. The active domain adaptation image classification method initializes labeled and unlabeled datasets based on the acquired source domain image dataset and target domain image dataset, respectively. After initializing and training a preset deep neural network model using the labeled dataset to obtain a target domain image classification model, an information scoring model and an information sampling model are established based on the labeled, unlabeled, and target domain image classification models. Active domain adaptation learning is then iteratively trained on the target domain image classification model, updating the labeled and unlabeled datasets until a preset iteration stopping condition is reached. The acquired image to be classified is then input into the target domain image classification model for classification prediction to obtain the image classification result. This method implements a target data partitioning strategy based on the differences in target domain samples, an information sampling mechanism based on a Gaussian mixture model, and a gradual classification learning training strategy, resulting in stable and efficient active domain adaptation image classification learning. This effectively improves the model's classification accuracy and generalization ability while adapting to different active domain adaptation scenarios, demonstrating strong versatility.

[0140] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0141] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this invention, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. An image classification method based on active domain adaptation, characterized in that, The method includes the following steps: The labeled and unlabeled datasets are initialized based on the obtained source domain image dataset and target domain image dataset, respectively; the source domain image dataset is a labeled image dataset; the target domain image dataset is an unlabeled image dataset. The preset deep neural network model is initialized and trained based on the labeled dataset to obtain a target domain image classification model; the preset deep neural network model includes a feature extractor and a classifier. Based on the labeled dataset, the unlabeled dataset, and the target domain image classification model, an information scoring model and an information sampling model are established, including: Based on the labeled dataset and the target domain image classification model, similarity labels are obtained for each sample in the unlabeled dataset. Based on the similarity labels and the corresponding model prediction probability vectors, the information scoring model is established. The information scoring model is a model that comprehensively scores the domain representativeness and uncertainty of each unlabeled sample based on the feature similarity between the unlabeled samples in the target domain image dataset and the samples in the source domain image dataset, and determines the category of the subset of the unlabeled dataset based on the information score. Based on the information scoring model, information scoring sets for the unlabeled dataset and the labeled dataset are obtained respectively, and the information sampling model is constructed based on the information scoring sets for the unlabeled dataset and the labeled dataset. Based on the information scoring model and the information sampling model, the target domain image classification model is subjected to active domain adaptive learning iterative training to update the labeled dataset and the unlabeled dataset until the preset iteration stopping condition is reached. The image to be classified is obtained and input into the target domain image classification model for classification prediction to obtain the image classification result.

2. The image classification method based on active domain adaptation as described in claim 1, characterized in that, The step of obtaining the similarity label of each sample in the unlabeled dataset based on the labeled dataset and the target domain image classification model includes: The labeled dataset is used to extract features using the target domain image classification model to obtain the corresponding labeled sample features; Based on the labeled sample features, the category centers are obtained; the category centers are represented as follows: in, Indicates category c Corresponding category center , C Indicates the total number of categories; Indicates an indicator function; A feature extractor representing a target domain image classification model; This indicates that the dataset has been labeled, and = , Represents the source domain image dataset ； This represents the labeled portion of the target domain image dataset during the training of the target domain image classification model. and These represent the image data of the labeled dataset and the predicted category of the corresponding target domain image classification model, respectively. Based on the category centers, the similarity of each sample in each category in the unlabeled dataset is calculated using the topk metric. The maximum class similarity among the class similarities of each sample is selected, and the class corresponding to the maximum class similarity is used as the corresponding similarity label; the similarity label is represented as: in, Indicates samples in the unlabeled dataset Similarity tags; The parentheses (·) indicate that the vector is sorted in descending order of its element values, and the first few elements are selected. A list of indices of elements; (·) represents the intersection-union ratio function.

3. The image classification method based on active domain adaptation as described in claim 2, characterized in that, The information scoring model is represented as follows: in, Indicates samples in the unlabeled dataset Information rating; Indicates samples in the unlabeled dataset Predicted probability vector The Each element.

4. The image classification method based on active domain adaptation as described in claim 1, characterized in that, The step of constructing the information sampling model based on the information scoring set of the unlabeled dataset and the labeled dataset includes: Based on the information score set of the unlabeled dataset, the unlabeled dataset is classified according to domain representativeness and uncertainty to obtain subset classification categories of the unlabeled dataset; the subset classification categories include confident and consistent class, uncertain and consistent class, uncertain and inconsistent class, and confident and inconsistent class; Based on the category division of the subset and the information scoring set of the labeled dataset, a sample observation label segmentation function is constructed; Based on the sample observation label segmentation function and the information scoring sets of the unlabeled dataset and the labeled dataset, a sampling model training set is constructed. The information sampling model is obtained by training a preset Gaussian mixture model based on the sampling model training set.

5. The image classification method based on active domain adaptation as described in claim 4, characterized in that, The step of performing active domain adaptive learning iterative training on the target domain image classification model based on the information scoring model and the information sampling model, and updating the labeled dataset and the unlabeled dataset, includes: Based on the information scoring model and the information sampling model, the unlabeled dataset is divided according to the subset division category to obtain corresponding unlabeled data subsets; the unlabeled data subsets include a confident and consistent subset, an uncertain and consistent subset, an uncertain and inconsistent subset, and a confident and inconsistent subset; The samples in the uncertain and inconsistent subset are sorted in descending order according to their corresponding classification probabilities. A preset number of unlabeled samples are obtained from top to bottom and labeled. Based on the corresponding labeling results, the labeled dataset and the unlabeled dataset are updated. Based on the updated labeled dataset and the unlabeled dataset, and a preset loss function, the parameters of the target domain image classification model are optimized, and active domain adaptive learning is performed iteratively based on the optimized target domain image classification model to update the labeled dataset and the unlabeled dataset.

6. The image classification method based on active domain adaptation as described in claim 5, characterized in that, The preset loss function includes the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function; The step of optimizing the parameters of the target domain image classification model based on the updated labeled dataset and the unlabeled dataset, and a preset loss function includes: The labeled dataset, the confident and consistent subset, and the uncertain and consistent subset are respectively input into the target domain image classification model for prediction analysis, and the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function are respectively established based on the corresponding prediction results. Based on the cross-entropy loss function, the regularization loss function, and the conditional entropy loss function, the total training loss value of the model is obtained, and the parameters of the target domain image classification model are optimized based on the total training loss value of the model.

7. An image classification system based on active domain adaptation, characterized in that, The system, employing the active neighborhood adaptive image classification method as described in claim 1, comprises: The data acquisition module is used to initialize a labeled dataset and an unlabeled dataset based on the acquired source domain image dataset and target domain image dataset, respectively; the source domain image dataset is a labeled image dataset; the target domain image dataset is an unlabeled image dataset; An initial training module is used to initialize and train a preset deep neural network model based on the labeled dataset to obtain a target domain image classification model; the preset deep neural network model includes a feature extractor and a classifier. The scoring and sampling module is used to establish an information scoring model and an information sampling model based on the labeled dataset, the unlabeled dataset, and the target domain image classification model. The learning and training module is used to perform active domain adaptive learning iterative training on the target domain image classification model based on the information scoring model and the information sampling model, and update the labeled dataset and the unlabeled dataset until a preset iteration stopping condition is reached. The image classification module is used to acquire the image to be classified, input the image to be classified into the target domain image classification model for classification prediction, and obtain the image classification result.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images