Active domain adaptation method based on large model uncertainty

Abstract

The invention discloses an active domain adaptation method based on large model uncertainty, and belongs to the technical field of deep learning. According to the method, screening and labeling of target domain samples are assisted through a large language model, and progressive adaptation under the cross-domain condition is achieved. Firstly, in order to reduce sample selection deviation and reduce large model calling overhead, a distance weighting strategy of an energy function is introduced on the basis of a nearest centroid classifier, and a target domain sample with potential labeling value is preliminarily screened out; next, aiming at a special cue word in the data set style design field, calling a large model in combination with a meta cue word to carry out multi-round labeling on a screening sample, generating a plurality of candidate results in each round, and then carrying out uncertainty modeling on the candidate results through Bayesian aggregation to determine a final unique labeling result; and finally, training a classifier by mixing the labeled target domain sample and the source domain sample to obtain a model with good performance on the target domain. According to the method, dependence on artificial experts in a traditional active domain adaptation task is reduced, active domain adaptation is achieved by combining energy distance screening and uncertainty modeling, and an automatic and efficient solution is provided for active domain adaptation related tasks.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, specifically designing an active domain adaptation method based on large model uncertainty. Background Technology

[0002] With the widespread application of deep learning in fields such as visual recognition, medical diagnosis, industrial inspection, and autonomous driving, models typically achieve high performance in the source domain. However, in actual deployments, the data distribution of the target application environment often differs significantly from the training data, a phenomenon known as domain shift, which leads to a significant drop in model performance in the target domain. Traditional transfer learning and unsupervised domain adaptation methods primarily mitigate domain shift by aligning feature distributions or through adversarial training. However, these methods often overlook the potential value of target domain samples at the annotation level, making it difficult to achieve fast and effective cross-domain adaptation at a limited cost. Proactive domain adaptation proposes actively selecting the most critical target domain samples for annotation to achieve better adaptation results with lower annotation costs. However, existing methods still heavily rely on manual annotation, resulting in high costs and low efficiency.

[0003] In recent years, large language models have demonstrated powerful capabilities in reasoning, understanding, and cross-modal processing, making it possible to use large models to replace human experts for automatic annotation. However, the output of large models is affected by cue word design, sample style differences, and contextual biases, leading to uncertainty and instability in their output. Furthermore, large models have high overhead, making them difficult to use frequently in active learning scenarios. On the other hand, traditional uncertainty estimation methods struggle to accurately identify the most valuable samples in the target domain, limiting the quality of the selection results. Therefore, combining the strong reasoning capabilities of large models with stable and reliable uncertainty modeling mechanisms to achieve efficient, low-cost, and robust active domain adaptation has become a critical problem that urgently needs to be solved. Summary of the Invention

[0004] To address the problems of insufficient sample selection accuracy, reliance on human experts for annotation, and instability of large-model annotation results in existing active domain adaptation methods, this invention proposes an active domain adaptation method based on the uncertainty of large models. This method uses a minimum centroid classifier with energy function optimization to initially screen target domain samples, reducing sample selection bias and the frequency of large model calls. Subsequently, it designs specific prompts based on task and data style, and uses a large language model to annotate the screened samples in multiple rounds, generating multiple candidate answers in each round. Then, it models the uncertainty of the candidate results using a Bayesian aggregation method to determine the final unique annotation result. Finally, it trains a classifier together with the finely annotated target domain samples and source domain data to obtain a model with superior performance and better generalization ability in the target domain. This invention effectively reduces the cost of manual annotation, improves the reliability of sample selection and annotation processes, and achieves more efficient and robust active domain adaptation in cross-domain scenarios, providing an automated, high-precision, and cost-controllable solution for target domain recognition tasks in practical applications. The technical problem addressed by this invention is solved as follows:

[0005] Specific content:

[0006] An active domain adaptation method based on large model uncertainty includes the following steps:

[0007] Step 1: Obtain the required source and target domain datasets, and perform preliminary processing on the images in the datasets to divide them into training and test sets;

[0008] Step 2: Preprocess the low-light data from Step 1 to match the model input, and then perform normalization processing; this process consists of the following 3 steps.

[0009] Step 2-1: Use imageio and opencv to read the image. Read the corresponding image file according to the input file name, file path and image size, and return an array containing the images.

[0010] Step 2-2: Use shuffle and filename to read the data obtained in step 2-1 in batches. Shuffle the index according to the number of data files, and then read the data according to the file corresponding to the index.

[0011] Step 2-3: Call the methods in transforms to process the data and normalize the data read in step 2-2;

[0012] Step 3: Introduce a distance-weighted strategy of energy function on the nearest centroid classifier to initially screen out target domain samples with potential labeling value; this process is divided into the following 4 steps;

[0013] Step 3-1: First, train the classifier on the source domain dataset to obtain the initial classifier;

[0014] Step 3-2: Calculate the energy value of the target domain sample using the energy function E(z). The closer the predicted result is to the true result, the smaller the energy value; conversely, the larger the energy value, the further the predicted result is from the true label. For a given sample feature representation z, the energy function E(z) is calculated as follows:

[0015] E c (z)=-f c (z), c = 1, 2, ..., C

[0016] Among them, f c (z) represents the predicted score of sample feature z in class c obtained by the model, where C is the total number of classes in the dataset.

[0017] Step 3-3: Calculate the energy value of the target domain data samples using the energy function, and then calculate the weighted weights of the samples. The calculation method is as follows:

[0018] w = E 2 (z)-E 1 (z)

[0019] Where w is the uncertainty weight, z represents the feature representation of the target domain, and E 1 E represents the lowest energy value calculated by z. 2 This represents the second lowest energy value calculated by z;

[0020] Steps 3-4 involve obtaining the uncertainty weight w and then performing a weighted calculation on the nearest centroid classifier:

[0021]

[0022] Where, r c Let y be the centroid of the corresponding category c. target Representing the target domain features z target The corresponding pseudo-tags, Let the target domain sample set be defined. After obtaining the class centroids, calculate the Euclidean distance d from the target domain sample to the class centroid. c :

[0023] d c =||z target -r c ||2, c = 1, 2, ..., C

[0024] The category corresponding to the minimum distance is taken as its pseudo-label, and the top 50% of the target domain samples are initially selected as samples with potential value.

[0025] Step 4: Design domain-specific prompts for the dataset style, and combine the meta-prompts with a large model to perform multiple rounds of annotation on the selected samples, generating multiple candidate results in each round; this process is divided into the following 3 steps;

[0026] Step 4-1, design the meta-prompt word template. The template design is as follows:

[0027] Based on the specified reference image, select K most likely category labels from the given label list: <candidate category list>.

[0028] Only provide your guess of the <output format description>, without any explanation. Results are listed in descending order of probability:

[0029] G1: <Index of most likely tags>

[0030] G2: <Index of the second most likely tag> ...

[0032] Gk: <Index of the Kth most likely label>

[0033] Here, <candidate class category> represents a predefined set of category labels, K represents the number of candidate labels to be returned in each inference, and <output format description> is used to constrain the format of the output content.

[0034] Step 4-2: Design different domain-style descriptions for different datasets and add them to the meta-cue word template. Specifically, use the domain description of the target domain. As constraint information, It consists of two parts: first, a "domain style description," which reflects the target domain's unique language style, visual context, or terminology; and second, "task constraints," which limit the output format, category granularity, or decision boundaries. Utilizing... In addition to the aforementioned meta-catch word templates, a set of domain-related candidate catch words P1, P2, ..., P are automatically generated by the large model. N Among them, P j Let N represent the j-th candidate prompt word, and N be the number of prompt words.

[0035] Step 4-3: Input the prompt words and target domain sample data into the large model to obtain multiple candidate answers through multi-round questioning. For each target domain sample x... i (where i is the sample index), and each candidate prompt word P j (where j is the index of the prompt word), the large model is called M times, and each time the K candidate labels are returned in order of confidence. All prompt words and the results of multiple calls are summarized to form a set of candidate labels.

[0036] Step 5: Model the uncertainty of the candidate label results obtained in Step 4 using Bayesian aggregation to determine the final labeling results; this process consists of the following 3 steps.

[0037] Step 5-1, for each candidate label S u Construct a latent score model, assuming the latent scores s of the label u Follows a Gaussian distribution:

[0038]

[0039] Where, μ u Indicates label S u The average ranking preference, i.e., the overall tendency score of the label under multiple prompts and multiple calls. σ represents the semantic uncertainty associated with the label. u Its standard deviation.

[0040] Step 5-2: Approximate the calculation of any two candidate labels S using Monte Carlo sampling. u With S v The pairwise preference probability P is as follows:

[0041]

[0042] Wherein, P(S) u >S v ) indicates the potential score s u ,s v Under the joint distribution, label s u Better than label s v The probability; symbol This represents the expectation of a Gaussian distribution. Since the expectation is difficult to solve analytically, Monte Carlo sampling is used for approximation. The specific method is as follows:

[0043]

[0044]

[0045] Where t∈1,2,…,T is the sampling index, T is the number of samples, and ∈ (t) Let be the random noise term generated in the t-th sampling, which comes from a standard normal distribution. Then, the sampled values ​​are used to approximate the pairwise preference probability:

[0046]

[0047] in, μ is an approximation of the pairwise preference probability. u Indicates label S u Average ranking preference, σu Indicates the relationship with label S u Related semantic uncertainty; μ v With σ v The meaning is similar.

[0048] Step 5-3: Construct a likelihood function using the pairwise preference statistics generated in Step 4, and learn μ by optimizing the negative log-likelihood function. u With σ u Ultimately, only those satisfying σ are retained. u <τ high confidence labels, select μ from them u The largest label is used as the final annotation result S. * :

[0049]

[0050] Tags that do not meet the criteria are considered unstable or semantically ambiguous and are removed, thereby improving the reliability and consistency of the final annotation results.

[0051] Step 6: Mix the high-quality target domain labeled samples obtained in Step 5 with the source domain samples to train the classifier, obtain the final model that can maintain high performance in the target domain, and achieve progressive adaptation under cross-domain conditions.

[0052] This invention proposes an active domain adaptation method based on large model uncertainty, which has the following advantages and beneficial effects compared with existing technologies:

[0053] This invention combines energy function filtering with secondary filtering using a large language model, enabling high-quality automatic labeling based on collected target domain data. Compared to existing methods, this invention improves the accuracy of target domain sample selection by weighting samples using an energy function and obtains more stable candidate labels through multi-cue word-driven large model inference. Bayesian uncertainty aggregation further refines the labeling results, effectively reducing bias and noise in the large model output, thereby significantly improving labeling quality and model adaptability in cross-domain scenarios. Attached Figure Description

[0054] Figure 1 This is the overall flowchart of the method of the present invention.

[0055] Figure 2 This is a schematic diagram of the overall network structure.

[0056] Figure 3 This is a diagram comparing different annotation methods.

[0057] Figure 4 These are images from a portion of the dataset.

[0058] Figure 5 This is a schematic diagram of the sample distribution T-SNE. Detailed Implementation

[0059] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0060] The present invention includes:

[0061] The overall process of the method proposed in this invention is as follows: Figure 1 As shown, the specific construction steps for low-light image enhancement in this embodiment of the invention are as follows:

[0062] Step 1: Obtain the required source and target domain datasets, and perform preliminary processing on the images in the datasets to divide them into training and test sets;

[0063] Step 2: Preprocess the low-light data from Step 1 to match the model input, and then perform normalization processing; this process consists of the following 3 steps.

[0064] Step 2-1: Use imageio and opencv to read the image. Read the corresponding image file according to the input file name, file path and image size, and return an array containing the images.

[0065] Step 2-2: Use shuffle and filename to read the data obtained in step 2-1 in batches. Shuffle the index according to the number of data files, and then read the data according to the file corresponding to the index.

[0066] Step 2-3: Call the methods in transforms to process the data and normalize the data read in step 2-2;

[0067] Step 3: Introduce a distance-weighted strategy of energy function on the nearest centroid classifier to initially screen out target domain samples with potential labeling value; this process is divided into the following 4 steps;

[0068] Step 3-1: First, train the classifier on the source domain dataset to obtain the initial classifier;

[0069] Step 3-2: Calculate the energy value of the target domain sample using the energy function E(z). The closer the predicted result is to the true result, the smaller the energy value; conversely, the larger the energy value, the further the predicted result is from the true label. For a given sample feature representation z, the energy function E(z) is calculated as follows:

[0070] E c (z) = -fc(z), c = 1, 2, ..., C

[0071] Among them, f c (z) represents the predicted score of sample feature z in class c obtained by the model, where C is the total number of classes in the dataset.

[0072] Step 3-3: Calculate the energy value of the target domain data samples using the energy function, and then calculate the weighted weights of the samples. The calculation method is as follows:

[0073] w = E 2 (z)-E 1 (z)

[0074] Where w is the uncertainty weight, z represents the feature representation of the target domain, and E 1 E represents the lowest energy value calculated by z. 2 This represents the second lowest energy value calculated by z;

[0075] Steps 3-4 involve obtaining the uncertainty weight w and then performing a weighted calculation on the nearest centroid classifier:

[0076]

[0077] Where, r c Let y be the centroid of the corresponding category c. target Representing the target domain features z target The corresponding pseudo-tags, Let the target domain sample set be defined. After obtaining the class centroids, calculate the Euclidean distance d from the target domain sample to the class centroid. c :

[0078] d c =||z target -r c ||2, c = 1, 2, ..., C

[0079] The category corresponding to the minimum distance is taken as its pseudo-label, and the top 50% of the target domain samples are initially selected as samples with potential value.

[0080] Step 4: Design domain-specific prompts for the dataset style, and combine the meta-prompts with a large model to perform multiple rounds of annotation on the selected samples, generating multiple candidate results in each round; this process is divided into the following 3 steps;

[0081] Step 4-1, design the meta-prompt word template. The template design is as follows:

[0082] Based on the specified reference image, select K most likely category labels from the given label list: <candidate category list>.

[0083] Only provide your guess of the <output format description>, without any explanation. Results are listed in descending order of probability:

[0084] G1: <Index of most likely tags>

[0085] G2: <Index of the second most likely tag> ...

[0087] Gk: <Index of the Kth most likely label>

[0088] Here, <candidate class category> represents a predefined set of category labels, K represents the number of candidate labels to be returned in each inference, and <output format description> is used to constrain the format of the output content.

[0089] Step 4-2: Design different domain-style descriptions for different datasets and add them to the meta-cue word template. Specifically, use the domain description of the target domain. As constraint information, It consists of two parts: first, a "domain style description," which reflects the target domain's unique language style, visual context, or terminology; and second, "task constraints," which limit the output format, category granularity, or decision boundaries. Utilizing... In addition to the aforementioned meta-catch word templates, a set of domain-related candidate catch words P1, P2, ..., P are automatically generated by the large model. N Among them, P j Let N represent the j-th candidate prompt word, and N be the number of prompt words.

[0090] Step 4-3: Input the prompt words and target domain sample data into the large model to obtain multiple candidate answers through multi-round questioning. For each target domain sample x... i (where i is the sample index), and each candidate prompt word P j (where j is the index of the prompt word), the large model is called M times, and each time the K candidate labels are returned in order of confidence. All prompt words and the results of multiple calls are summarized to form a set of candidate labels.

[0091] Step 5: Model the uncertainty of the candidate label results obtained in Step 4 using Bayesian aggregation to determine the final labeling results; this process consists of the following 3 steps.

[0092] Step 5-1, for each candidate label S u Construct a potential score model, assuming the potential score s of the standard answer. u Follows a Gaussian distribution:

[0093]

[0094] Where, μu Indicates label S u The average ranking preference, i.e., the overall tendency score of the label under multiple prompts and multiple calls. σ represents the semantic uncertainty associated with the label. u Its standard deviation.

[0095] Step 5-2: Approximate the calculation of any two candidate labels S using Monte Carlo sampling. u With S v The pairwise preference probability P is as follows:

[0096]

[0097] Wherein, P(S) u >S v ) indicates the potential score s u ,s v Under the joint distribution, label s u Better than label s v The probability; symbol This represents the expectation of a Gaussian distribution. Since the expectation is difficult to solve analytically, Monte Carlo sampling is used for approximation. The specific method is as follows:

[0098]

[0099]

[0100] Where t∈1,2,...,T is the sampling index, and T is the number of samples. (t) Let be the random noise term generated in the t-th sampling, which comes from a standard normal distribution. Then, the sampled values ​​are used to approximate the pairwise preference probability:

[0101]

[0102] in, μ is an approximation of the pairwise preference probability. u Indicates label S u Average ranking preference, σ u Indicates the relationship with label S u Related semantic uncertainty; μ u It has a similar meaning to σ.

[0103] Step 5-3: Construct a likelihood function using the pairwise preference statistics generated in Step 4, and learn μ by optimizing the negative log-likelihood function. u With σ u Ultimately, only those satisfying σ are retained. u <τ high confidence labels, from which μ is selected u The largest label is used as the final annotation result S.* :

[0104]

[0105] Tags that do not meet the criteria are considered unstable or semantically ambiguous and are removed, thereby improving the reliability and consistency of the final annotation results.

[0106] Step 6: Mix the high-quality target domain labeled samples obtained in Step 5 with the source domain samples to train the classifier, obtain the final model that can maintain high performance in the target domain, and achieve progressive adaptation under cross-domain conditions.

[0107] This invention combines an energy function screening mechanism with a secondary screening using a large language model, enabling automated and high-quality label generation based on collected target domain data. Compared to existing methods, this invention utilizes an energy function to accurately assess the importance of target domain samples during labeling, ensuring that the selected samples are more representative. It then obtains stable candidate label results through large-scale model inference driven by multiple prompt words. Subsequently, a Bayesian uncertainty aggregation method is employed to refine the candidate labels, effectively reducing noise and bias in the large model output, thereby significantly improving the labeling reliability and model adaptability in cross-domain scenarios.

[0108] This invention provides an active domain adaptation method based on large model uncertainty. The above description is only an embodiment of this invention and does not limit the patent scope of this invention. Any equivalent structural or procedural transformations made using the content of this specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this invention.

Claims

1. An active domain adaptation method based on large model uncertainty, characterized in that, Includes the following steps: Step 1: Obtain the required source and target domain datasets, and perform preliminary processing on the images in the datasets to divide them into training and test sets; Step 2: Preprocess the low-light data from Step 1 to match the model input, and then perform normalization processing; this process consists of the following 3 steps. Step 2-1: Use imageio and opencv to read the image. Read the corresponding image file according to the input file name, file path and image size, and return an array containing the images. Step 2-2: Use shuffle and filename to read the data obtained in step 2-1 in batches. Shuffle the index according to the number of data files, and then read the data according to the file corresponding to the index. Step 2-3: Call the methods in transforms to process the data and normalize the data read in step 2-2; Step 3: Introduce a distance-weighted strategy of energy function on the nearest centroid classifier to initially screen out target domain samples with potential labeling value; this process is divided into the following 4 steps; Step 3-1: First, train the classifier on the source domain dataset to obtain the initial classifier; Step 3-2: Calculate the energy value of the target domain sample using the energy function E(z). The closer the predicted result is to the true result, the smaller the energy value; conversely, the larger the energy value, the further the predicted result is from the true label. For a given sample feature representation z, the energy function E(z) is calculated as follows: E c (z)=-f c (z),c=1,2,...,C Among them, f c (z) represents the predicted score of sample feature z in class c obtained by the model, where C is the total number of classes in the dataset. Step 3-3: Calculate the energy value of the target domain data samples using the energy function, and then calculate the weighted weights of the samples. The calculation method is as follows: in=E 2 (That 1 (With) Where w is the uncertainty weight, z represents the feature representation of the target domain, and E 1 E represents the lowest energy value calculated by z. 2 This represents the second lowest energy value calculated by z; Steps 3-4 involve obtaining the uncertainty weight w and then performing a weighted calculation on the nearest centroid classifier: Where, r c Let y be the centroid of the corresponding category c. target Representing the target domain features z target The corresponding pseudo-tag, z target Let the target domain sample set be defined. After obtaining the class centroids, calculate the Euclidean distance d from the target domain sample to the class centroid. c : d c =||z target -r c ||2,c=1,2,…,C The category corresponding to the minimum distance is taken as its pseudo-label, and the top 50% of the target domain samples are initially selected as samples with potential value. Step 4: Design domain-specific prompts for the dataset style, and combine the meta-prompts with a large model to perform multiple rounds of annotation on the selected samples, generating multiple candidate results in each round; this process is divided into the following 3 steps; Step 4-1, design the meta-prompt word template. The template design is as follows: Based on the specified reference image, select K most likely category labels from the given label list: <candidate category list>. Only provide your guess of the <output format description>, without any explanation. Results are listed in descending order of probability: G1: <Index of most likely tags> G2: <Index of the second most likely tag> ... Gk: <Index of the Kth most likely label> Here, <candidate class category> represents a predefined set of category labels, K represents the number of candidate labels to be returned in each inference, and <output format description> is used to constrain the format of the output content. Step 4-2: Design different domain-style descriptions for different datasets and add them to the meta-cue word template. Specifically, use the domain description of the target domain. As constraint information, It consists of two parts: first, a "domain style description," used to reflect the target domain's unique language style, visual context, or terminology; and second, "task constraints," used to limit the output format, category granularity, or decision boundaries. Utilizing... In addition to the aforementioned meta-catch word templates, a set of domain-related candidate catch words P1, P2, ..., P are automatically generated by the large model. N Among them, P j Let N represent the j-th candidate prompt word, and N be the number of prompt words. Step 4-3: Input the prompt words and target domain sample data into the large model to obtain multiple candidate answers through multi-round questioning. For each target domain sample x... i (where i is the sample index), and each candidate prompt word P j (where j is the index of the prompt word), the large model is called M times, and each time the K candidate labels are returned in order of confidence. All prompt words and the results of multiple calls are summarized to form a set of candidate labels. Step 5: Model the uncertainty of the candidate label results obtained in Step 4 using Bayesian aggregation to determine the final labeling results; this process consists of the following 3 steps. Step 5-1, for each candidate label S u Construct a latent score model, assuming the latent scores s of the label u Follows a Gaussian distribution: Where, μ u Indicates label S u The average ranking preference, i.e., the overall tendency score of the label under multiple prompts and multiple calls. σ represents the semantic uncertainty associated with the label. u Its standard deviation. Step 5-2: Approximate the calculation of any two candidate labels S using Monte Carlo sampling. u With S v The pairwise preference probability P is as follows: Wherein, P(S) u >S v ) indicates the potential score s u s v Under the joint distribution, label s u Better than label s v The probability; symbol This represents the expectation of a Gaussian distribution. Since the expectation is difficult to solve analytically, Monte Carlo sampling is used for approximation. The specific method is as follows: Where t∈1,2,...,T is the sampling index, and T is the number of samples. (t) Let be the random noise term generated in the t-th sampling, which comes from a standard normal distribution. Then, the sampled values ​​are used to approximate the pairwise preference probability: in, μ is an approximation of the pairwise preference probability. u Indicates label S u Average ranking preference, σ u Indicates the relationship with label S u Related semantic uncertainty; μ v With σ v The meaning is similar. Step 5-3: Construct a likelihood function using the pairwise preference statistics generated in Step 4, and learn μ by optimizing the negative log-likelihood function. u With σ u Ultimately, only those satisfying σ are retained. u <τ high confidence labels, select μ from them u The largest label is used as the final annotation result S. * : Tags that do not meet the criteria are considered unstable or semantically ambiguous and are removed, thereby improving the reliability and consistency of the final annotation results. Step 6: Mix the high-quality target domain labeled samples obtained in Step 5 with the source domain samples to train the classifier, obtain the final model that can maintain high performance in the target domain, and achieve progressive adaptation under cross-domain conditions.

Images