Label shift adaptive correction method and apparatus

Abstract

This invention relates to a label shift adaptive correction method and apparatus, belonging to the field of image processing technology. The method includes: acquiring source data and target data; calculating estimated importance weights based on a traditional label shift method; training an initial target classifier using cross-entropy loss; calculating the class probability of the target data using the initial target classifier; approximating the target distribution corresponding to the target data based on the estimated importance weights; selecting high-confidence pseudo-label samples based on the approximation of the target distribution and the class probability; aligning the high-confidence pseudo-label samples with the label conditional distribution of the source data; combining the aligned pseudo-label samples with the source data to construct a new training set approximating an intermediate distribution; recalculating the estimated importance weights on the new training set based on the traditional label shift method; and training the final target classifier by minimizing empirical loss. This method can effectively alleviate label shift, correct pseudo-label bias, and align conditional distributions.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a label shift adaptive correction method and apparatus. Background Technology

[0002] In classic learning scenarios, models are typically trained on labeled source sets and predicted on unlabeled target sets. This approach is reasonably effective when the source and target sets have independent samples with the same distribution. However, in real-world scenarios, the same distribution assumption is rarely satisfied. For example, in medical diagnosis, disease distributions often differ across regions, while the symptoms of the same disease can be very similar. Another example is in bird identification... Figure 1 As shown, the distribution of bird species differs between spring and winter, but the physical appearance of birds remains constant with the seasons. These phenomena are commonly referred to as tag shifting, characterized by differences in the distribution of tag edges between the source and target domains. ), while the condition distribution remains consistent across domains ( ).

[0003] Due to changes in the marginal distribution of labels, the performance of traditional models tends to decline significantly. Therefore, to improve the generalization ability of models, traditional models must be adjusted to adapt to label shifting scenarios. Most advanced label shifting methods use importance weight estimation to recalibrate the source classifier, making it better suited to target samples. For example, BBSE uses importance-weighted empirical risk minimization to correct the source classifier, while RLLS mitigates the impact of limited target samples on weight estimation through a regularized weight estimator and derives a generalization error bound for limited samples. Some methods also employ maximum likelihood estimation or confidence calibration techniques, eliminating the need for model retraining. For instance, MLLS combines maximum likelihood with temperature scaling for bias correction to directly estimate the posterior probability of target samples, while SML addresses the problem of negative prior estimation that may result from inconsistent estimation of decision probabilities and confusion matrices. In addition, some strategies attempt to combine labeled source samples and unlabeled target samples to co-train the classifier, aiming to overcome the limitation of limited training samples.

[0004] However, existing methods still suffer from significant performance bottlenecks. Both theoretical analysis and empirical observations show that when faced with scenarios involving large label shifts or a limited number of training samples, the performance of both importance weight estimation-based methods and training strategies combining two sample sets degrades considerably. Furthermore, directly using unlabeled target samples to generate pseudo-labels for training can easily lead to a shift in the conditional distribution of labels due to sample selection bias, violating the label shift assumption and thus affecting the model's generalization performance. In addition, the noise inherent in the pseudo-labels themselves further exacerbates the bias in model training. These issues make it difficult for existing methods to maintain stable and reliable performance in complex real-world scenarios. Summary of the Invention

[0005] Therefore, it is necessary to provide a label shift adaptive correction method and apparatus that can effectively alleviate label shift, correct false label deviation, and align conditional distribution to address the aforementioned technical problems.

[0006] A tag shift adaptive correction method, the method comprising:

[0007] Acquire labeled source data and unlabeled target data, wherein the source data and target data satisfy the label shift assumption;

[0008] The importance weights are estimated based on the traditional label shifting method, and the initial target classifier is obtained by training with cross-entropy loss.

[0009] The initial target classifier is used to calculate the class probability of the target data, and the estimated importance weights are used to approximate the target distribution corresponding to the target data; high-confidence pseudo-label samples are selected based on the target distribution approximation and the class probability.

[0010] Kernel mean matching is used to align the label conditional distribution of the high-confidence pseudo-label samples with that of the source data, while sample weight constraints are applied to keep the marginal distribution of labels unchanged;

[0011] The aligned pseudo-label samples are combined with the source data to construct a new training set that approximates the intermediate distribution;

[0012] The importance weights are re-estimated on the new training set based on the traditional label shifting method, and the final target classifier is obtained by training by minimizing the empirical loss.

[0013] On the other hand, a tag shift adaptive correction device is also provided, comprising:

[0014] The data acquisition module is used to acquire labeled source data and unlabeled target data, wherein the source data and target data satisfy the label shift assumption;

[0015] The initial target classifier training module is used to calculate the estimated importance weights based on the traditional label shifting method and train the initial target classifier by combining it with cross-entropy loss.

[0016] The pseudo-label sample screening module is used to calculate the class probability of the target data through the initial target classifier, approximate the target distribution corresponding to the target data according to the estimated importance weight, and screen high-confidence pseudo-label samples according to the target distribution approximation result and the class probability.

[0017] The alignment module is used to align the label conditional distribution of the high-confidence pseudo-label samples with the source data using kernel mean matching, while applying sample weight constraints to keep the marginal distribution of labels unchanged.

[0018] A new training set module is constructed to combine the aligned pseudo-label samples with the source data to construct a new training set that approximates the intermediate distribution.

[0019] The final target classifier training module is used to recalculate the estimated importance weights on the new training set based on the traditional label shifting method, and train the final target classifier by minimizing the empirical loss.

[0020] Compared with existing technologies, the tag shift adaptive correction method and apparatus provided by the present invention have the following advantages:

[0021] 1. By constructing a new training set that approximates an intermediate distribution, the deviation problem of direct correction between the source distribution and the target distribution in traditional methods is avoided, effectively alleviating label shift. Even in scenarios with large label shifts or limited training samples, the classifier's performance can still be maintained stably, significantly improving the generalization ability under the target distribution.

[0022] 2. By calculating the class probability using the initial classifier and combining it with the approximate result of the target distribution to screen high-confidence pseudo-labels, reliable pseudo-label samples are selected, and pseudo-label bias is accurately corrected. At the same time, by constraining sample weights, the distribution shift problem caused by pseudo-label selection is avoided, effectively reducing the interference of pseudo-label noise on the training process and improving the utilization efficiency of pseudo-label samples.

[0023] 3. By using kernel mean matching technology, we achieved accurate alignment between high-confidence pseudo-label samples and the conditional distribution of source data labels. At the same time, we maintained the marginal distribution of labels by constraining sample weights, which strictly conformed to the core assumptions of the label shifting scenario and ensured the rationality and effectiveness of classifier training. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention, and those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram illustrating tag displacement in the context of bird identification in one embodiment;

[0026] Figure 2 This is a flowchart illustrating a tag shift adaptive correction method in one embodiment;

[0027] Figure 3This is a schematic diagram of the tag shift adaptive correction method framework in one embodiment;

[0028] Figure 4 This is a schematic diagram of the tag distribution deviation in a tag shift adaptive correction method according to one embodiment, wherein, Figure 4 (a) is a schematic diagram of the label distribution bias on the Tweak-One MNIST dataset. Figure 4 (b) is a schematic diagram of the label distribution bias on the Dirichlet shift CIFAR10 dataset. Figure 4 (c) is a schematic diagram of the label distribution bias on the Dirichlet CIFAR100 dataset;

[0029] Figure 5 This is a structural block diagram of a tag shift adaptive correction device in one embodiment;

[0030] Figure 6 This is an internal structural diagram of a computer device in one embodiment.

[0031] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0033] It should be noted that in this invention, the use of terms such as "first," "second," etc., is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0034] It is understood that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0035] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0036] Example 1

[0037] like Figure 2 and Figure 3 As shown, this embodiment provides a tag shift adaptive correction method, including the following steps:

[0038] Step 201: Obtain the labeled source data and the unlabeled target data, where the source data and target data satisfy the label shift assumption.

[0039] Step 202: Calculate the estimated importance weights based on the traditional label shifting method, and train the initial target classifier using cross-entropy loss.

[0040] Step 203: Calculate the class probability of the target data using the initial target classifier, approximate the target distribution corresponding to the target data based on the estimated importance weights, and select high-confidence pseudo-label samples based on the approximate target distribution and class probability.

[0041] Step 204: Kernel mean matching is used to align the label conditional distribution of the high-confidence pseudo-label samples with that of the source data, while sample weight constraints are applied to keep the marginal distribution of the labels unchanged.

[0042] Step 205: Combine the aligned pseudo-label samples with the source data to construct a new training set that approximates the intermediate distribution.

[0043] Step 206: Based on the traditional label shifting method, recalculate the estimated importance weights on the new training set, and train the final target classifier by minimizing the empirical loss.

[0044] As can be seen, the tag shift adaptive correction method disclosed in this embodiment proposes a general framework, abbreviated as ALS, such as... Figure 3 As shown, firstly, the concept of an intermediate distribution is introduced, whose label distribution lies between the source and target label distributions. Furthermore, a traditional label shifting method is used to approximate the target distribution, and a self-training method is employed to select high-confidence pseudo-label target data that conforms to the aforementioned distribution. To mitigate bias caused by sample selection, kernel mean matching (KMM) is used to align the conditional distribution. Based on this, a new training set is constructed, consisting of selected target and source data, which approximately follows the intermediate distribution. Finally, the target classifier is obtained on the new training set using the traditional label shifting method. Theoretical analysis is conducted to investigate the impact of pseudo-label bias, and it is demonstrated that ALS can achieve a more stringent generalization bound under mild conditions.

[0045] In the specific implementation of step 201, random variables are used. Modeling is performed separately for features and labels, where the feature space... ; tag space It is a discrete domain, equivalent to ; It is the feature dimension. It represents the total number of categories.

[0046] Suppose the labeled source data follows a source distribution. tag set Unlabeled target data follows a target distribution. unlabeled set ;in, The total number of samples in the source data. The total number of target data samples. (Used) and Represents the source distribution and target distribution The relevant probability density or mass function. The displacement is quantified using the exponents of the second-order and infinite-order Renyi divergences, expressed as:

[0047] ;

[0048] In the formula, Represents the second-order Renyi divergence between the target distribution and the source distribution; Represents the target distribution In the Marginal probability of class label; Represents source distribution In the Marginal probability of class label; Represents the infinite-order Renyi divergence between the target distribution and the source distribution.

[0049] Under the label shift assumption, the label edge distributions in the source and target domains differ, i.e. The condition distribution remains consistent across domains, i.e. ,in, Represents source distribution The marginal distribution of labels Represents the target distribution The marginal distribution of labels Represents source distribution The distribution of label conditions; Represents the target distribution The label conditional distribution. If given a hypothetical class... and a loss function The goal of the label shift setting is to find a hypothesis. To minimize the reweighted loss:

[0050] ;

[0051] In the formula, Represents the risk function; This represents the classifier to be trained; Indicates the target distribution The mathematical expectation operator is used to calculate the target data sample. The mean of the corresponding statistic; Represents the loss function; Represents the classifier to be trained For sample features The predicted output; This represents the characteristic data of the sample; Labels representing samples; Indicates the source distribution The mathematical expectation operator is used to calculate the source data sample. The mean of the corresponding statistic; Indicates importance weight, and .if = This indicates that the source domain and the target domain are the same, that is... If so, the above problems degenerate into the standard learning paradigm.

[0052] In reality, the true importance weight It is unknown. If the weights can be estimated as... The target classifier is obtained by minimizing the empirical loss, expressed as:

[0053] ;

[0054] In the formula, Represents the classifier to be trained In target distribution The empirical risk estimate is used to approximate the loss of the classifier on the target data using the source data; This indicates the estimated importance weights; Indicates the first The true label of each source data sample The corresponding estimated importance weights; Represents the classifier to be trained For the first Features of each source data sample The predicted output.

[0055] This step involves acquiring basic data that matches the label shift scenario, clarifying the data distribution characteristics and quantifying the degree of shift, thus providing accurate data input and a basis for measuring differences for the subsequent adaptive correction process.

[0056] In the specific implementation of step 202, as can be seen from step 201, the core of the reweighting framework is how to estimate the importance weights. .

[0057] Traditional label shifting methods include existing methods such as BBSE and RLLS; importance weights refer to the weights of the target distribution relative to the source distribution, satisfying... It is used to correct the marginal difference between the source and target labels.

[0058] set up Let be the confusion matrix, where, Represents a probability function. Represents the classifier to be trained Features of source data samples The prediction category This represents the true label of the source data sample. Represents the classifier to be trained The predicted category identifier for the source data samples, This indicates the true category identifier of the source data sample.

[0059] In a typical finite sample setting, the confusion matrix and probability mass function The corresponding estimated confusion matrix and estimating probability mass function Estimated from the existing sample, where, express The probability mass function; Represents the classifier to be trained Output for unlabeled target data.

[0060] Therefore, the expression for BBSE to calculate the estimated importance weights is: The expression for calculating the estimated importance weight shift using RLLS is: Then, based on the estimated importance weight shift... Obtain the estimated importance weights In the formula, Indicates the degree of importance weight shift; This represents the regularization coefficient.

[0061] In order to analyze the theoretical properties, let The generalizations of the BBSE and RLLS methods are summarized in the following unified form.

[0062] Lemma 1: Let and These are the final target classifier and the optimal target classifier obtained through training, respectively. The generalization bound below is at least based on probability. Established:

[0063] ;

[0064] in,

[0065] ;

[0066] ;

[0067] also, And Rademacher's complexity is defined as:

[0068] ;

[0069] In the formula, This represents the generalization error term; This represents the weight estimation error term; Represents the second-order Renyi divergence between the target distribution and the source distribution; Represents the infinite-order Renyi divergence between the target distribution and the source distribution; This represents the Rademacher complexity metric; Represents the class of weighted loss functions; Represents the infinite-order Renyi divergence between the target distribution and the intermediate distribution; Represents the second-order Renyi divergence between the target distribution and the intermediate distribution; This represents the total number of samples in the source data; Indicates importance weight; Indicates the total number of categories; This represents the total number of samples in the target data. This represents the smallest eigenvalue of the confusion matrix; Indicates the intermediate distribution The mathematical expectation operator; Represents the Rademacher random variable The mathematical expectation operator; Represents the weighted loss function; Indicates the first Each feature and label sample pair; Represents the set of all possible classifiers; This indicates the importance weight of the label. The universal constant factor is hidden.

[0070] As can be seen from Lemma 1, the label shift deviation The larger the value, the lower the model's generalization performance.

[0071] The initial target classifier is obtained by training using cross-entropy loss combined with importance weights, and its expression is:

[0072] ;

[0073] In the formula, Indicates the initial target classifier; Indicates the first Estimated importance weights for each class; Indicates the total number of categories; Indicates the data sample category; Indicates the first The source data sample corresponds to the first The class label indicates the variable; Indicates in the parameter Under the classifier, the first Individual source data samples Belongs to the The probability of a class category; Indicates the parameters of the classifier; Indicates the first Feature data of each source data sample; Indicates the sample number, where, This represents the total number of samples in the source data. This represents the total number of samples in the target data.

[0074] This step combines a mature weight estimation method with cross-entropy loss to quickly obtain an initial model that fits the target distribution trend, providing reliable support for subsequent category probability calculation and pseudo-label screening, ensuring the accuracy and efficiency of subsequent processes. At the same time, regularization terms are used to reduce the impact of limited target samples on weight estimation.

[0075] In the specific implementation of step 203, the category probability refers to the softmax probability of the target data output by the initial target classifier belonging to each category. The approximate result of the target distribution is obtained through the formula. Calculate, where For an approximate marginal distribution of target labels, The label distribution of the source data, The estimated importance weights are obtained in step 202.

[0076] Empirical evidence suggests that high soft probabilities lead to high-confidence pseudo-labels. Based on this, we propose the concept of confidence-guided pseudo-label selection, where the optimization objective for screening high-confidence pseudo-label samples is:

[0077] ;

[0078] In the formula, This represents a set of pseudo-labels for the target data; This represents the total number of samples in the target data. Indicates the first The true labels of each source data sample; Indicates the first Each source data sample is used in the initial target classifier. The probability of the category.

[0079] The global optimal solution to the above optimization objective, i.e., the selection criteria for high-confidence pseudo-label samples, is as follows:

[0080] ;

[0081] In the formula, Indicates the first The pseudo-label indicator vector of each target sample; Indicates the first The optimal class for each target sample; Indicates the first The confidence threshold for the class.

[0082] It is worth noting that the pseudo-label indicator vector It can be a single-hot vector or an all-zero vector, depending on whether the corresponding sample is selected. In particular, when A single-hot vector indicates that the sample is selected, while a vector of all zeros indicates that the sample is not selected. To determine sample selection, the first... kind The probability is used as the confidence level of the prediction. If the prediction satisfies... Then, with confidence, samples are selected and classified as .on the contrary, This corresponds to samples with lower credibility that are not selected.

[0083] Confidence threshold These parameters play a crucial role in controlling the learning and selection of pseudo-labels. This embodiment proposes a strategy for determining these parameters in a specific scenario. Specifically, it selects pseudo-labels with high confidence that satisfy the marginal probability. The sample. Because the real one. It is unknown, therefore use Let's calculate it.

[0084] In this embodiment, the confidence threshold The method for determining this is: setting the sample selection ratio. ,pass Determine the number of target samples to select from all classes; then the... The number of samples to be selected for each class is ,in This is a rounding function. For the first Approximate results of the target distribution of the class. The total number of samples in the target data. For the target data, the first The set of tags corresponding to the class; the first After sorting the class probabilities of the target data in descending order, take the top [number]. The minimum value among these probabilities is used as the confidence threshold. .

[0085] This step uses a three-layer logic of "probability calculation - distribution approximation - confidence screening" to accurately screen out high-reliability pseudo-label samples, effectively eliminate noise interference caused by low-quality samples, and provide a high-quality data foundation for subsequent distribution alignment. At the same time, the sample selection ratio is constrained by the target distribution approximation result to ensure that the distribution of pseudo-label samples is consistent with the trend of the target distribution.

[0086] In the specific implementation of step 204, it is assumed that the selected pseudo-label data conforms to... Distribution. However, due to selection bias, the label conditional distribution may change, i.e. This violates the label shift assumption. Therefore, this change affects the expected loss of the intermediate distribution, as described in Theorem 2.

[0087] Theorem 2: Assume that the classifier has an upper bound loss, that is, there exists at least one classifier to be trained. satisfy ,in, This represents the upper bound of the classifier loss.

[0088] For convenience, let's assume represent .exist Under the -loss assumption, for the learning model The following results were obtained:

[0089] ;

[0090] in, Represents source distribution The classifier to be trained Expected loss; Represents the target distribution The classifier to be trained Expected loss; Represents source distribution The marginal distribution of labels; Represents the target distribution The marginal distribution of labels; This represents the classifier to be trained under the intermediate distribution T. Expected loss; The marginal distribution of labels represents the intermediate distribution T; The operator representing the mathematical expectation under the intermediate distribution T; The total variational distance of the label-conditional distribution between the intermediate distribution and the target distribution; This represents the total variation (TV) distance between probability distributions, and .

[0091] According to Theorem 2, even if It is also difficult to guarantee that the expected difference will be met. This is due to the TV distance. This is the cause. However, due to It is difficult to optimize directly, but a sufficient condition is given by Theorem 3.

[0092] Theorem 3: If the conditional distribution satisfies ,but Established, among which It is the conditional distribution followed by the selected pseudo-label data.

[0093] Based on this, kernel mean matching is used to align the label conditional distribution of the high-confidence pseudo-label samples with that of the source data, as shown below:

[0094] ;

[0095] In the formula, Indicates the total number of categories; Indicates the data sample category; Indicates the source data number The kernel mapping expectation of features under class-conditional distribution; Represents the mathematical expectation operator; Representation of features Obey the source data Class label conditional distribution; Indicates the target data number Weighted feature kernel mapping expectation under class-conditional distribution; Representation of features Obey the target data Class label conditional distribution; Indicates the first The weights corresponding to pseudo-labeled samples.

[0096] To ensure that as many samples as possible are selected from the target set, A regularization term was added. However, the weights corresponding to the pseudo-labeled samples... This could alter the marginal distribution, which violates the condition. Therefore, constraints on sample weights were introduced. Theoretically, this guarantees the invariance of the marginal distribution, as shown in Theorem 4.

[0097] Theorem 4: Assumption yes The The element corresponding to the element The class of The weights of each data point. If sample weights are constrained. For each class If all are true, then ,in It is a weighted pseudo-label distribution;

[0098] The empirical form of expected loss can be expressed as:

[0099] ;

[0100] The above loss can be reparameterized as:

[0101] ;

[0102] Based on the above information, when kernel mean matching is used to align the label conditional distributions of the high-confidence pseudo-labeled samples with those of the source data, the final loss function is:

[0103] ;

[0104] In the formula, The kernel matrix representing the pseudo-labeled samples; Indicates the first The cross-core matrix between source data and pseudo-labeled samples; Indicates the first The regularization coefficient of the class; Indicates the transpose operation; Indicates the first The number of high-confidence pseudo-labeled samples; The set of in-class weights representing pseudo-labeled samples; Indicates the first Class 1 The weights of each pseudo-labeled sample; Represents the kernel mapping function; Indicates the first Class 1 One pseudo-label sample; Indicates the first The number of samples in the source data; Indicates the first Class 1 One source data sample.

[0105] The dimension of the optimal solution to the above problem is It can be obtained through quadratic programming (QP).

[0106] Assumption And the weight is ,analyze The boundary, which represents the distribution and The difference between them. In the formula, This represents the label conditional distribution alignment loss; Indicates estimated weights The corresponding label conditional distribution alignment loss; Indicates the source data number Mathematical expectation operator under class label conditional distribution; This indicates the first high-confidence pseudo-label sample. Mathematical expectation operator under class label conditional distribution.

[0107] Theorem 5: Assume that the kernel mapping has an upper bound on the intermediate set, i.e. It is obvious that At least with The probability of holds as follows:

[0108] ;

[0109] In the formula, Represents the optimal weight The corresponding label conditional distribution alignment loss; The upper bound of the deviation representing the alignment loss; Denotes the upper bound of the norm of the kernel mapping function; Indicates the first The number of high-confidence pseudo-labeled samples; Indicates the first The number of samples in the source data; This represents the total number of samples in the source data; This represents the total number of high-confidence pseudo-labeled samples; Indicates the total number of categories; This represents the confidence probability parameter.

[0110] This step addresses the conditional distribution shift caused by pseudo-label selection using kernel mean matching technology. It combines weight constraints to maintain the marginal distribution, strictly adhering to the label shift hypothesis and providing reliable data that conforms to the hypothesis for subsequent training. At the same time, regularization terms ensure the rationality of sample weights and avoid overfitting.

[0111] In the specific implementation of step 205, once the aligned pseudo-label samples are obtained... It is then easy to construct a new training set. ,in, Represents the set of source data samples labeled; Indicates the first The weights of high-confidence pseudo-labeled samples; This represents the set of high-confidence pseudo-labeled samples. The new training set is composed of labeled source data and weighted high-confidence pseudo-labeled samples, and its self-training distribution approximates the intermediate distribution.

[0112] As can be seen from Lemma 1, the label shift deviation The larger the value, the lower the model's generalization performance. Therefore, the concept of an intermediate distribution is introduced to alleviate label shifting. The intermediate distribution is defined as follows: given a distribution... , This corresponds to the probability density or mass function. If the label conditional distribution remains unchanged... Furthermore, the marginal distribution offset of the labels is mitigated. If ), then take It is an intermediate distribution.

[0113] Furthermore, to satisfy The sufficient condition for the distribution to be intermediate is as stated in Theorem 6.

[0114] Theorem 6: Intermediate Distribution The following two conditions must be met: (1) The label distribution conforms to (2) Intermediate distribution The conditional distribution conforms to ,in The mixing coefficient represents the intermediate distribution. The marginal distribution of labels represents the intermediate distribution T. Represent the label conditional distribution of the intermediate distribution T; if Then it is believed It is an intermediate distribution.

[0115] Subsequently, as described in Theorem 7, a distribution following the intermediate distribution was constructed. The specific set is shown, and the traditional generalization bound (Lemma 1) is proved to be more compact in the following three aspects: (1) increasing the number of training samples; (2) reducing the label distribution bias; (3) reducing the infinite and second-order Renyi divergence.

[0116] Theorem 7: Assume that... One target label, i.e. Then construct a new tag set. As a new training set subsequently used to train the classifier, it follows an intermediate distribution. and ;in, The labeled set representing high-confidence pseudo-labeled samples. This represents the set of high-confidence pseudo-labeled samples. This represents a new set of labels that follows the intermediate distribution T. This represents the source data sample set for labeling. If traditional label shifting methods (such as RLLS or BBSE) are used in the new training... If the classifier is trained on a higher level, the generalization bound will be tighter.

[0117] This step establishes a relationship between the source distribution and the target distribution by combining the source data and the aligned pseudo-label samples. This effectively alleviates the direct bias caused by label shifting, provides a high-quality data foundation for the final classifier training, and solves the problem of correction failure caused by excessive differences between the source distribution and the target distribution in traditional methods.

[0118] In the specific implementation of step 206, the recalculation of the estimated importance weights still uses traditional label shifting methods such as BBSE and RLLS, and the weights are recalibrated on the new training set. The function expression that minimizes the empirical loss is:

[0119] ;

[0120] In the formula, Represents the empirical loss function; This represents the classifier to be trained; This represents the estimated importance weights recalculated on the new training set; This represents the set of weights for the aligned pseudo-labeled samples. This represents the total number of samples in the source data; Represents the cross-entropy loss function; Indicates the first The true labels of each source data; This represents the total number of high-confidence pseudo-labeled samples; Indicates the first The weights of each pseudo-labeled sample.

[0121] This step involves retraining on a new training set that approximates the intermediate distribution, ultimately obtaining a classifier that fits the target distribution. This addresses the issue of significant performance degradation in existing methods under scenarios with large label shifts or limited samples, ensuring that the model has stable generalization ability. Furthermore, the theoretical reliability of the method is proven through rigorous generalization error bounds.

[0122] In one embodiment, to demonstrate the effectiveness of the proposed ALS framework, the Acc and F-scores were calculated for different proportions of pseudo-labeled samples on the Dirichlet and Tweak-one shift datasets. All methods were run on Python 3.7 and PyTorch frameworks. As can be seen from the results in Table 1, although the performance of different comparison methods varies across different datasets, the ALS framework consistently enhances the performance of existing label shifting methods in most cases, as indicated by the "Average Improvement" metric. It is worth noting that when using… On the Dirichlet shift CIFAR10 dataset, the ALS framework showed an improvement of over 6% in accuracy and 8% in F-score.

[0123] Furthermore, the method proposed in this invention shows greater improvement in scenarios involving large-scale tag transfer. For example, when using... On the Dirichlet shift MNIST dataset, an accuracy improvement of over 3% and an F-score improvement of over 5% were observed. However, when using... On the same class dataset, the improvement in accuracy and f-score was only close to 1%. These findings suggest that small label shifts may have the least impact on model performance, indicating that the importance weights of ALS correction are not significantly effective. Therefore, the effectiveness of the ALS framework in mitigating label shifting is demonstrated.

[0124] Furthermore, in scenarios involving large label shifts, experimental results may not consistently improve with an increase in the proportion of selected pseudo-label samples. This is because the accuracy of these samples tends to decrease as the number of pseudo-label samples increases. Therefore, more erroneous samples are introduced during training, which adversely affects the learning of the ALS classifier and the estimation of importance weights. For example, in all cases containing... The Dirichlet shift dataset contains Both the Acc and F-score values ​​are less than those containing The value of .

[0125] Furthermore, all label shifting methods improved prediction performance compared to WW, especially with larger label shifts. This observation provides compelling evidence for the effectiveness of the importance weight estimation strategy. Moreover, the ALS framework achieved better performance than the semi-supervised method CBST, particularly with label shifts. This finding further underscores the effectiveness of the weight estimation and pseudo-label selection strategies employed within the ALS framework.

[0126] Table 1. ACC values ​​on the Dirichlet dataset

[0127]

[0128] In some cases, the Acc metric may not fully capture the accuracy of minor categories. Therefore, the F-score metric was incorporated into the study, as shown in Table 2. Experimental results show that, in most cases, the improvement in F-score is greater than that in Acc, indicating that the ALS framework can handle minor categories more effectively. This indirectly suggests that ALS can mitigate label shift and avoid bias towards major categories.

[0129] Table 2. F-scores on the Dirichlet dataset

[0130]

[0131] To illustrate that the ALS proposed in this invention can alleviate label shift through a self-trained intermediate distribution, this embodiment demonstrates... Figure 4 Some visualizations are presented, showing the label shift w⁻¹ between the intermediate and source distributions. Experimental results demonstrate that the ALS framework can effectively reduce label distribution discrepancies, whether positive or negative.

[0132] Furthermore, to evaluate the performance of the ALS framework on medical images, a publicly available medical image dataset was used. This dataset was divided into two subsets: one containing 500 labeled samples and the other containing 2000 unlabeled samples. The proportion of medical images in the labeled dataset was determined by the offset parameter α ∈ [0.5, 1, 5] (taking values ​​of 0.5, 1, and 5, respectively). Ten random samples were performed for each distribution parameter, and a two-layer fully connected neural network was used as the base classifier. The experimental results are shown in Table 3, demonstrating that the ALS framework exhibits superior performance compared to existing label offset techniques in all offset scenarios. Notably, on the dataset with α = 0.9 and r = 0.5, the ALS framework achieved an average performance improvement of over 4%. These findings fully validate the reliability and effectiveness of the ALS framework.

[0133] Table 3. Acc and F-score values ​​on the medical imaging dataset.

[0134]

[0135] It should be understood that, although this embodiment Figure 2 The steps are shown sequentially as indicated by the arrows, but they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are performed; they can be executed in other orders. Figure 2At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0136] Example 2

[0137] Based on the tag shift adaptive correction method in Embodiment 1, this embodiment discloses a tag shift adaptive correction device, such as... Figure 5 As shown, the label shift adaptive correction device includes: a data acquisition module 401, an initial target classifier training module 402, a pseudo-label sample screening module 403, an alignment module 404, a new training set construction module 405, and a final target classifier training module 406, wherein:

[0138] The data acquisition module 401 is used to acquire labeled source data and unlabeled target data, wherein the source data and target data satisfy the label shift assumption;

[0139] The initial target classifier training module 402 is used to calculate the estimated importance weights based on the traditional label shifting method, and then train the initial target classifier by combining it with cross-entropy loss.

[0140] The pseudo-label sample screening module 403 is used to calculate the category probability of the target data through the initial target classifier, approximate the target distribution corresponding to the target data according to the estimated importance weight, and screen high-confidence pseudo-label samples according to the target distribution approximation result and the category probability.

[0141] Alignment module 404 is used to align the label conditional distribution of the high-confidence pseudo-label sample with the source data using kernel mean matching, while applying sample weight constraints to keep the marginal distribution of the labels unchanged;

[0142] The new training set construction module 405 is used to combine the aligned pseudo-label samples with the source data to construct a new training set that approximates the intermediate distribution.

[0143] The final target classifier training module 406 is used to recalculate the estimated importance weights on the new training set based on the traditional label shifting method, and train the final target classifier by minimizing the empirical loss.

[0144] In this embodiment, the specific working process and working principle of the data acquisition module 401, the initial target classifier training module 402, the pseudo-label sample screening module 403, the alignment module 404, the new training set construction module 405, and the final target classifier training module 406 are the same as those in Embodiment 1, and therefore will not be described again in this embodiment. Each unit module can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit module can be embedded in or independent of the processor in the computer device in hardware form, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above unit modules.

[0145] Example 3

[0146] like Figure 6 The diagram illustrates a terminal device disclosed in this embodiment, comprising a transmitter, a receiver, a memory, and a processor. The transmitter transmits instructions and data, the receiver receives instructions and data, the memory stores computer-executed instructions, and the processor executes the computer-executed instructions stored in the memory to implement the method described in Embodiment 1 above.

[0147] It is important to note that the aforementioned memory can be either standalone or integrated with the processor. When the memory is set up independently, the terminal device also includes a bus for connecting the memory and the processor.

[0148] Example 4

[0149] This embodiment discloses a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method in Embodiment 1 above.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A tag shift adaptive correction method, characterized in that, The method includes: Obtain labeled source data and unlabeled target data from a medical image dataset, wherein the source data and target data satisfy the label shift assumption; The importance weights are estimated based on the traditional label shifting method, and the initial target classifier is obtained by training with cross-entropy loss. The initial target classifier is used to calculate the class probability of the target data, and the estimated importance weights are used to approximate the target distribution corresponding to the target data; high-confidence pseudo-label samples are selected based on the target distribution approximation and the class probability. Kernel mean matching is used to align the label conditional distribution of the high-confidence pseudo-label samples with that of the source data, while sample weight constraints are applied to keep the marginal distribution of labels unchanged; The aligned pseudo-label samples are combined with the source data to construct a new training set that approximates the intermediate distribution; Based on the traditional label shifting method, the estimated importance weights are recalculated on the new training set, and the final target classifier is obtained by training by minimizing the empirical loss. The high-confidence pseudo-label samples are aligned with the label conditional distribution of the source data using kernel mean matching, as shown below: ； In the formula, Indicates the total number of categories; Indicates the data sample category; Indicates the source data number The kernel mapping expectation of features under class-conditional distribution; Represents the mathematical expectation operator; Representation of features Obey the source data Class label conditional distribution; Indicates the target data number Weighted feature kernel mapping expectation under class-conditional distribution; Representation of features Obey the target data Class label conditional distribution; Indicates the first The weights corresponding to pseudo-labeled samples; When kernel mean matching is used to align the label conditional distributions of the high-confidence pseudo-labeled samples with those of the source data, the final loss function is: ； In the formula, The kernel matrix representing the pseudo-labeled samples; Indicates the first The cross-core matrix between source data and pseudo-labeled samples; Indicates the first The regularization coefficient of the class; Indicates the transpose operation; Indicates the first The number of high-confidence pseudo-labeled samples; Indicates the first Number of samples in the source data.

2. The tag shift adaptive correction method according to claim 1, characterized in that, The expression for the initial target classifier is: ； In the formula, Indicates the initial target classifier; Indicates the first Estimated importance weights for each class; Indicates the total number of categories; Indicates the data sample category; Indicates the first The source data sample corresponds to the first The class label indicates the variable; Indicates in the parameter Under the classifier, the first Individual source data samples Belongs to the The probability of a class category; Indicates the parameters of the classifier; Indicates the first Feature data of each source data sample; Indicates the sample number, where, This represents the total number of samples in the source data. This represents the total number of samples in the target data.

3. The tag shift adaptive correction method according to claim 2, characterized in that, The optimization objective when screening high-confidence pseudo-labeled samples is: ； In the formula, This represents a set of pseudo-labels for the target data; Indicates the first The true labels of each source data sample; Indicates the first The confidence threshold for the class.

4. The tag shift adaptive correction method according to claim 2, characterized in that, The selection criteria for high-confidence pseudo-label samples are as follows: ； In the formula, Indicates the first The pseudo-label indicator vector of each target sample; Indicates the first The optimal class for each target sample; Indicates the first Each target sample is in the initial target classifier The probability of the following categories; Indicates the first The confidence threshold for the class.

5. The tag shift adaptive correction method according to claim 4, characterized in that, The confidence threshold The method for determining it is as follows: Set sample selection ratio Then the first The number of samples to be selected for each class is ,in This is a rounding function. For the first Approximate results of the target distribution of the class. The total number of samples in the target data. For the target data, the first The set of tags corresponding to the class; The first After sorting the class probabilities of the target data in descending order, take the top [number]. The minimum value among these probabilities is used as the confidence threshold. .

6. The tag shift adaptive correction method according to any one of claims 1 to 5, characterized in that, The final target classifier is obtained by training by minimizing the empirical loss. The function expression for minimizing the empirical loss is: ； In the formula, This represents the classifier to be trained; This represents the estimated importance weights recalculated on the new training set; This represents the set of weights for the aligned pseudo-labeled samples. This represents the total number of samples in the source data; Represents the cross-entropy loss function; Indicates the first The true labels of each source data; This represents the total number of high-confidence pseudo-labeled samples; Indicates the first The weights of each pseudo-labeled sample.

7. A tag shift adaptive correction device, characterized in that, The device employing the tag shift adaptive correction method according to any one of claims 1 to 6 includes: The data acquisition module is used to acquire centrally labeled source data and unlabeled target data in a medical image dataset, wherein the source data and target data satisfy the label shift assumption; The initial target classifier training module is used to calculate the estimated importance weights based on the traditional label shifting method and train the initial target classifier by combining it with cross-entropy loss. The pseudo-label sample screening module is used to calculate the class probability of the target data through the initial target classifier, approximate the target distribution corresponding to the target data according to the estimated importance weight, and screen high-confidence pseudo-label samples according to the target distribution approximation result and the class probability. The alignment module is used to align the label conditional distribution of the high-confidence pseudo-label samples with the source data using kernel mean matching, while applying sample weight constraints to keep the marginal distribution of labels unchanged. A new training set module is constructed to combine the aligned pseudo-label samples with the source data to construct a new training set that approximates the intermediate distribution. The final target classifier training module is used to re-estimate the importance weights on the new training set based on the traditional label shifting method, and obtain the final target classifier by minimizing the empirical loss.

Images