Class imbalance data model training method based on double views and semi-supervised learning

Abstract

The invention discloses a class imbalance data model training method based on double views and semi-supervised learning, and the method comprises the following core steps: firstly, carrying out the data enhancement of an original image, and generating a weak enhancement view and two strong enhancement views; secondly, generating a depolarization false label with smaller deviation by utilizing a prediction result of the weak enhancement view and combining a dynamically updated global category confidence library; meanwhile, according to the original prediction of the amblyopia, active sampling is carried out to generate a pseudo-negative label for reverse supervision; and finally, through a multi-task optimization framework, combining depolarization loss, negative learning loss and consistency loss to jointly guide model training. According to the method, false label deviation is reduced through depolarization learning, the model discrimination capability is enhanced through negative learning, the model robustness is improved through consistency learning, and the learning effect and generalization capability of the model on class imbalance data can be remarkably improved under the condition of a small amount of labeled data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and computer vision, in particular to a deep learning model training method for processing class imbalance data in a semi-supervised learning scenario. The present application can be widely applied to visual tasks such as image classification, target detection, semantic segmentation, etc. that require a large amount of labeled data. BACKGROUND

[0002] Deep learning models have achieved remarkable success in various computer vision tasks, but this usually relies on large-scale, high-quality human-labeled data sets. However, in practical applications, it is costly and time-consuming to obtain such data sets. Semi-supervised learning (Semi-Supervised Learning) aims to utilize a large amount of unlabeled data and a small amount of labeled data for model training, which is an effective way to reduce labeling costs.

[0003] However, in many real-world scenarios, there is a serious class imbalance problem in the data distribution, i.e. the number of samples of some classes (majority classes) is much larger than that of other classes (minority classes). When semi-supervised learning is superimposed with class imbalance, the challenge becomes particularly severe. Traditional semi-supervised learning methods, such as pseudo-labeling, tend to trust the high confidence predictions of the model. On class-imbalanced data, the model will naturally produce higher confidence for the majority class, thereby generating a large number of biased pseudo-labels that are biased towards the majority class. These biased pseudo-labels will further exacerbate the model's bias during the training process, leading to a significant decline in model performance, especially in the ability to recognize minority classes.

[0004] Therefore, how to effectively solve the pseudo-label bias caused by class imbalance in semi-supervised visual scenarios and improve the model's generalization ability on imbalanced data is a technical problem that needs to be solved in the current technical field. SUMMARY

[0005] To solve the above technical problems, the present application provides a class imbalance data model training method based on dual-view and semi-supervised learning. This method organically integrates three modules of debiasing learning, negative learning and consistency learning through a multi-task collaborative optimization framework, which can significantly reduce pseudo-label bias and improve the performance of the model in class imbalance scenarios.

[0006] The main technical scheme of the present application is: a class imbalance data model training method based on dual-view and semi-supervised learning, comprising the following steps: S1: obtaining the original image in the training data and performing data augmentation processing on the original image to generate a weak augmented view and two strong augmented views; The first task is to enhance the input image to generate multiple data samples of different perspectives, including weak enhancement and strong enhancement, which are used to generate stable predictions and diverse features, respectively. By generating diverse data samples with different noise and structural disturbances, the model can learn more robust feature representations and lay the foundation for subsequent consistency learning. For each input original image X, three views with different enhancement intensities are generated: wherein the weak enhancement view (X w ): uses slight random data enhancement operations such as RandomCrop and RandomHorizontalFlip. The purpose is to generate views closest to the original image in semantics, which are used to produce relatively stable and reliable prediction results. The strong enhancement view (X s1 ,X s2 ): uses more aggressive data enhancement operations such as ColorJitter, GaussianBlur, and RandomErasing. Two strong enhancement views X s1 and X s2 are generated independently. The purpose is to introduce greater visual disturbance to train the robustness of the model. S2: input the weak enhancement view into the model to be trained to generate a weak view prediction result. This result usually includes a logits vector z w before the Softmax layer and an original probability vector calculated by the Softmax function. S3: adjust the weak view prediction result based on the global class confidence library to generate a debiased pseudo label. S4: sample from the classes other than the highest confidence class based on the weak view prediction result to generate a pseudo-negative label. S5: input the two strong enhancement views into the model to be trained to obtain their respective strong view prediction results. S6: calculate the debiased loss L dcl based on the debiased pseudo label and the first strong view prediction result. S7: calculate the negative learning loss L ncl based on the pseudo-negative label and the second strong view prediction result. S8: calculate the consistency loss L con based on the two strong view prediction results. S9: update the parameters of the model to be trained based on the sum of the debiased loss, the negative learning loss, and the consistency loss.

[0007] As preferred, the specific process of generating the debiased pseudo label in step S3 is as follows: The weakly supervised prediction result (log vector) z w is adjusted to generate the debiased soft label p w according to the following formula: p w ’=σ(z w -φlogΘ); wherein σ(·) represents a softmax function, φ is a preset adjustment coefficient for controlling the debiasing intensity, φ is an empirical parameter, generally between 0.1 and 1, for example, set to 0.3, and Θ is a historical average confidence vector in the global class confidence library; this operation can adaptively reduce the confidence of the majority class (Θ value is high) and improve the confidence of the minority class (Θ value is low); Then, the One-Hot operation is performed on p w ’ to generate the debiased pseudo label y dbs : y dbs =One-Hot(p w ’), that is, the class with the highest probability is found to generate a hard label vector with 1 in the position of the class and 0 in the remaining positions, and this vector is the final debiased pseudo label y dbs .

[0008] The core of this step is to dynamically adjust the prediction of the model using a global class confidence library Θ to generate a pseudo label with less bias.

[0009] As preferred, the pseudo negative label y neg is generated in the following manner in step S4: S41: Based on the weakly supervised prediction result, the class with the highest confidence is determined as the predicted positive class; S42: Among the remaining classes except the predicted positive class, m classes are randomly selected as pseudo negative classes, and m is the number of pseudo negative labels set artificially, for example, m=3; S43: A vector with 1 in the positions corresponding to the pseudo negative classes and 0 in the remaining positions is generated, which is the pseudo negative label y neg .

[0010] This step aims to provide the model with a reverse supervision signal of "it is not what". The process uses the original, uncorrected weakly supervised prediction probability p w obtained in step S2 (directly input z w into the ssoftmax function).

[0011] As preferred, the calculation process of the debiased loss is as follows: The logarithm z s1According to formula p s1 '=σ(z s1 Symmetric enhancement is performed using +φlogΘ); then the debiasing loss L is calculated according to the following formula. dcl : L dcl =-I[max(p w ')≥δ]y dbs log(p s1 '); I[·] is the indicator function, δ is the confidence threshold, and y dbs To remove biased false labels. The meaning of this formula is: only when the model's confidence in the original prediction for that sample is sufficiently high (max(p...)) w Only when the value of ')≥δ) is the biased pseudo-label y used. dbs Supervise it.

[0012] This loss is used to utilize the biased pseudo-label y dbs The prediction of the first strong view is supervised. To make the training more stable, symmetric augmentation and confidence filtering mechanisms are introduced.

[0013] Preferably, the negative learning loss L cnl Determined by the following formula: L ncl =-y neg log(1-p s2 ); In the formula, y neg It's a pseudo-negative label, p s2 =σ(z s2 ),z s2 It is the logarithm of the second strong view prediction result. This loss is used to utilize the pseudo-negative label y. neg Negative supervision forces the model to reduce its prediction probability for these negative classes.

[0014] Preferably, the consistency loss L con The calculation method is as follows: L con =(p s1 -p s2 ) 2 ; In the formula, p s1 =σ(z s1 ), p s2 =σ(z s2 ), z s1 and z s2 These are the strong view predictions for each of the two strongly enhanced views. This loss is used to ensure that the model makes consistent predictions for different strongly enhanced versions of the same original image, thus improving the model's robustness.

[0015] Preferably, the sum of the bias reduction loss, negative learning loss, and consistency loss is calculated using the following formula: L total =L dcl +L ncl +λ con L con ; λ con The weighting coefficient for consistency loss is an empirical parameter, ranging from approximately 0.1 to 1.5.

[0016] Preferably, the global category confidence database records the historical average confidence of each category during the training process, and updates it using the momentum update formula Θ. c ←λΘ c +(1-λ)θ c t Update; where Θ is to the right of the arrow. c θ represents the historical average confidence level of category c before this update. c t λ is the average confidence level of category c calculated in this iteration t, and λ is the momentum coefficient (for example, it can be 0.999).

[0017] The substantial effects of this invention are: Effective reduction of pseudo-label bias: By introducing a dynamic probability redistribution mechanism based on a global category confidence library, this method can adaptively suppress the overconfidence of the majority class and improve the effective confidence of the minority class, thereby generating pseudo-labels with higher quality and less bias.

[0018] Enhancing the model's discriminative ability: Through an innovative pseudo-negative label learning mechanism, this method provides the model with rich reverse supervision signals, forcing the model to learn subtle features that distinguish similar categories, sharpening the decision boundary, and enhancing semantic discriminative ability.

[0019] Enhancing the robustness and stability of the model: The consistency learning strategy based on two strong views forces the model to maintain predictive invariance to visual perturbations in the input, reducing the risk of misjudgment caused by input perturbations and enhancing the robustness of the model.

[0020] Achieving efficient end-to-end training: This invention seamlessly integrates bias-reduction learning, negative learning, and consistency learning into a collaborative optimization framework, supporting an end-to-end training process without the need for complex, multi-stage training strategies, thereby improving training efficiency and automation. Attached Figure Description

[0021] Figure 1 This is a flowchart of a method for training a class-imbalanced data model based on dual-view and semi-supervised learning according to the present invention. Figure 2This is a schematic diagram of a training method for an imbalanced data model based on dual-view and semi-supervised learning according to the present invention. Detailed Implementation

[0022] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0023] Example: This example illustrates a method for training a class-imbalanced data model based on dual views and semi-supervised learning, such as... Figure 1 As shown, it includes the following steps: S1: Obtain the original images from the training data and perform data augmentation on the original images to generate one weakly augmented view and two strongly augmented views; S2: Input the weakly enhanced view into the model to be trained to generate a weakly enhanced view prediction result; S3: Based on the weak view prediction results and the preset global category confidence library, adjust the weak view prediction results to generate biased pseudo-labels; S4: Based on the weak view prediction results, sample from categories other than the highest confidence category to generate pseudo-negative labels; S5: Input the two strongly augmented views into the model to be trained to obtain the corresponding strongly augmented view prediction results; S6: Based on the debiased pseudo-labels and the prediction result of the first strong view, calculate the debiasing loss L. dcl ; S7: Based on the pseudo-negative label and the prediction result of the second strong view, calculate the negative learning loss L. ncl ; S8: Calculate the consistency loss L based on the prediction results of the two strong views. con ; S9: Update the parameters of the model to be trained based on the sum of the bias reduction loss, the negative learning loss, and the consistency loss.

[0024] like Figure 2 As shown, this plan can be divided into five tasks: Image preprocessing: The first task is to enhance the input image to generate data samples from multiple different perspectives. This includes two methods: weak enhancement (such as random cropping and flipping) and strong enhancement (such as color jittering and noise injection), which are used to generate stable predictions and diverse features, respectively.

[0025] By generating diverse data samples with varying noise and structural perturbations, the model learns more robust feature representations, laying the foundation for subsequent consistency learning. For each input original image X, three views with different enhancement intensities are generated: Weak Enhanced View (X) w): This employs mild random data augmentation operations, such as random cropping and random horizontal flipping. The aim is to generate a view that is semantically closest to the original image, thus producing relatively stable pseudo-labels.

[0026] Enhance vision Figure 1 (X s1 This approach employs more aggressive data augmentation techniques, including color jitter, Gaussian blur, and random erasing. The aim is to introduce greater visual perturbations to train the model's robustness to noise.

[0027] Enhance vision Figure 2 (X s2 ): Adopted with X s1 Similar but independent strong enhancement operations. (With X) s1 Together they form a dual-strong enhanced view, which is used for subsequent multi-view consistency loss calculation.

[0028] Debiased Learning: In the second task, a dynamic probability reassignment mechanism is introduced to alleviate the class bias problem in pseudo-label learning. This mechanism is based on a global class confidence library Θ, used to measure the historical performance of each class during training. This scheme also maintains a global class confidence library C={c1,c2,…,c…} N}, where N is the total number of categories, c k This represents the average confidence level or learning performance of the k-th class during the current training phase.

[0029] Θ c ←λΘ c +(1-λ)θ c t c∈{0,…,C-1}; Where θ c t λ ∈ (0,1) represents the average confidence score of class c at the t-th iteration. The parameter λ ∈ [0,1) is the momentum coefficient, which defaults to 0.999. It is worth noting that only samples from the target unlabeled data are used when calculating class confidence scores. The confidence score library can indirectly reflect the inherent biases of the model during training. A low confidence score indicates poor training performance for that class, while a very high score suggests potential overconfidence in assigning the corresponding class label.

[0030] For classes with better training performance (usually with high confidence), a conservative prediction strategy is adopted to moderately reduce their confidence and mitigate the bias caused by overconfidence. For minority classes with poor performance, an aggressive strategy is adopted to increase their effective confidence, thereby narrowing the decision boundary of the majority class and expanding the decision space of the minority class, achieving balanced optimization among classes.

[0031] Specifically, for the logits (logarithmic vector) z output by the weakly enhanced image model w Adjustments are made to generate the bias-corrected soft label p. w ': p w '=σ(z w -φlogΘ), y dbs =One-Hot(p w '), Where σ(·) represents the softmax function, Θ is the historical average confidence vector in the global category confidence library, and φ is the adjustment coefficient that controls the debiasing intensity, which is set to 0.3 here.

[0032] Next, a bias-reduced pseudo-label y is generated using a One-Hot operation. dbs =One-Hot(p w ')=[0,0,…,0,1,0,…,0] T , The predicted output p of the student model used to supervise the bias removal branch s1 ', which has undergone symmetric enhancement treatment: p s1 '=σ(z s1 +φlogΘ); The corresponding debiased contrast loss is defined as: L dcl =-I[max(p w ')≥δ]y dbs log(p s1 '); I[·] is the indicator function, δ is the confidence threshold, and y dbs To remove biased pseudo-labels.

[0033] Negative Learning: In the third task, an active pseudo-negative sample generation mechanism is proposed to enhance the model's discriminative ability from the perspective of reverse semantics. Unlike traditional methods that only use low-confidence samples to generate pseudo-labels, this method randomly selects several candidate categories that do not contain the currently predicted category (i.e., argmax(p)). w The model constructs "pseudo-negative class" supervision signals to guide the model to explicitly exclude these interference categories.

[0034] Let y be a pseudo-negative label neg Generated by the following function, pw y represents the predicted probability of a weakly enhanced image after passing through the model's output, where m is the number of artificially set pseudo-negative labels. neg ∈ψ(p w ,m), Where, ψ(p,m)={v|v∈{0,1} C ,Σ i C v i =m argmax p ≠1}; This indicates that m classes are randomly selected from all classes except the class with the highest confidence level as pseudo-negative classes, forming a sparse negative label vector y. neg =[0,1,…,0,1,0,…,1] T .

[0035] Subsequently, the output p of the student model on the negative branch is optimized using negative class cross-entropy loss. s2 :L ncl =-y neg log(1-p s2 ).

[0036] Multi-view Consistency Learning: The fourth task ensures that the model maintains consistent predictions across different augmentation perspectives through a consistency loss function. Specifically, it calculates the prediction difference between two strongly augmented versions, enabling the model to remain robust when facing ambiguous or uncertain samples and reducing the negative impact of erroneous pseudo-labels.

[0037] L con =(p s1 -p s2 ) 2 .

[0038] Gradient backpropagation optimization: Finally, the losses output by the above modules are integrated for gradient backpropagation, including the loss of pseudo-label removal, pseudo-negative sample label removal, and consistency constraint loss, to train the deep learning model.

[0039] L total =L dcl +L ncl +λ con L con .

[0040] This method can significantly reduce model bias caused by class imbalance without requiring extensive manual annotation, thereby improving the robustness and generalization ability of the model in practical applications.

[0041] (1) Verify the effectiveness of the method of the present invention on the Office-Home dataset, which contains four domains: art, clip art, products and real world, with a total of 65 categories and obvious class imbalance.

[0042] Data preparation and augmentation: The resolution of the input images is uniformly adjusted to 224×224. For each input image, a weakly augmented view (randomly cropped and flipped) and two independent strongly augmented views (applying strong augmentation strategies such as RandAugment) are generated.

[0043] Model and parameter settings: The model to be trained is ResNet-50. The adjustment coefficient φ is set to 0.3, the confidence threshold δ is set to 0.95, the number of pseudo-negative labels m is set to 3, the momentum coefficient λ is set to 0.999, and the consistency loss weight λ is set to... con Set it to 1.0.

[0044] Training process: End-to-end training is performed according to the steps described in the invention. In each training batch, the original prediction p of the weak view is first calculated. w and removing pseudo-labels y dbs Then generate pseudo-negative labels y. neg Next, calculate the prediction p for the two strong views. s1 and p s2 Finally, calculate L according to the formula. dcl ,L ncl ,L con And sum them up to get L total Perform gradient backpropagation.

[0045] Experimental results: The experimental results show that, compared with the baseline method, the model using the method of this invention achieves an average accuracy of 73.7% and 76.7% in scenarios with 1 / 4 labeled data and 1 / 2 labeled data, respectively, which is a significant performance improvement and proves the effectiveness of this invention in solving the semi-supervised imbalance problem.

[0046] (2) Validation of the object detection task on the Pascal VOC dataset.

[0047] Data and Model: The Pascal VOC dataset with a resolution of 256×256 was used. The model to be trained was a YOLOv5-based object detection model.

[0048] Training process: The loss function (L) proposed in this invention is applied... dcl ,L ncl ,L con This is integrated into the classification loss branch of YOLOv5 to optimize the training process for unlabeled data. The parameter settings are similar to those in Example 1.

[0049] Experimental results: Experimental results show that the method of the present invention can also effectively improve the performance of the model on the imbalanced object detection task. The average accuracy reaches 75.7% and 77.4% in scenarios with 1 / 4 labeled data and 1 / 2 labeled data, respectively, which is better than the existing mainstream methods.

[0050] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0051] Although this document uses terms such as enhanced view, debiased pseudo-label, and pseudo-negative label frequently, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of this invention; interpreting them as any additional limitation would contradict the spirit of this invention.

Claims

1. A method for training a class imbalance data model based on dual-view and semi-supervised learning, characterized in that, The method comprises the following steps: S1: obtaining an original image in training data and performing data enhancement processing on the original image to generate a weak enhanced view and two strong enhanced views; S2: inputting the weak enhanced view into a to-be-trained model to generate a weak view prediction result; S3: adjusting the weak view prediction result based on the weak view prediction result and a preset global class confidence library to generate a debiased pseudo label; S4: sampling in categories other than the highest confidence category according to the weak view prediction result to generate a pseudo negative label; S5: inputting the two strong enhanced views into the to-be-trained model respectively to obtain respective corresponding strong view prediction results; S6: calculating a debiased loss L based on the debiased pseudo label and the first strong view prediction result dcl ; S7: calculating a negative learning loss L based on the pseudo negative label and the second strong view prediction result ncl ; S8: Calculate consistency loss L based on two strong view prediction results con ; S9: updating parameters of the to-be-trained model according to a sum of the debiased loss, the negative learning loss and the consistency loss.

2. The class imbalance data model training method based on dual-view and semi-supervised learning according to claim 1, characterized in that, In the step S3, a specific process of generating the debiased pseudo label is as follows: z = z - z w The adjusted soft label p is generated by the following formula w : p w ’=σ(z w -φlogΘ). In the formula, σ(·) represents a softmax function, φ is a preset adjustment coefficient, and Θ is a historical average confidence vector in the global class confidence library; Then p w is One-Hot operated to generate the debiased pseudo label y dbs : y dbs = One-Hot(p w ’).

3. The class imbalance data model training method based on dual-view and semi-supervised learning according to claim 2, characterized in that, In the step S4, the pseudo negative label y neg is generated by the following manner: S41: determining a category with the highest confidence in the weak view prediction result as a predicted positive category; S42: randomly selecting m categories as pseudo negative categories in the remaining categories other than the predicted positive category, and m is a manually set number of pseudo negative labels; S43: generate a vector with 1 at the position corresponding to the pseudo-negative class and 0 at the other positions, which is the pseudo-negative label y neg .

4. The class imbalance data model training method based on dual-view and semi-supervised learning according to claim 2, characterized in that, A calculation process of the debiased loss is as follows: log z s1 symmetrically enhanced by p s1 = σ(z s1 + φ log Θ); then the debiasing loss L dcl is calculated as follows: L dcl = -I[max(p w ’)≥δ]y dbs log(p s1 ’) I[·] is an indicator function, δ is a confidence threshold, y dbs is the debiased pseudo-label.

5. The class imbalance data model training method based on dual-view and semi-supervised learning according to claim 3, characterized in that, The negative learning loss L cnl is determined by the following equation: L ncl =-y neg log(1-p s2 ); In the formula, y neg is a pseudo-negative label, p s2 = σ(z s2 ), z s2 is the logarithm of the second strong view prediction result.

6. The class imbalance data model training method based on dual-view and semi-supervised learning according to claim 2, characterized in that, The consistency loss L con The calculation method is as follows: L con = (p s1 -p s2 ) 2 ; where p s1 = σ(z s1 ), p s2 = σ(z s2 ), z s1 and z s2 are two strong view prediction results corresponding to the strong enhanced view respectively.

7. The class-imbalanced data model training method based on dual-view and semi-supervised learning according to any one of claims 1 to 6, characterized in that, The sum of the debiased loss, the negative learning loss and the consistency loss is calculated through the following formula: L total =L dcl +L ncl +λ con L con ; λ con is a weight coefficient for the consistency loss.

8. The class imbalance data model training method based on dual-view and semi-supervised learning according to claim 2 or 4 or 5 or 6, characterized in that, The global category confidence library records the historical average confidence of each category during the training process, and is updated by a momentum update formula Θ c ← λΘ c + (1-λ)θ c t is updated; wherein, Θ c on the right side of the arrow is the historical average confidence of category c before this update, θ c t is the average confidence of category c calculated in this iteration t, and λ is a momentum coefficient.

Images